This is a post I had been wanting to write for some time, and one I had promised in another similar post a few weeks ago.

When you start doing macroeconomic analysis —whether for research or for presentations— the usual workflow involves going to the data source (usually some international organization that aggregates information), downloading the dataset, and then importing it into your tool of choice to proceed with the analysis.

This workflow, however, has several drawbacks. First, it is not replicable: there are several manual steps that are not strictly documented. Moreover, it is not scalable, because if at some point a new variable becomes necessary for the analysis and was not included in the first data extraction, you need to repeat the whole process from the beginning.

Therefore, for the applied analyst, it becomes essential to automate the data extraction process, so that most of the work is focused on analysis and less on the plumbing required to gather the necessary information.

In this sense, the optimal solution is to rely on an API (application programming interface) that allows you to find, understand, and extract information quickly, so that the analysis is fully reproducible and scalable. For example, in previous posts I showed how, in a simplified way, to access the API of ECLAC and the World Economic Outlook published by the International Monetary Fund (IMF) to automate this workflow.

In this post, however, I want to broaden the data sources and present, at least conceptually, a general method that allows systematic access to data from international organizations. This is possible thanks to a standardized information model or protocol called SDMX, which allows us to access data in a more or less homogeneous way.

In what follows, I will review the SDMX model conceptually, show a relatively simplified workflow to extract information, and present some applied examples. Finally, for the IMF case, a library called imfapi has been created, which facilitates data extraction using the IMF’s SDMX API, and which I explore below.

What is SDMX?

As stated on its official website:

SDMX, which stands for Statistical Data and Metadata eXchange, is an ISO standard designed to describe statistical data and metadata, standardize their exchange, and improve their efficient distribution among statistical and similar organizations.

SDMX is sponsored by eight international organizations:

1. Bank for International Settlements (BIS),
2. European Central Bank (ECB),
3. Statistical Office of the European Union (Eurostat),
4. International Labour Organization (ILO),
5. International Monetary Fund (IMF),
6. Organisation for Economic Co-operation and Development (OECD),
7. United Nations Statistics Division (UNSD), and
8. World Bank (WB)

Each of these organizations has implemented SDMX in their statistical information systems and provides APIs following this structure. This can be seen in the table below:

Thus, analysts can access a wide range of macroeconomic and social data from different international sources using a standardized approach.

General workflow to extract SDMX data

Although each API may have its particularities and the SDMX model is quite broad, the general workflow for extracting data using SDMX APIs can be summarized in four steps. Each one corresponds to a key concept within the SDMX architecture and helps us understand how statistical agencies organize and expose their data.

1. Identifying the dataflow. The first step consists in identifying the dataflow. Conceptually, a dataflow is the container for a thematic dataset. It works like the “database” or the statistical domain published by an institution (for example, monetary statistics, bank balance sheets, exchange rates, or macroeconomic indicators). Identifying the appropriate dataflow involves reviewing the list of dataflows available in the API and selecting the one that contains the information relevant for the desired analysis. Without this step, it is impossible to proceed to a valid query, because every SDMX dataset starts from a specific dataflow.

2. Retrieving the data structure. Once the dataflow has been selected, the next step is to retrieve the associated data structure, known as the Data Structure Definition (DSD). Conceptually, the DSD is the logical schema of the dataset: it describes how observations are organized and which elements are involved in their identification. The DSD specifies the dimensions (such as country, indicator, frequency, or currency), which are key variables that classify the data and determine how they are combined to form series or observations. It also defines the main measure, which is the numerical value of the data (for example, a price index or a monetary balance), as well as additional attributes that provide contextual information (such as unit, seasonal adjustment method, or observation status). Understanding this structure is essential to know what filters can be applied and how to build the query.

3. Consulting the codelists. The third step consists in consulting the codelists. Each dimension defined in the DSD has an associated codelist, which is the authorized set of possible values for that dimension. Codelists act like a standard dictionary of codes that ensures interoperability between institutions and systems. For example, a codelist may contain all country codes, the types of time frequency (annual, quarterly, monthly), or the different indicators within a statistical domain. Consulting them is fundamental to know which values can be used when filtering a query and how they should be correctly encoded in the URL or request body. As we will see later, many times codelists show what is possible but not necessarily what is valid for a particular dataflow. Therefore, it is often necessary to investigate whether the DSD contains constraints on values and, if not, to resort to brute force by downloading a sample of the data and extracting unique values. In any case, the important thing is that the user becomes familiar with the data.

4. Extracting the data. Finally, the fourth step is to extract the data. Once we know the dataflow, its structure, and the codes allowed for each dimension, we can build a valid query to the SDMX API. This involves selecting the relevant combinations of codes (for example, country, indicator, and frequency) and applying time or detail filters as permitted by the API. The response is usually received in a standardized format, such as SDMX-JSON or SDMX-XML, which can later be processed in tools like R or Python for analysis, visualization, or integration into a reproducible pipeline. This step is where all the previous work materializes, as it turns the conceptual understanding of the SDMX structure into a concrete data request.

Some examples

In this section we will work through an example using the IMF APIs and, later, the European Central Bank (ECB) APIs. Although we go step by step, the key idea is that what we are going to do is:

1. Identify the available dataflows (“databases”)
2. Retrieve the data structure of the dataflow of interest (“the variables”)
3. Consult the codelists associated with the dimensions of interest (“the valid codes”)
4. Extract the data (“the final query”)

SDMX at the IMF

The IMF provides a fairly complete collection of SDMX 3.0 APIs that give access to a wide range of macroeconomic data. These can be found, after registering, here:

The IMF’s SDMX APIs—and in fact any SDMX 3.0 API—are designed following a hierarchical structure, where the URL acts like a path that moves from the most general to the most specific. This allows the user to progressively explore the statistical system, starting from complete catalogs and drilling down to a specific dataset or even a single time series.

The core idea is that each segment of the path adds a restriction, a filter, or a level of precision. Thus, a request can mean “show everything” or “give me exactly this data structure”, depending on how many parameters are included. Let’s go through this step by step.

Available data: the dataflow

The starting point is the base URL, which in this case is https://api.imf.org/external/sdmx/3.0. From this point we can say that the IMF API is divided into two big families: a structure family and a data family. The former allows you to explore catalogs and structures, while the latter allows you to extract specific data.

For example, if we append /structure/dataflow to the base URL, we get the complete catalog of dataflows available in the API—that is, we are asking “give me all the dataflows the IMF has”. If, on the other hand, we append /structure/datastructure, we are essentially asking “give me all the DSDs the IMF has”. This level is quite general: it does not apply any filters to the extraction, which is useful when we are not yet sure what we need.

To illustrate this, we will carry out the first step of the workflow: obtaining all the dataflows available in the IMF API and locating the dataflow corresponding to the World Economic Outlook (WEO). This can be done with the following R code:

# 1. Base URL SDMX 3.0 of the IMF
base_url <- "https://api.imf.org/external/sdmx/3.0"

# 2. Get all dataflows
flows_resp <- httr2::request(paste0(base_url, "/structure/dataflow")) |>
  httr2::req_perform()

flows_json <- flows_resp |>
  httr2::resp_body_string() |>
  jsonlite::fromJSON(flatten = TRUE)

flows <- flows_json$data$dataflows

# Dataflows table as a tibble
df_flows <- tibble::tibble(
  id      = flows$id,
  name    = flows$names.en,
  version = flows$version,
  agency  = flows$agencyID,
  dsd_urn = flows$structure
)

df_flows |> 
  dplyr::select(id, name, version, agency) |>
  dplyr::arrange(id) |> 
  kableExtra::kable() 

In the table above we can see all the dataflows available in the IMF API. Each dataflow has a unique identifier (id), a descriptive name (name), a version (version), a responsible agency (agency), and a URN (Uniform Resource Name) not shown in the previous table but pointing to its associated data structure (dsd_urn).

In our case we are interested in the WEO dataflow. We can filter the dataflow of interest as follows:

# Keep only the WEO dataflow
weo <- df_flows |>
  dplyr::filter(id == "WEO")

weo |> 
  kableExtra::kable()

Data structure: the dimensions

Now that we have the dataflow of interest, the next step is to understand which variables (dimensions) are available in its data structure. To do this, the API we need to query has the following form: /structure/datastructure/{agency}/{dsd_id}/{version}, where {agency}, {dsd_id}, and {version} are the components we must extract from the WEO dataflow URN. Note that if we do not specify these three fields, we run the risk of getting outdated information.

How do we obtain this information? The good thing is that, when we query the dataflow, this information is contained in the associated URN:

# Parse the URN of WEO's datastructure
urn <- weo$dsd_urn
urn

## [1] "urn:sdmx:org.sdmx.infomodel.datastructure.DataStructure=IMF.RES:DSD_WEO(9.0+.0)"

Note that here we have the three essential pieces:

1. The agency responsible for the DSD: IMF.RES
2. The DSD identifier: DSD_WEO
3. The DSD version: 9.0+.0

To systematically extract these pieces we can use regular expressions in R as follows:

m     <- base::regexec("DataStructure=([^:]+):([^()]+)\\(([^)]+)\\)", urn)
parts <- base::regmatches(urn, m)[[1]]

agency_dsd  <- parts[2]   # e.g. "IMF.RES"
dsd_id      <- parts[3]   # e.g. "DSD_WEO"
version_dsd <- parts[4]   # e.g. "9.0+.0"

# Download WEO's datastructure and inspect dimensions
dsd_url <- paste0(
  base_url,
  "/structure/datastructure/",
  agency_dsd, "/", dsd_id, "/", version_dsd
)
dsd_url

## [1] "https://api.imf.org/external/sdmx/3.0/structure/datastructure/IMF.RES/DSD_WEO/9.0+.0"

Now that we have the URL for WEO’s datastructure, we can download it and inspect the available dimensions:

dsd_resp <- httr2::request(dsd_url) |>
  httr2::req_perform()

dsd_json <- dsd_resp |>
  httr2::resp_body_string() |>
  jsonlite::fromJSON(flatten = TRUE)

# "Regular" dimensions: COUNTRY, INDICATOR, FREQUENCY
dims_main <- dsd_json$data$dataStructures$dataStructureComponents.dimensionList.dimensions[[1]] |>
  tibble::as_tibble() |>
  dplyr::select(position, id, type)

# Time dimension: TIME_PERIOD
time_dim <- dsd_json$data$dataStructures$dataStructureComponents.dimensionList.timeDimensions[[1]] |>
  tibble::as_tibble() |>
  dplyr::select(position, id, type)

# Full dimension order
dims_clean <- dplyr::bind_rows(dims_main, time_dim) |>
  dplyr::arrange(position)

dims_clean |> 
  kableExtra::kable()

In the table above we can see the available dimensions in the WEO dataflow (COUNTRY, INDICATOR, FREQUENCY, and TIME_PERIOD). These dimensions are key for understanding how the data are organized and what filters we can apply when extracting specific information.

Note also that, although the query ran without errors, we first explored dsd_json to understand where and how the information was stored.

Codelists: the valid codes

As a third step, and before extracting data, it is important to know the valid codes for each dimension of the dataset. This is essential because, when building the final query, we must make sure to use the exact values that the API recognizes. Otherwise, it will return an error. For example, if we want to obtain real GDP growth for a specific country, we need to know how the IMF codes the country, the indicator, and the frequency. It is not enough to write “Bolivia” or “Real GDP growth”: we must use the official codes defined by the dataset’s structure.

Each of the WEO dimensions—COUNTRY, INDICATOR, FREQUENCY, and TIME_PERIOD—has an authorized set of possible values: the codelist. For example, country codes might be BOL, ARG, or USA; indicators could be NGDP_RPCH, LUR, and so on; and valid frequencies are typically A (annual), Q (quarterly), etc. An important detail is that each dataset can have different codelists, even for conceptually similar items. The IMF might use one list of countries for WEO, another for BOP (Balance of Payments), and another for monetary statistics. This means that codes that work for one dataflow will not necessarily work for another.

Therefore, if we want to query WEO data, we cannot make up codes or rely on old lists: we need to query the official codelists from the IMF. The only safe way to do this is to start from the dataflow, retrieve its structure (the DSD), and from there obtain the correct lists of codes for each dimension. This procedure ensures that the query is reproducible, accurate, and fully compatible with the IMF’s internal definitions.

To obtain the codelists associated with the WEO dataflow, we use a specific URL from the IMF’s SDMX API. The general pattern is: /structure/dataflow/{agency}/{dataflow_id}/+, where {agency} and {dataflow_id} come from the dataflow itself (IMF.RES and WEO, respectively, in our case). The + symbol indicates that we want the latest available version. But the most important part is the parameters we add at the end: ?detail=full&references=descendants¹.

For our case, the full URL to obtain WEO’s codelists is:

weo_struct_url <- paste0(
  base_url,
  "/structure/dataflow/",
  weo$agency, "/", weo$id, "/+",
  "?detail=full&references=descendants"
)
weo_struct_url

## [1] "https://api.imf.org/external/sdmx/3.0/structure/dataflow/IMF.RES/WEO/+?detail=full&references=descendants"

When we send a request to this URL, we get a response that includes all the codelists associated with the WEO dataflow. We can process this response to extract the specific codelists we need for each dimension:

weo_struct_resp <- httr2::request(weo_struct_url) |>
  httr2::req_perform()

weo_struct_json <- weo_struct_resp |>
  httr2::resp_body_string() |>
  jsonlite::fromJSON(flatten = TRUE)

# Codelists table
codelists_tbl <- weo_struct_json$data$codelists

codelists_tbl |> 
  dplyr::select(id, name, version, agencyID) |> 
  head() |> 
  kableExtra::kable()

Note that in the table above we have extracted quite a bit of information about the codelists available for the WEO dataflow. In the following code, we summarize what we obtained:

# Summary view of the codelists
weo_codelists_overview <- tibble::tibble(
  cl_id   = codelists_tbl$id,
  cl_name = ifelse(is.na(codelists_tbl$names.en), 
                   yes = codelists_tbl$name,
                   no = codelists_tbl$names.en
                   ),
  n_codes = sapply(codelists_tbl$codes, nrow)
)

weo_codelists_overview |> 
  dplyr::arrange(desc(cl_id)) |> 
  kableExtra::kable()

In the table above we can see a summary of the codelists available for the WEO dataflow. Each codelist has an identifier (cl_id), a descriptive name (cl_name), and the number of codes it contains (n_codes). In our case, we are particularly interested in the codelists associated with the COUNTRY, INDICATOR, and FREQUENCY dimensions: CL_WEO_COUNTRY, CL_WEO_INDICATOR, and CL_FREQUENCY.

For example, for CL_WEO_COUNTRY:

weo_country_codes <- codelists_tbl$codes[[which(codelists_tbl$id == "CL_WEO_COUNTRY")]] |>
  tibble::as_tibble() |>
  dplyr::transmute(
    code    = id,
    name_en = names.en  # in CL_WEO_COUNTRY it appears as names.en1
  )
weo_country_codes |>
  head(20) |> 
  kableExtra::kable()

In the table above we can see the valid country codes for the WEO dataflow. Each country has a code (code) and an English name (name_en). These codes are essential for building precise queries to the IMF API.

Next, we look for the available INDICATOR codes in WEO:

weo_indicator_codes <- codelists_tbl$codes[[which(codelists_tbl$id == "CL_WEO_INDICATOR")]] |>
  tibble::as_tibble() |>
  dplyr::transmute(
    code    = id,
    name_en = names.en
  )

weo_indicator_codes |>
  kableExtra::kable()

Finally, for the frequency:

weo_frequency_codes <- codelists_tbl$codes[[which(codelists_tbl$id =="CL_FREQ")]] |>
  tibble::as_tibble() |>
  dplyr::transmute(
    code    = id,
    name_en = names.en
  )
weo_frequency_codes |>
  kableExtra::kable()

Once we have the valid codes for the dimensions of interest, we are ready to proceed to the extraction of specific data from the WEO dataflow. This is the final step of the SDMX workflow and will allow us to obtain the time series needed for our macroeconomic analysis.

Extracting data: the final query

With valid codes in hand for the COUNTRY, INDICATOR, and FREQUENCY dimensions, we are now in a position to build the final query to extract specific data from the WEO dataflow. From the IMF API’s perspective, the general data endpoint has the form:

/data/{context}/{agencyID}/{resourceID}/{version}/{key}[?c]

In our case, {context} will be dataflow, {agencyID} will be IMF.RES, {resourceID} will be WEO, and {version} will be the latest stable version (+). That is, everything before {key} is already known from the previous steps. What we still need to define is the key and, optionally, the filter parameter c.

The key parameter is the central piece of the query: it represents the combination of dimension values that identifies a series or a subset of the “cube” of data. The API defines it as “the combination of dimension values identifying series or slices of the cube”. In practice, this translates into a string where dimension values are concatenated following the order defined in the DSD.

In the case of WEO, we saw that the order is COUNTRY.INDICATOR.FREQUENCY, so a key like BOL.NGDP_RPCH.A means: country Bolivia (BOL), indicator “Real GDP, percent change” (NGDP_RPCH), and annual frequency (A). The API also allows the use of wildcards (*), for example BOL.*.A (all annual series for Bolivia) or *.NGDP_RPCH.A (annual real GDP growth for all countries). This makes the key a very compact way of specifying which dimension combinations we want.

The c parameter, in turn, allows additional filtering using a syntax like c[DIM]=value. The documentation describes it as “Filter data by component value (e.g. c[FREQ]=A)”, and it also allows logical operators. In our simple example, we do not use c because we already restrict country, indicator, and frequency directly in the key, and we let the API return the full history of the series. However, c is very useful when you want to apply additional filters (for example, narrowing the time range, selecting only certain currencies, methods, or observation statuses) without complicating the key, or when you are working with richer dimensional structures.

Suppose we want to obtain real GDP growth (percent change) for Bolivia at annual frequency.

# Code for Bolivia (according to the WEO codelist)
bol_code <- "BOL"

# Code for the "Real GDP, percent change" indicator
rgdp_code <- "NGDP_RPCH"

# Annual frequency (A) according to the FREQUENCY dimension
freq_code <- "A"

# SDMX key for WEO: COUNTRY.INDICATOR.FREQUENCY
weo_key <- paste(bol_code, rgdp_code, freq_code, sep = ".")
weo_key

## [1] "BOL.NGDP_RPCH.A"

Now that we have the key, we proceed to make the query:

# SDMX 3.0 data extraction URL (correct data query)
weo_data_url <- paste0(
  base_url,
  "/data/dataflow/",
  weo$agency, "/", weo$id, "/+/",
  weo_key
)

weo_data_resp <- httr2::request(weo_data_url) |>
  httr2::req_url_query(
    dimensionAtObservation = "TIME_PERIOD",
    attributes             = "dsd",
    measures               = "all",
    includeHistory         = "false"
  ) |>
  httr2::req_perform()

weo_data_json <- weo_data_resp |>
  httr2::resp_body_string() |>
  jsonlite::fromJSON(flatten = FALSE)

In weo_data_json we have the full API response containing the requested data. The structure of this object is quite complex, as it is a list of lists with several different levels of depth.

str(weo_data_json, max.level = 4)

## List of 2
##  $ meta: Named list()
##  $ data:List of 2
##   ..$ dataSets  :'data.frame':	1 obs. of  4 variables:
##   .. ..$ structure               : int 0
##   .. ..$ action                  : chr "Replace"
##   .. ..$ series                  :'data.frame':	1 obs. of  1 variable:
##   .. .. ..$ 0:0:0:'data.frame':	1 obs. of  2 variables:
##   .. ..$ dimensionGroupAttributes:'data.frame':	1 obs. of  1 variable:
##   .. .. ..$ :0:::List of 1
##   ..$ structures:'data.frame':	1 obs. of  6 variables:
##   .. ..$ dataSets   :List of 1
##   .. .. ..$ : int 0
##   .. ..$ links      :List of 1
##   .. .. ..$ :'data.frame':	2 obs. of  2 variables:
##   .. ..$ dimensions :'data.frame':	1 obs. of  2 variables:
##   .. .. ..$ series     :List of 1
##   .. .. ..$ observation:List of 1
##   .. ..$ measures   :'data.frame':	1 obs. of  1 variable:
##   .. .. ..$ observation:List of 1
##   .. ..$ attributes :'data.frame':	1 obs. of  3 variables:
##   .. .. ..$ dimensionGroup:List of 1
##   .. .. ..$ series        :List of 1
##   .. .. ..$ observation   :List of 1
##   .. ..$ annotations:List of 1
##   .. .. ..$ : list()

The next step, therefore, is to transform this response into a more manageable format, such as a tibble in R that contains time observations and their corresponding values. First, we extract the dataSets object from the previous list and everything it contains:

# Series 0:0:0 (the only one we requested in WEO)
obs_df <- weo_data_json$data$dataSets$series$`0:0:0`$observations

obs_values <- obs_df[1, ] |>
  unlist(use.names = FALSE) |>
  as.numeric()

# TIME_PERIOD from the structure
time_dim    <- weo_data_json$data$structures$dimensions$observation[[1]]$values |> 
  unlist(use.names = FALSE)

# Final tibble (year, value)
weo_bol_rgdp <- tibble::tibble(
  time  = as.integer(time_dim),
  value = obs_values
) |>
  dplyr::arrange(time)

weo_bol_rgdp |> 
  kableExtra::kable()

Finally, we can also make a richer query. For example, we can extract both real GDP growth and the primary deficit as a percentage of GDP for Bolivia, Spain, and the United States between 2015 and 2025:

# 1. Query parameters

countries  <- c("BOL", "ESP", "USA")
indicators <- c("NGDP_RPCH", "GGXONLB_NGDP")  # Real GDP % and deficit % of GDP
freq_code  <- "A"                             # Annual frequency

country_key   <- paste(countries,  collapse = "+")
indicator_key <- paste(indicators, collapse = "+")
weo_key_multi <- paste(country_key, indicator_key, freq_code, sep = ".")

# SDMX 3.0 data URL (IMF WEO)
weo_data_url_multi <- paste0(
  base_url,
  "/data/dataflow/",
  weo$agency, "/", weo$id, "/+/",
  weo_key_multi
)


# 2. API call (1980–2030)

weo_data_resp_multi <- httr2::request(weo_data_url_multi) |>
  httr2::req_url_query(
    dimensionAtObservation = "TIME_PERIOD",
    attributes             = "dsd",
    measures               = "all",
    includeHistory         = "false",
    startPeriod            = "1980",
    endPeriod              = "2030"
  ) |>
  httr2::req_perform()

weo_data_json_multi <- weo_data_resp_multi |>
  httr2::resp_body_string() |>
  jsonlite::fromJSON(flatten = FALSE)


# 3. Dimension structures

series_dims_resp <- weo_data_json_multi$data$structures$dimensions$series[[1]]
obs_dim_resp     <- weo_data_json_multi$data$structures$dimensions$observation[[1]]

time_values <- obs_dim_resp$values |>
  unlist(use.names = FALSE)


# 4. Extract the series

series_df <- weo_data_json_multi$data$dataSets$series

# Each element is a data.frame with $attributes and $observations
series_list <- lapply(series_df, function(col) col$observations)
names(series_list) <- names(series_df)

# Function to unnest a series
extract_series_tbl <- function(obs_df, series_name) {
  # Dimension indices from "0:0:0"
  idx <- as.integer(strsplit(series_name, ":", fixed = TRUE)[[1]]) + 1L
  
  # Map indices to COUNTRY / INDICATOR / FREQUENCY codes
  dim_values <- purrr::map2_chr(
    seq_along(series_dims_resp$id),
    idx,
    ~ series_dims_resp$values[[.x]]$id[.y]
  )
  names(dim_values) <- series_dims_resp$id
  
  # Number of observations
  n_obs <- ncol(obs_df)
  if (is.null(n_obs) || n_obs == 0) {
    return(tibble::tibble(
      COUNTRY     = character(0),
      INDICATOR   = character(0),
      FREQ        = character(0),
      TIME_PERIOD = integer(0),
      value       = numeric(0)
    ))
  }
  
  # Map column names "0,1,2,...,41,..." to time_values indices
  pos_idx     <- as.integer(colnames(obs_df)) + 1L
  these_times <- as.integer(time_values[pos_idx])
  
  values_num <- obs_df[1, ] |>
    unlist(use.names = FALSE) |>
    as.numeric()
  
  tibble::tibble(
    COUNTRY     = dim_values[["COUNTRY"]],
    INDICATOR   = dim_values[["INDICATOR"]],
    FREQ        = dim_values[["FREQUENCY"]],
    TIME_PERIOD = these_times,
    value       = values_num
  )
}

# Apply to all series
weo_multi_tidy_raw <- purrr::imap_dfr(
  series_list,
  extract_series_tbl
)


# 5. Filter 2015–2025

weo_multi_tidy <- weo_multi_tidy_raw |>
  dplyr::filter(TIME_PERIOD >= 2015, TIME_PERIOD <= 2025) |>
  dplyr::arrange(COUNTRY, INDICATOR, TIME_PERIOD)

# Wide version (one column per indicator)
weo_multi_wide <- weo_multi_tidy |>
  tidyr::pivot_wider(
    id_cols    = c(COUNTRY, TIME_PERIOD),
    names_from  = INDICATOR,
    values_from = value
  )

weo_multi_wide |> 
  kableExtra::kable()

SDMX at the ECB

The European Central Bank (ECB) also offers a fairly complete SDMX 3.0 API to access its statistical data. In addition, the ECB has information on its website explaining each of its dataflows, which makes exploration easier².

The workflow for extracting ECB data is similar to that of the IMF, but with some differences in the URL structure and specific parameters. In addition, the responses from the APIs come in .xml format and the data can be downloaded as .csv files, which must be taken into account when processing the information.

Available data: the dataflow

First, we look up the dataflows available in the ECB API.

# 1. Base URL
base_url_ecb <- "https://data-api.ecb.europa.eu/service/"

# 2. Request dataflows (XML only)
flows_resp_ecb <- httr2::request(
  paste0(base_url_ecb, "dataflow")
) |>
  httr2::req_perform()

xml <- xml2::read_xml(
  httr2::resp_body_string(flows_resp_ecb)
)

# 3. Extract Dataflow nodes
df_nodes <- xml2::xml_find_all(
  xml,
  ".//*[local-name()='Dataflow']"
)

# 4. Extract basic fields + the DSD reference
flows_tbl <- tibble::tibble(
  id      = purrr::map_chr(df_nodes, ~ xml2::xml_attr(.x, "id")),
  agency  = purrr::map_chr(df_nodes, ~ xml2::xml_attr(.x, "agencyID")),
  version = purrr::map_chr(df_nodes, ~ xml2::xml_attr(.x, "version")),
  
  # extract the DSD reference
  dsd_id = purrr::map_chr(
    df_nodes,
    ~ xml2::xml_attr(
        xml2::xml_find_first(.x, ".//*[local-name()='Structure']/*[local-name()='Ref']"),
        "id"
      )
  ),
  
  dsd_agency = purrr::map_chr(
    df_nodes,
    ~ xml2::xml_attr(
        xml2::xml_find_first(.x, ".//*[local-name()='Structure']/*[local-name()='Ref']"),
        "agencyID"
      )
  ),
  
  dsd_version = purrr::map_chr(
    df_nodes,
    ~ xml2::xml_attr(
        xml2::xml_find_first(.x, ".//*[local-name()='Structure']/*[local-name()='Ref']"),
        "version"
      )
  ),
  
  # Extract Name with namespace-independent XPath
  name    = purrr::map_chr(
    df_nodes,
    ~ xml2::xml_text(
        xml2::xml_find_first(.x, ".//*[local-name()='Name']")
      )
  )
)

# 5. Show sorted table
flows_tbl |>
  dplyr::arrange(id) |>
  dplyr::select(agency, version, dsd_id, name) |>
  kableExtra::kable()

In the table above we can see all the dataflows available in the ECB API. Each dataflow has a unique identifier (id), a responsible agency (agency), a version (version), a dsd_id that will be used to identify the data structure, and a descriptive name (name).

Let’s suppose we are interested in the EST dataflow, which shows the euro short-term interest rate (€STR) and reflects the wholesale unsecured euro funding costs of euro area banks for overnight borrowing.

# Keep only the EST dataflow
est <- flows_tbl |> 
  dplyr::filter(id == "EST")

est |>
  kableExtra::kable()

Data structure: the dimensions

As mentioned earlier, the next step is to understand which variables (dimensions) are available in the data structure of the EST dataflow. In this particular case, the ECB has detailed information on its website about what this dataflow contains. However, we are going to verify its dimensions via the API:

# 1. Define DSD location explicitly

base_url_ecb <- "https://data-api.ecb.europa.eu/service/"
dsd_id       <- "ECB_EST1"
dsd_agency   <- "ECB"
dsd_version  <- "latest"   # or "1.0" if you want a fixed version


# 2. Download the DSD (Structure with children: codelists, concepts)

url_dsd <- paste0(
  base_url_ecb,
  "datastructure/", dsd_agency, "/", dsd_id, "/", dsd_version,
  "?references=children"
)

dsd_resp <- httr2::request(url_dsd) |>
  httr2::req_perform()

xml_dsd <- xml2::read_xml(
  httr2::resp_body_string(dsd_resp)
)


# 3. Extract series-level dimensions only

series_dims <- xml2::xml_find_all(
  xml_dsd,
  ".//*[local-name()='DataStructure']
     /*[local-name()='DataStructureComponents']
     /*[local-name()='DimensionList']
     /*[local-name()='Dimension']"
)


# 4. Extract Dimension ID and ConceptIdentity/Ref

dimensions_tbl <- tibble::tibble(
  id = purrr::map_chr(
    series_dims,
    ~ xml2::xml_attr(.x, "id")
  ),
  
  concept_ref = purrr::map_chr(
    series_dims,
    ~ {
      concept_node <- xml2::xml_find_first(
        .x,
        ".//*[local-name()='ConceptIdentity']/*[local-name()='Ref']"
      )
      xml2::xml_attr(concept_node, "id")
    }
  ),
  
  position = seq_along(series_dims)
)

dimensions_tbl |> 
  kableExtra::kable()

Codelists: the valid codes

Once we know the dimensions of the EST dataflow, the next step is to extract the codelists associated with each dimension. In some cases, the DSD contains additional information about the codelists, such as constraints that help us understand which codes are valid. In other cases, it does not.

Suppose the DSD did not show the codelist constraints. In that case, we would have to use a more pragmatic approach and extract a data sample to understand what values our dimensions actually take. This is relatively easy with the ECB because the API returns structured data in .csv format. We can also use the lastNObservations parameter to control the sample size:

est_sample <- httr2::request(
  "https://data-api.ecb.europa.eu/service/data/EST"
) |>
  httr2::req_url_query(
    format = "csvdata",
    lastNObservations = 100
  ) |>
  httr2::req_perform() |>
  httr2::resp_body_string() |>
  readr::read_csv(show_col_types = FALSE)

est_sample |> 
  dplyr::select(KEY, FREQ, BENCHMARK_ITEM, DATA_TYPE_EST, TIME_PERIOD, OBS_VALUE) |>
  head() |> 
  kableExtra::kable()

To check the unique values, we run the following query on the dataflow dimensions:

est_sample |> 
  dplyr::distinct(FREQ, BENCHMARK_ITEM, DATA_TYPE_EST, TITLE) |> 
  kableExtra::kable()

Extracting data: the final query

Now suppose we want to extract the euro short-term compounded interest rate (€STR) with a maturity of 12 months. According to the sample above, the relevant codes are:

est_all <- httr2::request(
  "https://data-api.ecb.europa.eu/service/data/EST/B.EU000A2QQF57.CR"
) |>
  httr2::req_url_query(
    format = "csvdata",
    startPeriod = "2025-01-01",
    endPeriod   = "2025-12-06"
  ) |>
  httr2::req_perform() |>
  httr2::resp_body_string() |>
  readr::read_csv(show_col_types = FALSE)

est_all |> 
  head() |> 
  kableExtra::kable()

We can now plot the results:

library(ggplot2)

est_all |> 
  ggplot(aes(x = TIME_PERIOD, y = OBS_VALUE)) +
  geom_line(color = "steelblue", linewidth = 1.2) +
  labs(
    title = "Euro short-term compounded interest rate (€STR) - 12 months",
    x = "Date",
    y = "Interest rate (%)"
  ) +
  theme_bw()

A new library: imfapi

A few months ago, someone in my LinkedIn network shared a post by Christopher Smith announcing the creation of a new library that “provides user-friendly functions for programmatic access to macroeconomic data from the ‘SDMX 3.0 IMF Data API’ of the International Monetary Fund.” The library in question is available on CRAN and works stably. Furthermore, it is part of the EconDataverse, which I recommend exploring. Below I will show, in a simplified way, how to access the same WEO data we saw earlier.

Using imfapi to extract WEO data

First, we load the library:

library(imfapi)

Then we retrieve the dataflows available in the IMF API:

dataflows <- imfapi::imf_get_dataflows()

dataflows |> 
  dplyr::glimpse()

## Rows: 72
## Columns: 6
## $ id           <chr> "IMTS", "IL", "MFS_OFC", "ISORA_LATEST_DATA_PUB", "WPCPER…
## $ name         <chr> "International Trade in Goods (by partner country) (IMTS)…
## $ description  <chr> "The International trade in goods by partner country data…
## $ version      <chr> "1.0.0", "13.0.1", "7.0.0", "4.0.0", "6.0.0", "1.0.0", "9…
## $ agency       <chr> "IMF.STA", "IMF.STA", "IMF.STA", "ISORA", "IMF.STA", "IMF…
## $ last_updated <chr> "2025-03-28T17:18:44.429573Z", "2025-08-29T09:08:22.00741…

Now we filter for the WEO dataflow:

weo_flow <- dataflows |> 
  dplyr::filter(id == "WEO")

weo_flow |> 
  kableExtra::kable()

And we obtain the dimensions of the WEO dataflow:

dsd_weo <- imfapi::imf_get_datastructure(dataflow_id = "WEO")
dsd_weo |> 
  kableExtra::kable()

Next, we extract the codelists associated with the WEO dataflow:

dim_list <- list()

for (dim_id in dsd_weo$dimension_id) {
  dim_list[[dim_id]] <- imfapi::imf_get_codelists(
    dimension_ids = dim_id,
    dataflow_id   = "WEO"
  )
}

dim_list |> 
  dplyr::bind_rows() |> 
  dplyr::group_by(dimension_id) |> 
  dplyr::summarise(n_codes = dplyr::n()) |> 
  kableExtra::kable()

In this case we have 338 codes for the COUNTRY dimension, 6 codes for the FREQUENCY dimension, and 145 valid codes for INDICATOR.

Finally, we extract WEO data for Bolivia and real GDP growth:

data <- imfapi::imf_get(
  dataflow_id = "WEO",
  dimensions  = list(
    COUNTRY   = "BOL",
    INDICATOR = "NGDP_RPCH",
    FREQUENCY = "A"
  )
)

data |>
  kableExtra::kable()

In the table above we can see the WEO data for Bolivia and real GDP growth at annual frequency. The imfapi library considerably simplifies the data extraction process from the IMF API, allowing users to focus on data analysis rather than on the technical details of the SDMX API.

Conclusions

The aim of this post was to show that, for the applied analyst, working with SDMX APIs is not an end in itself but a way to build cleaner, more reproducible, and more scalable workflows. Instead of relying on manual downloads and poorly documented steps, SDMX allows us to find, understand, and extract macroeconomic data from different organizations under a common logic.

Through the IMF and ECB examples we saw that the general process—identifying the dataflow, reviewing its structure, consulting the codelists, and finally extracting the data—is repeated consistently, which makes it easier to automate and apply to other datasets. Moreover, tools like imfapi show how this approach can be simplified even further, reducing the technical burden so that most of the effort can be dedicated to analysis.