The tools ecosystem

The tools ecosystem

The ecosystem includes a collection of open source SDMX software tools cooperating to solve official statistics use cases.

Fusion Metadata Registry (FMR)

Fusion Metadata Registry (FMR) is primarily a structural metadata registry - a database designed specifically for storing SDMX artefacts including Concepts, Codelists and Data Structure definitions. A registry is useful in many cases where statistics are collected, produced and exchanged for externalising, centralising and controlling the structural metadata.

Benefits of using a registry for managing structural metadata include:

  • sharing and re-use efficiencies
  • standardisation
  • harmonisation of concepts making datasets comparable and consistent
  • improved metadata and data governance

FMR provides several additional features in addition to its core registry role that help to support practical use cases:

Web user interface for creating and maintaining SDMX structure artefacts

FMR’s web user interface provides metadata managers and data modellers with a graphical way to explore the registry’s content, create new structures and maintain existing ones.

Many artefacts can also be exported to Excel, modified and reloaded - particularly useful for maintaining large Codelists and Constraints.

Data validation

Datasets can be loaded and validated - checked that they comply with the Checking data quality helps to deliver effective data governance and improves the efficiency of data exchanges, avoiding the costly re-processing or re-transmission.

FMR applies nine standard validation rules:

Rule Test Applied
Syntactically compliant The XML, JSON, CSV or Excel is well formed
Duplicate observations Uniqueness - there is only one observation value reported for each time period
Mandatory attributes All mandatory attributes are reported
Obs status OBS_STATUS is consistent with the observation value
Time period format E.g. FREQ=M means the TIME_PERIOD format must be YYYY-MM
Valid calculations Balance equalities defined used Validation Schemes
Valid constraints The data is within the universe defined by Data Constraints
Valid representation Each component complies with the representation defined in the DSD
Valid structure The dimensions and attributes are consistent with the DSD

Conversion of SDMX data between transmission formats

Datasets can be converted between most SDMX formats including:

  • SDMX-ML (XML) 2.0 and 2.1
  • SDMX-JSON versions v1 (SDMX 2.1) and v2 (SDMX 3.0)
  • SDMX-CSV versions v1 (SDMX 2.1) and v2 (SDMX 3.0)
  • EDI
  • Excel (note that data worksheets must follow a specific layout)

Data mapping

Datasets can be mapped (transformed) to different DSDs using SDMX Structure Mapping.

Typical use cases include:

  • Data collection - transforming received datasets to the DSDs needed for internal processing
  • Data reporting - transforming reporting datasets into the structures specified by the data collector (e.g. IMF SDDS ECOFIN, or Eurosystem NAMAIN for National Accounts)
  • Public data dissemination - transforming datasets that use an internal DSD with many dimensions to a simpler DSD suitable for external publication
  • Statistics production - harmonising the structure and coding of datasets

More about FMR

FusionXL (FXL)

FusionXL is an Excel interface for FMR that complements and provides an alternative to the web user interface. The following functions are supported:

  • Structure Explorer - browse the structural metadata and work with artefacts such as Concepts, Codelists and DSDs in Excel worksheets. Create new structures and maintain existing ones, for instance: add new codes to a Codelist.
  • Data Author, Validate, Convert - create SDMX datasets using helpers to set the correct structure and pick codes. Validate the dataset and convert it to any of the standard SDMX transmission formats including XML, JSON and EDI.

Fusion Transformer (FT)

Fusion Transformer is a command line application for converting SDMX data between transmission formats. It provides an alternative to the FMR data conversion service finding application in scripts and data processing automation.

.Stat Suite

The .Stat Suite is a standard-based, componentised, open-source platform for the efficient production and dissemination of high-quality statistical data. It is:

  • An environment to manage the data lifecycle for official stats (design, collect, process, disseminate).
  • An environment to explore data and develop various reporting and dissemination experiences.
  • An ‘SDMX-native’ environment, building on best practices in statistical data modelling.

More about .Stat Suite

Fusion Workbench (FWB)

The FMR Workbench is a remote registry browser and metadata maintenance tool which brings the strengths and capabilities of the FMR User Interface to any SDMX compliant registry, implementing an SDMX RESTful API. One significant use case for FMR Workbench users is the ability to connect to .Stat-Suite spaces for browsing and maintaining .Stat-Suite structural metadata.

More about FWB