Skip to content

Open Energy Metadata#

Access the Open Energy Metadata

https://github.com/OpenEnergyPlatform/oemetadata

What is it and what is it good for?#

Open Energy Metadata (OEMetadata) is a metadata standard designed specifically to be used on data for energy (systems) research. For Science, a metadata standard can provide unambiguity, transparency, objectivity, reliability, verifiability, openness, integrity and novelty. In short - it can help with good scientific practice. OEMetadata adhere to the FAIR principles, i.e. they ensure Findability, Accessibility, Interoperability, and Reuse of digital assets.

Structural Design#

Data and metadata come in different levels of structuredness.

Image title

Levels of structuredness

OEMetadata are semi-structured and designed to accompany the data themselves. They can describe every structural element of tabular data

Image title

Tabular Data

When designing OEMetadata the following existing standards and agreements were considered:

  • Dublin Core -> Documenting digital documents
  • Frictionless Data Package -> A container format for in a single 'package'.
  • ISO_19115 -> Geodata
  • INSPIRE -> Regulation on administrative and other specialized Geodata
  • DataCite -> Metadata Schema for data citations
  • schema.org -> Schemas for structured data markup on web pages
  • PROV -> W3C specification providing a vocabulary to interchange provenance information
  • DCAT-AP -> Application profile for data portals in Europe based on the Data Catalog Vocabulary

They shaped OEMetadata to varying degrees. Some of them were too general, others too specific. The following requirements lead us to define our own standard:

  • Compatibility with csv and database tables
  • machine- and human readability
  • Coverage of all aspects of metadata
  • Coverage of all data and tailoring to energy system analysis
  • Compliance with FAIR criteria
  • Extensibility
  • Well defined compatibility with ontology and linked open data
  • Compatibility with DCAT-AP was originally planned, but the standard was found partly incompatible with datapackages
  • Compatible with all: timeseries, geodata, parameter collections, data produced by machines, data collaboratively collected

Our concept to include ontology references is depiced in a poster (pdf) which was created during the development stage. The resulting standard is based on Data Packages. The file format is JSON (and JSON-LD). In it's simplest form a Tabular Data Package is a csv file containing data, accompanied by a JSON file which describes the name and structure of the data. OEMetadata take the standard set of keys and possible values and extend it with ones useful for energy research. It is inspired by Dublin Core, INSPIRE and DataCite. The development process is organized on GitHub and open for everyone to see and participate in. The repository contains the following useful files:

metadata_key_description.md - contains a description of each metadata key template.json - contains an empty metadata string example.json - contains a basic metadata example with filled fields schema.json - JSON schema ensures a well defined standard

Creation and management#

  • Creating a table on the OEP can be done through the wizard. The menu has a section that helps you fill out OEMetadata to accompany your data
  • To help with the creation of a standalone metadata file, the OEP has a metadata creator (You will need to be logged in to use it)
  • There is a review process to maintain any given metadata on the OEP. This process was created to replace the now deprecated process on GitHub. As a owner of a table on the OEP, you can ask for a review which will start a guided review process. At the end of the process a badge will be assigned to the metadata depicting its level of completeness:
  • Iron – Technically required for data structure
  • Bronze – Basic description of the data
  • Silver – Supplement description of the data
  • Gold – Extended description of the context
  • Platinum – Ontological annotation

Metadata keys with a description and example#

The standard is under active development and currently available in version 1.6.0. The table with a full key description is shown here for convenience, but may not be as up to date as in the repository.