Introduction to OpenEnergyMetadata (OEM)¶

The development of tutorials for the Open Energy Family takes place publicly in a dedicated tutorial repository. Please report bugs and suggestions as new issues. If Jupyter Notebooks are new to you and you'd like to get an introduction, have a look at this less than 10 minute introduction video. Official installation instructions are available on jupyter's readthedocs page.

license: GNU Affero General Public License Version 3 (AGPL-3.0)
copyright: Reiner Lemoine Institut
authors: jh-RLI

Takeaways¶

• Understand what OEMetadata is and where it is applied.
• How to use OEMetadata.
• Find helpful resources.
• Where to start contributing as a developer.

The OpenEnergyMetadata¶

Introduction¶

OpenenEnergyMetadata(OEM) is a standardised metadata format. OEM are used to provide relevant information about a dataset on the OEP "It´s data explaining the data :)". OEM are mandatory for oficially publishing data on the OEP, because this will clarify any licensing issues in advance and also provide context for the review that is part of every publishing process. You can find out how to publish your data in the publishing guidebook. There are descriptions on how to upload OEM on the Open Energy Platform itself with a wizard and a more technical description) for using the API in scrips. If a dataset has already been uploaded to the OEP, then the metadata can be seen in the view of the table by clicking on the tab Meta Information right next to Data. The OEP also offers functions for editing and downloading metadata, which can be accessed in the Meta Information view . Make sure to sign in to the OEP in advance to be able to edit the data. You can see what permissions you have in the options on the rigth hand side by clicking on Permissions.

Compatible with frictionless data¶

OEMetadata development is partially based on the specification of frictionless data. The advantage is that OEM is compatible with frictionless datapackage format and thus a solid basis for applicability is ensured. In the easiest case, a frictionless datapackage is a JSON-file that is placed directly next to the actual data - the former describing the latter.

Developer information¶

The oemetadata is developed in JSON format and is published on GitHub under the MIT License. The current version of the JSON string can be found in the oemetadata repository. As an open source project, suggestions for improvement and forking are welcome and encouraged. Please take note of our development philosophy when you start contributing.

Structure of the OEMetadata¶

The OEMetadata are, generally speaking, divided into six sections:

1. general
   • Describes the dataset
   • Links to related data tables on the OEP
   • References further documentation
2. spatiotemporal
   • Summarizes spatial and temporal information of the dataset
3. sources
   • Documents used sources with attribution and licenses
4. license
   • The license for the data
   • An open license must be available for OEP upload
5. history
   • Information on who created/edited the metadata
6. data model
   • Structure of the table in which the data is stored,
   • Technical documentation of the table
   • Manual fillout require basic database (SQL) knowledge

There is a detailed overview in which all fields that occur in the metadata are described. You will also find sample values there.

Suggested workflow to get started¶

There are several ways to create the OEMetadata. One way is to download the template JSON file and fill it manually in your text editor of choice. Another option is to use a tool on the OEP website called wizard. There, a table can be created, data can be uploaded from an excel or CSV file and the metadata can be filled in. The wizard is much more user friendly for beginners, so we recommend using the tool. Once the metadata has been created, in order to go into review it should be downloaded and put in an inssue on GitHub to start the review process. If you are unable to do this, create an issue and get in touch with a reviewer who will help you with those steps.

The metadata can be filled in to the best of your knowledge in a first step. If there are any difficulties, it is sufficient to create an issue on GitHub to get in contact with the OEP developers. The issue template can be used for this purpose. Unfortunately at this stage of development a GitHub account is still required for this.