API tutorial 3 - Plot data and spatial data¶

Repository: https://github.com/openego/oedialect

Please report bugs and improvements here: https://github.com/OpenEnergyPlatform/examples/issues
How to get started with Jupyter Notebooks can be found here: https://realpython.com/jupyter-notebook-introduction/
Please ensure you have read the Terms of use here: https://openenergy-platform.org/legal/tou/

In [1]:

__copyright__ = "Reiner Lemoine Institut, Zentrum für nachhaltige Energiesysteme Flensburg"
__license__   = "GNU Affero General Public License Version 3 (AGPL-3.0)"
__url__       = "https://github.com/openego/data_processing/blob/master/LICENSE"
__author__    = "wolfbunke, Ludee"

__copyright__ = "Reiner Lemoine Institut, Zentrum für nachhaltige Energiesysteme Flensburg" __license__ = "GNU Affero General Public License Version 3 (AGPL-3.0)" __url__ = "https://github.com/openego/data_processing/blob/master/LICENSE" __author__ = "wolfbunke, Ludee"

Introduction¶

This is an important information!

This is an information!

This is your task!

This tutorial gives you an overview of the OpenEnergy Platform and how you can work with the REST-full-HTTP API in Python.
The full API documentaion can be found on ReadtheDocs.io.

Part III - How to work with the OpenEnergy Platform (OEP)¶

0 Setup token
1 Select data
2 Make a pandas dataframe
3 Plot a dataframe (geo plot)

Part III¶

0. Setup token¶

The token is used to verify the API interaction for your OEP user.

Do not push your token to GitHub!

In [ ]:

import requests
import pandas as pd
from IPython.core.display import HTML

from token_config import oep_url, get_oep_token

# token
your_token = get_oep_token()

import requests import pandas as pd from IPython.core.display import HTML from token_config import oep_url, get_oep_token # token your_token = get_oep_token()

1. Select data¶

In [ ]:

import geopandas as gpd
from shapely.geometry import Point
import shapely.wkt
from shapely import wkb
from geoalchemy2.shape import to_shape

import geopandas as gpd from shapely.geometry import Point import shapely.wkt from shapely import wkb from geoalchemy2.shape import to_shape

In [ ]:

# select powerplant data
schema = 'supply'
table = 'ego_dp_conv_powerplant'
where = 'version=v0.2.10'

conv_powerplants = requests.get(oep_url+'/api/v0/schema/'+schema+'/tables/'+table+'/rows/?where='+where, )
conv_powerplants.status_code

# select powerplant data schema = 'supply' table = 'ego_dp_conv_powerplant' where = 'version=v0.2.10' conv_powerplants = requests.get(oep_url+'/api/v0/schema/'+schema+'/tables/'+table+'/rows/?where='+where, ) conv_powerplants.status_code

Response [200] succesfully selected data!
Response [404] table doesn't exist!

In [ ]:

# select borders
schema = 'boundaries'
table = 'bkg_vg250_2_lan_mview'

vg = requests.get(oep_url+'/api/v0/schema/'+schema+'/tables/'+table+'/rows/')
vg.status_code

# select borders schema = 'boundaries' table = 'bkg_vg250_2_lan_mview' vg = requests.get(oep_url+'/api/v0/schema/'+schema+'/tables/'+table+'/rows/') vg.status_code

Response [200] succesfully selected data!
Response [404] table doesn't exist!

2. Make a pandas dataframe¶

Create pandas dataframes for each data set returned as API result!

In [ ]:

# Create dataframe from json format
df_pp = pd.DataFrame(conv_powerplants.json())
df_vg = pd.DataFrame(vg.json())

# Create dataframe from json format df_pp = pd.DataFrame(conv_powerplants.json()) df_vg = pd.DataFrame(vg.json())

Let's take a look into our data!

In [ ]:

# Show metadata for a specific dataframe.
df_pp.info()

# Show metadata for a specific dataframe. df_pp.info()

In [ ]:

# List all column names for a specific dataframe. 
df_pp.columns

# List all column names for a specific dataframe. df_pp.columns

In [ ]:

#Print the df_pp dataframe as table.
df_vg

#Print the df_pp dataframe as table. df_vg

3. Plot a dataframe (geo plot)¶

In [ ]:

import geopandas as gpd
import shapely
import matplotlib.pyplot as plt
%matplotlib inline

import geopandas as gpd import shapely import matplotlib.pyplot as plt %matplotlib inline

If we want to apply a change to every entity in a column we can use Pandas apply function.

In [ ]:

# transform WKB to WKT / Geometry
df_pp['geom'] = df_pp['geom'].apply(lambda x:shapely.wkb.loads(x, hex=True))
df_vg['geom'] = df_vg['geom'].apply(lambda x:shapely.wkb.loads(x, hex=True))

# transform WKB to WKT / Geometry df_pp['geom'] = df_pp['geom'].apply(lambda x:shapely.wkb.loads(x, hex=True)) df_vg['geom'] = df_vg['geom'].apply(lambda x:shapely.wkb.loads(x, hex=True))

Let's plot our data!

In [ ]:

# plot powerplants
crs = {'init' :'epsg:4326'}
gdf_pp = gpd.GeoDataFrame(df_pp, crs=crs, geometry=df_pp.geom)
base1 = gdf_pp.plot(color='white', edgecolor='black',figsize=(8, 8))
gdf_pp.plot(ax=base1, color='green')
plt.show()

# plot powerplants crs = {'init' :'epsg:4326'} gdf_pp = gpd.GeoDataFrame(df_pp, crs=crs, geometry=df_pp.geom) base1 = gdf_pp.plot(color='white', edgecolor='black',figsize=(8, 8)) gdf_pp.plot(ax=base1, color='green') plt.show()

In [ ]:

# plot borders
crs = {'init' :'epsg:4326'}
gdf_vg = gpd.GeoDataFrame(df_vg, geometry=df_vg.geom)
base2 = gdf_vg.plot(color='white', edgecolor='black',figsize=(8, 8))
gdf_vg.plot(ax=base2)

plt.show()

# plot borders crs = {'init' :'epsg:4326'} gdf_vg = gpd.GeoDataFrame(df_vg, geometry=df_vg.geom) base2 = gdf_vg.plot(color='white', edgecolor='black',figsize=(8, 8)) gdf_vg.plot(ax=base2) plt.show()

Now we can create a map with two layers.

In [ ]:

# transform WKB to WKT / Geometry
crs1 = {'init' :'epsg:4326'}
crs2 = {'init' :'epsg:3035'}

gdf_pp = gpd.GeoDataFrame(df_pp, crs=crs1, geometry=df_pp.geom)
gdf_vg = gpd.GeoDataFrame(df_vg, crs=crs2, geometry=df_vg.geom)


base = gdf_vg.plot(color='white', edgecolor='black',figsize=(10, 10))

gdf_pp.plot(ax=base, marker='o', markersize=5)
# gdf_vg.plot(ax=base)

plt.show()

# transform WKB to WKT / Geometry crs1 = {'init' :'epsg:4326'} crs2 = {'init' :'epsg:3035'} gdf_pp = gpd.GeoDataFrame(df_pp, crs=crs1, geometry=df_pp.geom) gdf_vg = gpd.GeoDataFrame(df_vg, crs=crs2, geometry=df_vg.geom) base = gdf_vg.plot(color='white', edgecolor='black',figsize=(10, 10)) gdf_pp.plot(ax=base, marker='o', markersize=5) # gdf_vg.plot(ax=base) plt.show()

This plot does not display the data corretly. We need to change the Projections(crs) to something similar.

bug under ubuntu

In [ ]:

from shapely import geos
from geoalchemy2.shape import to_shape
from shapely.geometry import Point

# from ipywidgets import widgets
from IPython.display import display
from IPython.core.display import HTML
from geoalchemy2 import Geometry, WKTElement
import requests
import pandas as pd
# import mplleaflet

from shapely import geos from geoalchemy2.shape import to_shape from shapely.geometry import Point # from ipywidgets import widgets from IPython.display import display from IPython.core.display import HTML from geoalchemy2 import Geometry, WKTElement import requests import pandas as pd # import mplleaflet

In [ ]:

plants_data = requests.get(oep_url+'/api/v0/schema/model_draft/tables/ego_dp_supply_conv_powerplant/rows/?where=scenario=Status+Quo&limit=910',)
regions  =  requests.get(oep_url+'/api/v0/schema/model_draft/tables/renpass_gis_parameter_region/rows/?where=stat_level=999',)
regions.status_code
plants_data.status_code

plants_data = requests.get(oep_url+'/api/v0/schema/model_draft/tables/ego_dp_supply_conv_powerplant/rows/?where=scenario=Status+Quo&limit=910',) regions = requests.get(oep_url+'/api/v0/schema/model_draft/tables/renpass_gis_parameter_region/rows/?where=stat_level=999',) regions.status_code plants_data.status_code

Response [200] succesfully selected data!
Response [404] table doesn't exist!

Let´s transform the crs and plot the data again.

In [ ]:

sq_plants = pd.DataFrame(plants_data.json())
renpass_region_df = pd.DataFrame(regions.json())

# transform WKB to WKT / Geometry
crs = {'init' :'epsg:4326'}

sq_plants['geom'] =sq_plants['geom'].apply(lambda x:shapely.wkb.loads(x, hex=True))
renpass_region_df['geom'] =renpass_region_df['geom'].apply(lambda x:shapely.wkb.loads(x, hex=True))

gdf_plants = gpd.GeoDataFrame(sq_plants, crs=crs, geometry=sq_plants.geom)
gdf_regions = gpd.GeoDataFrame(renpass_region_df, crs=crs, geometry=renpass_region_df.geom)

base = gdf_regions.plot(color='white', edgecolor='black',figsize=(10, 10))

gdf_plants.plot(ax=base)


plt.show()

sq_plants = pd.DataFrame(plants_data.json()) renpass_region_df = pd.DataFrame(regions.json()) # transform WKB to WKT / Geometry crs = {'init' :'epsg:4326'} sq_plants['geom'] =sq_plants['geom'].apply(lambda x:shapely.wkb.loads(x, hex=True)) renpass_region_df['geom'] =renpass_region_df['geom'].apply(lambda x:shapely.wkb.loads(x, hex=True)) gdf_plants = gpd.GeoDataFrame(sq_plants, crs=crs, geometry=sq_plants.geom) gdf_regions = gpd.GeoDataFrame(renpass_region_df, crs=crs, geometry=renpass_region_df.geom) base = gdf_regions.plot(color='white', edgecolor='black',figsize=(10, 10)) gdf_plants.plot(ax=base) plt.show()

Point Plot¶

In [ ]:

import folium
from folium import plugins
import matplotlib.pyplot as plt
%matplotlib inline

import folium from folium import plugins import matplotlib.pyplot as plt %matplotlib inline

In [ ]:

# define map region
map = folium.Map(location=[51, 9], zoom_start=6)

# define map region map = folium.Map(location=[51, 9], zoom_start=6)

In [ ]:

# Use column lon / lat in order to plot map
for name, row in gdf_pp.iloc[:1000].iterrows():
    folium.Marker([row["lat"], row["lon"]], popup=row["type"] ).add_to(map)
#map.create_map('plants.html')
map

# Use column lon / lat in order to plot map for name, row in gdf_pp.iloc[:1000].iterrows(): folium.Marker([row["lat"], row["lon"]], popup=row["type"] ).add_to(map) #map.create_map('plants.html') map

Heat plot for locations¶

In [ ]:

stops_heatmap = folium.Map(location=[51, 9], zoom_start=6)
stops_heatmap.add_child(plugins.HeatMap([[row["lat"], row["lon"]] for capacity, row in df_pp.iloc[:1000].iterrows()]))
stops_heatmap.save("heatmap.html")
stops_heatmap

stops_heatmap = folium.Map(location=[51, 9], zoom_start=6) stops_heatmap.add_child(plugins.HeatMap([[row["lat"], row["lon"]] for capacity, row in df_pp.iloc[:1000].iterrows()])) stops_heatmap.save("heatmap.html") stops_heatmap

Make some statistics¶

Make an interesting API-example you need!

In [ ]: