Building Electricity Networks

Building Electricity Networks#

The preparation process of the PyPSA-Eur energy system model consists of a group of snakemake rules which are briefly outlined and explained in detail in the sections below.

Not all data dependencies are shipped with the git repository. Instead we provide separate data bundles which can be obtained using the retrieve* rules (Retrieving Data). Having downloaded the necessary data,

  • build_shapes generates GeoJSON files with shapes of the countries, exclusive economic zones and NUTS3 areas.

  • build_cutout prepares smaller weather data portions from ERA5 for cutout europe-2013-sarah3-era5 and SARAH for cutout europe-2013-sarah.

With these and the externally extracted ENTSO-E online map topology (data/entsoegridkit), it can build a base PyPSA network with the following rules:

  • base_network builds and stores the base network with all buses, HVAC lines and HVDC links, and determines Voronoi cells for all substations.

Then the process continues by calculating conventional power plant capacities, potentials, and per-unit availability time series for variable renewable energy carriers and hydro power plants with the following rules:

The central rule add_electricity then ties all the different data inputs together into a detailed PyPSA network stored in networks/base_s_{clusters}_elec.nc.

Rule build_cutout#

Create cutouts with atlite.

For this rule to work you must have

See also

For details on the weather data read the atlite documentation. If you need help specifically for creating cutouts the corresponding section in the atlite documentation should be helpful.

Relevant Settings#

atlite:
    nprocesses:
    cutouts:
        {cutout}:

See also

Documentation of the configuration file config/config.yaml at atlite

Inputs#

None

Outputs#

  • cutouts/{cutout}: weather data from either the ERA5 reanalysis weather dataset or SARAH-3 satellite-based historic weather data with the following structure:

ERA5 cutout:

Field

Dimensions

Unit

Description

pressure

time, y, x

Pa

Surface pressure

temperature

time, y, x

K

Air temperature 2 meters above the surface.

soil temperature

time, y, x

K

Soil temperature between 1 meters and 3 meters depth (layer 4).

influx_toa

time, y, x

Wm**-2

Top of Earth’s atmosphere TOA incident solar radiation

influx_direct

time, y, x

Wm**-2

Total sky direct solar radiation at surface

runoff

time, y, x

m

Runoff (volume per area)

roughness

y, x

m

Forecast surface roughness (roughness length)

height

y, x

m

Surface elevation above sea level

albedo

time, y, x

–

Albedo measure of diffuse reflection of solar radiation. Calculated from relation between surface solar radiation downwards (Jm**-2) and surface net solar radiation (Jm**-2). Takes values between 0 and 1.

influx_diffuse

time, y, x

Wm**-2

Diffuse solar radiation at surface. Surface solar radiation downwards minus direct solar radiation.

wnd100m

time, y, x

ms**-1

Wind speeds at 100 meters (regardless of direction)
_images/era5.png

A SARAH-3 cutout can be used to amend the fields temperature, influx_toa, influx_direct, albedo, influx_diffuse of ERA5 using satellite-based radiation observations.

Description#

Rule clean_osm_data#

This script is used to clean OpenStreetMap (OSM) data for creating a PyPSA-Eur ready network.

The script performs various cleaning operations on the OSM data, including: - Cleaning voltage, circuits, cables, wires, and frequency columns - Splitting semicolon-separated cells into new rows - Distributing values to circuits based on the number of splits - Adding line endings to substations based on line data

Rule build_osm_network#

Rule base_network#

Creates the network topology from a ENTSO-E map extract. (March 2022) or OpenStreetMap data (Aug 2024) as a PyPSA network.

Relevant Settings#

countries:

electricity:
    voltages:

lines:
    types:
    s_max_pu:
    under_construction:

links:
    p_max_pu:
    under_construction:

transformers:
    x:
    s_nom:
    type:

See also

Documentation of the configuration file config/config.yaml at snapshots, Top-level configuration, electricity, load, conventional, links, transformers_cf

Inputs#

  • data/entsoegridkit: Extract from the geographical vector data of the online ENTSO-E Interactive Map by the GridKit toolkit dating back to March 2022.

  • data/parameter_corrections.yaml: Corrections for data/entsoegridkit

  • data/links_p_nom.csv: confer links

  • resources/country_shapes.geojson: confer Rule build_shapes

  • resources/offshore_shapes.geojson: confer Rule build_shapes

  • resources/europe_shape.geojson: confer Rule build_shapes

Outputs#

  • networks/base.nc

  • resources/regions_onshore.geojson:

  • resources/regions_offshore.geojson:

Description#

Creates the network topology from an ENTSO-E map extract, and create Voronoi shapes for each bus representing both onshore and offshore regions.

Rule build_transmission_projects#

Gets the transmission projects defined in the config file, concatenates and deduplicates them. Projects are later included in add_electricity.py.

Relevant Settings#

transmission_projects:
include:

#tyndp: true # For later, when other TYNDP projects are combined with new version nep: true

status: - confirmed - in_permitting - under_construction

#- under_consideration

link_under_construction: zero

See also

Documentation of the configuration file config/config.yaml at transmission_projects

Inputs#

  • networks/base_network.nc: Base network topology for the electricity grid. This is processed in base_network.py.

  • data/transmission_projects/"project_name"/: Takes the transmission projects from the subfolder of data/transmission_projects. The subfolder name is the project name.

  • offshore_shapes.geojson: Shapefile containing the offshore regions. Used to determine if a new bus should be added for a new line or link.

  • europe_shape.geojson: Shapefile containing the shape of Europe. Used to determine if a project is within the considered countries.

Outputs#

  • transmission_projects/new_lines.csv: New project lines to be added to the network. This includes new lines and upgraded lines.

  • transmission_projects/new_links.csv: New project links to be added to the network. This includes new links and upgraded links.

  • transmission_projects/adjust_lines.csv: For lines which are upgraded, the decommissioning year of the existing line is adjusted to the build year of the upgraded line.

  • transmission_projects/adjust_links.csv: For links which are upgraded, the decommissioning year of the existing link is adjusted to the build year of the upgraded link.

  • transmission_projects/new_buses.csv: For some links, we have to add new buses (e.g. North Sea Wind Power Hub).

Rule build_shapes#

Creates GIS shape files of the countries, exclusive economic zones and `NUTS3 < https://en.wikipedia.org/wiki/Nomenclature_of_Territorial_Units_for_Statistics> `_ areas.

Relevant Settings#

countries:

See also

Documentation of the configuration file config/config.yaml at Top-level configuration

Inputs#

  • data/bundle/naturalearth/ne_10m_admin_0_countries.shp: World country shapes

  • ``data/eez/World_EEZ_v12_20231025_gpkg/eez_v12.gpkg ``: World exclusive economic zones (EEZ)

  • data/bundle/NUTS_2013_60M_SH/data/NUTS_RG_60M_2013.shp: Europe NUTS3 regions

  • data/bundle/nama_10r_3popgdp.tsv.gz: Average annual population by NUTS3 region (eurostat)

  • data/bundle/nama_10r_3gdp.tsv.gz: Gross domestic product (GDP) by NUTS 3 regions (eurostat)

  • data/ch_cantons.csv: Mapping between Swiss Cantons and NUTS3 regions

  • data/bundle/je-e-21.03.02.xls: Population and GDP data per Canton (BFS - Swiss Federal Statistical Office )

Outputs#

  • resources/country_shapes.geojson: country shapes out of country selection

  • resources/offshore_shapes.geojson: EEZ shapes out of country selection

  • resources/europe_shape.geojson: Shape of Europe including countries and EEZ

  • resources/nuts3_shapes.geojson: NUTS3 shapes out of country selection including population and GDP data.

Description#

Rule build_gdp_pop_non_nuts3#

Maps the per-capita GDP and population values to non-NUTS3 regions.

The script takes as input the country code, a GeoDataFrame containing the regions, and the file paths to the datasets containing the GDP and POP values for non-NUTS3 countries.

Rule build_powerplants#

Retrieves conventional powerplant capacities and locations from powerplantmatching, assigns these to buses and creates a .csv file. It is possible to amend the powerplant database with custom entries provided in data/custom_powerplants.csv. Lastly, for every substation, powerplants with zero-initial capacity can be added for certain fuel types automatically.

Relevant Settings#

electricity:
  powerplants_filter:
  custom_powerplants:
  everywhere_powerplants:

See also

Documentation of the configuration file config/config.yaml at Rule add_electricity

Inputs#

Outputs#

  • resource/powerplants_s_{clusters}.csv: A list of conventional power plants (i.e. neither wind nor solar) with fields for name, fuel type, technology, country, capacity in MW, duration, commissioning year, retrofit year, latitude, longitude, and dam information as documented in the powerplantmatching README; additionally it includes information on the closest substation/bus in networks/base_s_{clusters}.nc.

Description#

The configuration options electricity: powerplants_filter and electricity: custom_powerplants can be used to control whether data should be retrieved from the original powerplants database or from custom amendmends. These specify pandas.query commands. In addition the configuration option electricity: everywhere_powerplants can be used to place powerplants with zero-initial capacity of certain fuel types at all substations.

  1. Adding all powerplants from custom:

    powerplants_filter: false
custom_powerplants: true

  2. Replacing powerplants in e.g. Germany by custom data:

    powerplants_filter: Country not in ['Germany']
custom_powerplants: true

    or

    powerplants_filter: Country not in ['Germany']
custom_powerplants: Country in ['Germany']

  3. Adding additional built year constraints:

    powerplants_filter: Country not in ['Germany'] and YearCommissioned <= 2015
custom_powerplants: YearCommissioned <= 2015

  4. Adding powerplants at all substations for 4 conventional carrier types:

    everywhere_powerplants: ['Natural Gas', 'Coal', 'nuclear', 'OCGT']

Rule build_electricity_demand#

This rule downloads the load data from Open Power System Data Time series. For all countries in the network, the per country load timeseries are extracted from the dataset. After filling small gaps linearly and large gaps by copying time-slice of a given period, the load data is exported to a .csv file.

Relevant Settings#

snapshots:

load:
    interpolate_limit: time_shift_for_large_gaps: manual_adjustments:

See also

Documentation of the configuration file config/config.yaml at load

Inputs#

  • data/electricity_demand_raw.csv:

Outputs#

  • resources/electricity_demand.csv:

Rule build_monthly_prices#

This script extracts monthly fuel prices of oil, gas, coal and lignite, as well as CO2 prices.

Inputs#

  • data/energy-price-trends-xlsx-5619002.xlsx: energy price index of fossil fuels

  • emission-spot-primary-market-auction-report-2019-data.xls: CO2 Prices spot primary auction

Outputs#

  • data/validation/monthly_fuel_price.csv

  • data/validation/CO2_price_2019.csv

Description#

The rule build_monthly_prices collects monthly fuel prices and CO2 prices and translates them from different input sources to pypsa syntax

Data sources:

[1] Fuel price index. Destatis https://www.destatis.de/EN/Home/_node.html [2] average annual fuel price lignite, ENTSO-E https://2020.entsos-tyndp-scenarios.eu/fuel-commodities-and-carbon-prices/ [3] CO2 Prices, Emission spot primary auction, EEX https://www.eex.com/en/market-data/environmental-markets/eua-primary-auction-spot-download

Data was accessed at 16.5.2023

Rule build_ship_raster#

Transforms the global ship density data from the `World Bank Data Catalogue.

<https://datacatalog.worldbank.org/search/dataset/0037580/Global-Shipping-Traffic-Density>`_ to the size of the considered cutout. The global ship density raster is later used for the exclusion when calculating the offshore potentials.

Relevant Settings#

renewable:
    {technology}:
        cutout:

See also

Documentation of the configuration file config/config.yaml at renewable

Inputs#

Outputs#

  • resources/europe_shipdensity_raster.nc: Reduced version of global shipping traffic density from World Bank Data Catalogue to reduce computation time.

Description#

Rule determine_availability_matrix_MD_UA#

Rule determine_availability_matrix#

The script performs a land eligibility analysis of what share of land is availability for developing the selected technology at each cutout grid cell. The script uses the atlite library and several GIS datasets like the CORINE land use data, LUISA land use data, Natura2000 nature reserves, GEBCO bathymetry data, and shipping lanes.

Relevant settings#

atlite:
    nprocesses:

renewable:
    {technology}:
        cutout: corine: luisa: grid_codes: distance: natura: max_depth:
        min_depth: max_shore_distance: min_shore_distance: resource:

See also

Documentation of the configuration file config/config.yaml at atlite, renewable

Inputs#

Outputs#

  • resources/availability_matrix_{clusters_{technology}.nc

Rule build_renewable_profiles#

Calculates for each clustered region the (i) installable capacity (based on land-use from determine_availability_matrix), (ii) the available generation time series (based on weather data), and (iii) the average distance from the node for onshore wind, AC-connected offshore wind, DC-connected offshore wind and solar PV generators.

Note

Hydroelectric profiles are built in script build_hydro_profiles.

Relevant settings#

snapshots:

atlite:
    nprocesses:

renewable:
    {technology}:
        cutout: capacity_per_sqkm: correction_factor: min_p_max_pu:
        clip_p_max_pu: resource:

See also

Documentation of the configuration file config/config.yaml at snapshots, atlite, renewable

Inputs#

  • resources/availability_matrix_{clusters}_{technology}.nc: see determine_availability_matrix

  • resources/offshore_shapes.geojson: confer Rule build_shapes

  • resources/regions_onshore_base_s_{clusters}.geojson: (if not offshore wind), confer busregions

  • resources/regions_offshore_base_s_{clusters}.geojson: (if offshore wind), busregions

  • "cutouts/" + params["renewable"][{technology}]['cutout']: Rule build_cutout

  • networks/_base_s_{clusters}.nc: Rule base_network

Outputs#

  • resources/profile_{technology}.nc with the following structure

    Field

    Dimensions

    Description

    profile

    bus, time

    the per unit hourly availability factors for each bus

    p_nom_max

    bus

    maximal installable capacity at the bus (in MW)

    average_distance

    bus

    average distance of units in the region to the grid bus for onshore technologies and to the shoreline for offshore technologies (in km)

    • profile

    _images/profile_ts.png

    • p_nom_max

    _images/p_nom_max_hist.png

    • average_distance

    _images/distance_hist.png

Description#

This script functions at two main spatial resolutions: the resolution of the clustered network regions, and the resolution of the cutout grid cells for the weather data. Typically the weather data grid is finer than the network regions, so we have to work out the distribution of generators across the grid cells within each region. This is done by taking account of a combination of the available land at each grid cell (computed in determine_availability_matrix) and the capacity factor there.

Based on the availability matrix, the script first computes how much of the technology can be installed at each cutout grid cell. To compute the layout of generators in each clustered region, the installable potential in each grid cell is multiplied with the capacity factor at each grid cell. This is done since we assume more generators are installed at cells with a higher capacity factor.

_images/offwinddc-gridcell.png _images/offwindac-gridcell.png _images/onwind-gridcell.png _images/solar-gridcell.png

This layout is then used to compute the generation availability time series from the weather data cutout from atlite.

The maximal installable potential for the node (p_nom_max) is computed by adding up the installable potentials of the individual grid cells.

Rule build_hydro_profile#

Build hydroelectric inflow time-series for each country.

Relevant Settings#

countries:

renewable:
    hydro:
        cutout:
        clip_min_inflow:

See also

Documentation of the configuration file config/config.yaml at Top-level configuration, renewable

Inputs#

  • data/bundle/eia_hydro_annual_generation.csv: Hydroelectricity net generation per country and year (EIA)

  • resources/country_shapes.geojson: confer Rule build_shapes

  • "cutouts/" + config["renewable"]['hydro']['cutout']: confer Rule build_cutout

Outputs#

  • resources/profile_hydro.nc:

    Field

    Dimensions

    Description

    inflow

    countries, time

    Inflow to the state of charge (in MW), e.g. due to river inflow in hydro reservoir.
    _images/inflow-ts.png _images/inflow-box.png

Description#

See also

build_renewable_profiles

Contents