Release Notes¶
Upcoming Release¶
- Carriers of generators can now be excluded from aggregation in clustering network and simplify network.
- Fix EQ constraint for the case no hydro inflow is available
- Bugfix in the reserve constraint will increase demand related reserve requirements
PyPSA-Eur 0.6.1 (20th September 2022)¶
- Individual commits are now tested against pre-commit hooks. This includes black style formatting, sorting of package imports, Snakefile formatting and others. Installation instructions can for the pre-commit can be found here.
- Pre-commit CI is now part of the repository’s CI.
- The software now supports running the workflow with different settings within
the same directory. A new config section
run
was created that specifies under which scenarioname
the created resources, networks and results should be stored. Ifname
is not specified, the workflow uses the default paths. The entryshared_cutouts
specifies whether the run should use cutouts from the default root directory or use run-specific cutouts. - The heuristic distribution of today’s renewable capacity installations is now enabled by default.
- The marginal costs of conventional generators are now taking the plant-specific efficiency into account where available.
PyPSA-Eur 0.6.0 (10th September 2022)¶
- Functionality to consider shipping routes when calculating the available area for offshore technologies were added. Data for the shipping density comes from the Global Shipping Traffic Density dataset.
- When transforming all transmission lines to a unified voltage level of 380kV, the workflow now preserves the transmission capacity rather than electrical impedance and reactance.
- Memory resources are now specified for all rules.
- Filtering of power plant data was adjusted to new versions of
powerplantmatching
. - The resolution of land exclusion calculation is now a configurable option. See
setting
excluder_resolution
.
PyPSA-Eur 0.5.0 (27th July 2022)¶
New Features
New network topology extracted from the ENTSO-E interactive map.
Added existing renewable capacities for all countries based on IRENA statistics (IRENASTAT) using new
powerplantmatching
version: * The correspondingconfig
entries changed, cf.config.default.yaml
:- old:
estimate_renewable_capacities_from_capacity_stats
- new:
estimate_renewable_capacities
- The estimation is endabled by setting the subkey
enable
toTrue
. - Configuration of reference year for capacities can be configured (default:
2020
) - The list of renewables provided by the OPSD database can be used as a basis,
using the tag
from_opsd: True
. This adds the renewables from the database and fills up the missing capacities with the heuristic distribution. - Uniform expansion limit of renewable build-up based on existing capacities
can be configured using
expansion_limit
option (default:false
; limited to determined renewable potentials) - Distribution of country-level capacities proportional to maximum annual energy yield for each bus region
- The config key
renewable_capacities_from_OPSD
is deprecated and was moved under the section,estimate_renewable_capacities
. To enable it, setfrom_opsd
toTrue
.
- old:
Add operational reserve margin constraint analogous to GenX implementation. Can be activated with config setting
electricity: operational_reserve:
.Implement country-specific Energy Availability Factors (EAFs) for nuclear power plants based on IAEA 2018-2020 reported country averages. These are specified
data/nuclear_p_max_pu.csv
and translate to staticp_max_pu
values.Add function to add global constraint on use of gas in
prepare_network
. This can be activated by including the keywordCH4L
in the{opts}
wildcard which enforces the limit set inelectricity: gaslimit:
given in MWh thermal. Alternatively, it is possible to append a number in the{opts}
wildcard, e.g.CH4L200
which limits the gas use to 200 TWh thermal.Add option to alter marginal costs of a carrier through
{opts}
wildcard:<carrier>+m<factor>
, e.g.gas+m2.5
, will multiply the default marginal cost for gas by factor 2.5.Hierarchical clustering was introduced. Distance metric is calculated from renewable potentials on hourly (feature entry ends with
-time
) or annual (feature entry in config end with-cap
) values.Greedy modularity clustering was introduced. Distance metric is based on electrical distance taking into account the impedance of all transmission lines of the network.
Techno-economic parameters of technologies (e.g. costs and efficiencies) will now be retrieved from a separate repository PyPSA/technology-data that collects assumptions from a variety of sources. It is activated by default with
enable: retrieve_cost_data: true
and controlled withcosts: year:
andcosts: version:
. The location of this data changed fromdata/costs.csv
toresources/costs.csv
[#184].A new section
conventional
was added to the config file. This section contains configurations for conventional carriers.Add configuration option to implement arbitrary generator attributes for conventional generation technologies.
Add option to set CO2 emission prices through
{opts}
wildcard:Ep<number>
, e.g.Ep180
, will set the EUR/tCO2 price.
Changes
- Add an efficiency factor of 88.55% to offshore wind capacity factors as a proxy for wake losses. More rigorous modelling is planned [#277].
- Following discussion in #285 we have disabled the correction factor for solar PV capacity factors by default while satellite data is used. A correction factor of 0.854337 is recommended if reanalysis data like ERA5 is used.
- The default deployment density of AC- and DC-connected offshore wind capacity is reduced from 3 MW/sqkm to a more conservative estimate of 2 MW/sqkm [#280].
- The inclusion of renewable carriers is now specified in the config entry
renewable_carriers
. Before this was done by commenting/uncommenting sub-sections in therenewable
config section. - Now, all carriers that should be extendable have to be listed in the config
entry
extendable_carriers
. Before, renewable carriers were always set to be extendable. For backwards compatibility, the workflow is still looking at the listed carriers under therenewable
key. In the future, all of them have to be listed underextendable_carriers
. - It is now possible to set conventional power plants as extendable by adding
them to the list of extendable
Generator
carriers in the config. - Listing conventional carriers in
extendable_carriers
but not inconventional_carriers
, sets the corresponding conventional power plants as extendable without a lower capacity bound of today’s capacities. - Now, conventional carriers have an assigned capital cost by default.
- The
build_year
andlifetime
column are now defined for conventional power plants. - Use updated SARAH-2 and ERA5 cutouts with slightly wider scope to east and additional variables.
- Resource definitions for memory usage now follow Snakemake standard resource
definition
mem_mb
rather thanmem
. - The powerplants that have been shut down by 2021 are filtered out.
- Updated historical EIA hydro generation data.
- Network building is made deterministic by supplying a fixed random state to network clustering routines.
- Clustering strategies for generator and bus attributes can now be specified directly in the
config.yaml
. - Iterative solving with impedance updates is skipped if there are no expandable lines.
- The unused argument
simple_hvdc_costs
inadd_electricity
was removed. - Switch from Germany to Belgium for continuous integration and tutorial to save resources.
- It is now possible to skip the progressbar for land eligibility calculations for additional speedup.
Bugs and Compatibility
- Fix crs bug. Change crs 4236 to 4326.
powerplantmatching>=0.5.1
is now required forIRENASTATS
.- Update rasterio version to correctly calculate exclusion raster.
- It is now possible to run the workflow with only landlocked countries.
- Bugfixes for manual load adjustments across years.
- Enable parallel computing with new dask version.
- Restore compatibility of
mock_snakemake
with latest Snakemake versions. - Script
build_bus_regions
: move voronoi partition from vresutils to script. - Script
add_electricity
: removevresutils.costdata.annuity
dependency. - Fix the plot_network snakemake rule.
- Compatibility with pandas 1.4. Address deprecations.
- Restore Windows compatibility by using
shutil.move
rather thanmv
.
Synchronisation Release - Ukraine and Moldova (17th March 2022)¶
On March 16, 2022, the transmission networks of Ukraine and Moldova have successfully been synchronised with the continental European grid. We have taken this as an opportunity to add the power systems of Ukraine and Moldova to PyPSA-Eur. This includes:

- the transmission network topology from the ENTSO-E interactive map.
- existing power plants (incl. nuclear, coal, gas and hydro) from the powerplantmatching tool
- country-level load time series from ENTSO-E through the OPSD platform, which are then distributed heuristically to substations by GDP and population density.
- wind and solar profiles based on ERA5 and SARAH-2 weather data
- hydro profiles based on historical EIA generation data
- a simplified calculation of wind and solar potentials based on the Copernicus Land Cover dataset.
- electrical characteristics of 750 kV transmission lines
The Crimean power system is currently disconnected from the main Ukrainian grid and, hence, not included.
This release is not on the master
branch. It can be used with
git clone https://github.com/pypsa/pypsa-eur
git checkout synchronisation-release
PyPSA-Eur 0.4.0 (22th September 2021)¶
New Features and Changes
- With this release, we change the license from copyleft GPLv3 to the more liberal MIT license with the consent of all contributors [#276].
- Switch to the new major
atlite
release v0.2. The version upgrade comes along with significant speed up for the rulebuild_renewable_profiles.py
(~factor 2). A lot of the code which calculated the land-use availability is now outsourced and does not rely onglaes
,geokit
anymore. This facilitates the environment building and version compatibility ofgdal
,libgdal
with other packages [#224]. - Implemented changes to
n.snapshot_weightings
in new PyPSA version v0.18 (cf. PyPSA/PyPSA/#227) [#259]. - Add option to pre-aggregate nodes without power injections (positive or
negative, i.e. generation or demand) to electrically closest nodes or neighbors
in
simplify_network
. Defaults toFalse
. This affects nodes that are no substations or have no offshore connection. - In
simplify_network
, bus columns with no longer correct entries are removed (symbol, tags, under_construction, substation_lv, substation_off) [#219] - Add option to include marginal costs of links representing fuel cells, electrolysis, and battery inverters [#232].
- The rule and script
build_country_flh
are removed as they are no longer used or maintained. - The connection cost of generators in
simplify_network
are now reported inresources/connection_costs_s{simpl}.csv
[#261]. - The tutorial cutout was renamed from
cutouts/europe-2013-era5.nc
tocutouts/be-03-2013-era5.nc
to accommodate tutorial and productive cutouts side-by-side. - The flag
keep_all_available_areas
in the configuration for renewable potentials was deprecated and now defaults toTrue
. - Update dependencies in
envs/environment.yaml
[#257] - Continuous integration testing switches to Github Actions from Travis CI [#252].
- Documentation on readthedocs.io is now built with
pip
only and no longer requiresconda
[#267]. - Use
Citation.cff
[#273].
Bugs and Compatibility
- Support for PyPSA v0.18 [#268].
- Minimum Python version set to
3.8
. - Removed
six
dependency [#245]. - Update
plot_network
andmake_summary
rules to latest PyPSA versions [#270]. - Keep converter links to store components when using the
ATK
wildcard and only remove DC links [#214]. - Value for
co2base
inconfig.yaml
adjusted to 1.487e9 t CO2-eq (from 3.1e9 t CO2-eq). The new value represents emissions related to the electricity sector for EU+UK+Balkan. The old value was too high and used when the emissions wildcard in{opts}
was used [#233]. - Add escape in
base_network
if all TYNDP links are already contained in the network [#246]. - In
solve_operations_network
the optimised capacities are now fixed for all extendable links, not only HVDC links [#244]. - The
focus_weights
are now also considered when pre-clustering in thesimplify_network
rule [#241]. - in
build_renewable_profile
where offshore wind profiles could no longer be created [#249]. - Lower expansion limit of extendable carriers is now set to the
existing capacity, i.e.
p_nom_min = p_nom
(0 before). Simultaneously, the upper limit (p_nom_max
) is now the maximum of the installed capacity (p_nom
) and the previous estimate based on land availability (p_nom_max
) [#260]. - Solving an operations network now includes optimized store capacities as well. Before only lines, links, generators and storage units were considered [#269].
- With
load_shedding: true
in the solving options ofconfig.yaml
load shedding generators are only added at the AC buses, excluding buses for H2 and battery stores [#269]. - Delete duplicated capital costs at battery discharge link [#240].
- Propagate the solver log file name to the solver. Previously, the PyPSA network solving functions were not told about the solver logfile specified in the Snakemake file [#247]
PyPSA-Eur 0.3.0 (7th December 2020)¶
New Features
Using the {opts}
wildcard for scenarios:
- An option is introduced which adds constraints such that each country or node produces on average a minimal share of its total consumption itself.
For example
EQ0.5c
set in the{opts}
wildcard requires each country to produce on average at least 50% of its consumption. Additionally, the optionATK
requires autarky at each node and removes all means of power transmission through lines and links.ATKc
only removes cross-border transfer capacities. [#166]. - Added an option to alter the capital cost (
c
) or installable potentials (p
) of carriers by a factor viacarrier+{c,p}factor
in the{opts}
wildcard. This can be useful for exploring uncertain cost parameters. Example:solar+c0.5
reduces the capital cost of solar to 50% of original values [#167, #207]. - Added an option to the
{opts}
wildcard that applies a time series segmentation algorithm based on renewables, hydro inflow and load time series to produce a given total number of adjacent snapshots of varying lengths. This feature is an alternative to downsampling the temporal resolution by simply averaging and uses the tsam package [#186].
More OPSD integration:
- Add renewable power plants from OPSD to the network for specified technologies.
This will overwrite the capacities calculated from the heuristic approach in
estimate_renewable_capacities()
[#212]. - Electricity consumption data is now retrieved directly from the OPSD website using the rule
build_load_data
. The user can decide whether to take the ENTSO-E power statistics data (default) or the ENTSO-E transparency data [#211].
Other:
- Added an option to use custom busmaps in rule
cluster_network
. To use this feature setenable: custom_busmap: true
. Then, the rule looks for custom busmaps atdata/custom_busmap_elec_s{simpl}_{clusters}.csv
, which should have the same format asresources/busmap_elec_s{simpl}_{clusters}.csv
. i.e. the index should contain the buses ofnetworks/elec_s{simpl}.nc
[#193]. - Line and link capacities can be capped in the
config.yaml
atlines: s_nom_max:
andlinks: p_nom_max
: [#166]. - Added Google Cloud Platform tutorial (for Windows users) [#177].
Changes
- Don’t remove capital costs from lines and links, when imposing a line volume limit (
lv
) or a line cost limit (lc
). Previously, these were removed to move the expansion in direction of the limit [#183]. - The mappings for clustered lines and buses produced by the
simplify_network
andcluster_network
rules changed from Hierarchical Data Format (.h5
) to Comma-Separated Values format (.csv
) for ease of use. [#198] - The N-1 security margin for transmission lines is now fixed to a provided value in
config.yaml
, removing an undocumented linear interpolation between 0.5 and 0.7 in the range between 37 and 200 nodes. [#199]. - Modelling hydrogen and battery storage with Store and Link components is now the default, rather than using StorageUnit components with fixed power-to-energy ratio [#205].
- Use
mamba
(https://github.com/mamba-org/mamba) for faster Travis CI builds [#196]. - Multiple smaller changes: Removed unused
{network}
wildcard, moved environment files to dedicatedenvs
folder, removed sector-coupling components from configuration files, updated documentation colors, minor refactoring and code cleaning [#190].
Bugs and Compatibility
- Add compatibility for pyomo 5.7.0 in
cluster_network
andsimplify_network
[#172]. - Fixed a bug for storage units such that individual store and dispatch efficiencies are correctly taken account of rather than only their round-trip efficiencies.
In the cost database (
data/costs.csv
) the efficiency of battery inverters should be stated as per discharge/charge rather than per roundtrip [#202]. - Corrected exogenous emission price setting (in
config: cost: emission price:
), which now correctly accounts for the efficiency and effective emission of the generators [#171]. - Corrected HVDC link connections (a) between Norway and Denmark and (b) mainland Italy, Corsica (FR) and Sardinia (IT) as well as for East-Western and Anglo-Scottish interconnectors [#181, #206].
- Fix bug of clustering
offwind-{ac,dc}
generators in the option of high-resolution generators for renewables. Now, there are more sites foroffwind-{ac,dc}
available than network nodes. Before, they were clustered to the resolution of the network (elec_s1024_37m.nc
: 37 network nodes, 1024 generators) [#191]. - Raise a warning if
tech_colors
in the config are not defined for all carriers [#178].
PyPSA-Eur 0.2.0 (8th June 2020)¶
- The optimization is now performed using the
pyomo=False
setting in thepypsa.lopf.network_lopf()
. This speeds up the solving process significantly and consumes much less memory. The inclusion of additional constraints were adjusted to the new implementation. They are all passed to thenetwork_lopf()
function via theextra_functionality
argument. The ruletrace_solve_network
was integrated into the rulesolve_network
and can be activated via configuration withsolving: options: track_iterations: true
. The charging and discharging capacities of batteries modelled as store-link combination are now coupled [#116]. - An updated extract of the ENTSO-E Transmission System Map (including Malta) was added to the repository using the GridKit tool. This tool has been updated to retrieve up-to-date map extracts using a single script. The update extract features 5322 buses, 6574 lines, 46 links. [#118].
- Added FSFE REUSE compliant license information. Documentation now licensed under CC-BY-4.0 [#160].
- Added a 30 minute video introduction and a 20 minute video tutorial
- Networks now store a color and a nicely formatted name for each carrier, accessible via
n.carrier['color']
andn.carrier['nice_name'] ``(networks after ``elec.nc
). - Added an option to skip iterative solving usually performed to update the line impedances of expanded lines at
solving: options: skip_iterations:
. snakemake
rules for retrieving cutouts and the natura raster can now be disabled independently from their respective rules to build them; viaconfig.*yaml
[#136].- Removed the
id
column for custom power plants indata/custom_powerplants.csv
to avoid custom power plants with conflicting ids getting attached to the wrong bus [#131]. - Add option
renewables: {carrier}: keep_all_available_areas:
to use all available weather cells for renewable profile and potential generation. The default ignores weather cells where only less than 1 MW can be installed [#150]. - Added a function
_helpers.load_network()
which loads a network with overridden components specified insnakemake.config['override_components']
[#128]. - Bugfix in
base_network
which now finds all closest links, not only the first entry [#143]. - Bugfix in
cluster_network
which now skips recalculation of link parameters if there are no links [#149]. - Added information on pull requests to contribution guidelines [#151].
- Improved documentation on open-source solver setup and added usage warnings.
- Updated
conda
environment regardingpypsa
,pyproj
,gurobi
,lxml
. This release requires PyPSA v0.17.0.
PyPSA-Eur 0.1.0 (9th January 2020)¶
This is the first release of PyPSA-Eur, a model of the European power system at the transmission network level. Recent changes include:
- Documentation on installation, workflows and configuration settings is now available online at pypsa-eur.readthedocs.io [#65].
- The
conda
environment files were updated and extended [#81]. - The power plant database was updated with extensive filtering options via
pandas.query
functionality [#84 and #94]. - Continuous integration testing with Travis CI is now included for Linux, Mac and Windows [#82].
- Data dependencies were moved to zenodo and are now versioned [#60].
- Data dependencies are now retrieved directly from within the snakemake workflow [#86].
- Emission prices can be added to marginal costs of generators through the keyworks
Ep
in the{opts}
wildcard [#100]. - An option is introduced to add extendable nuclear power plants to the network [#98].
- Focus weights can now be specified for particular countries for the network clustering, which allows to set a proportion of the total number of clusters for particular countries [#87].
- A new rule
add_extra_components
allows to add additional components to the network only after clustering. It is thereby possible to model storage units (e.g. battery and hydrogen) in more detail via a combination ofStore
,Link
andBus
elements [#97]. - Hydrogen pipelines (including cost assumptions) can now be added alongside clustered network connections in the rule
add_extra_components
. Setelectricity: extendable_carriers: Link: [H2 pipeline]
and ensure hydrogen storage is modelled as aStore
. This is a first simplified stage [#108]. - Logfiles for all rules of the
snakemake
workflow are now written in the folderlog/
[#102]. - The new function
_helpers.mock_snakemake
creates asnakemake
object which mimics the actualsnakemake
object produced by workflow by parsing theSnakefile
and setting all paths for inputs, outputs, and logs. This allows running all scripts within a (I)python terminal (or just by callingpython <script-name>
) and thereby facilitates developing and debugging scripts significantly [#107].
Release Process¶
- Checkout a new release branch
git checkout -b release-v0.x.x
. - Finalise release notes at
doc/release_notes.rst
. - Update
envs/environment.fixed.yaml
viaconda env export -n pypsa-eur -f envs/environment.fixed.yaml --no-builds
from an up-to-date pypsa-eur environment. - Update version number in
doc/conf.py
,CITATION.cff
and*config.*.yaml
. - Open, review and merge pull request for branch
release-v0.x.x
. Make sure to close issues and PRs or the release milestone with it (e.g. closes #X). - Tag a release on Github via
git tag v0.x.x
,git push
,git push --tags
. Include release notes in the tag message. - Upload code to zenodo code repository with MIT license.
- Create pre-built networks for
config.default.yaml
by runningsnakemake -j 1 extra_components_all_networks
. - Upload pre-built networks to zenodo data repository with CC BY 4.0 license.
- Send announcement on the PyPSA and PyPSA-Eur mailing list.