Introduction#
Note
Find the introductory slides here.
Warning
The video only introduces the electricity-only part of PyPSA-Eur.
Workflow#
The generation of the model is controlled by the open workflow management system
Snakemake. In a nutshell, the Snakefile
declares for each script in the scripts
directory a rule which describes
which files the scripts consume and produce (their corresponding input and
output files). The snakemake
tool then runs the scripts in the correct order
according to the rules’ input and output dependencies. Moreover, snakemake
will track what parts of the workflow have to be regenerated when files or
scripts were modified.
For instance, an invocation to
.../pypsa-eur % snakemake -call results/networks/elec_s_128_ec_lvopt_Co2L-3H.nc
follows this dependency graph

to solve an electricity system model.
The blocks represent the individual rules which are required to create the file referenced in the command above. The arrows indicate the outputs from preceding rules which another rule takes as input data.
Note
The dependency graph was generated using
snakemake --dag results/networks/elec_s_128_ec_lvopt_Co2L-3H.nc -F | sed -n "/digraph/,/}/p" | dot -Tpng -o doc/img/intro-workflow.png
For the use of snakemake
, it makes sense to familiarize yourself quickly
with the basic tutorial and then
read carefully through the documentation of the command line interface, noting the
arguments -j
, -c
, -f
, -F
, -n
, -r
, --dag
and -t
in particular.
Scenarios, Configuration and Modification#
It is easy to run PyPSA-Eur for multiple scenarios using the wildcards feature
of snakemake
. Wildcards allow to generalise a rule to produce all files that
follow a regular expression pattern, which defines
a particular scenario. One can think of a wildcard as a parameter that shows
up in the input/output file names and thereby determines which rules to run,
what data to retrieve and what files to produce. Details are explained in
Wildcards and run.
The model also has several further configuration options collected in the
config/config.yaml
file located in the root directory, which that are not part of
the scenarios. Options are explained in Configuration.
Folder Structure#
scripts
: Includes all the Python scripts executed by thesnakemake
rules.rules
: Includes all thesnakemake
rules loaded in theSnakefile
.envs
: Includes all theconda
environment specifications to run the workflow.data
: Includes input data that is not produced by anysnakemake
rule.cutouts
: Stores raw weather data cutouts fromatlite
.resources
: Stores intermediate results of the workflow which can be picked up again by subsequent rules.results
: Stores the solved PyPSA network data, summary files and plots.logs
: Stores log files.benchmarks
: Storessnakemake
benchmarks.test
: Includes the test configuration files used for continuous integration.doc
: Includes the documentation of PyPSA-Eur.
System Requirements#
Building the model with the scripts in this repository runs on a regular computer. But optimising for investment and operation decisions across many scenarios requires a strong interior-point solver like Gurobi or CPLEX with more memory. Open-source solvers like HiGHS <https://highs.dev> can also be used for smaller problems.