Installation#

Mamba/Conda installation#

In order to install ESMValCore and its dependencies from conda-forge, you will first need to install the mamba package manager. We recommend using mamba instead of conda because it is faster and offers the same commands. For a minimal mamba installation (recommended) go to https://mamba.readthedocs.io/en/latest/installation/micromamba-installation.html.

Tip

It is recommended that you always use the latest version of mamba, as problems have been reported when trying to use older versions.

Once you have installed mamba, you can install ESMValCore by running:

mamba install -c conda-forge esmvalcore

It is also possible to create a new Conda environment and install ESMValCore into it with a single command:

mamba create --name esmvalcore -c conda-forge esmvalcore

Don’t forget to activate the newly created environment after the installation:

mamba activate esmvalcore

Of course it is also possible to choose a different name than esmvalcore for the environment.

Note

Creating a new Conda environment is often much faster and more reliable than trying to update an existing Conda environment.

Pip installation#

It is also possible to install ESMValCore from PyPI. However, this requires first installing dependencies that are not available on PyPI in some other way. The list of required dependencies can be found in pyproject.toml.

Warning

It is recommended to use the installation with mamba instead, as it may not be easy to install the correct versions of all dependencies.

After installing the dependencies that are not available from PyPI, install ESMValCore and any remaining Python dependencies with the command:

pip install esmvalcore

Docker installation#

ESMValCore is also provided through DockerHub in the form of docker containers. See https://docs.docker.com for more information about docker containers and how to run them.

You can get the latest release with

docker pull esmvalgroup/esmvalcore:stable

If you want to use the current main branch, use

docker pull esmvalgroup/esmvalcore:latest

To run a container using those images, use:

docker run esmvalgroup/esmvalcore:stable --help

Note that the container does not see the data or environmental variables available in the host by default. You can make data available with -v /path:/path/in/container and environmental variables with -e VARNAME.

For example, the following command would run a recipe

docker run -e HOME -v "$HOME":"$HOME" -v /data:/data esmvalgroup/esmvalcore:stable ~/recipes/recipe_example.yml

with the environmental variable $HOME available inside the container and the data in the directories $HOME and /data, so these can be used to find the configuration, recipe, and data.

It might be useful to define a bash alias or script to abbreviate the above command, for example

alias esmvaltool="docker run -e HOME -v $HOME:$HOME -v /data:/data esmvalgroup/esmvalcore:stable"

would allow using the esmvaltool command without even noticing that the tool is running inside a Docker container.

Singularity installation#

Docker is usually forbidden in clusters due to security reasons. However, there is a more secure alternative to run containers that is usually available on them: Singularity.

Singularity can use docker containers directly from DockerHub with the following command

singularity run docker://esmvalgroup/esmvalcore:stable ~/recipes/recipe_example.yml

Note that the container does not see the data available in the host by default. You can make host data available with -B /path:/path/in/container.

It might be useful to define a bash alias or script to abbreviate the above command, for example

alias esmvaltool="singularity run -B $HOME:$HOME -B /data:/data docker://esmvalgroup/esmvalcore:stable"

would allow using the esmvaltool command without even noticing that the tool is running inside a Singularity container.

Some clusters may not allow to connect to external services, in those cases you can first create a singularity image locally:

singularity build esmvalcore.sif docker://esmvalgroup/esmvalcore:stable

and then upload the image file esmvalcore.sif to the cluster. To run the container using the image file esmvalcore.sif use:

singularity run esmvalcore.sif ~/recipes/recipe_example.yml

Installation from source#

Note

If you would like to install the development version of ESMValCore alongside ESMValTool, please have a look at these instructions.

To install from source for development, follow these instructions.

  • Install pixi

  • Clone the ESMValCore Git repository: git clone https://github.com/ESMValGroup/ESMValCore.git

  • Go to the source code directory: cd ESMValCore

  • Create the development environment and start a shell in this environment by running: pixi shell

  • Test that your installation was successful by running esmvaltool -h.

  • Install the Pre-commit hooks by running: pre-commit install.

Tip

To exit the pixi environment, run exit or press Ctrl+D.

Pre-installed versions on HPC clusters / other servers#

If you would like to use pre-installed versions on HPC clusters (currently CEDA-JASMIN and DKRZ-Levante), and other servers (currently Met Office Linux estate), please have a look at these instructions.