Installation

The flex_extract software package contains python and shell scripts as well as a Fortran program. These components rely on several libraries which need to be available before starting the installation process. Currently, the software is only tested for a GNU/Linux environment. Feel free to try it out on other platforms.

At first, find out to which user group you belong and follow the instructions at Registration at ECMWF to obtain an account at ECMWF (if you don’t have it already). Depending on the user group and the way to access the ECWMF MARS archive, there are four possible Application modes:

More information can be found in Application modes.

Note

If you encounter any problems in the installation process, you can ask for Support.

Registration at ECMWF

The registration depends on the Application modes and in case of the local mode also on the data set you’d like to retrieve. The following table gives an overview where you need to register.

Data sets Member-state user Public user
Remote Gateway Local Local
Operational 1 1 1,2
ERA-Interim 1 1 1,2 2
CERA-20C 1 1 1,2 2
ERA5 1 1 3

Registration options:

1.) Access through a member-state user account granted by the Computing Representative. The credentials have to be provided during installation.

2.) Access through the ECMWF Web API. Need to sign in at ECMWF Web API and configure the ECMWF key as described. Member-state users can sign in with their credentials from the Computing Representative and public users have to fill out the registration form to get an account.

3.) Access through the CDS API. Extra registration for member-state users is required at Copernicus Climate Data Store including the configurations of the CDS key as described. This mode is currently not available for public users. See the note at user group for information.

Accept licences for public datasets

Each ECMWF public dataset has its own licence whose acceptance requires user activity, regardless of the user group.

For the ERA-Interim and CERA-20C datasets this can be done at the ECMWF website Available ECMWF Public Datasets. Log in and follow the licence links on the right side for each dataset and accept it.

For the ERA5 dataset this has to be done at the Climate Data Store (CDS) website. Log in and select, on the left panel, product type “Reanalysis” for finding ERA5 datasets. Then follow any link with ERA5 to the full dataset record, click on tab “Download data” and scroll down. There is a section “Terms of use” where you have to click the Accept terms button.

Download flex_extract

There are two options to download flex_extract:

tar ball

You can download a tar ball with the latest release from the flex_extract page from our FLEXPART community website and then untar the file.

tar -xvf <flex_extract_vX.X.tar>
git repo

Alternatively, if you have git installed on your machine, and if you are interested to keep the code in a version control system, you may clone the latest version from our git repository master branch.

$ git clone --single-branch --branch master https://www.flexpart.eu/gitmob/flex_extract

Dependencies

The software required to run flex_extract depends on the Application modes and therefore is described in the respective specific installation sections.

Generally speaking, flex_extract requires Python 3 and Fortran together with certain modules / libraries. We tested flex_extract with the Python3 package from the the GNU/Linux distribution and Anaconda Python. The required Python3 modules should prefarably be installed as distribution packages, or alternatively using Python’s own package manager pip (this may mess up some aspects of your python installation, especially if you use pip as root. Think about using virtual environments.).

Before installing the system packages check the availability with (Debian-based system) dpkg -s <package-name> |  grep Status or (Redhat-based system) rpm -q <package_name>. For example:

$ dpkg -s libeccodes-dev |  grep Status
# or
$ rpm -q libeccodes-dev

Installation of flex_extract

The actual installation of flex_extract is done by executing a shell script called setup.sh. It defines some parameters and calls a Python script passing the parameters as command line arguments. For details, see The installation script - setup.sh.

For each application mode installation section we describe the requirements for the explicit environment and how it is installed, test if it works and how the actual flex_extract installation has to be done. At the users local side not all software has to be present for flex_extract.

Select one of the following modes to install:

Flex_extract in combination with FLEXPART

Some users might wish to incorporate flex_extract directly into the FLEXPART distribution. Then the installation path has to be changed by setting the parameter installdir in the setup.sh file to the script directory in the FLEXPART root directoy.

Test installation

Fortran program test

To check whether the Fortran program calc_etadot has been compiled and runs properly, it can be applied to a prepared minimal dataset.

For this, go from the flex_extract root directory to the Testing/Installation/Calc_etadot/ directory and execute the Fortran program.

Note

Remember that you might have to log in at the ECMWF server if you used the installation for the remote or gateway mode. There you find the flex_extract root directory in your $HOME directory.

cd Testing/Installation/Calc_etadot
# execute the Fortran progam without arguments
../../../Source/Fortran/calc_etadot

The installation was successfull if you obtain on standard output:

STATISTICS:          98842.4598  98709.7359   5120.5385
STOP SUCCESSFULLY FINISHED calc_etadot: CONGRATULATIONS

Now go back to the root directory:

$ cd ../../../

Full test

see Usage