Installation

Installing the EMPD-admin locally

The EMPD-admin runs as a webapp (see EMPD-data pull requests), so you actually do not have to install it on your own laptop. However, the webapp has limitations in it’s performance. So you might want to install it locally to speed up the data contribution.

At the moment, we support installation from source or using the `edmp2/empd-admin docker image`_. The latter has the advantage that it avoids the installation of additional dependencies (that are quite heavy) and it runs in a virtual machine, separated from the rest of your system.

Installation through Docker

The easiest way to get the EMPD-admin is to use the docker image that we provide as empd2/empd-admin. Here you do not have to install anything, instead you pull the latest image from the docker hub. Install docker for your system, and then pull the latest image via:

docker pull empd2/empd-admin

And then run the command line utility via:

docker run empd2/empd-admin empd-admin

The docker image also has the latest version of the EMPD-data repository under /opt/empd-data. This is what the empd-admin command uses. If you want to use your own data contribution, mount it at that directory via:

docker run -v PATH-TO-YOUR-REPO:/opt/empd-data empd2/empd-admin empd-admin

For example, to use the test-data branch of the EMPD2/EMPD-data repository, as we use it in our Getting started guide, you can clone it via:

git clone -q https://github.com/EMPD2/EMPD-data -b test-data

And then test it via:

docker run -v `pwd`/EMPD-data:/opt/empd-data empd2/empd-admin empd-admin test

Installation from source

The dependencies of the EMPD-admin are quite heavy, in particularly we rely on

We highly recommend to use conda to install these dependencies. Here you can either use the anaconda or miniconda installer.

When conda is installed on your system, download the conda environment file and create a new conda environment for the EMPD-admin via:

conda env create -f PATH-TO-DOWNLOADED-FILE

Now, activate the environment via:

conda activate empd-admin

To get the source files, you have to clone the EMPD-admin repository from Github via:

git clone https://github.com/EMPD2/EMPD-admin.git

Alternatively you can download the zipped source files of the master branch.

Finally install the EMPD-admin via:

pip install EMPD-admin

where EMPD-admin is the path to the folder you just downloaded.

That’s it! Now you can then run the empd-admin via

$ empd-admin --help
usage: empd-admin [-h] [-d DIRECTORY]
                  {test,fix,createdb,rebuild,rebase,finish,accept,unaccept,query,diff,generate,merge-meta,help}
                  ...

optional arguments:
  -h, --help            show this help message and exit
  -d DIRECTORY, --directory DIRECTORY
                        Path to the local EMPD2/EMPD-data repository. Default:
                        .

Commands:
  {test,fix,createdb,rebuild,rebase,finish,accept,unaccept,query,diff,generate,merge-meta,help}
    test                test the database
    fix                 fix the database
    createdb            Create a postgres database out of the data
    rebuild             Rebuild the fixed tables of the postgres database
    rebase              Merge the master branch of EMPD2/EMPD-data into the
                        current branch to resolve merge conflicts
    finish              Finish this PR and merge the data into meta.tsv
    accept              Mark incomplete or erroneous meta data as accepted
    unaccept            Reverse the acceptance of incomplete or erroneous meta
                        data.
    query               Query and display the meta data
    diff                Compare two EMPD meta files
    generate            Generate the EMPD data out of a postgres dump
    merge-meta          Merge two metafiles
    help                Print the help on a command

Note

You can also skip the installation of the dependencies through conda, and directly run pip install EMPD-admin, but do that on your own risk. There is no guarantee that packages like netCDF4 or pandas work through pip installation

Building the docs

The docs are also built through the docker image. Clone the EMPD-admin repository:

git clone https://github.com/EMPD2/EMPD-admin.git
cd EMPD-admin

Then use the Dockerfile in docs/Dockerfile to built the empd-admin-docs image:

docker build -t empd-admin-docs docs

Now mount the directory where you want the documentation to be generated (in the example below docs/_build/html) as /opt/empd-admin-docs and run the `` build-empd-admin-docs``command:

docker run -v `pwd`/docs/_build/html:/opt/empd-admin-docs build-empd-admin-docs /opt/empd-admin-docs

Note

Depending in how you set up docker, you might have to change permissions to make the mounted directory writable from within the docker image. On OS X and Linux, you can do this via:

mkdir -p docs/_build/html && chmod 0777 docs/_build/html

Testing the EMPD-admin

Testing the EMPD-admin requires a Github API token to test the webapp features. Login to Github and create a token (without any scopes) at https://github.com/settings/tokens. Copy the token and run the tests of the EMPD-admin through the docker image:

docker run -e GH_TOKEN=YOUR-SECRET-TOKEN empd2/empd-admin test-empd-admin

Alternatively, if you installed the EMPD-admin from source, you can run the tests by executing pytest from within the downloaded Github repository, i.e.:

git clone https://github.com/EMPD2/EMPD-admin.git
cd EMPD-admin
GH_TOKEN=YOUR-SECRET-TOKEN pytest