Debugging the download Process¶
Go through this tutorial step by step.
- Step 0: Install malleefowl in debug mode
- Step 1: Start the malleefowl demo service
- Step 2: Install birdy
- Step 3: Check if birdy works
- Step 4: Run the download process
- Step 5: Install Phoenix
- Step 6: Login to Phoenix
- Step 7: Register your WPS demo service
- Step 8: Copy the twitcher access token in Phoenix
- Step 9: Access demo service behind the OWS proxy with access token
- Step 10: Get an ESGF certificate using Phoenix
- Step 11: Download a file from ESGF
Step 0: Install malleefowl in debug mode¶
# get the source code
$ git clone https://github.com/bird-house/malleefowl.git
$ cd malleefowl
# create conda env
$ conda env create
# activate malleefowl env
$ source activate malleefowl
# install malleefowl package in develop mode
$ python setup.py develop
# check if the demo service is available
$ malleefowl -h
Step 1: Start the malleefowl demo service¶
You might do this more often when debugging. Make sure you are in the malleefowl conda env.
# start service
$ malleefowl
# open the capabilities document
$ firefox http://localhost:5000/wps?service=WPS&request=GetCapabilities
The service is started in debug mode. See the Werkzeug documenation how to work with this.
You can stop the service with CTRL-c
.
The service is automatically restarted on source changes.
Step 2: Install birdy¶
We are using birdy in the examples, a WPS command line client.
# install it via conda
$ conda install -c birdhouse birdhouse-birdy
Step 3: Check if birdy works¶
# point birdy to the malleefowl service url
$ export WPS_SERVICE=http://localhost:5000/wps
# show a list of available command (wps processes)
$ birdy -h
Step 4: Run the download process¶
Make sure birdy works and is pointing to malleefowl … see above.
# show the description of the download process
$ birdy download -h
# download a netcdf file from a public thredds service
$ birdy download --resource \
https://www.esrl.noaa.gov/psd/thredds/fileServer/Datasets/ncep.reanalysis2/surface/mslp.1979.nc
Step 5: Install Phoenix¶
Phoenix is a web client for WPS and comes by default with an WPS security proxy (twitcher).
$ git clone https://github.com/bird-house/pyramid-phoenix.git
$ cd pyramid-phoenix
$ make clean install
$ make restart
Step 6: Login to Phoenix¶
# login ... by default admin password is "qwerty"
$ firefox https://localhost:8443/account/login
Step 7: Register your WPS demo service¶
Go to the registration page: https://localhost:8443/services/register
Register your service with the following parameters:
- Service URL: http://localhost:5000/wps
- Service Name: demo
Step 8: Copy the twitcher access token in Phoenix¶
- Go to your profile.
- Choose the
Twitcher access token
tab. - Copy the access token.
Step 9: Access demo service behind the OWS proxy with access token¶
# configure wps service
$ export WPS_SERVICE=https://localhost:8443/ows/proxy/demo
# check if it works
$ birdy -h
# run the download again ... you need the access token
$ birdy \
--token 3d8c24eeebb143b3a199ba8a0e045f93 \
download --resource \
https://www.esrl.noaa.gov/psd/thredds/fileServer/Datasets/ncep.reanalysis2/surface/mslp.1979.nc
Step 10: Get an ESGF certificate using Phoenix¶
- Go to your profile.
- Choose the
ESGF credentials
tab. - Use the green button
Update credentials
. - Choose your ESGF provider, enter your account details and press
Submit
.
Step 11: Download a file from ESGF¶
Make sure birdy works and points to the proxy url of demo service … see above.
Choose a file from the ESGF archive you would like to download and make sure you have dowload permissions.
You can choose the ESGF search browser in Phoenix or an ESGF portal.
# try the download ... in this example with a CORDEX file.
# make sure your twitcher token and your ESGF cert are still valid.
$ birdy \
--token 3d8c24eeebb143b3a199ba8a0e045f93 \
download --resource \
http://esgf1.dkrz.de/thredds/fileServer/cordex/cordex/output/EUR-44/MPI-CSC/MPI-M-MPI-ESM-LR/historical/r1i1p1/MPI-CSC-REMO2009/v1/mon/tas/v20150609/tas_EUR-44_MPI-M-MPI-ESM-LR_historical_r1i1p1_MPI-CSC-REMO2009_v1_mon_200101-200512.nc
You can also try this in WPS synchronous mode when your process is not long running:
$ birdy \
--sync \
--token 3d8c24eeebb143b3a199ba8a0e045f93 \
download --resource \
http://esgf1.dkrz.de/thredds/fileServer/cordex/cordex/output/EUR-44/MPI-CSC/MPI-M-MPI-ESM-LR/historical/r1i1p1/MPI-CSC-REMO2009/v1/mon/tas/v20150609/tas_EUR-44_MPI-M-MPI-ESM-LR_historical_r1i1p1_MPI-CSC-REMO2009_v1_mon_200101-200512.nc
… and with debug
option to see more log message:
$ birdy \
--sync \
--debug \
--token 3d8c24eeebb143b3a199ba8a0e045f93 \
download --resource \
http://esgf1.dkrz.de/thredds/fileServer/cordex/cordex/output/EUR-44/MPI-CSC/MPI-M-MPI-ESM-LR/historical/r1i1p1/MPI-CSC-REMO2009/v1/mon/tas/v20150609/tas_EUR-44_MPI-M-MPI-ESM-LR_historical_r1i1p1_MPI-CSC-REMO2009_v1_mon_200101-200512.nc