malleefowl.download

TODO: handle parallel downloads

Module Contents

malleefowl.download.LOGGER

malleefowl.download.download_with_archive(url, credentials=None)

Downloads file. Checks before downloading if file is already in local esgf archive.

malleefowl.download.download(url, use_file_url=False, credentials=None)

Downloads url and returns local filename.

Parameters:

• url – url of file
• use_file_url – True if result should be a file url “file://”, otherwise use system path.
• credentials – path to credentials if security is needed to download file

Returns:

downloaded file with either file:// or system path

malleefowl.download.wget(url, use_file_url=False, credentials=None)

Downloads url and returns local filename.

TODO: refactor cache handling.

Parameters:

• url – url of file
• use_file_url – True if result should be a file url “file://”, otherwise use system path.
• credentials – path to credentials if security is needed to download file

Returns:

downloaded file with either file:// or system path

malleefowl.download.download_files(urls=[], credentials=None, monitor=None)

malleefowl.download.download_files_from_thredds(url, recursive=False, monitor=None)

class malleefowl.download.DownloadManager(monitor=None)

Bases:object

show_status(self, message, progress)

threader(self)

download_job(self, url, credentials)

download(self, urls, credentials=None)