API

api.BROKER_URL = 'https://gateway.prod.wekeo2.eu/hda-broker/'
class hda.api.Paginator(request)

A class to iterate over paginated results, following the HDA specification.

class hda.api.SearchPaginator(request)

Bases: Paginator

class hda.api.DatasetPaginator(request)

Bases: Paginator

class hda.api.DataOrderRequest(client)

Bases: object

Runner class for a data order request. A data order request is performed in order to retrieve actual files for a given result returned in the data request phase.

class hda.api.SearchResults(client, results, dataset)

A wrapper to a data request response payload.

It adds aggregated information, like the total size and lenght of the results, and the ability to slice them.

Please refer to the Usage page for examples.

Parameters:

  • client (hda.api.Client) – The hda.api.Client instance to be used to perform the download.

  • results (list) – The results list coming from the data request.

  • dataset (string) – The dataset identifier.

download(download_dir: str = '.', force=False, *, to_s3=False, s3_bucket=None, s3_key_prefix='', s3_endpoint=None, s3_access_key_id=None, s3_secret_access_key=None, s3_verify_ssl=True)

Downloads the results into the given download directory or S3 bucket.

The process is executed concurrently using hda.api.Client.max_workers threads.

get_download_urls(limit: int | None = None)

Utility function to return the list of final download URLs. Useful in the context of the Serverless Functions service. If the list of results is long, it might take a long time. In that case, either subset the results or set a value for limit.

class hda.api.Configuration(url=None, user=None, password=None, verify=True, path=None)

Service class to wrap up the client configuration.

The main purpose is to allow multiple ways of injecting basic client parameters.

Please refer to the Usage page for examples.

Parameters:

  • url (str) – The base API URL. This should be set only for testing purposes. It defaults to BROKER_URL

  • user (str) – The API username to use. A valid WEkEO account is needed.

  • password (str) – The API password to use. A valid WEkEO account is needed.

  • verify (bool) – Whether to complain for an invalid SSL certificate. Usually only set for testing purposes.

  • path (str) – A path to an optional configuration file that will override the given inputs. Please refer to the Usage page for examples.

class hda.api.Client(config=None, timeout=None, retry_max=500, sleep_max=120, progress=True, max_workers=2)

HTTP client to request data from the WEkEO HDA API.

Parameters:

  • config (class:hda.api.Configuration) – A hda.api.Configuration instance. By default None is passed, which means that a $HOME/.hdarc configuration file will be read.

  • timeout (int, optional) – The timeout of each request in seconds. None means no timeout.

  • retry_max (int, optional) – The number of retries on request failure.

  • sleep_max (int, optional) – The maximum sleep time between failed requests.

  • progress (bool, optional) – Whether to show a progress bar when the download starts.

  • max_workers (int, optional) – The number of threads used during the download phase.

accept_tac(dataset_id)

Implicitly accept the terms and conditions of the service.

dataset(dataset_id)

Returns a JSON object that includes the abstract, the datasetId and other properties of the given dataset.

Parameters:

dataset_id (str) – The dataset ID

datasets(limit=None)

Returns the full list of available datasets. Each element of the list is a JSON object that includes the abstract, the dataset ID and other properties.

Parameters:

limit (int) – The maximum number of results to return. Set to None to return all results (default)

full_url(*args)

Returns the full URL of the API by appending the args to the configured base URL.

Parameters:

args (list) – A list of URL parts that will be joined to the base URL.

Returns:

The full URL

Return type:

str

get(*args, **kwargs)

Submits a GET request.

Parameters:

args (list) – The list of URL parts.

Returns:

A response object

head(*args)

Submits a HEAD request.

Parameters:

args (list) – The list of URL parts.

Returns:

A response object

metadata(dataset_id)

Returns the metadata object for the given dataset.

Parameters:

dataset_id (str) – The dataset ID

post(message, *args)

Submits a POST request.

Parameters:

  • message (json) – The POST payload, in JSON format.

  • args (list) – The list of URL parts.

Returns:

A response object

put(message, *args)

Submits a PUT request.

Parameters:

  • message (json) – The PUT payload, in JSON format.

  • args (list) – The list of URL parts.

Returns:

A response object

robust(call)

A robust way of submitting the call to the API by retrying it in case of failure. An exponential-backoff strategy is used to delay subsequent requests up to the hda.Client.sleep_max value.

Parameters:

call (callable) – The request call function, like get, post or put.

Returns:

The response object.

search(query, limit=None)

Submits a search request with the given query.

Parameters:

  • query (json) – The JSON object representing the query.

  • limit (int) – The maximum number of results to return. Set to None to return all results (default)

Returns:

An hda.api.SearchResults instance

property session

The requests library session object, with the attached authentication.

stream(download_id: str, size: int, download_dir: str = '.', force: bool = False, *, to_s3: bool = False, s3_bucket: str | None = None, s3_key_prefix: str = '', s3_endpoint: str | None = None, s3_access_key_id: str | None = None, s3_secret_access_key: str | None = None, s3_verify_ssl=True)

Streams the given URL into the specified download directory or S3 bucket. Usually, this is not called directly but through the download() method.

Parameters:

  • download_id – The download id as returned by the search API.

  • size (int) – The expected size of the resource.

  • download_dir (str, optional) – The directory into which the resource must be downloaded.

  • force (bool, optional) – Whether to override the product if a local file already exists.

  • to_s3 – Whether to download the product directly to S3 (needs optional dependencies).

  • s3_bucket (str) – The S3 bucket to stream the product to.

  • s3_endpoint (str, optional) – A custom endpoint when using MinIO or other S3-like service.

  • s3_access_key_id (str, optional) – The access key to S3. Overrides the AWS Credentials file.

  • s3_secret_access_key (str, optional) – The secrect access key to S3. Overrides the AWS Credentials file.

  • s3_verify_ssl (bool) – Whether to verify the SSL Certificate.

property token

The access token to access the API.