Data Helpers

mtlearn.data contains download and local dataset resolution helpers used by examples and notebooks.

Dataset download helpers for mtlearn examples and notebooks.

class mtlearn.data.DatasetSpec(key, description, target, url=None, url_env_var=None, local_env_var=None, access_note=None, download_by_default=True)[source]

Bases: object

Parameters:

  • key (str)

  • description (str)

  • target (tuple[str, ...])

  • url (str | None)

  • url_env_var (str | None)

  • local_env_var (str | None)

  • access_note (str | None)

  • download_by_default (bool)

key: str
description: str
target: tuple[str, ...]
url: str | None = None
url_env_var: str | None = None
local_env_var: str | None = None
access_note: str | None = None
download_by_default: bool = True
mtlearn.data.repo_root(start=None)[source]
Return type:

Path | None

Parameters:

start (Path | None)

mtlearn.data.default_data_dir(start=None)[source]
Return type:

Path

Parameters:

start (Path | None)

mtlearn.data.dataset_path(key, data_dir=None)[source]
Return type:

Path

Parameters:

  • key (str)

  • data_dir (Path | None)

mtlearn.data.require_local_dataset(name, data_dir=None, *, env_var=None, description=None)[source]

Resolve a local dataset that is not part of the public download registry.

Return type:

Path

Parameters:

  • name (str)

  • data_dir (Path | PathLike[str] | str | None)

  • env_var (str | None)

  • description (str | None)

mtlearn.data.dropbox_download_url(url)[source]
Return type:

str

Parameters:

url (str)

mtlearn.data.format_size(num_bytes)[source]
Return type:

str

Parameters:

num_bytes (int | None)

mtlearn.data.download_file(url, destination)[source]
Return type:

None

Parameters:

  • url (str)

  • destination (Path)

mtlearn.data.extracted_content_root(extract_dir)[source]
Return type:

Path

Parameters:

extract_dir (Path)

mtlearn.data.replace_directory(source, destination)[source]
Return type:

None

Parameters:

  • source (Path)

  • destination (Path)

mtlearn.data.has_existing_files(path)[source]
Return type:

bool

Parameters:

path (Path)

mtlearn.data.ensure_dataset(key, data_dir=None, *, force=False, keep_archive=False, url=None)[source]
Return type:

Path

Parameters:

  • key (str)

  • data_dir (Path | None)

  • force (bool)

  • keep_archive (bool)

  • url (str | None)

mtlearn.data.parse_args(argv)[source]
Return type:

Namespace

Parameters:

argv (list[str])

mtlearn.data.list_datasets(data_dir=None)[source]
Return type:

None

Parameters:

data_dir (Path | None)

mtlearn.data.main(argv=None)[source]
Return type:

int

Parameters:

argv (list[str] | None)