Skip to content

edsnlp.training.loggers

CSVTracker [source]

Bases: GeneralTracker

log

Logs values to the CSV file, at an optional step.

  • If it's the first call, the columns are inferred from the keys in values plus a "step" column if the user provides step.
  • All subsequent calls must use the same columns. Any missing columns get written as empty, any new columns generate a warning.

JSONTracker [source]

Bases: GeneralTracker

log

Logs values along with a step (if provided).

On every call, we: 1. Append a new record to our in-memory list. 2. Write out the entire JSON file containing all records.

RichTracker [source]

Bases: GeneralTracker

log

Logs values in the Rich table. If step is provided, we include it in the logged data.

finish

Finalize the table (e.g., stop rendering).

sanitize_hparams [source]

TensorBoard only supports scalar hyperparameters. Convert unsupported types to strings.

JSONLogger

A simple JSON-based logger that writes logs to a JSON file as a list of dictionaries. By default, with edsnlp.train the JSON file is located under a local directory ${CWD}/artifact/metrics.json.

This method is not recommended for large and frequent logging, as it re-writes the entire JSON file on every call. Prefer CSVLogger for frequent and heavy logging.

Parameters

PARAMETER DESCRIPTION
logging_dir

Directory in which to store the JSON file.

TYPE: str or PathLike

file_name

Name of the JSON file. Defaults to "metrics.json".

TYPE: str DEFAULT: 'metrics.json'

CSVLogger

A simple CSV-based logger that writes logs to a CSV file. By default, with edsnlp.train the CSV file is located under a local directory ${CWD}/artifact/metrics.csv.

Consistent Keys

This logger expects that the values dictionary passed to log has consistent keys across all calls. If a new key is encountered in a subsequent call, it will be ignored and a warning will be issued.

Parameters

PARAMETER DESCRIPTION
logging_dir

Directory in which to store the CSV.

TYPE: str or PathLike

file_name

Name of the CSV file. Defaults to "metrics.csv".

TYPE: str DEFAULT: 'metrics.csv'

RichLogger

A logger that displays logs in a Rich-based table using rich-logger. This logger is also available via the loggers registry as rich.

No Disk Logging

This logger doesn't save logs to disk. It's meant for displaying logs in a pretty table during training. If you need to save logs to disk, consider combining this logger with any other logger.

Parameters

PARAMETER DESCRIPTION
fields

Field descriptors containing goal ("lower_is_better" or "higher_is_better"), format and display name The key is a regex that will be used to match the fields to log Each entry of the dictionary should match the following scheme:

  • key: a regex to match columns
  • value: either a Dict or False to hide the column, the dict format is
    • name: the name of the column
    • goal: "lower_is_better" or "higher_is_better"

This defaults to a set of metrics and stats that are commonly logged during EDS-NLP training.

TYPE: Dict[str, Union[Dict, bool]] DEFAULT: None

key

Key to group the logs

TYPE: Optional[str] DEFAULT: None

hijack_tqdm

Whether to replace the tqdm progress bar with a rich progress bar. Indeed, rich progress bars integrate better with the rich table.

TYPE: bool DEFAULT: True

AimLogger

Logger for Aim.

Parameters

PARAMETER DESCRIPTION
project_name

Name of the project.

TYPE: str

logging_dir

Directory in which to store the Aim logs. The environment variable AIM_LOGGING_DIR takes precedence over this argument.

TYPE: Optional[Union[str, PathLike]] DEFAULT: None

kwargs

Additional keyword arguments to pass to the Aim init function.

DEFAULT: {}

TensorBoardLogger

Logger for TensorBoard. This logger is also available via the loggers registry as tensorboard.

Parameters

PARAMETER DESCRIPTION
project_name

Name of the project.

TYPE: str

logging_dir

Directory in which to store the TensorBoard logs. Logs of different runs will be stored in logging_dir/project_name. The environment variable TENSORBOARD_LOGGING_DIR takes precedence over this argument.

TYPE: Optional[Union[str, PathLike]] DEFAULT: None

WandBLogger

Logger for Weights & Biases. This logger is also available via the loggers registry as wandb.

Parameters

PARAMETER DESCRIPTION
project_name

Name of the project. This will become the project parameter in wandb.init.

TYPE: str

kwargs

Additional keyword arguments to pass to the WandB init function.

DEFAULT: {}

RETURNS DESCRIPTION
WandBTracker

MLflowLogger

Logger for MLflow. This logger is also available via the loggers registry as mlflow.

Parameters

PARAMETER DESCRIPTION
project_name

Name of the project. This will become the mlflow experiment name.

TYPE: str

logging_dir

Directory in which to store the MLflow logs.

TYPE: Optional[Union[str, PathLike]] DEFAULT: None

run_id

If specified, get the run with the specified UUID and log parameters and metrics under that run. The run’s end time is unset and its status is set to running, but the run’s other attributes (source_version, source_type, etc.) are not changed. Environment variable MLFLOW_RUN_ID has priority over this argument.

TYPE: Optional[str] DEFAULT: None

tags

An optional dict of str keys and values, or a str dump from a dict, to set as tags on the run. If a run is being resumed, these tags are set on the resumed run. If a new run is being created, these tags are set on the new run. Environment variable MLFLOW_TAGS has priority over this argument.

TYPE: Optional[Union[Dict[str, Any], str]] DEFAULT: None

nested_run

Controls whether run is nested in parent run. True creates a nested run. Environment variable MLFLOW_NESTED_RUN has priority over this argument.

TYPE: Optional[bool] DEFAULT: False

run_name

Name of new run (stored as a mlflow.runName tag). Used only when run_id is unspecified.

TYPE: Optional[str] DEFAULT: None

description

An optional string that populates the description box of the run. If a run is being resumed, the description is set on the resumed run. If a new run is being created, the description is set on the new run.

TYPE: Optional[str] DEFAULT: None

RETURNS DESCRIPTION
MLflowTracker

CometMLLogger

Logger for CometML. This logger is also available via the loggers registry as cometml.

Parameters

PARAMETER DESCRIPTION
project_name

Name of the project.

TYPE: str

kwargs

Additional keyword arguments to pass to the CometML Experiment object.

DEFAULT: {}

RETURNS DESCRIPTION
CometMLTracker