scvi.train.TrainingPlan#

class scvi.train.TrainingPlan(module, *, optimizer='Adam', optimizer_creator=None, fused_optimizer=False, lr=0.001, update_only_decoder=False, weight_decay=1e-06, eps=0.01, n_steps_kl_warmup=None, n_epochs_kl_warmup=400, reduce_lr_on_plateau=False, lr_factor=0.6, lr_patience=30, lr_threshold=0.0, lr_scheduler_metric='elbo_validation', lr_min=0, max_kl_weight=1.0, min_kl_weight=0.0, compile=False, compile_kwargs=None, on_step=False, on_epoch=True, **loss_kwargs)[source]#

Bases: LightningModule

Lightning module task to train scvi-tools modules.

The training plan is a PyTorch Lightning Module that is initialized with a scvi-tools module object. It configures the optimizers, defines the training step and validation step, and computes metrics to be recorded during training. The training step and validation step are functions that take data, run it through the model, and return the loss, which will then be used to optimize the model parameters in the Trainer. Overall, custom training plans can be used to develop complex inference schemes on top of modules.

The following developer tutorial will familiarize you more with training plans and how to use them: Constructing a high-level model.

Parameters:
module BaseModuleClass

A module instance from class BaseModuleClass.

optimizer Literal['Adam', 'AdamW', 'Custom'] (default: 'Adam')

One of “Adam” (Adam), “AdamW” (AdamW), or “Custom”, which requires a custom optimizer creator callable to be passed via optimizer_creator.

optimizer_creator Callable[[Iterable[Tensor]], Optimizer] | None (default: None)

A callable taking in parameters and returning a Optimizer. This allows using any PyTorch optimizer with custom hyperparameters.

fused_optimizer bool (default: False)

Whether to use the fused implementation of the optimizer when the backend supports it, falling back to the standard one when it does not. The fused path issues far fewer kernels per step, which matters most on mps.

lr float (default: 0.001)

Learning rate used for optimization when optimizer_creator is None.

weight_decay float (default: 1e-06)

Weight decay used in optimization, when optimizer_creator is None.

eps float (default: 0.01)

eps used for optimization, when optimizer_creator is None.

n_steps_kl_warmup int (default: None)

Number of training steps (minibatches) to scale weight on KL divergences from min_kl_weight to max_kl_weight. Only activated when n_epochs_kl_warmup is set to None.

n_epochs_kl_warmup int (default: 400)

Number of epochs to scale weight on KL divergences from min_kl_weight to max_kl_weight. Overrides n_steps_kl_warmup when both are not None.

reduce_lr_on_plateau bool (default: False)

Whether to monitor validation loss and reduce learning rate when validation set lr_scheduler_metric plateaus.

lr_factor float (default: 0.6)

Factor to reduce the learning rate.

lr_patience int (default: 30)

Number of epochs with no improvement after which the learning rate will be reduced.

lr_threshold float (default: 0.0)

Threshold for measuring the new optimum.

lr_scheduler_metric Literal['elbo_validation', 'reconstruction_loss_validation', 'kl_local_validation'] (default: 'elbo_validation')

Which metric to track for learning rate reduction.

lr_min float (default: 0)

Minimum learning rate allowed.

max_kl_weight float (default: 1.0)

Maximum scaling factor on KL divergence during training.

min_kl_weight float (default: 0.0)

Minimum scaling factor on KL divergence during training.

compile bool (default: False)

Whether to compile the model using torch.compile.

on_step bool | None (default: False)

None auto-logs for training_step but not validation/test_step. The default value is determined by the hook.

on_epoch bool | None (default: True)

None auto-logs for val/test step but not training_step. The default value is determined by the hook.

**loss_kwargs

Keyword args to pass to the loss method of the module. kl_weight should not be passed here and is handled automatically.

Attributes table#

kl_weight

Scaling factor on KL divergence during training

n_obs_training

Number of observations in the training set.

n_obs_validation

Number of observations in the validation set.

use_sync_dist

training

Methods table#

compute_and_log_metrics(loss_output, ...)

Computes and logs metrics.

configure_optimizers()

Configure optimizers for the model.

forward(*args, **kwargs)

Passthrough to the module's forward method.

get_optimizer_creator()

Get the optimizer creator for the model.

initialize_train_metrics()

Initialize the train-related metrics.

initialize_val_metrics()

Initialize val-related metrics.

on_train_end()

Undo the process-wide dynamo state that compile=True turned on.

prepare_scib_autotune(loss_outputs, stage)

training_step(batch, batch_idx)

Training step for the model.

validation_step(batch, batch_idx)

Validation step for the model.

Attributes#

TrainingPlan.kl_weight#

Scaling factor on KL divergence during training

TrainingPlan.n_obs_training#

Number of observations in the training set.

This will update the loss kwargs for loss rescaling.

Notes

This can get set after initialization

TrainingPlan.n_obs_validation#

Number of observations in the validation set.

This will update the loss kwargs for loss rescaling.

Notes

This can get set after initialization

TrainingPlan.use_sync_dist#
TrainingPlan.training#

Methods#

TrainingPlan.compute_and_log_metrics(loss_output, metrics, mode)[source]#

Computes and logs metrics.

Parameters:
loss_output LossOutput

LossOutput object from scvi-tools module

metrics dict[str, ElboMetric]

Dictionary of metrics to update

mode str

Postfix string to add to the metric name of extra metrics

TrainingPlan.configure_optimizers()[source]#

Configure optimizers for the model.

TrainingPlan.forward(*args, **kwargs)[source]#

Passthrough to the module’s forward method.

TrainingPlan.get_optimizer_creator()[source]#

Get the optimizer creator for the model.

TrainingPlan.initialize_train_metrics()[source]#

Initialize the train-related metrics.

TrainingPlan.initialize_val_metrics()[source]#

Initialize val-related metrics.

TrainingPlan.on_train_end()[source]#

Undo the process-wide dynamo state that compile=True turned on.

TrainingPlan.prepare_scib_autotune(loss_outputs, stage)[source]#
TrainingPlan.training_step(batch, batch_idx)[source]#

Training step for the model.

TrainingPlan.validation_step(batch, batch_idx)[source]#

Validation step for the model.