scvi.train.TrainingPlan#
- class scvi.train.TrainingPlan(module, *, optimizer='Adam', optimizer_creator=None, fused_optimizer=False, lr=0.001, update_only_decoder=False, weight_decay=1e-06, eps=0.01, n_steps_kl_warmup=None, n_epochs_kl_warmup=400, reduce_lr_on_plateau=False, lr_factor=0.6, lr_patience=30, lr_threshold=0.0, lr_scheduler_metric='elbo_validation', lr_min=0, max_kl_weight=1.0, min_kl_weight=0.0, compile=False, compile_kwargs=None, on_step=False, on_epoch=True, **loss_kwargs)[source]#
Bases:
LightningModuleLightning module task to train scvi-tools modules.
The training plan is a PyTorch Lightning Module that is initialized with a scvi-tools module object. It configures the optimizers, defines the training step and validation step, and computes metrics to be recorded during training. The training step and validation step are functions that take data, run it through the model, and return the loss, which will then be used to optimize the model parameters in the Trainer. Overall, custom training plans can be used to develop complex inference schemes on top of modules.
The following developer tutorial will familiarize you more with training plans and how to use them: Constructing a high-level model.
- Parameters:
- module
BaseModuleClass A module instance from class
BaseModuleClass.- optimizer
Literal['Adam','AdamW','Custom'] (default:'Adam') One of “Adam” (
Adam), “AdamW” (AdamW), or “Custom”, which requires a custom optimizer creator callable to be passed viaoptimizer_creator.- optimizer_creator
Callable[[Iterable[Tensor]],Optimizer] |None(default:None) A callable taking in parameters and returning a
Optimizer. This allows using any PyTorch optimizer with custom hyperparameters.- fused_optimizer
bool(default:False) Whether to use the fused implementation of the optimizer when the backend supports it, falling back to the standard one when it does not. The fused path issues far fewer kernels per step, which matters most on
mps.- lr
float(default:0.001) Learning rate used for optimization when
optimizer_creatoris None.- weight_decay
float(default:1e-06) Weight decay used in optimization, when
optimizer_creatoris None.- eps
float(default:0.01) eps used for optimization, when
optimizer_creatoris None.- n_steps_kl_warmup
int(default:None) Number of training steps (minibatches) to scale weight on KL divergences from
min_kl_weighttomax_kl_weight. Only activated whenn_epochs_kl_warmupis set to None.- n_epochs_kl_warmup
int(default:400) Number of epochs to scale weight on KL divergences from
min_kl_weighttomax_kl_weight. Overridesn_steps_kl_warmupwhen both are notNone.- reduce_lr_on_plateau
bool(default:False) Whether to monitor validation loss and reduce learning rate when validation set
lr_scheduler_metricplateaus.- lr_factor
float(default:0.6) Factor to reduce the learning rate.
- lr_patience
int(default:30) Number of epochs with no improvement after which the learning rate will be reduced.
- lr_threshold
float(default:0.0) Threshold for measuring the new optimum.
- lr_scheduler_metric
Literal['elbo_validation','reconstruction_loss_validation','kl_local_validation'] (default:'elbo_validation') Which metric to track for learning rate reduction.
- lr_min
float(default:0) Minimum learning rate allowed.
- max_kl_weight
float(default:1.0) Maximum scaling factor on KL divergence during training.
- min_kl_weight
float(default:0.0) Minimum scaling factor on KL divergence during training.
- compile
bool(default:False) Whether to compile the model using torch.compile.
- on_step
bool|None(default:False) Noneauto-logs for training_step but not validation/test_step. The default value is determined by the hook.- on_epoch
bool|None(default:True) Noneauto-logs for val/test step but nottraining_step. The default value is determined by the hook.- **loss_kwargs
Keyword args to pass to the loss method of the
module.kl_weightshould not be passed here and is handled automatically.
- module
Attributes table#
Scaling factor on KL divergence during training |
|
Number of observations in the training set. |
|
Number of observations in the validation set. |
|
Methods table#
|
Computes and logs metrics. |
Configure optimizers for the model. |
|
|
Passthrough to the module's forward method. |
Get the optimizer creator for the model. |
|
Initialize the train-related metrics. |
|
Initialize val-related metrics. |
|
Undo the process-wide dynamo state that |
|
|
|
|
Training step for the model. |
|
Validation step for the model. |
Attributes#
- TrainingPlan.kl_weight#
Scaling factor on KL divergence during training
- TrainingPlan.n_obs_training#
Number of observations in the training set.
This will update the loss kwargs for loss rescaling.
Notes
This can get set after initialization
- TrainingPlan.n_obs_validation#
Number of observations in the validation set.
This will update the loss kwargs for loss rescaling.
Notes
This can get set after initialization
- TrainingPlan.use_sync_dist#
- TrainingPlan.training#
Methods#
- TrainingPlan.compute_and_log_metrics(loss_output, metrics, mode)[source]#
Computes and logs metrics.
- Parameters:
- loss_output
LossOutput LossOutput object from scvi-tools module
- metrics
dict[str,ElboMetric] Dictionary of metrics to update
- mode
str Postfix string to add to the metric name of extra metrics
- loss_output