scvi.model.PEAKVI#
- class scvi.model.PEAKVI(adata, n_hidden=None, n_latent=None, n_layers_encoder=2, n_layers_decoder=2, dropout_rate=0.1, model_depth=True, region_factors=True, use_batch_norm='none', use_layer_norm='both', latent_distribution='normal', deeply_inject_covariates=False, encode_covariates=False, **model_kwargs)[source]#
Peak Variational Inference [Ashuach21]
- Parameters
- adata :
AnnData
AnnData object that has been registered via
setup_anndata()
.- n_hidden :
int
|None
Optional
[int
] (default:None
) Number of nodes per hidden layer. If None, defaults to square root of number of regions.
- n_latent :
int
|None
Optional
[int
] (default:None
) Dimensionality of the latent space. If None, defaults to square root of n_hidden.
- n_layers_encoder :
int
(default:2
) Number of hidden layers used for encoder NN.
- n_layers_decoder :
int
(default:2
) Number of hidden layers used for decoder NN.
- dropout_rate :
float
(default:0.1
) Dropout rate for neural networks
- model_depth :
bool
(default:True
) Model sequencing depth / library size (default: True)
- region_factors :
bool
(default:True
) Include region-specific factors in the model (default: True)
- latent_distribution : {‘normal’, ‘ln’}
Literal
[‘normal’, ‘ln’] (default:'normal'
) One of
'normal'
- Normal distribution (Default)'ln'
- Logistic normal distribution (Normal(0, I) transformed by softmax)
- deeply_inject_covariates :
bool
(default:False
) Whether to deeply inject covariates into all layers of the decoder. If False (default), covariates will only be included in the input layer.
- **model_kwargs
Keyword args for
PEAKVAE
- adata :
Examples
>>> adata = anndata.read_h5ad(path_to_anndata) >>> scvi.model.PEAKVI.setup_anndata(adata, batch_key="batch") >>> vae = scvi.model.PEAKVI(adata) >>> vae.train()
Notes
See further usage examples in the following tutorials:
Attributes table#
Data attached to model instance. |
|
Manager instance associated with self.adata. |
|
The current device that the module's params are on. |
|
Returns computed metrics during training. |
|
Whether the model has been trained. |
|
Observations that are in test set. |
|
Observations that are in train set. |
|
Observations that are in validation set. |
Methods table#
|
A unified method for differential accessibility analysis. |
|
Impute the full accessibility matrix. |
|
Retrieves the |
|
Return the ELBO for the data. |
|
Returns the object in AnnData associated with the key in the data registry. |
|
Return the latent representation for each cell. |
|
Return library size factors. |
|
Return the marginal LL for the data. |
|
Return the reconstruction error for the data. |
Return region-specific factors. |
|
|
Instantiate a model from the saved output. |
|
Online update of a reference model with scArches algorithm [Lotfollahi21]. |
|
Prepare data for query integration. |
|
Registers an |
|
Save the state of the model. |
|
Sets up the |
|
Move model to device. |
|
Trains the model using amortized variational inference. |
|
Print summary of the setup for the initial AnnData or a given AnnData object. |
|
Print args used to setup a saved model. |
Attributes#
adata#
adata_manager#
- PEAKVI.adata_manager#
Manager instance associated with self.adata.
- Return type
device#
history#
- PEAKVI.history#
Returns computed metrics during training.
is_trained#
test_indices#
train_indices#
validation_indices#
Methods#
differential_accessibility#
- PEAKVI.differential_accessibility(adata=None, groupby=None, group1=None, group2=None, idx1=None, idx2=None, mode='change', delta=0.05, batch_size=None, all_stats=True, batch_correction=False, batchid1=None, batchid2=None, fdr_target=0.05, silent=False, two_sided=True, **kwargs)[source]#
A unified method for differential accessibility analysis.
Implements “vanilla” DE [Lopez18] and “change” mode DE [Boyeau19].
- Parameters
- adata :
AnnData
|None
Optional
[AnnData
] (default:None
) AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.
- groupby :
str
|None
Optional
[str
] (default:None
) The key of the observations grouping to consider.
- group1 :
Iterable
[str
] |None
Optional
[Iterable
[str
]] (default:None
) Subset of groups, e.g. [‘g1’, ‘g2’, ‘g3’], to which comparison shall be restricted, or all groups in groupby (default).
- group2 :
str
|None
Optional
[str
] (default:None
) If None, compare each group in group1 to the union of the rest of the groups in groupby. If a group identifier, compare with respect to this group.
- idx1 :
Sequence
[int
] |Sequence
[bool
] |str
|None
Union
[Sequence
[int
],Sequence
[bool
],str
,None
] (default:None
) idx1 and idx2 can be used as an alternative to the AnnData keys. Custom identifier for group1 that can be of three sorts: (1) a boolean mask, (2) indices, or (3) a string. If it is a string, then it will query indices that verifies conditions on adata.obs, as described in
pandas.DataFrame.query()
If idx1 is not None, this option overrides group1 and group2.- idx2 :
Sequence
[int
] |Sequence
[bool
] |str
|None
Union
[Sequence
[int
],Sequence
[bool
],str
,None
] (default:None
) Custom identifier for group2 that has the same properties as idx1. By default, includes all cells not specified in idx1.
- mode : {‘vanilla’, ‘change’}
Literal
[‘vanilla’, ‘change’] (default:'change'
) Method for differential expression. See user guide for full explanation.
- delta :
float
(default:0.05
) specific case of region inducing differential expression. In this case, we suppose that \(R \setminus [-\delta, \delta]\) does not induce differential expression (change model default case).
- batch_size :
int
|None
Optional
[int
] (default:None
) Minibatch size for data loading into model. Defaults to scvi.settings.batch_size.
- all_stats :
bool
(default:True
) Concatenate count statistics (e.g., mean expression group 1) to DE results.
- batch_correction :
bool
(default:False
) Whether to correct for batch effects in DE inference.
- batchid1 :
Iterable
[str
] |None
Optional
[Iterable
[str
]] (default:None
) Subset of categories from batch_key registered in
setup_anndata
, e.g. [‘batch1’, ‘batch2’, ‘batch3’], for group1. Only used if batch_correction is True, and by default all categories are used.- batchid2 :
Iterable
[str
] |None
Optional
[Iterable
[str
]] (default:None
) Same as batchid1 for group2. batchid2 must either have null intersection with batchid1, or be exactly equal to batchid1. When the two sets are exactly equal, cells are compared by decoding on the same batch. When sets have null intersection, cells from group1 and group2 are decoded on each group in group1 and group2, respectively.
- fdr_target :
float
(default:0.05
) Tag features as DE based on posterior expected false discovery rate.
- silent :
bool
(default:False
) If True, disables the progress bar. Default: False.
- two_sided :
bool
(default:True
) Whether to perform a two-sided test, or a one-sided test.
- **kwargs
Keyword args for
scvi.model.base.DifferentialComputation.get_bayes_factors()
- adata :
- Return type
- Returns
Differential accessibility DataFrame with the following columns: prob_da
the probability of the region being differentially accessible
- is_da_fdr
whether the region passes a multiple hypothesis correction procedure with the target_fdr threshold
- bayes_factor
Bayes Factor indicating the level of significance of the analysis
- effect_size
the effect size, computed as (accessibility in population 2) - (accessibility in population 1)
- emp_effect
the empirical effect, based on observed detection rates instead of the estimated accessibility scores from the PeakVI model
- est_prob1
the estimated probability of accessibility in population 1
- est_prob2
the estimated probability of accessibility in population 2
- emp_prob1
the empirical (observed) probability of accessibility in population 1
- emp_prob2
the empirical (observed) probability of accessibility in population 2
get_accessibility_estimates#
- PEAKVI.get_accessibility_estimates(adata=None, indices=None, n_samples_overall=None, region_list=None, transform_batch=None, use_z_mean=True, threshold=None, normalize_cells=False, normalize_regions=False, batch_size=128, return_numpy=False)[source]#
Impute the full accessibility matrix.
Returns a matrix of accessibility probabilities for each cell and genomic region in the input (for return matrix A, A[i,j] is the probability that region j is accessible in cell i).
- Parameters
- adata :
AnnData
|None
Optional
[AnnData
] (default:None
) AnnData object that has been registered with scvi. If None, defaults to the AnnData object used to initialize the model.
- indices :
Sequence
[int
] |None
Optional
[Sequence
[int
]] (default:None
) Indices of cells in adata to use. If None, all cells are used.
- n_samples_overall :
int
|None
Optional
[int
] (default:None
) Number of samples to return in total
- region_list :
Sequence
[str
] |None
Optional
[Sequence
[str
]] (default:None
) Return accessibility estimates for this subset of regions. if None, all regions are used. This can save memory when dealing with large datasets.
- transform_batch :
int
|str
|None
Union
[int
,str
,None
] (default:None
) Batch to condition on. If transform_batch is:
None, then real observed batch is used
int, then batch transform_batch is used
- use_z_mean :
bool
(default:True
) If True (default), use the distribution mean. Otherwise, sample from the distribution.
- threshold :
float
|None
Optional
[float
] (default:None
) If provided, values below the threshold are replaced with 0 and a sparse matrix is returned instead. This is recommended for very large matrices. Must be between 0 and 1.
- normalize_cells :
bool
(default:False
) Whether to reintroduce library size factors to scale the normalized probabilities. This makes the estimates closer to the input, but removes the library size correction. False by default.
- normalize_regions :
bool
(default:False
) Whether to reintroduce region factors to scale the normalized probabilities. This makes the estimates closer to the input, but removes the region-level bias correction. False by default.
- batch_size :
int
(default:128
) Minibatch size for data loading into model
- return_numpy :
bool
(default:False
) If True and threshold=None, return
ndarray
. If True and threshold is given, returncsr_matrix
. If False, returnDataFrame
. DataFrame includes regions names as columns.
- adata :
- Return type
DataFrame
|ndarray
|csr_matrix
Union
[DataFrame
,ndarray
,csr_matrix
]
get_anndata_manager#
- PEAKVI.get_anndata_manager(adata, required=False)#
Retrieves the
AnnDataManager
for a given AnnData object specific to this model instance.Requires
self.id
has been set. Checks for anAnnDataManager
specific to this model instance.- Parameters
- Return type
get_elbo#
- PEAKVI.get_elbo(adata=None, indices=None, batch_size=None)#
Return the ELBO for the data.
The ELBO is a lower bound on the log likelihood of the data used for optimization of VAEs. Note, this is not the negative ELBO, higher is better.
- Parameters
- adata :
AnnData
|None
Optional
[AnnData
] (default:None
) AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.
- indices :
Sequence
[int
] |None
Optional
[Sequence
[int
]] (default:None
) Indices of cells in adata to use. If None, all cells are used.
- batch_size :
int
|None
Optional
[int
] (default:None
) Minibatch size for data loading into model. Defaults to scvi.settings.batch_size.
- adata :
- Return type
get_from_registry#
- PEAKVI.get_from_registry(adata, registry_key)#
Returns the object in AnnData associated with the key in the data registry.
AnnData object should be registered with the model prior to calling this function via the
self._validate_anndata
method.
get_latent_representation#
- PEAKVI.get_latent_representation(adata=None, indices=None, give_mean=True, mc_samples=5000, batch_size=None)#
Return the latent representation for each cell.
This is denoted as \(z_n\) in our manuscripts.
- Parameters
- adata :
AnnData
|None
Optional
[AnnData
] (default:None
) AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.
- indices :
Sequence
[int
] |None
Optional
[Sequence
[int
]] (default:None
) Indices of cells in adata to use. If None, all cells are used.
- give_mean :
bool
(default:True
) Give mean of distribution or sample from it.
- mc_samples :
int
(default:5000
) For distributions with no closed-form mean (e.g., logistic normal), how many Monte Carlo samples to take for computing mean.
- batch_size :
int
|None
Optional
[int
] (default:None
) Minibatch size for data loading into model. Defaults to scvi.settings.batch_size.
- adata :
- Return type
- Returns
-latent_representation (
ndarray
) Low-dimensional representation for each cell
get_library_size_factors#
- PEAKVI.get_library_size_factors(adata=None, indices=None, batch_size=128)[source]#
Return library size factors.
- Parameters
- adata :
AnnData
|None
Optional
[AnnData
] (default:None
) AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.
- indices :
Sequence
[int
] |None
Optional
[Sequence
[int
]] (default:None
) Indices of cells in adata to use. If None, all cells are used.
- batch_size :
int
(default:128
) Minibatch size for data loading into model. Defaults to scvi.settings.batch_size.
- adata :
- Return type
- Returns
Library size factor for expression and accessibility
get_marginal_ll#
- PEAKVI.get_marginal_ll(adata=None, indices=None, n_mc_samples=1000, batch_size=None)#
Return the marginal LL for the data.
The computation here is a biased estimator of the marginal log likelihood of the data. Note, this is not the negative log likelihood, higher is better.
- Parameters
- adata :
AnnData
|None
Optional
[AnnData
] (default:None
) AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.
- indices :
Sequence
[int
] |None
Optional
[Sequence
[int
]] (default:None
) Indices of cells in adata to use. If None, all cells are used.
- n_mc_samples :
int
(default:1000
) Number of Monte Carlo samples to use for marginal LL estimation.
- batch_size :
int
|None
Optional
[int
] (default:None
) Minibatch size for data loading into model. Defaults to scvi.settings.batch_size.
- adata :
- Return type
get_reconstruction_error#
- PEAKVI.get_reconstruction_error(adata=None, indices=None, batch_size=None)#
Return the reconstruction error for the data.
This is typically written as \(p(x \mid z)\), the likelihood term given one posterior sample. Note, this is not the negative likelihood, higher is better.
- Parameters
- adata :
AnnData
|None
Optional
[AnnData
] (default:None
) AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.
- indices :
Sequence
[int
] |None
Optional
[Sequence
[int
]] (default:None
) Indices of cells in adata to use. If None, all cells are used.
- batch_size :
int
|None
Optional
[int
] (default:None
) Minibatch size for data loading into model. Defaults to scvi.settings.batch_size.
- adata :
- Return type
get_region_factors#
load#
- classmethod PEAKVI.load(dir_path, adata=None, use_gpu=None, prefix=None)#
Instantiate a model from the saved output.
- Parameters
- dir_path :
str
Path to saved outputs.
- adata :
AnnData
|None
Optional
[AnnData
] (default:None
) AnnData organized in the same way as data used to train model. It is not necessary to run setup_anndata, as AnnData is validated against the saved scvi setup dictionary. If None, will check for and load anndata saved with the model.
- use_gpu :
str
|int
|bool
|None
Union
[str
,int
,bool
,None
] (default:None
) Load model on default GPU if available (if None or True), or index of GPU to use (if int), or name of GPU (if str), or use CPU (if False).
- prefix :
str
|None
Optional
[str
] (default:None
) Prefix of saved file names.
- dir_path :
- Returns
Model with loaded state dictionaries.
Examples
>>> model = ModelClass.load(save_path, adata) # use the name of the model class used to save >>> model.get_....
load_query_data#
- classmethod PEAKVI.load_query_data(adata, reference_model, inplace_subset_query_vars=False, use_gpu=None, unfrozen=False, freeze_dropout=False, freeze_expression=True, freeze_decoder_first_layer=True, freeze_batchnorm_encoder=True, freeze_batchnorm_decoder=False, freeze_classifier=True)#
Online update of a reference model with scArches algorithm [Lotfollahi21].
- Parameters
- adata :
AnnData
AnnData organized in the same way as data used to train model. It is not necessary to run setup_anndata, as AnnData is validated against the
registry
.- reference_model :
str
|BaseModelClass
Union
[str
,BaseModelClass
] Either an already instantiated model of the same class, or a path to saved outputs for reference model.
- inplace_subset_query_vars :
bool
(default:False
) Whether to subset and rearrange query vars inplace based on vars used to train reference model.
- use_gpu :
str
|int
|bool
|None
Union
[str
,int
,bool
,None
] (default:None
) Load model on default GPU if available (if None or True), or index of GPU to use (if int), or name of GPU (if str), or use CPU (if False).
- unfrozen :
bool
(default:False
) Override all other freeze options for a fully unfrozen model
- freeze_dropout :
bool
(default:False
) Whether to freeze dropout during training
- freeze_expression :
bool
(default:True
) Freeze neurons corersponding to expression in first layer
- freeze_decoder_first_layer :
bool
(default:True
) Freeze neurons corersponding to first layer in decoder
- freeze_batchnorm_encoder :
bool
(default:True
) Whether to freeze batchnorm weight and bias during training for encoder
- freeze_batchnorm_decoder :
bool
(default:False
) Whether to freeze batchnorm weight and bias during training for decoder
- freeze_classifier :
bool
(default:True
) Whether to freeze classifier completely. Only applies to SCANVI.
- adata :
prepare_query_anndata#
- static PEAKVI.prepare_query_anndata(adata, reference_model, return_reference_var_names=False, inplace=True)#
Prepare data for query integration.
This function will return a new AnnData object with padded zeros for missing features, as well as correctly sorted features.
- Parameters
- adata :
AnnData
AnnData organized in the same way as data used to train model. It is not necessary to run setup_anndata, as AnnData is validated against the
registry
.- reference_model :
str
|BaseModelClass
Union
[str
,BaseModelClass
] Either an already instantiated model of the same class, or a path to saved outputs for reference model.
- return_reference_var_names :
bool
(default:False
) Only load and return reference var names if True.
- inplace :
bool
(default:True
) Whether to subset and rearrange query vars inplace or return new AnnData.
- adata :
- Return type
- Returns
Query adata ready to use in load_query_data unless return_reference_var_names in which case a pd.Index of reference var names is returned.
register_manager#
- classmethod PEAKVI.register_manager(adata_manager)#
Registers an
AnnDataManager
instance with this model class.Stores the
AnnDataManager
reference in a class-specific manager store. Intended for use in thesetup_anndata()
class method followed up by retrieval of theAnnDataManager
via the_get_most_recent_anndata_manager()
method in the model init method.Notes
Subsequent calls to this method with an
AnnDataManager
instance referring to the same underlying AnnData object will overwrite the reference to previousAnnDataManager
.
save#
- PEAKVI.save(dir_path, prefix=None, overwrite=False, save_anndata=False, **anndata_write_kwargs)#
Save the state of the model.
Neither the trainer optimizer state nor the trainer history are saved. Model files are not expected to be reproducibly saved and loaded across versions until we reach version 1.0.
- Parameters
- dir_path :
str
Path to a directory.
- prefix :
str
|None
Optional
[str
] (default:None
) Prefix to prepend to saved file names.
- overwrite :
bool
(default:False
) Overwrite existing data or not. If False and directory already exists at dir_path, error will be raised.
- save_anndata :
bool
(default:False
) If True, also saves the anndata
- anndata_write_kwargs
Kwargs for
write()
- dir_path :
setup_anndata#
- classmethod PEAKVI.setup_anndata(adata, batch_key=None, labels_key=None, categorical_covariate_keys=None, continuous_covariate_keys=None, layer=None, **kwargs)[source]#
- Sets up the
AnnData
object for this model. A mapping will be created between data fields used by this model to their respective locations in adata.
None of the data in adata are modified. Only adds fields to adata.
- Parameters
- batch_key :
str
|None
Optional
[str
] (default:None
) key in adata.obs for batch information. Categories will automatically be converted into integer categories and saved to adata.obs[‘_scvi_batch’]. If None, assigns the same batch to all the data.
- labels_key :
str
|None
Optional
[str
] (default:None
) key in adata.obs for label information. Categories will automatically be converted into integer categories and saved to adata.obs[‘_scvi_labels’]. If None, assigns the same label to all the data.
- layer :
str
|None
Optional
[str
] (default:None
) if not None, uses this as the key in adata.layers for raw count data.
- categorical_covariate_keys :
List
[str
] |None
Optional
[List
[str
]] (default:None
) keys in adata.obs that correspond to categorical data.
- continuous_covariate_keys :
List
[str
] |None
Optional
[List
[str
]] (default:None
) keys in adata.obs that correspond to continuous data.
- batch_key :
- Sets up the
to_device#
- PEAKVI.to_device(device)#
Move model to device.
- Parameters
Examples
>>> adata = scvi.data.synthetic_iid() >>> model = scvi.model.SCVI(adata) >>> model.to_device('cpu') # moves model to CPU >>> model.to_device('cuda:0') # moves model to GPU 0 >>> model.to_device(0) # also moves model to GPU 0
train#
- PEAKVI.train(max_epochs=500, lr=0.0001, use_gpu=None, train_size=0.9, validation_size=None, batch_size=128, weight_decay=0.001, eps=1e-08, early_stopping=True, early_stopping_patience=50, save_best=True, check_val_every_n_epoch=None, n_steps_kl_warmup=None, n_epochs_kl_warmup=50, plan_kwargs=None, **kwargs)[source]#
Trains the model using amortized variational inference.
- Parameters
- max_epochs :
int
(default:500
) Number of passes through the dataset.
- lr :
float
(default:0.0001
) Learning rate for optimization.
- use_gpu :
str
|int
|bool
|None
Union
[str
,int
,bool
,None
] (default:None
) Use default GPU if available (if None or True), or index of GPU to use (if int), or name of GPU (if str, e.g., ‘cuda:0’), or use CPU (if False).
- train_size :
float
(default:0.9
) Size of training set in the range [0.0, 1.0].
- validation_size :
float
|None
Optional
[float
] (default:None
) Size of the test set. If None, defaults to 1 - train_size. If train_size + validation_size < 1, the remaining cells belong to a test set.
- batch_size :
int
(default:128
) Minibatch size to use during training.
- weight_decay :
float
(default:0.001
) weight decay regularization term for optimization
- eps :
float
(default:1e-08
) Optimizer eps
- early_stopping :
bool
(default:True
) Whether to perform early stopping with respect to the validation set.
- early_stopping_patience :
int
(default:50
) How many epochs to wait for improvement before early stopping
- save_best :
bool
(default:True
) Save the best model state with respect to the validation loss (default), or use the final state in the training procedure
- check_val_every_n_epoch :
int
|None
Optional
[int
] (default:None
) Check val every n train epochs. By default, val is not checked, unless early_stopping is True. If so, val is checked every epoch.
- n_steps_kl_warmup :
int
|None
Optional
[int
] (default:None
) Number of training steps (minibatches) to scale weight on KL divergences from 0 to 1. Only activated when n_epochs_kl_warmup is set to None. If None, defaults to floor(0.75 * adata.n_obs).
- n_epochs_kl_warmup :
int
|None
Optional
[int
] (default:50
) Number of epochs to scale weight on KL divergences from 0 to 1. Overrides n_steps_kl_warmup when both are not None.
- plan_kwargs :
dict
|None
Optional
[dict
] (default:None
) Keyword args for
TrainingPlan
. Keyword arguments passed to train() will overwrite values present in plan_kwargs, when appropriate.- **kwargs
Other keyword args for
Trainer
.
- max_epochs :
view_anndata_setup#
- PEAKVI.view_anndata_setup(adata=None, hide_state_registries=False)#
Print summary of the setup for the initial AnnData or a given AnnData object.