scvi.external.SCAR#
- class scvi.external.SCAR(adata, ambient_profile=None, n_hidden=150, n_latent=15, n_layers=2, dropout_rate=0.0, gene_likelihood='b', latent_distribution='normal', scale_activation='softplus_sp', sparsity=0.9, **model_kwargs)[source]#
Ambient RNA removal in scRNA-seq data [Sheng et al., 2022].
Original Github: https://github.com/Novartis/scar. The models are parameter matched in architecture, activations, dropout, sparsity, and batch normalization.
- Parameters:
adata (AnnData) – AnnData object that has been registered via
setup_anndata()
.ambient_profile (Union[str, ndarray, DataFrame, tensor]) – The probability of occurrence of each ambient transcript. If None, averaging cells to estimate the ambient profile, by default None.
n_hidden (int) – Number of nodes per hidden layer.
n_latent (int) – Dimensionality of the latent space.
n_layers (int) – Number of hidden layers used for encoder and decoder NNs.
dropout_rate (float) – Dropout rate for neural networks.
gene_likelihood (Literal['zinb', 'nb', 'b', 'poisson']) – One of: *
'b'
- Binomial distribution *'nb'
- Negative binomial distribution *'zinb'
- Zero-inflated negative binomial distribution *'poisson'
- Poisson distributionlatent_distribution (Literal['normal', 'ln']) – One of: *
'normal'
- Normal distribution *'ln'
- Logistic normal distribution (Normal(0, I) transformed by softmax)scale_activation (Literal['softmax', 'softplus', 'softplus_sp']) – Activation layer to use for px_scale_decoder
sparsity (float) – The sparsity of expected native signals. It varies between datasets, e.g. if one prefilters genes – use only highly variable genes – the sparsity should be low; on the other hand, it should be set high in the case of unflitered genes.
**model_kwargs – Keyword args for
SCAR
Examples
>>> adata = anndata.read_h5ad(path_to_anndata) >>> raw_adata = anndata.read_h5ad(path_to_raw_anndata) >>> scvi_external.SCAR.setup_anndata(adata, batch_key="batch") >>> scvi_external.SCAR.get_ambient_profile(adata=adata, raw_adata=raw_adata, prob=0.995) >>> vae = scvi_external.SCAR(adata) >>> vae.train() >>> adata.obsm["X_scAR"] = vae.get_latent_representation() >>> adata.layers['denoised'] = vae.get_denoised_counts()
Attributes table#
Data attached to model instance. |
|
Manager instance associated with self.adata. |
|
The current device that the module's params are on. |
|
Returns computed metrics during training. |
|
Whether the model has been trained. |
|
Observations that are in test set. |
|
Observations that are in train set. |
|
Observations that are in validation set. |
Methods table#
|
Converts a legacy saved model (<v0.15.0) to the updated save format. |
|
. |
|
Calculate ambient profile for relevant features. |
|
Retrieves the |
|
Generate observation samples from the posterior predictive distribution. |
|
Return the ELBO for the data. |
|
Generate gene-gene correlation matrix using scvi uncertainty and expression. |
|
Returns the object in AnnData associated with the key in the data registry. |
|
Returns the latent library size for each cell. |
|
Return the latent representation for each cell. |
|
Estimates for the parameters of the likelihood \(p(x \mid z)\). |
|
Return the marginal LL for the data. |
|
Returns the normalized (decoded) gene expression. |
|
Return the reconstruction error for the data. |
|
Instantiate a model from the saved output. |
|
Return the full registry saved with the model. |
|
Generate observation samples from the posterior predictive distribution. |
|
Registers an |
|
Save the state of the model. |
|
Sets up the |
|
Move model to device. |
|
Train the model. |
|
Print summary of the setup for the initial AnnData or a given AnnData object. |
|
Print args used to setup a saved model. |
Attributes#
adata
adata_manager
device
history
is_trained
test_indices
train_indices
validation_indices
Methods#
convert_legacy_save
- classmethod SCAR.convert_legacy_save(dir_path, output_dir_path, overwrite=False, prefix=None)[source]#
Converts a legacy saved model (<v0.15.0) to the updated save format.
differential_expression
- SCAR.differential_expression(adata=None, groupby=None, group1=None, group2=None, idx1=None, idx2=None, mode='change', delta=0.25, batch_size=None, all_stats=True, batch_correction=False, batchid1=None, batchid2=None, fdr_target=0.05, silent=False, **kwargs)[source]#
.
A unified method for differential expression analysis.
Implements
'vanilla'
DE [Lopez et al., 2018] and'change'
mode DE [Boyeau et al., 2019].adata
AnnData object with equivalent structure to initial AnnData. If None, defaults to the AnnData object used to initialize the model.
- groupby
The key of the observations grouping to consider.
- group1
Subset of groups, e.g. [
'g1'
,'g2'
,'g3'
], to which comparison shall be restricted, or all groups ingroupby
(default).- group2
If
None
, compare each group ingroup1
to the union of the rest of the groups ingroupby
. If a group identifier, compare with respect to this group.- idx1
idx1
andidx2
can be used as an alternative to the AnnData keys. Custom identifier forgroup1
that can be of three sorts: (1) a boolean mask, (2) indices, or (3) a string. If it is a string, then it will query indices that verifies conditions onadata.obs
, as described inpandas.DataFrame.query()
Ifidx1
is notNone
, this option overridesgroup1
andgroup2
.- idx2
Custom identifier for
group2
that has the same properties asidx1
. By default, includes all cells not specified inidx1
.- mode
Method for differential expression. See user guide for full explanation.
- delta
specific case of region inducing differential expression. In this case, we suppose that \(R \setminus [-\delta, \delta]\) does not induce differential expression (change model default case).
- batch_size
Minibatch size for data loading into model. Defaults to
scvi.settings.batch_size
.- all_stats
Concatenate count statistics (e.g., mean expression group 1) to DE results.
- batch_correction
Whether to correct for batch effects in DE inference.
- batchid1
Subset of categories from
batch_key
registered insetup_anndata
, e.g. ['batch1'
,'batch2'
,'batch3'
], forgroup1
. Only used ifbatch_correction
isTrue
, and by default all categories are used.- batchid2
Same as
batchid1
for group2.batchid2
must either have null intersection withbatchid1
, or be exactly equal tobatchid1
. When the two sets are exactly equal, cells are compared by decoding on the same batch. When sets have null intersection, cells fromgroup1
andgroup2
are decoded on each group ingroup1
andgroup2
, respectively.- fdr_target
Tag features as DE based on posterior expected false discovery rate.
- silent
If True, disables the progress bar. Default: False.
- **kwargs
Keyword args for
scvi.model.base.DifferentialComputation.get_bayes_factors()
Differential expression DataFrame.
get_ambient_profile
- static SCAR.get_ambient_profile(adata, raw_adata, prob=0.995, min_raw_counts=2, iterations=3, n_batch=1, sample=50000)[source]#
Calculate ambient profile for relevant features.
Identify the cell-free droplets through a multinomial distribution. See EmptyDrops [Lun et al., 2019] for details.
- Parameters:
adata (AnnData) – A filtered adata object, loaded from filtered_feature_bc_matrix using
scanpy.read
, gene filtering is recommended to save memory.raw_adata (AnnData) – A raw adata object, loaded from raw_feature_bc_matrix using
read()
.prob (float) – The probability of each gene, considered as containing ambient RNA if greater than prob (joint prob euqals to the product of all genes for a droplet), by default 0.995.
min_raw_counts (int) – Total counts filter for raw_adata, filtering out low counts to save memory, by default 2.
iterations (int) – Total iterations, by default 3.
n_batch (int) – Total number of batches, set it to a bigger number when out of memory issue occurs, by default 1.
sample (int) – Randomly sample droplets to test, if greater than total droplets, use all droplets, by default 50000.
- Returns:
The relevant ambient profile is added in
adata.varm
get_anndata_manager
- SCAR.get_anndata_manager(adata, required=False)[source]#
Retrieves the
AnnDataManager
for a given AnnData object specific to this model instance.Requires
self.id
has been set. Checks for anAnnDataManager
specific to this model instance.
get_denoised_counts
- SCAR.get_denoised_counts(adata=None, n_samples=1, batch_size=None)[source]#
Generate observation samples from the posterior predictive distribution.
The posterior predictive distribution is written as \(p(\hat{x} \mid x)\).
- Parameters:
adata (Optional[AnnData]) – AnnData object with equivalent structure to initial AnnData. If
None
, defaults to the AnnData object used to initialize the model.n_samples (int) – Number of samples for each cell.
batch_size (Optional[int]) – Minibatch size for data loading into model. Defaults to
scvi.settings.batch_size
.
- Returns:
x_denoised :
torch.Tensor
tensor with shape (n_cells, n_genes)- Return type:
get_elbo
- SCAR.get_elbo(adata=None, indices=None, batch_size=None)[source]#
Return the ELBO for the data.
The ELBO is a lower bound on the log likelihood of the data used for optimization of VAEs. Note, this is not the negative ELBO, higher is better.
- Parameters:
adata (Optional[AnnData]) – AnnData object with equivalent structure to initial AnnData. If
None
, defaults to the AnnData object used to initialize the model.indices (Optional[Sequence[int]]) – Indices of cells in adata to use. If
None
, all cells are used.batch_size (Optional[int]) – Minibatch size for data loading into model. Defaults to
scvi.settings.batch_size
.
- Return type:
get_feature_correlation_matrix
- SCAR.get_feature_correlation_matrix(adata=None, indices=None, n_samples=10, batch_size=64, rna_size_factor=1000, transform_batch=None, correlation_type='spearman')[source]#
Generate gene-gene correlation matrix using scvi uncertainty and expression.
- Parameters:
adata (Optional[AnnData]) – AnnData object with equivalent structure to initial AnnData. If
None
, defaults to the AnnData object used to initialize the model.indices (Optional[Sequence[int]]) – Indices of cells in adata to use. If
None
, all cells are used.n_samples (int) – Number of posterior samples to use for estimation.
batch_size (int) – Minibatch size for data loading into model. Defaults to
scvi.settings.batch_size
.rna_size_factor (int) – size factor for RNA prior to sampling gamma distribution.
transform_batch (Optional[Sequence[Union[int, float, str]]]) –
Batches to condition on. If transform_batch is:
None, then real observed batch is used.
int, then batch transform_batch is used.
list of int, then values are averaged over provided batches.
correlation_type (Literal['spearman', 'pearson']) – One of “pearson”, “spearman”.
- Returns:
Gene-gene correlation matrix
- Return type:
get_from_registry
- SCAR.get_from_registry(adata, registry_key)[source]#
Returns the object in AnnData associated with the key in the data registry.
AnnData object should be registered with the model prior to calling this function via the
self._validate_anndata
method.
get_latent_library_size
- SCAR.get_latent_library_size(adata=None, indices=None, give_mean=True, batch_size=None)[source]#
Returns the latent library size for each cell.
This is denoted as \(\ell_n\) in the scVI paper.
- Parameters:
adata (Optional[AnnData]) – AnnData object with equivalent structure to initial AnnData. If
None
, defaults to the AnnData object used to initialize the model.indices (Optional[Sequence[int]]) – Indices of cells in adata to use. If
None
, all cells are used.give_mean (bool) – Return the mean or a sample from the posterior distribution.
batch_size (Optional[int]) – Minibatch size for data loading into model. Defaults to
scvi.settings.batch_size
.
- Return type:
get_latent_representation
- SCAR.get_latent_representation(adata=None, indices=None, give_mean=True, mc_samples=5000, batch_size=None, return_dist=False)[source]#
Return the latent representation for each cell.
This is typically denoted as \(z_n\).
- Parameters:
adata (Optional[AnnData]) – AnnData object with equivalent structure to initial AnnData. If
None
, defaults to the AnnData object used to initialize the model.indices (Optional[Sequence[int]]) – Indices of cells in adata to use. If
None
, all cells are used.give_mean (bool) – Give mean of distribution or sample from it.
mc_samples (int) – For distributions with no closed-form mean (e.g.,
logistic normal
), how many Monte Carlo samples to take for computing mean.batch_size (Optional[int]) – Minibatch size for data loading into model. Defaults to
scvi.settings.batch_size
.return_dist (bool) – Return (mean, variance) of distributions instead of just the mean. If
True
, ignoresgive_mean
andmc_samples
. In the case of the latter,mc_samples
is used to compute the mean of a transformed distribution. Ifreturn_dist
is true the untransformed mean and variance are returned.
- Returns:
Low-dimensional representation for each cell or a tuple containing its mean and variance.
- Return type:
get_likelihood_parameters
- SCAR.get_likelihood_parameters(adata=None, indices=None, n_samples=1, give_mean=False, batch_size=None)[source]#
Estimates for the parameters of the likelihood \(p(x \mid z)\).
- Parameters:
adata (Optional[AnnData]) – AnnData object with equivalent structure to initial AnnData. If
None
, defaults to the AnnData object used to initialize the model.indices (Optional[Sequence[int]]) – Indices of cells in adata to use. If
None
, all cells are used.n_samples (Optional[int]) – Number of posterior samples to use for estimation.
give_mean (Optional[bool]) – Return expected value of parameters or a samples
batch_size (Optional[int]) – Minibatch size for data loading into model. Defaults to
scvi.settings.batch_size
.
- Return type:
get_marginal_ll
- SCAR.get_marginal_ll(adata=None, indices=None, n_mc_samples=1000, batch_size=None)[source]#
Return the marginal LL for the data.
The computation here is a biased estimator of the marginal log likelihood of the data. Note, this is not the negative log likelihood, higher is better.
- Parameters:
adata (Optional[AnnData]) – AnnData object with equivalent structure to initial AnnData. If
None
, defaults to the AnnData object used to initialize the model.indices (Optional[Sequence[int]]) – Indices of cells in adata to use. If
None
, all cells are used.n_mc_samples (int) – Number of Monte Carlo samples to use for marginal LL estimation.
batch_size (Optional[int]) – Minibatch size for data loading into model. Defaults to
scvi.settings.batch_size
.
- Return type:
get_normalized_expression
- SCAR.get_normalized_expression(adata=None, indices=None, transform_batch=None, gene_list=None, library_size=1, n_samples=1, n_samples_overall=None, batch_size=None, return_mean=True, return_numpy=None)[source]#
Returns the normalized (decoded) gene expression.
This is denoted as \(\rho_n\) in the scVI paper.
- Parameters:
adata (Optional[AnnData]) – AnnData object with equivalent structure to initial AnnData. If
None
, defaults to the AnnData object used to initialize the model.indices (Optional[Sequence[int]]) – Indices of cells in adata to use. If
None
, all cells are used.transform_batch (Optional[Sequence[Union[int, float, str]]]) –
Batch to condition on. If transform_batch is:
None, then real observed batch is used.
int, then batch transform_batch is used.
gene_list (Optional[Sequence[str]]) – Return frequencies of expression for a subset of genes. This can save memory when working with large datasets and few genes are of interest.
library_size (Union[float, Literal['latent']]) – Scale the expression frequencies to a common library size. This allows gene expression levels to be interpreted on a common scale of relevant magnitude. If set to
"latent"
, use the latent library size.n_samples (int) – Number of posterior samples to use for estimation.
batch_size (Optional[int]) – Minibatch size for data loading into model. Defaults to
scvi.settings.batch_size
.return_mean (bool) – Whether to return the mean of the samples.
return_numpy (Optional[bool]) – Return a
ndarray
instead of aDataFrame
. DataFrame includes gene names as columns. If eithern_samples=1
orreturn_mean=True
, defaults toFalse
. Otherwise, it defaults toTrue
.n_samples_overall (int) –
- Returns:
If
n_samples
> 1 andreturn_mean
is False, then the shape is(samples, cells, genes)
. Otherwise, shape is(cells, genes)
. In this case, return type isDataFrame
unlessreturn_numpy
is True.- Return type:
get_reconstruction_error
- SCAR.get_reconstruction_error(adata=None, indices=None, batch_size=None)[source]#
Return the reconstruction error for the data.
This is typically written as \(p(x \mid z)\), the likelihood term given one posterior sample. Note, this is not the negative likelihood, higher is better.
- Parameters:
adata (Optional[AnnData]) – AnnData object with equivalent structure to initial AnnData. If
None
, defaults to the AnnData object used to initialize the model.indices (Optional[Sequence[int]]) – Indices of cells in adata to use. If
None
, all cells are used.batch_size (Optional[int]) – Minibatch size for data loading into model. Defaults to
scvi.settings.batch_size
.
- Return type:
load
- classmethod SCAR.load(dir_path, adata=None, use_gpu=None, prefix=None, backup_url=None)[source]#
Instantiate a model from the saved output.
- Parameters:
dir_path (str) – Path to saved outputs.
adata (Optional[Union[AnnData, MuData]]) – AnnData organized in the same way as data used to train model. It is not necessary to run setup_anndata, as AnnData is validated against the saved
scvi
setup dictionary. If None, will check for and load anndata saved with the model.use_gpu (Optional[Union[str, int, bool]]) – Load model on default GPU if available (if None or True), or index of GPU to use (if int), or name of GPU (if str), or use CPU (if False).
backup_url (Optional[str]) – URL to retrieve saved outputs from if not present on disk.
- Returns:
Model with loaded state dictionaries.
Examples
>>> model = ModelClass.load(save_path, adata) # use the name of the model class used to save >>> model.get_....
load_registry
- static SCAR.load_registry(dir_path, prefix=None)[source]#
Return the full registry saved with the model.
posterior_predictive_sample
- SCAR.posterior_predictive_sample(adata=None, indices=None, n_samples=1, gene_list=None, batch_size=None)[source]#
Generate observation samples from the posterior predictive distribution.
The posterior predictive distribution is written as \(p(\hat{x} \mid x)\).
- Parameters:
adata (Optional[AnnData]) – AnnData object with equivalent structure to initial AnnData. If
None
, defaults to the AnnData object used to initialize the model.indices (Optional[Sequence[int]]) – Indices of cells in adata to use. If
None
, all cells are used.n_samples (int) – Number of samples for each cell.
gene_list (Optional[Sequence[str]]) – Names of genes of interest.
batch_size (Optional[int]) – Minibatch size for data loading into model. Defaults to
scvi.settings.batch_size
.
- Returns:
x_new :
torch.Tensor
tensor with shape (n_cells, n_genes, n_samples)- Return type:
register_manager
- classmethod SCAR.register_manager(adata_manager)[source]#
Registers an
AnnDataManager
instance with this model class.Stores the
AnnDataManager
reference in a class-specific manager store. Intended for use in thesetup_anndata()
class method followed up by retrieval of theAnnDataManager
via the_get_most_recent_anndata_manager()
method in the model init method.Notes
Subsequent calls to this method with an
AnnDataManager
instance referring to the same underlying AnnData object will overwrite the reference to previousAnnDataManager
.- Parameters:
adata_manager (AnnDataManager) –
save
- SCAR.save(dir_path, prefix=None, overwrite=False, save_anndata=False, **anndata_write_kwargs)[source]#
Save the state of the model.
Neither the trainer optimizer state nor the trainer history are saved. Model files are not expected to be reproducibly saved and loaded across versions until we reach version 1.0.
- Parameters:
dir_path (str) – Path to a directory.
prefix (Optional[str]) – Prefix to prepend to saved file names.
overwrite (bool) – Overwrite existing data or not. If
False
and directory already exists atdir_path
, error will be raised.save_anndata (bool) – If True, also saves the anndata
anndata_write_kwargs – Kwargs for
write()
setup_anndata
- classmethod SCAR.setup_anndata(adata, layer=None, size_factor_key=None, **kwargs)[source]#
Sets up the
AnnData
object for this model.A mapping will be created between data fields used by this model to their respective locations in adata. None of the data in adata are modified. Only adds fields to adata.
- Parameters:
layer (Optional[str]) – if not
None
, uses this as the key inadata.layers
for raw count data.batch_key – key in
adata.obs
for batch information. Categories will automatically be converted into integer categories and saved toadata.obs['_scvi_batch']
. IfNone
, assigns the same batch to all the data.labels_key – key in
adata.obs
for label information. Categories will automatically be converted into integer categories and saved toadata.obs['_scvi_labels']
. IfNone
, assigns the same label to all the data.size_factor_key (Optional[str]) – key in
adata.obs
for size factor information. Instead of using library size as a size factor, the provided size factor column will be used as offset in the mean of the likelihood. Assumed to be on linear scale.adata (AnnData) –
to_device
- SCAR.to_device(device)[source]#
Move model to device.
- Parameters:
device (Union[str, int]) – Device to move model to. Options: ‘cpu’ for CPU, integer GPU index (eg. 0), or ‘cuda:X’ where X is the GPU index (eg. ‘cuda:0’). See torch.device for more info.
Examples
>>> adata = scvi.data.synthetic_iid() >>> model = scvi.model.SCVI(adata) >>> model.to_device('cpu') # moves model to CPU >>> model.to_device('cuda:0') # moves model to GPU 0 >>> model.to_device(0) # also moves model to GPU 0
train
- SCAR.train(max_epochs=None, use_gpu=None, train_size=0.9, validation_size=None, batch_size=128, early_stopping=False, plan_kwargs=None, **trainer_kwargs)[source]#
Train the model.
- Parameters:
max_epochs (Optional[int]) – Number of passes through the dataset. If
None
, defaults tonp.min([round((20000 / n_cells) * 400), 400])
use_gpu (Optional[Union[str, int, bool]]) – Use default GPU if available (if None or True), or index of GPU to use (if int), or name of GPU (if str, e.g.,
'cuda:0'
), or use CPU (if False).train_size (float) – Size of training set in the range [0.0, 1.0].
validation_size (Optional[float]) – Size of the test set. If
None
, defaults to 1 -train_size
. Iftrain_size + validation_size < 1
, the remaining cells belong to a test set.batch_size (int) – Minibatch size to use during training.
early_stopping (bool) – Perform early stopping. Additional arguments can be passed in
**kwargs
. SeeTrainer
for further options.plan_kwargs (Optional[dict]) – Keyword args for
TrainingPlan
. Keyword arguments passed totrain()
will overwrite values present inplan_kwargs
, when appropriate.**trainer_kwargs – Other keyword args for
Trainer
.
view_anndata_setup
- SCAR.view_anndata_setup(adata=None, hide_state_registries=False)[source]#
Print summary of the setup for the initial AnnData or a given AnnData object.
view_setup_args