scvi.dataloaders.DataSplitter#
- class scvi.dataloaders.DataSplitter(adata_manager, train_size=None, validation_size=None, shuffle_set_split=True, load_sparse_tensor=False, pin_memory=False, external_indexing=None, **kwargs)[source]#
Bases:
LightningDataModuleCreates data loaders
train_set,validation_set,test_set.If
train_size + validation_set < 1, thentest_setis non-empty.- Parameters:
- adata_manager
AnnDataManager AnnDataManagerobject that has been created viasetup_anndata.- train_size
float|None(default:None) float, or None (default is None, which is practically 0.9 and potentially adding a small last batch to validation cells)
- validation_size
float|None(default:None) float, or None (default is None)
- shuffle_set_split
bool(default:True) Whether to shuffle indices before splitting. If
False, the val, train, and test set are split in the sequential order of the data according tovalidation_sizeandtrain_sizepercentages.- load_sparse_tensor
bool(default:False) EXPERIMENTALIfTrue, loads sparse CSR or CSC arrays in the input dataset as sparseTensorwith the same layout. Can lead to significant speedups in transferring data to GPUs, depending on the sparsity of the data.- pin_memory
bool(default:False) Whether to copy tensors into device-pinned memory before returning them. Passed into
AnnDataLoader.- external_indexing
list[ndarray] |None(default:None) A list of data split indices in the order of training, validation, and test sets. Validation and test set are not required and can be left empty.
- **kwargs
Keyword args for data loader. If adata has labeled data, the data loader class is
SemiSupervisedDataLoader, else the data loader class isAnnDataLoader.
- adata_manager
Examples
>>> adata = scvi.data.synthetic_iid() >>> scvi.model.SCVI.setup_anndata(adata) >>> adata_manager = scvi.model.SCVI(adata).adata_manager >>> splitter = DataSplitter(adata) >>> splitter.setup() >>> train_dl = splitter.train_dataloader()
Attributes table#
Methods table#
|
Converts sparse tensors to dense if necessary. |
|
Split indices in train/test/val sets. |
Create a test data loader. |
|
Create a train data loader. |
|
|
Densifies sparse tensors before the transfer if the device can't hold them. |
Create validation data loader. |
Attributes#
Methods#
- DataSplitter.on_after_batch_transfer(batch, dataloader_idx)[source]#
Converts sparse tensors to dense if necessary.
- DataSplitter.transfer_batch_to_device(batch, device, dataloader_idx)[source]#
Densifies sparse tensors before the transfer if the device can’t hold them.
MPS has no sparse CSR/CSC tensor support at all (unlike a missing single-op kernel, the whole layout is unimplemented there), so the default transfer would crash trying to move the still-sparse tensor onto the device; densify first in that case.