Note

This page was generated from PeakVI.ipynb. Interactive online version: .

# PeakVI: Analyzing scATACseq data¶

PeakVI is used for analyzing scATACseq data. This tutorial walks through how to read, set-up and train the model, accessing and visualizing the latent space, and differential accessibility. We use the 5kPBMC sample dataset from 10X but these steps can be easily adjusted for other datasets.

```
[1]:
```

```
import sys
# If True, will install via pypi, else will install from source
stable = False
IN_COLAB = "google.colab" in sys.modules
if IN_COLAB and stable:
!pip install --quiet scvi-tools[tutorials]
elif IN_COLAB and not stable:
!pip install --quiet --upgrade jsonschema
!pip install --quiet git+https://github.com/yoseflab/scvi-tools@master#egg=scvi-tools[tutorials]
```

First we need to download the sample data. This block will do this for a google colab session, but if you’re running it in a different platform you might need to adjust it, or download and unpack the data manually.

```
[2]:
```

```
!wget https://cf.10xgenomics.com/samples/cell-atac/1.2.0/atac_pbmc_5k_nextgem/atac_pbmc_5k_nextgem_filtered_peak_bc_matrix.tar.gz
!tar -xvf atac_pbmc_5k_nextgem_filtered_peak_bc_matrix.tar.gz
```

```
--2021-10-01 17:37:31-- https://cf.10xgenomics.com/samples/cell-atac/1.2.0/atac_pbmc_5k_nextgem/atac_pbmc_5k_nextgem_filtered_peak_bc_matrix.tar.gz
Resolving cf.10xgenomics.com (cf.10xgenomics.com)... 104.18.1.173, 104.18.0.173, 2606:4700::6812:ad, ...
Connecting to cf.10xgenomics.com (cf.10xgenomics.com)|104.18.1.173|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 114015463 (109M) [application/x-tar]
Saving to: ‘atac_pbmc_5k_nextgem_filtered_peak_bc_matrix.tar.gz.1’
atac_pbmc_5k_nextge 100%[===================>] 108.73M 90.6MB/s in 1.2s
2021-10-01 17:37:34 (90.6 MB/s) - ‘atac_pbmc_5k_nextgem_filtered_peak_bc_matrix.tar.gz.1’ saved [114015463/114015463]
filtered_peak_bc_matrix/
filtered_peak_bc_matrix/matrix.mtx
filtered_peak_bc_matrix/peaks.bed
filtered_peak_bc_matrix/barcodes.tsv
```

```
[3]:
```

```
import scvi
import anndata
import scipy
import numpy as np
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt
import random
random.seed(420)
```

```
Global seed set to 0
```

## loading data¶

PeakVI expects as input an AnnData object with a cell-by-region matrix. There are various pipelines that handle preprocessing of scATACseq to obtain this matrix from the sequencing data. If the data was generated by 10X genomics, this matrix is among the standard outputs of CellRanger. Other pipelines, like SnapATAC and ArchR, also generate similar matrices.

In the case of 10X data, PeakVI has a special reader function `scvi.data.read_10x_atac`

that reads the files and creates an AnnData object, demonstrated below. For conveniece, we also demonstrate how to initialize an AnnData object from scratch.

Throughout this tutorial, we use sample scATACseq data from 10X of 5K PBMCs.

```
[4]:
```

```
# read the count matrix into a sparse matrix, and the cell and region annotations as pandas DataFrames
counts = scipy.io.mmread("filtered_peak_bc_matrix/matrix.mtx").T
regions = pd.read_csv("filtered_peak_bc_matrix/peaks.bed", sep='\t', header=None, names=['chr','start','end'])
cells = pd.read_csv("filtered_peak_bc_matrix/barcodes.tsv", header=None, names=['barcodes'])
# then initialize a new AnnData object
adata = anndata.AnnData(X=counts, obs=cells, var=regions)
# or use this methods to read 10x data directly
adata = scvi.data.read_10x_atac("filtered_peak_bc_matrix")
```

```
/data/yosef2/users/jhong/miniconda3/envs/r_tutorial/lib/python3.7/site-packages/anndata/_core/anndata.py:119: ImplicitModificationWarning: Transforming to str index.
warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

we can use scanpy functions to handle, filter, and manipulate the data. In our case, we might want to filter out peaks that are rarely detected, to make the model train faster:

```
[5]:
```

```
print(adata.shape)
# compute the threshold: 5% of the cells
min_cells = int(adata.shape[0] * 0.05)
# in-place filtering of regions
sc.pp.filter_genes(adata, min_cells=min_cells)
print(adata.shape)
```

```
(4585, 115554)
(4585, 33142)
```

## set up, training, saving, and loading¶

We can now set up the AnnData object, which will ensure everything the model needs is in place for training.

This is also the stage where we can condition the model on additional covariates, which encourages the model to remove the impact of those covariates from the learned latent space. Our sample data is a single batch, so we won’t demonstrate this directly, but it can be done simply by setting the `batch_key`

argument to the annotation to be used as a batch covariate (must be a valid key in `adata.obs`

) .

```
[6]:
```

```
scvi.model.PEAKVI.setup_anndata(adata)
```

```
INFO No batch_key inputted, assuming all cells are same batch
INFO No label_key inputted, assuming all cells have same label
INFO Using data from adata.X
INFO Successfully registered anndata object containing 4585 cells, 33142 vars, 1 batches,
1 labels, and 0 proteins. Also registered 0 extra categorical covariates and 0 extra
continuous covariates.
INFO Please do not further modify adata until model is trained.
```

We can now create a PeakVI model object and train it!

**Importantly**: the default max epochs is set to 500, but in practice PeakVI stops early once the model converges, which rarely requires that many, especially for large datasets (which require fewer epochs to converge, since each epoch includes letting the model view more data). *So the estimated runtime is usually a substantial overestimate of the actual runtime.* In the case of the data we use for this tutorial, it used less than half of the max epochs!

```
[7]:
```

```
pvi = scvi.model.PEAKVI(adata)
pvi.train()
```

```
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1,2]
```

```
Epoch 253/500: 51%|█████ | 253/500 [07:24<07:13, 1.76s/it, loss=1.91e+08, v_num=1]
```

since training a model can take a while, we recommend saving the trained model after training, just in case.

```
[8]:
```

```
pvi.save("trained_model", overwrite=True)
```

We can then load the model later, which require providing an AnnData object that is structured similarly to the one used for training (or, in most cases, the same one):

```
[9]:
```

```
pvi = scvi.model.PEAKVI.load("trained_model", adata)
```

```
INFO Using data from adata.X
INFO Registered keys:['X', 'batch_indices', 'labels']
INFO Successfully registered anndata object containing 4585 cells, 33142 vars, 1 batches,
1 labels, and 0 proteins. Also registered 0 extra categorical covariates and 0 extra
continuous covariates.
```

## visualizing and analyzing the latent space¶

We can now use the trained model to visualize, cluster, and analyze the data. We first extract the latent representation from the model, and save it back into our AnnData object:

```
[10]:
```

```
latent = pvi.get_latent_representation()
adata.obsm["X_PeakVI"] = latent
print(latent.shape)
```

```
(4585, 13)
```

We can now use scanpy functions to cluster and visualize our latent space:

```
[11]:
```

```
# compute the k-nearest-neighbor graph that is used in both clustering and umap algorithms
sc.pp.neighbors(adata, use_rep="X_PeakVI")
# compute the umap
sc.tl.umap(adata, min_dist=0.2)
# cluster the space (we use a lower resolution to get fewer clusters than the default)
sc.tl.leiden(adata, key_added="cluster_pvi", resolution=0.2)
```

```
[12]:
```

```
sc.pl.umap(adata, color='cluster_pvi')
```

```
... storing 'chr' as categorical
```

## differential accessibility¶

Finally, we can use PeakVI to identify regions that are differentially accessible. There are many different ways to run this analysis, but the simplest is comparing one cluster against all others, or comparing two clusters to each other. In the first case we’ll be looking for marker-regions, so we’ll mostly want a one-sided test (the significant regions will only be the ones preferentially accessible in our target cluster). In the second case we’ll use a two-sided test to find regions that are differentially accessible, regardless of direction.

We demonstrate both of these next, and do this in two different ways: (1) more convenient but less flexible: using an existing factor to group the cells, and then comparing groups. (2) more flexible: using cell indices directly.

**If the data includes multiple batches**, we encourage setting `batch_correction=True`

so the model will sample from multiple batches when computing the differential signal. We do this below despite the data only having a single batch, as a demonstration.

```
[13]:
```

```
# (1.1) using a known factor to compare two clusters
## two-sided is True by default, but included here for emphasis
da_res11 = pvi.differential_accessibility(groupby='cluster_pvi', group1='3', group2='0', two_sided=True)
# (1.2) using a known factor to compare a cluster against all other clusters
## if we only provide group1, group2 is all other cells by default
da_res12 = pvi.differential_accessibility(groupby='cluster_pvi', group1='3', two_sided=False)
# (2.1) using indices to compare two clusters
## we can use boolean masks or integer indices for the `idx1` and `idx2` arguments
da_res21 = pvi.differential_accessibility(
idx1 = adata.obs.cluster_pvi == '3',
idx2 = adata.obs.cluster_pvi == '0',
two_sided=True,
)
# (2.2) using indices to compare a cluster against all other clusters
## if we don't provide idx2, it uses all other cells as the contrast
da_res22 = pvi.differential_accessibility(
idx1 = np.where(adata.obs.cluster_pvi == '3'),
two_sided=False,
)
da_res22.head()
```

```
DE...: 100%|██████████| 1/1 [00:14<00:00, 14.21s/it]
DE...: 100%|██████████| 1/1 [00:14<00:00, 14.20s/it]
DE...: 100%|██████████| 1/1 [00:15<00:00, 15.58s/it]
DE...: 100%|██████████| 1/1 [00:15<00:00, 15.22s/it]
```

```
[13]:
```

prob_da | is_da_fdr | bayes_factor | effect_size | emp_effect | est_prob1 | est_prob2 | emp_prob1 | emp_prob2 | |
---|---|---|---|---|---|---|---|---|---|

chr1:713460-715296 | 0.2470 | False | -1.114677 | 0.075389 | 0.049756 | 0.650814 | 0.726204 | 0.350679 | 0.400434 |

chr1:752299-753057 | 0.0360 | False | -3.287572 | 0.110416 | 0.053858 | 0.026006 | 0.136422 | 0.011312 | 0.065170 |

chr1:762004-763374 | 0.1322 | False | -1.881645 | 0.188169 | 0.100717 | 0.381069 | 0.569238 | 0.192308 | 0.293024 |

chr1:804843-805776 | 0.4788 | False | -0.084851 | -0.054666 | -0.033183 | 0.386094 | 0.331429 | 0.196833 | 0.163650 |

chr1:839252-841274 | 0.2620 | False | -1.035599 | 0.053273 | 0.027211 | 0.310811 | 0.364084 | 0.147059 | 0.174270 |

Note that `da_res11`

and `da_res21`

are equivalent, as are `da_res12`

and `da_res22`

. The return value is a pandas DataFrame with the differential results and basic properties of the comparison:

`prob_da`

in our case is the probability of cells from cluster 0 being more than 0.05 (the default minimal effect) more accessible than cells from the rest of the data.

`is_da_fdr`

is a conservative classification (True/False) of whether a region is differential accessible. This is one way to threshold the results.

`bayes_factor`

is a statistical significance score. It doesn’t have a commonly acceptable threshold (e.g 0.05 for p-values), bu we demonstrate below that it’s well calibrated to the effect size.

`effect_size`

is the effect size, calculated as `est_prob1 - est_prob2`

.

`emp_effect`

is the empirical effect size, calculated as `emp_prob1 - emp_prob2`

.

`est_prob{1,2}`

are the estimated probabilities of accessibility in group1 and group2.

`emp_prob{1,2}`

are the empirical probabilities of detection (how many cells in group X was the region detected in).

We can make sure the probability of DA is well calibrated, and look at the regions that are identified as differentially accessible:

```
[14]:
```

```
plt.scatter(da_res22.effect_size, da_res22.prob_da, s=1)
plt.xlabel("effect size")
plt.ylabel("probability of DA")
plt.show()
da_res22.loc[da_res22.is_da_fdr].sort_values('prob_da', ascending=False).head(10)
```

```
[14]:
```

prob_da | is_da_fdr | bayes_factor | effect_size | emp_effect | est_prob1 | est_prob2 | emp_prob1 | emp_prob2 | |
---|---|---|---|---|---|---|---|---|---|

chr22:22522044-22523273 | 1.0000 | True | 18.420681 | -0.804578 | -0.407230 | 0.915299 | 0.110721 | 0.461538 | 0.054308 |

chr1:1071624-1073485 | 1.0000 | True | 18.420681 | -0.661622 | -0.335611 | 0.836944 | 0.175322 | 0.420814 | 0.085204 |

chr4:115519248-115520882 | 1.0000 | True | 18.420681 | -0.749133 | -0.382350 | 0.786780 | 0.037647 | 0.400452 | 0.018103 |

chr3:122294823-122297353 | 1.0000 | True | 18.420681 | -0.757865 | -0.412656 | 0.897258 | 0.139393 | 0.488688 | 0.076032 |

chr5:150503270-150506342 | 1.0000 | True | 18.420681 | -0.697132 | -0.391740 | 0.951799 | 0.254667 | 0.527149 | 0.135409 |

chr14:64701787-64703599 | 1.0000 | True | 18.420681 | -0.925818 | -0.548805 | 0.962763 | 0.036944 | 0.567873 | 0.019068 |

chr10:5882094-5883416 | 0.9998 | True | 8.516943 | -0.843671 | -0.440539 | 0.878818 | 0.035147 | 0.461538 | 0.020999 |

chr19:2710133-2711582 | 0.9998 | True | 8.516943 | -0.795542 | -0.413570 | 0.843327 | 0.047785 | 0.438914 | 0.025344 |

chr1:110553989-110555202 | 0.9998 | True | 8.516943 | -0.796103 | -0.453863 | 0.929749 | 0.133646 | 0.529412 | 0.075549 |

chr1:110926104-110928161 | 0.9998 | True | 8.516943 | -0.761378 | -0.383103 | 0.810353 | 0.048975 | 0.407240 | 0.024137 |

We can now examine these regions to understand what is happening in the data, using various different annotation and enrichment methods. For instance, `chr11:60222766-60223569`

, one of the regions preferentially accessible in cluster 0, is the promoter region of ``MS4A1`

<https://www.genecards.org/cgi-bin/carddisp.pl?gene=MS4A1>`__, also known as `CD20`

, a known B-cell surface marker, indicating that cluster 0 are probably B-cells.

```
[ ]:
```

```
```