114 lines (106 with data), 12.7 kB
U
ùqþc¼8 ã @ sl d Z ddlZddlZddlZddlZddlmZ ddlm Z ddl
m Z ddl m
Z
mZ G dd deZdS ) a
Author: Pinar Demetci
Principal Investigator: Ritambhara Singh, Ph.D. from Brown University
08 August 2021
Updated: 23 February 2023
SCOTv2 algorithm: Single Cell alignment using Optimal Transport version 2
Correspondence: pinar_demetci@brown.edu, ritambhara@brown.edu
é N)Údijkstra)Ú
csr_matrix)Úkneighbors_graph)ÚStandardScalerÚ normalizec @ st e Zd ZdZdd Zdd Zd$dd Zd%d
dZdd Zdd Z d&ddZ
d'ddZ d d Z d(d d Z
d)d"d#ZdS )*ÚSCOTv2a
SCOT algorithm for unsupervised alignment of single-cell multi-omic data.
https://www.biorxiv.org/content/10.1101/2020.04.28.066787v2 (original preprint)
https://www.liebertpub.com/doi/full/10.1089/cmb.2021.0446 (Journal of Computational Biology publication through RECOMB 2021 conference)
Input: domain1, domain2 in form of numpy arrays/matrices, where the rows correspond to samples and columns correspond to features.
Returns: aligned domain 1, aligned domain 2 in form of numpy arrays/matrices projected on domain 1
Example use:
# Given two numpy matrices, domain1 and domain2, where the rows are cells and columns are different genomic features:
scot= SCOT(domain1, domain2)
aligned_domain1, aligned_domain2 = scot.align(k=20, e=1e-3)
#If you can't pick the parameters k and e, you can try out our unsupervised self-tuning heuristic by running:
scot= SCOT(domain1, domain2)
aligned_domain1, aligned_domain2 = scot.align(selfTune=True)
Required parameters:
- k: Number of neighbors to be used when constructing kNN graphs. Default= min(min(n_1, n_2), 50), where n_i, for i=1,2 corresponds to the number of samples in the i^th domain.
- e: Regularization constant for the entropic regularization term in entropic Gromov-Wasserstein optimal transport formulation. Default= 1e-3
Optional parameters:
- normalize= Determines whether to normalize input data ahead of alignment. True or False (boolean parameter). Default = True.
- norm= Determines what sort of normalization to run, "l2", "l1", "max", "zscore". Default="l2"
- mode: "connectivity" or "distance". Determines whether to use a connectivity graph (adjacency matrix of 1s/0s based on whether nodes are connected) or a distance graph (adjacency matrix entries weighted by distances between nodes). Default="connectivity"
- metric: Sets the metric to use while constructing nearest neighbor graphs. some possible choices are "correlation", "minkowski". "correlation" is Pearson's correlation and "minkowski" is equivalent to Euclidean distance in its default form (). Default= "correlation".
- verbose: Prints loss while optimizing the optimal transport formulation. Default=True
- XontoY: Determines the direction of barycentric projection. True or False (boolean parameter). If True, projects domain1 onto domain2. If False, projects domain2 onto domain1. Default=True.
Note: If you want to specify the marginal distributions of the input domains and not use uniform distribution, please set the attributes p and q to the distributions of your choice (for domain 1, and 2, respectively)
after initializing a SCOT class instance and before running alignment and set init_marginals=False in .align() parameters
c C sT t |tkrt|dks td|| _g | _g | _g | _g | _g | _ g | _
g | _ d S )Né ay As input, SCOTv2 requires a list, containing at least two numpy arrays to be aligned. Each numpy array/matrix corresponds to a dataset, with samples (cells) in rows and features (latent representations or genomic features) in columns. We recommend using latent representations (e.g. principal components for RNA-seq and topics - via cisTopic- for ATAC-seq/Methyl-seq).) ÚtypeÚlistÚlenÚAssertionErrorÚdataÚ marginalsÚgraphsÚ
graphDistsÚ couplingsZgwdistsÚflagsZ aligned_data)Úselfr
© r ú&/Users/CCMB/Desktop/SCOT/src/scotv2.pyÚ__init__; s zSCOTv2.__init__c C sD t t| jD ].}| j| jd }t |¡| }| j |¡ q| jS )Nr )Úranger r
ÚshapeÚtorchÚonesr Úappend)r ÚiZ num_cellsZ marginalDistr r r Ú_init_marginalsJ s
zSCOTv2._init_marginalsÚl2Tc C s |dkst dtt| jD ]^}|dkrHt }| | j| ¡| j|< q |dksX|d kr^d}nd}t| j| ||d| j|< q | jS )N)Úl1r ÚmaxÚzscorea~ Norm argument has to be either one of 'max', 'l1', 'l2' or 'zscore'. If you would like to perform another type of normalization, please give SCOT the normalized data and set the argument 'normalize=False' when running the algorithm. We have found l2 normalization to empirically perform better with single-cell sequencing datasets, including when using latent representations. r! Té r )ÚnormÚaxis)r r r r
r Ú
fit_transformr )r r# ÚbySampler Zscalerr$ r r r Ú
_normalizeR s zSCOTv2._normalizeé Ú connectivityÚ correlationc
C s\ |dkst d|dkr d}nd}tt| jD ]$}| j t| j| ||||d¡ q0| jS )N)r) ÚdistancezENorm argument has to be either one of 'connectivity', or 'distance'. r) TF)Z n_neighborsÚmodeÚmetricÚ include_self)r r r r
r r r )r Úkr, r- r. r r r r Úconstruct_graphc s "zSCOTv2.construct_graphc C sf t t| jD ]P}tt| j| ddd}t ||tjk ¡}||||k< | j
|| ¡ ¡ q| j S )NF)ÚcsgraphÚdirectedÚreturn_predecessors) r r r
r r r ÚnpÚnanmaxÚinfr r r )r r Z shortestPathZMax_distr r r Úinit_graph_distanceso s zSCOTv2.init_graph_distancesc C s¶ |dks|dkr&t |¡t |¡ }}||| | ¡ ||| | ¡ } | d|dd
df |ddd
f | |dd
df |ddd
f ¡ } d| | dd|| d|| }
| |
} || || }}t|
D ]z}| ¡ }t d||| ¡dd|| }t d||| ¡dd|| }| ¡ | ¡ ¡ ¡ ¡ | | k rì qhqì|dd
df |ddd
f | |dd
df |ddd
f }|||fS ) a}
Parameters
----------
- ecost: torch.Tensor of size [size_X, size_Y]
Exponential kernel generated from the local cost based on the current coupling.
- u: torch.Tensor of size [size_X[0]].
First dual potential defined on X.
- v: torch.Tensor of size [size_Y[0]].
Second dual potential defined on Y.
- mass: torch.Tensor of size [1].
Mass of the current coupling.
- nits_sinkhorn: int.
Maximum number of iterations to update Sinkhorn potentials in inner loop.
- tol_sinkhorn: float
Tolerance on convergence of Sinkhorn potentials.
Returns
----------
u: torch.Tensor of size [size_X[0]]
First dual potential of Sinkhorn algorithm
v: torch.Tensor of size [size_Y[0]]
Second dual potential of Sinkhorn algorithm
logpi: torch.Tensor of size [size_X, size_Y]
Optimal transport plan in log-space.
Nr g à?g @zij,i->jg ð¿ç ð?úij,j->i)
r Ú ones_likeÚsumr ÚcloneÚeinsumÚlogÚabsr Úitem)r ÚecostÚuÚvÚaÚbZmassÚepsÚrhoÚrho2Ú
nits_sinkhornÚ tol_sinkhornr/ ÚzÚjZu_prevÚpir r r Ú_exp_sinkhorn_solver{ s ,P( ""$DzSCOTv2._exp_sinkhorn_solverç{®Gáz?r8 Né¸ çíµ ÷Æ°>c C s4 |d kr |}|d d
d f |d d d
f | ¡ | ¡ ¡ } t | ¡}
d\}}t|D ]Î}| ¡ }
| ¡ }t d|t d|| ¡¡}t | | |d d
d f |d d d
f d ¡ ¡}tj | ddtj | dd }}t d|d |¡}t d
|d |¡}|d d
d f |d d d
f d | || }|td k r`||t ||| d ¡ ¡ }|td k r||t ||| d ¡ ¡ }| || ¡ }|d dkrºt
d
| | ||||||||||
| ¡ \}}} d}t t
| ¡¡rød}|| ¡ ¡ | } | |
¡ ¡ ¡ | k rZ q,qZ| |fS )N)NNz ij,kj->ikz kl,jl->kjg»½×Ùß|Û=r" )Údimr r9 r zkl,l->kÚInfé
zUnbalanced GW step:TF)r; Úsqrtr Ú
zeros_liker r< r= r> ÚfloatÚexpÚprintrN ÚanyÚisnanr? r r@ )r rD ÚdxrE ÚdyrF rG rH Ú nits_planÚtol_planrI rJ rM Zpi_prevÚupZvpr ÚmpZdistxyZkl_piÚmuÚnuZdistxxZdistyyZlcostrA Úflagr r r Úexp_unbalanced_gw¨ s: 4
6 0""
$zSCOTv2.exp_unbalanced_gwc
C s |r| j ||d | ¡ td | j|||d | ¡ tt| jd D ]²}
td t | j
d ¡t | j
|
d ¡ } } t | j d ¡t | j |
d ¡ }
}| j | |
| |||| ddddd \}}| j
|¡ | j |¡ |d
krLtd ||| f d qL| j
S )
N)r# r& z&computing intra-domain graph distances)r/ r, r- r" z#running pairwise dataset alignmentsr rP rQ )rF rG rH r^ r_ rI rJ Fz4Solver got NaN plan with params (eps, rho, rho2) = z . Try increasing argument eps)r' r rY r0 r7 r r r
r ZTensorr r re r r r Ú Exception)r r r# r& r/ r, r- rF rG rH r rD rE r\ r] Úcouplingrd r r r Úfind_correspondencesÒ s$ &&$ ÿzSCOTv2.find_correspondencesc C st | j d g}tdt| jD ]R}t | j| ¡ ¡}tj|dd}t ||d d
d f | j d ¡}| |¡ q |S )Nr r" )r$ )
r
r r r r4 Ú transposeÚnumpyr; Úmatmulr )r Zaligned_datasetsr rg ÚweightsZprojected_datar r r Úbarycentric_projectioné s " z SCOTv2.barycentric_projectionrT c C sv t | j}g }g }t|d D ](}| j| t | j| ¡d | j|< q t|D ]°}| j| | j| j | j| j| j| k¡ | j| | j| j| j| k¡ }t |
¡ ¡}t |dk¡} d|| || < t t
|t t |¡d ¡¡¡}
| |
| ¡ qPg } g } t|d D ]t}| t t
t t t | j| ¡d ¡¡| j| ¡¡¡ | t t
| j| t t | j| ¡d ¡¡¡¡ qtd }
|d || d }|d || d }td|d D ]D}t |
| j| f¡}
t||| || | }||| | }qÊtj |¡\}}|d }t |d ¡}t
|t
|t |¡¡¡}tj |¡\}}|d }t |d ¡}t
|t
|t |¡¡¡}t
|t
|
|¡¡}tj |¡\}}}dg}t|d D ] }| || t t| ¡ qÈ|dd
dt
f t |¡dd
dt
f }}t
||¡}t
||¡}g }t|d D ]$}| ||| ||d
¡ qB| |¡ |S )z`
Co-embeds datasets in a shared space.
Implementation is based on Cao et al 2022 (Pamona)
r" r éÿÿÿÿgê-q=g à¿N)r r
r r r4 r r ÚTÚmultiplyÚarrayÚtodenseÚwhereÚdiagÚdotr r ri rg ÚvstackÚ
block_diagÚlinalgÚeigÚsvdZ
output_dim) r ÚLambdaÚout_dimÚ
n_datasetsZH0ÚLr Z
graph_dataÚWZ index_posÚDZSigma_xZSigma_yZS_xyZS_xxZS_yyrC ÚQÚVZH_xZH_yÚHÚUÚsigmaÚnumÚfxÚfyÚintegrated_datar r r Úcoembed_datasetsò s^
& , ÿ":8 0 "
zSCOTv2.coembed_datasetsÚ embeddingc
C sb |
dkst d| j||||||||| d td| j |
dkrP| j| | d}
n| ¡ }
|
| _|
S )N)r Z barycentricz®The input to the parameter 'projMethod' needs to be one of 'embedding' (if co-embedding them in a new shared space) or 'barycentric' (if using barycentric projection)) r r# r& r/ r, r- rF rG rH ÚFLAGSr )r{ r| )r rh rY r r rm r )r r r# r& r/ r, r- rF rG rH Ú
projMethodr{ r| r r r r Úalign2 s z SCOTv2.align)r T)r( r) r* )rO r8 NrP rQ rP rQ ) Tr Tr( r) r* rO r8 N)r8 rT ) Tr Tr( r) r* rO r8 Nr r8 rT )Ú__name__Ú
__module__Ú __qualname__Ú__doc__r r r' r0 r7 rN re rh rm r r r r r r r s "
-
*
@r )r rj r4 r ZotÚscipyZscipy.sparse.csgraphr Ú scipy.sparser Úsklearn.neighborsr Úsklearn.preprocessingr r Úobjectr r r r r Ú<module> s
Datasets
Models
