282 lines (262 with data), 20.8 kB
B
ðaFk ã @ s¼ U d dl mZ d dlZd dlZd dlZd dlZd dlZ d dl
Z d dl Z d dl
mZ d dlmZ d dlmZ d dlmZ d dlmZmZmZmZmZ d dl m Z d dl m Z m Z m!Z! d d l"m#Z# d dl$Z$d dl%Z%e$j&d
d Z'ede(f e)d < e'dkre$j&d
d Z'e(e)d < e *e+¡Z,dddddddddgddddd
dddd d d d d e- fe j.e(e/e(ee0 e/e/ee/df ee/ e1e/e/ee/df e(e0e0e(ede(ej2j3ej2j4f e-ee/ee0def f d d d!Z5dddddddddgddddd
dddd d d d d e- fe j.e(e/e(ee0 e/e/ee/df ee/ e1e/e/ee/df e(e0e0e(ede(ej2j3ej2j4f e-ee/ee0def f d d"d#Z6ede(f e(d$d%d&Z7d'd( Z8dLe j.e/e(e(e1ej9d*d+d,Z:dMe j.e/d-d.d/Z;dNe j.e/e/e(e0e(ede(f e1ee0e0e0f d1 d2d3Z<d4d5 Z=d6d7 Z>dOe j.e/e/e(e(e(e(ede(f e1e0d:
d;d<Z?dPe j.ee0 e/e/e/ee/e0e0f d=d>d?Z@dd d d@dAd
d e- fe j.ee/ee/ df e0e0e/e/e1e(e-edef dB
dCdDZAdQe/e(e1e1e(ee( dIdJdKZBdS )Ré )ÚinfN)ÚFigure)Úspmatrix)Úchi2)ÚMappingÚSequenceÚTupleÚIterableÚUnion)Úissparse)Úadjusted_rand_scoreÚ normalized_mutual_info_scoreÚsilhouette_samples)ÚNearestNeighborsF)ZlogicalÚ
_cpu_countTÚdeltaé g{®Gáz?g{®Gáz?g{®Gáz¤?g{®Gáz´?g{®GázÄ?g{®GázÔ?g{®Gázä?ÚleidenÚ
cell_typesÚ
batch_indicesÚumapZjpgi, g333333Ó?é )ÚadataÚn_epochÚ
embedding_keyÚ n_neighborsÚ resolutionsÚclustering_methodÚ
cell_type_colÚ batch_colÚcolor_byÚ
return_figÚ
plot_fnameÚ
plot_ftypeÚplot_dirÚplot_dpiÚmin_distÚspreadÚn_jobsÚ random_stateÚ umap_kwargsÚreturnc ! C s® |r4t jj | j| ¡s4| j| t¡ d¡| j|< |rht jj | j| ¡sh| j| t¡ d¡| j|< t| |||dd || jkrªt|dkrªt | ||||d\}}}n d } }}| j|
¡ dkrZt |dkrÚ| j n| j
| | j| }|| jd < t |¡}tjj| ||d
}|rb|rb| jj||d d d }| dk rb| tj | |
d
¡¡ nd}d}|ox| j|
¡ dk}|rÐt| |||d|d}t| |||d|dd } tjj| |d|dd} t| |d} nd } } } } | sð| dk r|dkr|r||gn|g}t |}d| j krNx*| j d D ] } | |kr.| d| ¡ q.W |dk rb|g| }t | |||| |
t| d| | |
|d } nd} t |||||| | | | d S )aÕ Evaluates the clustering and batch correction performance of the given
embeddings, and optionally plots the embeddings.
Embeddings will be plotted if return_fig is True or plot_dir is provided.
When tensorboard_dir is provided, will also save the embeddings using a
tensorboard SummaryWriter.
NOTE: Set n_jobs to 1 if you encounter pickling error.
Args:
adata: the dataset with the embedding to be evaluated.
embedding_key: the key to the embedding. Must be in adata.obsm.
n_neighbors: #neighbors used when computing neithborhood graph and
calculating entropy of batch mixing / kBET.
resolutions: a sequence of resolutions used for clustering.
clustering_method: clustering method used. Should be one of 'leiden' or
'louvain'.
cell_type_col: a key in adata.obs to the cell type column.
batch_col: a key in adata.obs to the batch column.
return_fig: whether to return the Figure object. Useful for visualizing
the plot.
color_by: a list of adata.obs column keys to color the embeddings by.
If None, will look up adata.uns['color_by']. Only used if is
drawing.
plot_fname: file name of the generated plot. Only used if is drawing.
plot_ftype: file type of the generated plot. Only used if is drawing.
plot_dir: directory to save the generated plot. If None, do not save
the plot.
plot_dpi: dpi to save the plot.
writer: an initialized SummaryWriter to save the UMAP plot to. Only
used if is drawing.
min_dist: the min_dist argument in sc.tl.umap. Only used is drawing.
spread: the spread argument in sc.tl.umap. Only used if is drawing.
n_jobs: # jobs to generate. If <= 0, this is set to the number of
physical cores.
random_state: random state for knn calculation.
umap_kwargs: other kwargs to pass to sc.pl.umap.
Returns:
A dict storing the ari, nmi, asw, ebm and k_bet of the cell embeddings
with key "ari", "nmi", "asw", "ebm", "k_bet", respectively. If draw is
True and return_fig is True, will also store the plotted figure with
key "fig".
ÚcategoryT)Úuse_repr r) Úcalc_knnr )r r r r Nr ÚXZsilhouette_width)Ú group_keyÚembedÚmean)ÚindexÚcolumnsÚvaluesZaggfuncz.csvg F)r- r r r. r( é Ú cell_type)Z batch_keyr0 r1 Úverbose)r- r Ú.) r r r& r' Úckpt_dirÚfnamer! Údpir* ) ÚariÚnmiÚaswÚasw_2ÚebmÚk_betÚ batch_aswÚbatch_graph_scoreÚfig)!ÚpdÚapiÚtypesÚis_categorical_dtypeÚobsÚastypeÚstrÚ_get_knn_indicesÚlenÚ
clusteringÚnuniquer r/ ÚobsmÚnpr2 ÚscibÚmeZ
silhouetteZ pivot_tableÚto_csvÚosÚpathÚjoinÚ calculate_entropy_batch_mixingÚcalculate_kbetZsilhouette_batchÚget_graph_connectivityÚlistÚunsÚinsertÚdraw_embeddingsÚdict)!r r r r r r r r r r! r" r# r$ r% r& r' r( r) r* Ú cluster_keyÚbest_ariÚbest_nmiÚswr? r@ Zsw_tableZ
need_batchrA rB rC rD ÚcolrE © rf ú6C:\Users\Xin\Desktop\mscETM\Demo code_BS\eval_utils.pyÚevaluate s A
rh c C sº |r4t jj | j| ¡s4| j| t¡ d¡| j|< |rht jj | j| ¡sh| j| t¡ d¡| j|< t| |||dd || jkrªt|dkrªt | ||||d\}}}n d } }}|S )aÕ Evaluates the clustering and batch correction performance of the given
embeddings, and optionally plots the embeddings.
Embeddings will be plotted if return_fig is True or plot_dir is provided.
When tensorboard_dir is provided, will also save the embeddings using a
tensorboard SummaryWriter.
NOTE: Set n_jobs to 1 if you encounter pickling error.
Args:
adata: the dataset with the embedding to be evaluated.
embedding_key: the key to the embedding. Must be in adata.obsm.
n_neighbors: #neighbors used when computing neithborhood graph and
calculating entropy of batch mixing / kBET.
resolutions: a sequence of resolutions used for clustering.
clustering_method: clustering method used. Should be one of 'leiden' or
'louvain'.
cell_type_col: a key in adata.obs to the cell type column.
batch_col: a key in adata.obs to the batch column.
return_fig: whether to return the Figure object. Useful for visualizing
the plot.
color_by: a list of adata.obs column keys to color the embeddings by.
If None, will look up adata.uns['color_by']. Only used if is
drawing.
plot_fname: file name of the generated plot. Only used if is drawing.
plot_ftype: file type of the generated plot. Only used if is drawing.
plot_dir: directory to save the generated plot. If None, do not save
the plot.
plot_dpi: dpi to save the plot.
writer: an initialized SummaryWriter to save the UMAP plot to. Only
used if is drawing.
min_dist: the min_dist argument in sc.tl.umap. Only used is drawing.
spread: the spread argument in sc.tl.umap. Only used if is drawing.
n_jobs: # jobs to generate. If <= 0, this is set to the number of
physical cores.
random_state: random state for knn calculation.
umap_kwargs: other kwargs to pass to sc.pl.umap.
Returns:
A dict storing the ari, nmi, asw, ebm and k_bet of the cell embeddings
with key "ari", "nmi", "asw", "ebm", "k_bet", respectively. If draw is
True and return_fig is True, will also store the plotted figure with
key "fig".
r, T)r- r r) r. r )r r r r N)
rF rG rH rI rJ rK rL rM rN rO )r r r r r r r r r r! r" r# r$ r% r& r' r( r) r* ra rb rc rf rf rg Ú evaluate_ari¸ s A ri )r( r+ c C s | dkr dS | dkr t | S tS )zAIf n_jobs <= 0, set it as the number of physical cores _cpu_countNr r )Úintr )r( rf rf rg Ú _eff_n_jobs s rk c C s¢ |j d }| jd }t |df¡}xzt|D ]n}t || |d d
f ¡jddj}|| } t || d | ¡}
dt
|
|¡ } |
||df< | ||df< q,W |S )Nr r r6 F)Úsort) ÚsizeÚshaperR ÚzerosÚrangerF ÚSeriesÚ value_countsr5 Úsumr Úcdf) Ú knn_indicesÚ attr_valuesÚ
ideal_distr ÚdofÚnsÚresultsÚiZobserved_countsZexpected_countsÚstatZp_valuerf rf rg Ú _calculate_kbet_for_one_chunk s
$ r} é )r r- r r) r. r+ c C sÚ |r|dks$|| j ks$t| dt | ¡}|j|d||dd |j| jd< |j| jd< |j| j d< ddd||dd d
d | j d < nJd | j kstd
| j d d d |ksÐtd| j d d d d| | j d S )Nr/ z! not in adata.obsm and is not "X"T)r Zknnr- r) Zwrite_knn_indicesÚ distancesÚconnectivitiesru Z euclideanr )r r- ÚmetricÚmethod)Zconnectivities_keyZ
distances_keyZknn_indices_keyÚparamsÚ neighborszNo precomputed knn exists.r r z pre-computed n_neighbors is z, which is smaller than )
rQ ÚAssertionErrorÚscZ NeighborsZcompute_neighborsr Zobspr ru r] )r r- r r) r. r rf rf rg rM $ s&
,rM )r r- c C s$ t jj| |d tjj| dd}|S )N)r- r7 )Z label_key)r Úppr rS rT Zgraph_connectivity)r r- Úscorerf rf rg r[ D s r[ ç©?) r r- r r Úalphar) r( r. r+ c
s¢ t d¡ || jkst| j| jjdkrRt d| d¡ | j| d¡| j|< | j| jdddj | j
d }j } | j| j ¡ t
| _t| |||tt||}tj|d td
|| }
|| } x4t
|D ](} | |
| | k röd nd | d < qÜW dd lm}
mm} |d |d
2 t |
fddt
|D ¡}W dQ R X |jdd}|d }|d }|dd
d f |k ¡ | }|||fS )a² Calculates the kBET metric of the data.
kBET measures if cells from different batches mix well in their local
neighborhood.
Args:
adata: annotated data matrix.
use_rep: the embedding to be used. Must exist in adata.obsm.
batch_col: a key in adata.obs to the batch column.
n_neighbors: # nearest neighbors.
alpha: acceptance rate threshold. A cell is accepted if its kBET
p-value is greater than or equal to alpha.
random_state: random seed. Used only if method is "hnsw".
n_jobs: # jobs to generate. If <= 0, this is set to the number of
physical cores.
calc_knn: whether to re-calculate the kNN graph or reuse the one stored
in adata.
Returns:
stat_mean: mean kBET chi-square statistic over all cells.
pvalue_mean: mean kBET p-value over all cells.
accept_rate: kBET Acceptance rate of the sample.
zCalculating kbet...r, zMaking the column z of adata.obs categorical.TF)Ú normalizerl r r )Údtype)ÚParallelÚdelayedÚparallel_backendÚloky)r( c 3 s< | ]4}t | |d
dd
f V qdS )r N)r} )Ú.0r{ )rv r rw ru r Ústartsrf rg ú <genexpr> s z!calculate_kbet.<locals>.<genexpr>N)Úaxis) Ú_loggerÚinforJ r
r ÚnameÚwarningrK rr r5 rn rm Úcopyrp Ú
categoriesrM Úminrk rR ro rj Újoblibr r r Ú concatenater2 rs )r r- r r r r) r( r. ÚnsampleZnbatchZquotientÚ remainderr{ r r ZkBET_arrÚresZ stat_meanZ pvalue_meanZ accept_raterf )rv r rw ru r r rg rZ L s8 "
( rZ c C s6 t j| dd\}}|| ¡ }| t |d ¡ ¡ S )NT)Ú
return_countsg ÂëþKH´9)rR Úuniquers Úlog)Z hist_dataÚ_ÚcountsZfreqsrf rf rg Ú_entropy s r¦ c s6 t jjt |¡|dt fddt|D ¡S )N)rm c s g | ]}t | qS rf )r¦ )r r{ )ÚbatchesÚindicesru rf rg ú
<listcomp>¤ s z6_entropy_batch_mixing_for_one_pool.<locals>.<listcomp>)rR ÚrandomÚchoiceÚaranger2 rp )r§ ru r Ún_samples_per_poolrf )r§ r¨ ru rg Ú"_entropy_batch_mixing_for_one_pool s
r® é2 éd )
r r- r r Ún_poolsr r) r( r. r+ c
s~ t d¡ jt ||||ddlm} mm}
|
d|dd2 t | fddt
|D ¡} W d Q R X | S )
aq Calculates the entropy of batch mixing of the data.
kBET measures if cells from different batches mix well in their local
neighborhood.
Args:
adata: annotated data matrix.
use_rep: the embedding to be used. Must exist in adata.obsm.
batch_col: a key in adata.obs to the batch column.
n_neighbors: # nearest neighbors.
n_pools: #pools of cells to calculate entropy of batch mixing.
n_samples_per_pool: #cells per pool to calculate within-pool entropy.
random_state: random seed. Used only if method is "hnsw".
n_jobs: # jobs to generate. If <= 0, this is set to the number of
physical cores.
calc_knn: whether to re-calculate the kNN graph or reuse the one stored
in adata.
Returns:
score: the mean entropy of batch mixing, averaged from n_pools samples.
z#Calculating batch mixing entropy...r )r r r r r )r( Zinner_max_num_threadsc 3 s& | ] }t j V qd S )N)r® rJ )r r¤ )r r r ru r r rf rg r Ô s z1calculate_entropy_batch_mixing.<locals>.<genexpr>N) r r Ún_obsrM r r r r rR r2 rp ) r r- r r r± r r) r( r. r r r rf )r r r ru r r rg rY ª s !
rY )r r r r r r+ c C s< t |dkstd|dkr&tjj}n|dkr8tjj}ntd|| jksXt| ddt t }}}xº|D ]²} | d| }
|| | |
d t | j| | j|
} t
| j| | j|
} | j|
¡ }
| |krÜ| }| }| |krè| }|| jkr | j| ¡ d
kr t | j| | j|
}qrd}qrW | d| ||fS ) a¼ Clusters the data and calculate agreement with cell type and batch
variable.
This method cluster the neighborhood graph (requires having run sc.pp.
neighbors first) with "clustering_method" algorithm multiple times with the
given resolutions, and return the best result in terms of ARI with cell
type.
Other metrics such as NMI with cell type, ARi with batch are logged but not
returned. (TODO: also return these metrics)
Args:
adata: the dataset to be clustered. adata.obsp shouhld contain the keys
'connectivities' and 'distances'.
resolutions: a list of leiden/louvain resolution parameters. Will
cluster with each resolution in the list and return the best result
(in terms of ARI with cell type).
clustering_method: Either "leiden" or "louvain".
cell_type_col: a key in adata.obs to the cell type column.
batch_col: a key in adata.obs to the batch column.
Returns:
best_cluster_key: a key in adata.obs to the best (in terms of ARI with
cell type) cluster assignment column.
best_ari: the best ARI with cell type.
best_nmi: the best NMI with cell type.
r z%Must specify at least one resolution.r ÚlouvainzDPlease specify louvain or leiden for the clustering method argument.z not in adata.obsNr¤ )Ú
resolutionZ key_addedr ) rN r
r Útlr r³ Ú
ValueErrorrJ r r r
rP )r r r r r Zclustering_funcZbest_resrb rc r re r= r> Zn_uniqueZ ari_batchÚarf rf rg rO Ý s. "
rO r9 zumap.pdf)
r r r& r' r: r; r! r< r* r+ c
C s t jj| ||d t jj| f|ddd|} |dk rltj |¡sRtd| d| jtj ||¡|dd |rt| S |
¡ t | ¡ dS )
aÁ Embeds, plots and optionally saves the neighborhood graph with UMAP.
Requires having run sc.pp.neighbors first.
Args:
adata: the dataset to draw. adata.obsp shouhld contain the keys
'connectivities' and 'distances'.
color_by: a str or a list of adata.obs keys to color the points in the
scatterplot by. E.g. if both cell_type_col and batch_col is in
color_by, then we would have two plots colored by cell type and
batch variables, respectively.
min_dist: The effective minimum distance between embedded points.
Smaller values will result in a more clustered/clumped embedding
where nearby points on the manifold are drawn closer together,
while larger values will result on a more even dispersal of points.
spread: The effective scale of embedded points. In combination with
`min_dist` this determines how clustered/clumped the embedded
points are.
ckpt_dir: where to save the plot. If None, do not save the plot.
fname: file name of the saved plot. Only used if ckpt_dir is not None.
return_fig: whether to return the Figure object. Useful for visualizing
the plot.
dpi: the dpi of the saved plot. Only used if ckpt_dir is not None.
umap_kwargs: other kwargs to pass to sc.pl.umap.
Returns:
If return_fig is True, return the figure containing the plot.
)r& r' FT)ÚcolorÚshowr! Nz ckpt_dir z does not exist.Ztight)r< Z bbox_inches)
r rµ r ÚplrV rW Úexistsr
ZsavefigrX ZclfÚpltÚclose)
r r r& r' r: r; r! r< r* rE rf rf rg r_ s (
r_ Úaggéx é
©rÀ rÀ )Úmatplotlib_backendr< ÚframeonÚvector_friendlyÚfontsizeÚfigsizec C s" t | ¡ tj|||||d dS )u¤ Set figure parameters.
Args
backend: the backend to switch to. This can either be one of th
standard backend names, which are case-insensitive:
- interactive backends:
GTK3Agg, GTK3Cairo, MacOSX, nbAgg,
Qt4Agg, Qt4Cairo, Qt5Agg, Qt5Cairo,
TkAgg, TkCairo, WebAgg, WX, WXAgg, WXCairo
- non-interactive backends:
agg, cairo, pdf, pgf, ps, svg, template
or a string of the form: ``module://my.module.name``.
dpi: resolution of rendered figures â this influences the size of
figures in notebooks.
frameon: add frames and axes labels to scatter plots.
vector_friendly: plot scatter plots using `png` backend even when
exporting as `pdf` or `svg`.
fontsize: the fontsize for several `rcParams` entries.
figsize: plt.rcParams['figure.figsize'].
)r< rÆ rÅ rà rÄ N)Ú
matplotlibZuser Úset_figure_params)r r< rà rÄ rÅ rÆ rf rf rg rÈ U s
rÈ )r r~ r T)r )r r r~ r r NT)r r r¯ r¯ r° r NT)r r r )r¾ r¿ TTrÀ rÁ )CÚmathr rV ÚloggingÚnumpyrR Zscanpyr ZanndataÚadÚpandasrF rÇ Zmatplotlib.figurer Úmatplotlib.pyplotÚpyplotr¼ Zscipy.sparse.csrr Z scipy.statsr Útypingr r r r r
Z scipy.sparser Zsklearn.metricsr r
r Zsklearn.neighborsr ZpsutilrS Ú cpu_countr rj Ú__annotations__Ú getLoggerÚ__name__r r` ZAnnDatarL ÚfloatÚboolrª Ú RandomStateÚ Generatorrh ri rk r} ÚndarrayrM r[ rZ r¦ r® rY rO r_ rÈ rf rf rf rg Ú<module> sÒ
Z Z @ D
, =(.
Datasets
Models
