Datasets
Models
Downloads: 1
371 lines (339 with data), 19.5 kB
3
ö└A]5M Ń @ sý d Z ddljZddljZddlZddl Z
ddl Z ddl
jZddlmZ ddlZddljZddlZddlZddlZddlmZ ejâ G ddä dâZ ddä Z d*d
d äZ d d
ä Z ddä Z ddä Z!d+ddäZ"G ddä dâZ#d,d d äZ$d-d(d)äZ%dS ).zs
visualize.py
=======================
Plots SHAP outputs, UMAP embeddings, and overlays predictions on top of WSI.
Ú N)┌Image)┌joinc @ s@ e Zd ZdZddä Zdddddd gd
d g fd d
äZdddäZdS )┌
PlotlyPlotzCreates plotly html plots.c C s
g | _ d S )N)┌plots)┌selfę r ˙E/Users/joshualevy/Documents/GitHub/PathFlowAI/pathflowai/visualize.py┌__init__ s zPlotlyPlot.__init__N┌color┌name┌x┌y┌zÚ g ?c sz g } t |â\}
} } || jtjkr~| jtj||
|| || ddt|| ||dtddŹdŹ|t|âkrn|| n|| dŹâ n˘|| j â }
t
j dt |
ââë tj
d d
ä ł D ââë |rÂ|ë ç fd d ätt|
ââD â}xá|jâ D ]ö\}}| jtj||
|| |k || || |k || || |k t|âdt|||d
Źdt|âkÉrT|j|| |k n|| || |k dŹâ q┌W |dk Érjg g g }}}x┤|jâ D ]Ę}|d |jjkÉrľ|d |jjkÉrľ||j|d |
f |j|d |
f dg7 }||j|d | f |j|d | f dg7 }||j|d | f |j|d | f dg7 }ÉqľW | jtj|||dtjjdddŹddŹâ | jj| â dS )aŤ Adds plotting data to be plotted.
Parameters
----------
t_data_df:dataframe
3-D transformed dataframe.
G:nx.Graph
Networkx graph.
color_col:str
Column to use to color points.
name_col:str
Column to use to name points.
xyz_cols:list
3 columns that denote x,y,z coords.
size:int
Marker size.
opacity:float
Marker opacity.
custom_colors:list
Custom colors to supply.
┌ ┌markers┌ViridisZColorbar)┌title)r
┌size┌opacity┌
colorscale┌colorbar)r r
r r ┌mode┌marker┌text┌hlsc S s6 g | ].}d j djtj|âd jtâjtâjâ ââĹqS )zrgb({})˙,Ú )┌formatr ┌np┌array┌astype┌int┌str┌tolist)┌.0Zc_ir r r ˙
<listcomp>; s z'PlotlyPlot.add_plot.<locals>.<listcomp>c s i | ]\}}ł | |ôqS r r )r% ┌ir )┌cr r ˙
<dictcomp>> s z'PlotlyPlot.add_plot.<locals>.<dictcomp>)r
r r r Nr Ú ┌lineszrgb(210,210,210)r )r
┌width┌none)r r
r r ┌line┌ hoverinfo)┌tuple┌dtyper ┌float64┌append┌go┌ Scatter3d┌dict┌list┌unique┌sns┌
color_palette┌lenr ┌ enumerate┌sorted┌itemsr# ┌index┌edges┌values┌loc┌ scatter3d┌Liner ┌extend)r Z t_data_df┌GZ color_colZname_colZxyz_colsr r Z
custom_colorsr r r
r ┌colorsZ
color_dictr ┌colZXedZYedZZed┌edger )r( r ┌add_plot sF 0 (J
$**0
zPlotlyPlot.add_plotFc C sÇ |r^t j| jt jttddddddddŹtddddddddŹtddddddddŹdŹdŹdŹ}nt j| jdŹ}tj||dd Ź d
S ) zÄPlot embedding of patches to html file.
Parameters
----------
output_fname:str
Output html file.
axes_off:bool
Remove axes.
r TF)r ┌ autorange┌showgrid┌zeroline┌showline┌ticks┌showticklabels)┌xaxis┌yaxis┌zaxis)┌scene)┌data┌layout)rU )┌filename┌ auto_openN)r4 ┌Figurer ┌Layoutr6 ┌py┌plot)r Z output_fname┌axes_off┌figr r r r\ W s "$zPlotlyPlot.plot)F)┌__name__┌
__module__┌ __qualname__┌__doc__r rJ r\ r r r r r s >r c C s t j| jdâdâS )zpNumpy array to pil.
Parameters
----------
arr:array
Numpy array.
Returns
-------
Image
PIL Image.
┌uint8┌RGB)r ┌ fromarrayr! )┌arrr r r ┌to_pilj s rg š Ó?c C s || d| | S )zŢBlend 2 arrays together, mixing with alpha.
Parameters
----------
arr1:array
Image 1.
arr2:array
Image 2.
alpha:float
Higher alpha makes image more like image 1.
Returns
-------
array
Resulting image.
g ?r )┌arr1┌arr2┌alphar r r ┌blendz s rl c C s6 || â}x(t dâD ] }t|| d â|d|f< qW |S )a Convert probability score to rgb image.
Parameters
----------
prob:float
Between 0 and 1 score.
palette:palette
Pallet converts between prob and color.
arr:array
Original array.
Returns
-------
array
New image colored by prediction score.
Ú r .)┌ranger" )┌prob┌paletterf rH r' r r r ┌prob2rbgÄ s rq c C s& || | âdddůf d j tâ}|S )a Color each pixel by segmentation class.
Parameters
----------
seg:array
Segmentation mask.
palette:palette
Color to RGB map.
n_segmentation_classes:int
Total number segmentation classes.
Returns
-------
array
Returned segmentation image.
.Nrm r )r! r" )┌segrp ┌n_segmentation_classes┌imgr r r ┌seg2rgbą s "ru c C s6 || }x(t dâD ] } t|| d â|d| f< qW |S )z╬Go from annotation of patch to color.
Parameters
----------
i:int
Annotation index.
palette:palette
Index to color mapping.
arr:array
Image array.
Returns
-------
array
Resulting image.
rm r .)rn r" )r' rp rf rH r r r ┌annotation2rgb╝ s rv š @˙test.pngc C sć ddl m}m} ddl}| jdâs4|| dddddŹn|| â}t|j|jâ tt j
|j dd ů â| j t
âjâ â|jd
Źâ}|j|â dS ) zăPlots entire SVS/other image.
Parameters
----------
image_file:str
Image file.
compression_factor:float
Amount to shrink each dimension of image.
test_image_name:str
Output image file.
r )┌svs2dask_array┌npy2daNz.npyiŔ TF)┌ tile_size┌overlap┌ remove_last┌allow_unknown_chunksizesr )┌dsize┌
interpolation)┌pathflowai.utilsry rz ┌cv2┌endswithrg ┌resize┌computer0 r r ┌shaper! r" r$ ┌ INTER_CUBIC┌save)┌
image_file┌compression_factor┌test_image_namery rz ré rf rj r r r ┌ plot_image_Ë s
$<rî c
@ s< e Zd ZdZdd
d äZd d
ä Zddä Zddä ZdddäZdS )┌PredictionPlottera Plots predictions over entire image.
Parameters
----------
dask_arr_dict:dict
Stores all dask arrays corresponding to all of the images.
patch_info_db:str
Patch level information, eg. prediction.
compression_factor:float
How much to compress image by.
alpha:float
Low value assigns higher weight to prediction over original image.
patch_size:int
Patch size.
no_db:bool
Don't use patch information.
plot_annotation:bool
Plot annotations from patch information.
segmentation:bool
Plot segmentation mask.
n_segmentation_classes:int
Number segmentation classes.
input_dir:str
Input directory.
annotation_col:str
Annotation column to plot.
scaling_factor:float
Multiplies the prediction scores to make them appear darker on the images when predicting.
rm š Ó?ÚÓ FÚ r ┌
annotationš ?c
s2 || _ | | _d | _t| â| _tjdddŹ| _|s˙|| _|| _ || _
t j |â}
t
jdj|â|
dŹ}|
jâ ddä t|d jâ jâ âD â| _|| _tjtt| jjâ ââd Ź| _d
|jkr╝d |d
< |d d
ddd| g | _ | j t j | j d t j t|jâ âââ | _ | j Érç fddä|jâ D â| _ddä |j â D â| _!d S )Nr T)┌startZas_cmapzselect * from "{}";)┌conc S s i | ]\}}|t |âôqS r )r# )r% r' ┌ar r r r) s z.PredictionPlotter.__init__.<locals>.<dictcomp>rĹ )┌n_colors┌y_predg ┌IDr r
┌
patch_sizec s. i | ]&}t jtjtł d j|ââddŹâ|ôqS )z {}_mask.npyzr+)┌ mmap_mode)┌da┌
from_arrayr ┌loadr r )r% ┌slide)┌ input_dirr r r) s c S s" i | ]\}}|d ddůf |ôqS ).Nrm r )r% ┌k┌vr r r r) " s )"┌ segmentation┌scaling_factor┌segmentation_maps┌floatrs r9 Zcubehelix_palette┌ pred_paletterŐ rk rÖ ┌sqlite3┌connect┌pd┌read_sqlr ┌closer< r8 r$ ┌ annotations┌plot_annotationr: r; r7 ┌keysrp ┌columns┌
patch_infor ┌isinr r> ┌
dask_arr_dict)r r▓ Z
patch_info_dbrŐ rk rÖ ┌no_dbrş ró rs rč ┌annotation_colrú ┌connr░ r )rč r r s,
(zPredictionPlotter.__init__c C s t jtj|ddŹâ| j|< dS )zőReplace segmentation mask with new custom segmentation.
Parameters
----------
basename:str
Patient ID
npy:str
Numpy mask.
zr+)rÜ N)rŤ rť r rŁ rĄ )r ┌basename┌npyr r r ┌add_custom_segmentation$ s
z)PredictionPlotter.add_custom_segmentationc C s▓ | j | j d |k }| j| }tj|jâjtâ}|ddů | j < |jtâj â }t
j d|ddů dâ}Éx@t |jd âD É],}|j
| j â \}}}} }
} t|| j ât|| j â } }
tj| | dfâ}| jÉrt| j| ||| ů||| ůf jâ | j| jâ}n4| jÉs"t| | j | j|ânt| jt| â | j|â}|||| ů||| ůf jâ }t ||| j âj d
â}t |â}| | j } t| â} |j | | fâ}|j!|| |
fdd Ź q|W |S ) z╩Generate the image array for the whole slide image with predictions overlaid.
Parameters
----------
ID:str
patient ID.
Returns
-------
array
Resulting overlaid whole slide image.
rś Nr rd ┌whiter rm r* )┌box┌mask)r* r r )"r░ r▓ r r rć r! rą rŐ r" r$ r ┌newrn ┌iloc┌zerosró ru rĄ rů rŽ rs rş rq rú rv rČ r# rp rl rk ┌ transposerg rä ┌paste)r rś r░ Zdask_arr┌ arr_shapert r' r r
rÖ rĹ ┌pred┌x_new┌y_new┌imagerf Z
blended_patchZblended_patch_pilr r r ┌generate_image0 s*
44
z PredictionPlotter.generate_imagec C sf | j s,| j| ||| ů||| ůf jâ n0t| j| ||| ů||| ůf jâ | j| jâ}t|âS )zŮReturn one single patch instead of entire image.
Parameters
----------
ID:str
Patient ID
x:int
X coordinate.
y:int
Y coordinate.
patch_size:int
Patch size.
Returns
-------
array
Image.
)ró r▓ rů ru rĄ rŽ rs rg )r rś r r
rÖ rt r r r ┌ return_patch] s ^z PredictionPlotter.return_patchc C s4 |r&ddl m} ||tj|âddŹ n
|j|â dS )zŤOutput calculated image to file.
Parameters
----------
img:array
Image.
filename:str
Output file name.
tif:bool
Store in TIF format?
r )┌imwrite┌rgb)Z photometricN)Ztifffiler╚ r r rł )r rt rW Ztifr╚ r r r ┌ output_images s z PredictionPlotter.output_imageN)
rm rÄ rĆ FFFrÉ r rĹ rĺ )F) r_ r` ra rb r rŞ rĂ ră r╩ r r r r rŹ š s
-rŹ r* ┌deepš Ú Fc
$ s\ ddl }
ddlm} ddlëddlm} ddl}
ddlm} ddl }ddl m
} ddl m } t
| j| jdd ä d
Ź}|jd â}|jd âë|f |Ä}|d
r░|jł|dŹ}t|jâë| ||dłdkr─dndłdkrÍ||ânddŹë ttł ââ\}}|dkÉr|
j|ttł ââd gdâ}ttł ââ\}}|
jjâ Ér@|jâ }|jâ }| dkÉrv|
jjâ Ér^| jâ }|| ||ââjâ jâ }|jâ }|dkÉrĄ|
j| |â}|j |łdŹ} n,|dkÉrđ|
j | |||dŹ}|j |ł|dŹ} |j d dkÉrý|j ddŹ}łdkÉr | \} } n
| | } } łdkÉr0łj łj | dd%âddâg} nçfd d ä| D â} |jâ jâ jâ } | j!d&â} xTt"| j d âD ]B}!| |!d f łj#|d â9 < | |!d f łj#|d! â7 < ÉqlW | j!d'â} łj łj | dd(âddâ}"| dkÉrŕ|j$t%â}#n łj#ç ççfd"d ä| D ââ}#dÉr4t|#j âdk És,|#j d dkÉr4|#j&â }#|j'â |
j(| |"|#â |j)|d#d$Ź dS ))aD Plot shapley attributions overlaid on images for classification tasks.
Parameters
----------
model:nn.Module
Pytorch model.
dataset_opts:dict
Options used to configure dataset
transform_opts:dict
Options used to configure transformers.
batch_size:int
Batch size for training.
outputfilename:str
Output filename.
n_outputs:int
Number of top outputs.
method:str
Gradient or deep explainer.
local_smoothing:float
How much to smooth shapley map.
n_samples:int
Number shapley samples to draw.
pred_out:bool
Label images with binary prediction score?
r N)┌
functional)┌
DataLoader)┌DynamicImageDataset)┌pyplot)┌ImbalancedDatasetSamplerc S s | S )Nr )r r r r ┌<lambda>¬ s zplot_shap.<locals>.<lambda>)┌sigmoid┌softmaxr- ┌binary_threshold┌ num_targets┌classify_annotations)rÎ rÍ Ú
r* TF)┌
batch_size┌ num_workers┌shuffle┌sampler┌gradientr- r╦ )┌ranked_outputs)r┌ ┌local_smoothing)r▀ ┌nsamples)┌axisr c s$ g | ] }ł j ł j |d dâd dâĹqS )r* r Ú )┌swapaxes)r% ┌s)r r r r& █ s z plot_shap.<locals>.<listcomp>rm .┌std┌meanc s& g | ] ë çç çfd dät łâD âĹqS )c s, g | ]$}łd kr ł j jł| ntłâĹqS )r* )┌dataset┌targetsr# )r% ┌j)┌dataloader_valr' rÎ r r r& š s z(plot_shap.<locals>.<listcomp>.<listcomp>)rn )r% )rŰ ┌ n_outputsrÎ )r' r r& š s i, )┌dpirŃ )r r rm r* )r rm r* r rŃ )*┌torch┌torch.nnr╬ ┌numpy┌torch.utils.datar¤ ┌shap┌pathflowai.datasetsrđ ┌
matplotlibrĐ Zpathflowai.samplerrĎ r6 rď rŇ ┌popZbinarize_annotationsr; rÚ ┌next┌iter┌cat┌cuda┌ is_available┌detach┌cpuZ
DeepExplainer┌ shap_valuesZGradientExplainerrć ┌argmaxrń r┐ rn r r! r# ┌flatten┌figureZ
image_plot┌savefig)$┌model┌ dataset_opts┌transform_optsr┌ ┌outputfilenamerý ┌methodrÓ ┌ n_samples┌pred_outrţ ┌Fr¤ r˛ rđ r˘ ┌pltrĎ Z
out_transformrÍ rŔ Z binarizer┌
backgroundZ y_background┌X_test┌y_testZmodel2┌erň rř ┌idxZ
shap_numpyZ X_test_numpyr' Z
test_numpy┌labelsr )rŰ rý r rÎ r ┌ plot_shapů st
.
$
&r š ?š └r@˙output_embedding.pngTr š{«Gßzä?šÜÖÖÖÖÖę?r┘ ┌ascc
; sţ ddl }
ddl}ddlm} ddlm} ddlm} ddl}ddl ëddl
}ddl m ë ddl
}|jdâ ddl
m} tjd d
Ź çfd d äë|
j|â}|d
}|d }| rŠłj|| jâ}| dkrĎ|ddd$ů }|j| }|j| }|Ér|d |kj}|j| }|j| }|Ér:|| d| kj}|j| }|j| }|d|
dŹ}|j|j|jddůdd%ůf jâddg|jdŹ} g } xpt|jd âD ]^} |j| d }|j| dddg jj â \} } }!| | | | |! ů| | |! ůf }"| j |"â ÉqŐW |â }#|j | â} |#j â |Érrddl m!ëm"ë ç ççfddä}$|j#â \}%}&|$| d j| d j| d |&| d Ź tj$â |j%|d d Ź Énx| jddůdf }'| jddůdf }(çfd d ä| D â} t&d!d ä | D ââ})t&d"d ä | D ââ}*|'j'â |'j&â }+},|(j'â |(j&â }-}.|,|+ }/|.|- }0|/|0kÉr |/t(|0â t) }1t)}2nt)}1|0t(|/â t) }2łj*|1|) |2|* d#fâ| }3łj+|+|,|1â}4łj+|-|.|2â}5xnt,|'|(| âD ]^\} } }6|6jddů \}7}8łj-| |4 d â}9łj-| |5 d â}:|6|3|9|9|7 ů|:|:|8 ůf< ÉqzW |j.j/||3â dS )&a╠ Make UMAP embedding plot, overlaid with images.
Parameters
----------
dask_arr_dict:dict
Stored dask arrays for each WSI.
embeddings_file:str
Embeddings pickle file stored from running using after trainign the model.
ID:str
Patient ID.
cval:float
Deprecated
image_res:float
Image resolution.
outputfname:str
Output image file.
mpl_scatter:bool
Recommended: Use matplotlib for scatter plot.
remove_background_annotation:str
Remove the background annotations. Enter for annotation to remove.
max_background_area:float
Maximum backgrund area in each tile for inclusion.
zoom:float
How much to zoom in on each patch, less than 1 is zoom out.
n_neighbors:int
Number of neighbors for UMAP embedding.
sort_col:str
Patch info column to sort on.
sort_mode:str
Sort ascending or descending.
Returns
-------
type
Description of returned object.
Inspired by: https://gist.github.com/lukemetz/be6123c7ee3b366e333a
WIP!! Needs testing.r N)┌Client)┌UMAP)r )rä ┌Agg)rĐ r╣ )┌stylec sz t t| jddů â\}}t||gâ|krv||krTł | tt|| | âât|âfâ} n"ł | t|âtt|| | ââfâ} | S )zM
Resize an image so that it is size along the minimum spatial dimension.
Nr )┌maprą rć ┌minr" ┌round)rt r ┌w┌h)rä r r ┌
min_resize$ s $"z$plot_umap_images.<locals>.min_resize┌
embeddingsr░ ┌descr* rś g ?r )┌ n_components┌ n_neighborsr r
)r» r? rÖ )┌ OffsetImage┌AnnotationBboxc sä g }x^t t| ââD ]N}| | || }}|| } ł| |dŹ}
ł |
||fdddŹ} |j|j| ââ qW |jłj| |gââ |jâ d S )N)┌zoomrU F)ZxycoordsZframeon)rn r; r3 Z
add_artistZupdate_datalim┌ column_stack┌ autoscale) r r
┌ax┌ imageDatar( ┌imagesr' ┌x0┌y0rt r┼ ┌ab)r' r& r r r ┌ imscatterU s z#plot_umap_images.<locals>.imscatter)r, r+ r( i, )rÝ c s g | ]}ł |t âĹqS r )Zimg_res)r% r┼ )r! r r r& l s z$plot_umap_images.<locals>.<listcomp>c S s g | ]}|j d ĹqS )r )rć )r% r┼ r r r r& m s c S s g | ]}|j d ĹqS )r* )rć )r% r┼ r r r r& n s rm rŃ rŃ )0rţ ┌dask┌dask.distributedr ┌umapr ┌pathflowai.visualizer ┌pandasr Z
skimage.ioZskimage.transformrä r˘ ┌userĐ r9 ┌setrŁ ┌argsortrA rŻ rB ┌ DataFrame┌
fit_transformr? rn rć r$ r3 rů rź Zmatplotlib.offsetboxr& r' ┌subplotsZdespiner ┌maxr rą ┌res┌ones┌linspace┌zip┌argmin┌ioZimsave);r▓ ┌embeddings_filerś ┌cval┌ image_res┌ outputfname┌ mpl_scatter┌ remove_background_annotation┌max_background_arear( r% ┌sort_col┌ sort_moderţ r2 r r r rę Zskimager˘ r
┌embeddings_dictr" r░ r ┌ removal_boolr4 ┌t_datar- r' r r
rÖ rf r( r1 r^ r+ ┌xx┌yy┌ max_width┌
max_height┌x_min┌x_max┌y_min┌y_max┌sx┌syZres_xZres_y┌canvasZx_coordsZy_coordsr┼ r r ┌x_idxZy_idxr )r' r& r! r rä r ┌plot_umap_images sÄ '
0
"
"r\ )rh )rw rx )r* r╦ r╠ r═ F) Nr r r Tr r r r┘ r r )&rb ┌plotly.graph_objs┌
graph_objsr4 Zplotly.offline┌offliner[ r6 rę r r Znetworkx┌nx┌
dask.arrayr rŤ ┌PILr r˘ ┌matplotlib.pyplotrĐ r
┌seabornr9 rž ┌os.pathr r8 r rg rl rq ru rv rî rŹ r r\ r r r r ┌<module> s.
V
k