
# Hierarchical MRI tumor segmentation with densely connected 3D CNN

By Lele Chen, Yue Wu, [Adora M. DSouza](https://www.rochester.edu/college/gradstudies/profiles/adora-dsouza.html),Anas Z. Abidin, [Axel W. E. Wismuelle](https://www.urmc.rochester.edu/people/27063859-axel-w-e-wismueller), [Chenliang Xu](https://www.cs.rochester.edu/~cxu22/).

University of Rochester.

### Table of Contents
0. [Introduction](#introduction)
0. [Citation](#citation)
0. [Running](#running)
0. [Model](#model)
0. [Disclaimer and known issues](#disclaimer-and-known-issues)
0. [Results](#results)

### Introduction

This repository contains the original models (dense24, dense48, no-dense) described in the paper "Hierarchical MRI tumor segmentation with densely connected 3D CNN" (https://arxiv.org/abs/1802.02427). This code can be applied directly in [BTRAS2017](http://braintumorsegmentation.org/). 

![model](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/spie.gif?raw=true)


### Citation

If you use these models or the ideas in your research, please cite:
    
    @inproceedings{DBLP:conf/miip/ChenWDAWX18,
      author    = {Lele Chen and
               Yue Wu and
               Adora M. DSouza and
               Anas Z. Abidin and
               Axel Wism{\"{u}}ller and
               Chenliang Xu},
      title     = {{MRI} tumor segmentation with densely connected 3D {CNN}},
      booktitle = {Medical Imaging 2018: Image Processing, Houston, Texas, United States,
               10-15 February 2018},
      pages     = {105741F},
      year      = {2018},
      crossref  = {DBLP:conf/miip/2018},
      url       = {https://doi.org/10.1117/12.2293394},
      doi       = {10.1117/12.2293394},
      timestamp = {Tue, 06 Mar 2018 10:50:01 +0100},
      biburl    = {https://dblp.org/rec/bib/conf/miip/ChenWDAWX18},
      bibsource = {dblp computer science bibliography, https://dblp.org}
    }
### Running


0. Pre-installation:[Tensorflow](https://www.tensorflow.org/install/),[Ants](https://github.com/ANTsX/ANTs),[nibabel](http://nipy.org/nibabel/),[sklearn](http://scikit-learn.org/stable/),[numpy](http://www.numpy.org/)

0. Download and unzip the training data from [BTRAS2017](http://braintumorsegmentation.org/)

0. Use N4ITK to correct the data: `python n4correction.py /mnt/disk1/dat/lchen63/spie/Brats17TrainingData/HGG`
0. Train the model:  `python train.py`
    - `-gpu`: gpu id
    - `-bs`: batch size 
    - `-mn`: model name, 'dense24' or 'dense48' or 'no-dense' or 'dense24_nocorrection'
    - `-nc`:  [n4ITK bias correction](https://www.ncbi.nlm.nih.gov/pubmed/20378467),True or False
    - `-e`: epoch number 
    - `-r`: data path
    - `-sp`: save path/name
    - ...

For example:
`python train.py -bs 2 -gpu 0  -mn dense24 -nc True -sp dense48_correction -e 5  -r /mnt/disk1/dat/lchen63/spie/Brats17TrainingData/HGG`

0. Test the model: `python test.py`
    - `-gpu`: gpu id
    - `-m`: model path, the saved model name
    - `-mn`: model name, 'dense24' or 'dense48' or 'no-dense' or 'dense24_nocorrection'
    - `-nc`:  [n4ITK bias correction](https://www.ncbi.nlm.nih.gov/pubmed/20378467), True or False
    - `-r`: data path
    - ...

For example:
`python test.py -m Dense24_correction-2 -mn dense24 -gpu 0 -nc True  -r /mnt/disk1/dat/lchen63/spie/Brats17TrainingData/HGG`


### Model

0. Hierarchical segmentation
    ![model](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/2.png?raw=true)

    
0. 3D densely connected CNN

    ![model](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/1.png?raw=true)

### Disclaimer and known issues

0. These codes are implmented in Tensorflow
0. In this paper, we only use the glioblastoma (HGG) dataset.
0. I didn't config nipype.interfaces.ants.segmentation. So if you need to use `n4correction.py` code, you need to copy it to the bin directory where antsRegistration etc are located. Then run `python n4correction.py`
0. If you want to train these models using this version of tensorflow without modifications, please notice that:
    - You need at lest 12 GB GPU memory.
    - There might be some other untested issues.
    

### Results
0. Result visualization :
    ![visualization](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/h.png?raw=true)
    ![visualization](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/v.png?raw=true)

0. Quantitative results:

    model|whole|peritumoral edema (ED)|FGD-enhan. tumor (ET)
    :---:|:---:|:---:|:---:
    Dense24 |0.74| 0.81| 0.80
    Dense48 | 0.61|0.78|0.79
    no-dense|0.61|0.77|0.78
    dense24+n4correction|0.72|0.83|0.81

	a/README.md		b/README.md
1	# Hierarchical MRI tumor segmentation with densely connected 3D CNN	1	# Hierarchical MRI tumor segmentation with densely connected 3D CNN
2		2
3	By Lele Chen, Yue Wu, [Adora M. DSouza](https://www.rochester.edu/college/gradstudies/profiles/adora-dsouza.html),Anas Z. Abidin, [Axel W. E. Wismuelle](https://www.urmc.rochester.edu/people/27063859-axel-w-e-wismueller), [Chenliang Xu](https://www.cs.rochester.edu/~cxu22/).	3	By Lele Chen, Yue Wu, [Adora M. DSouza](https://www.rochester.edu/college/gradstudies/profiles/adora-dsouza.html),Anas Z. Abidin, [Axel W. E. Wismuelle](https://www.urmc.rochester.edu/people/27063859-axel-w-e-wismueller), [Chenliang Xu](https://www.cs.rochester.edu/~cxu22/).
4		4
5	University of Rochester.	5	University of Rochester.
6		6
7	### Table of Contents	7	### Table of Contents
8	0. [Introduction](#introduction)	8	0. [Introduction](#introduction)
9	0. [Citation](#citation)	9	0. [Citation](#citation)
10	0. [Running](#running)	10	0. [Running](#running)
11	0. [Model](#model)	11	0. [Model](#model)
12	0. [Disclaimer and known issues](#disclaimer-and-known-issues)	12	0. [Disclaimer and known issues](#disclaimer-and-known-issues)
13	0. [Results](#results)	13	0. [Results](#results)
14		14
15	### Introduction	15	### Introduction
16		16
17	This repository contains the original models (dense24, dense48, no-dense) described in the paper "Hierarchical MRI tumor segmentation with densely connected 3D CNN" (https://arxiv.org/abs/1802.02427). This code can be applied directly in [BTRAS2017](http://braintumorsegmentation.org/).	17	This repository contains the original models (dense24, dense48, no-dense) described in the paper "Hierarchical MRI tumor segmentation with densely connected 3D CNN" (https://arxiv.org/abs/1802.02427). This code can be applied directly in [BTRAS2017](http://braintumorsegmentation.org/).
18		18
19	![model](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/spie.gif)	19	![model](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/spie.gif?raw=true)
20		20
21		21
22	### Citation	22	### Citation
23		23
24	If you use these models or the ideas in your research, please cite:	24	If you use these models or the ideas in your research, please cite:
25		25
26	@inproceedings{DBLP:conf/miip/ChenWDAWX18,	26	@inproceedings{DBLP:conf/miip/ChenWDAWX18,
27	author = {Lele Chen and	27	author = {Lele Chen and
28	Yue Wu and	28	Yue Wu and
29	Adora M. DSouza and	29	Adora M. DSouza and
30	Anas Z. Abidin and	30	Anas Z. Abidin and
31	Axel Wism{\"{u}}ller and	31	Axel Wism{\"{u}}ller and
32	Chenliang Xu},	32	Chenliang Xu},
33	title = {{MRI} tumor segmentation with densely connected 3D {CNN}},	33	title = {{MRI} tumor segmentation with densely connected 3D {CNN}},
34	booktitle = {Medical Imaging 2018: Image Processing, Houston, Texas, United States,	34	booktitle = {Medical Imaging 2018: Image Processing, Houston, Texas, United States,
35	10-15 February 2018},	35	10-15 February 2018},
36	pages = {105741F},	36	pages = {105741F},
37	year = {2018},	37	year = {2018},
38	crossref = {DBLP:conf/miip/2018},	38	crossref = {DBLP:conf/miip/2018},
39	url = {https://doi.org/10.1117/12.2293394},	39	url = {https://doi.org/10.1117/12.2293394},
40	doi = {10.1117/12.2293394},	40	doi = {10.1117/12.2293394},
41	timestamp = {Tue, 06 Mar 2018 10:50:01 +0100},	41	timestamp = {Tue, 06 Mar 2018 10:50:01 +0100},
42	biburl = {https://dblp.org/rec/bib/conf/miip/ChenWDAWX18},	42	biburl = {https://dblp.org/rec/bib/conf/miip/ChenWDAWX18},
43	bibsource = {dblp computer science bibliography, https://dblp.org}	43	bibsource = {dblp computer science bibliography, https://dblp.org}
44	}	44	}
45	### Running	45	### Running
46		46
47		47
48	0. Pre-installation:[Tensorflow](https://www.tensorflow.org/install/),[Ants](https://github.com/ANTsX/ANTs),[nibabel](http://nipy.org/nibabel/),[sklearn](http://scikit-learn.org/stable/),[numpy](http://www.numpy.org/)	48	0. Pre-installation:[Tensorflow](https://www.tensorflow.org/install/),[Ants](https://github.com/ANTsX/ANTs),[nibabel](http://nipy.org/nibabel/),[sklearn](http://scikit-learn.org/stable/),[numpy](http://www.numpy.org/)
49		49
50	0. Download and unzip the training data from [BTRAS2017](http://braintumorsegmentation.org/)	50	0. Download and unzip the training data from [BTRAS2017](http://braintumorsegmentation.org/)
51		51
52	0. Use N4ITK to correct the data: `python n4correction.py /mnt/disk1/dat/lchen63/spie/Brats17TrainingData/HGG`	52	0. Use N4ITK to correct the data: `python n4correction.py /mnt/disk1/dat/lchen63/spie/Brats17TrainingData/HGG`
53	0. Train the model: `python train.py`	53	0. Train the model: `python train.py`
54	- `-gpu`: gpu id	54	- `-gpu`: gpu id
55	- `-bs`: batch size	55	- `-bs`: batch size
56	- `-mn`: model name, 'dense24' or 'dense48' or 'no-dense' or 'dense24_nocorrection'	56	- `-mn`: model name, 'dense24' or 'dense48' or 'no-dense' or 'dense24_nocorrection'
57	- `-nc`: [n4ITK bias correction](https://www.ncbi.nlm.nih.gov/pubmed/20378467),True or False	57	- `-nc`: [n4ITK bias correction](https://www.ncbi.nlm.nih.gov/pubmed/20378467),True or False
58	- `-e`: epoch number	58	- `-e`: epoch number
59	- `-r`: data path	59	- `-r`: data path
60	- `-sp`: save path/name	60	- `-sp`: save path/name
61	- ...	61	- ...
62		62
63	For example:	63	For example:
64	`python train.py -bs 2 -gpu 0 -mn dense24 -nc True -sp dense48_correction -e 5 -r /mnt/disk1/dat/lchen63/spie/Brats17TrainingData/HGG`	64	`python train.py -bs 2 -gpu 0 -mn dense24 -nc True -sp dense48_correction -e 5 -r /mnt/disk1/dat/lchen63/spie/Brats17TrainingData/HGG`
65		65
66	0. Test the model: `python test.py`	66	0. Test the model: `python test.py`
67	- `-gpu`: gpu id	67	- `-gpu`: gpu id
68	- `-m`: model path, the saved model name	68	- `-m`: model path, the saved model name
69	- `-mn`: model name, 'dense24' or 'dense48' or 'no-dense' or 'dense24_nocorrection'	69	- `-mn`: model name, 'dense24' or 'dense48' or 'no-dense' or 'dense24_nocorrection'
70	- `-nc`: [n4ITK bias correction](https://www.ncbi.nlm.nih.gov/pubmed/20378467), True or False	70	- `-nc`: [n4ITK bias correction](https://www.ncbi.nlm.nih.gov/pubmed/20378467), True or False
71	- `-r`: data path	71	- `-r`: data path
72	- ...	72	- ...
73		73
74	For example:	74	For example:
75	`python test.py -m Dense24_correction-2 -mn dense24 -gpu 0 -nc True -r /mnt/disk1/dat/lchen63/spie/Brats17TrainingData/HGG`	75	`python test.py -m Dense24_correction-2 -mn dense24 -gpu 0 -nc True -r /mnt/disk1/dat/lchen63/spie/Brats17TrainingData/HGG`
76		76
77		77
78	### Model	78	### Model
79		79
80	0. Hierarchical segmentation	80	0. Hierarchical segmentation
81	![model](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/2.png)	81	![model](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/2.png?raw=true)
82		82
83		83
84	0. 3D densely connected CNN	84	0. 3D densely connected CNN
85		85
86	![model](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/1.png)	86	![model](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/1.png?raw=true)
87		87
88	### Disclaimer and known issues	88	### Disclaimer and known issues
89		89
90	0. These codes are implmented in Tensorflow	90	0. These codes are implmented in Tensorflow
91	0. In this paper, we only use the glioblastoma (HGG) dataset.	91	0. In this paper, we only use the glioblastoma (HGG) dataset.
92	0. I didn't config nipype.interfaces.ants.segmentation. So if you need to use `n4correction.py` code, you need to copy it to the bin directory where antsRegistration etc are located. Then run `python n4correction.py`	92	0. I didn't config nipype.interfaces.ants.segmentation. So if you need to use `n4correction.py` code, you need to copy it to the bin directory where antsRegistration etc are located. Then run `python n4correction.py`
93	0. If you want to train these models using this version of tensorflow without modifications, please notice that:	93	0. If you want to train these models using this version of tensorflow without modifications, please notice that:
94	- You need at lest 12 GB GPU memory.	94	- You need at lest 12 GB GPU memory.
95	- There might be some other untested issues.	95	- There might be some other untested issues.
96		96
97		97
98	### Results	98	### Results
99	0. Result visualization :	99	0. Result visualization :
100	![visualization](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/h.png)	100	![visualization](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/h.png?raw=true)
101	![visualization](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/v.png)	101	![visualization](https://github.com/lelechen63/MRI-tumor-segmentation-Brats/blob/master/image/v.png?raw=true)
102		102
103	0. Quantitative results:	103	0. Quantitative results:
104		104
105	model\|whole\|peritumoral edema (ED)\|FGD-enhan. tumor (ET)	105	model\|whole\|peritumoral edema (ED)\|FGD-enhan. tumor (ET)
106	:---:\|:---:\|:---:\|:---:	106	:---:\|:---:\|:---:\|:---:
107	Dense24 \|0.74\| 0.81\| 0.80	107	Dense24 \|0.74\| 0.81\| 0.80
108	Dense48 \| 0.61\|0.78\|0.79	108	Dense48 \| 0.61\|0.78\|0.79
109	no-dense\|0.61\|0.77\|0.78	109	no-dense\|0.61\|0.77\|0.78
110	dense24+n4correction\|0.72\|0.83\|0.81	110	dense24+n4correction\|0.72\|0.83\|0.81