Switch to unified view

a/README.md b/README.md
1
# RotCAtt-TransUNet++: Novel Deep Neural Network for Cardiac Segmentation
1
# RotCAtt-TransUNet++: Novel Deep Neural Network for Cardiac Segmentation
2
2
3
## MAPR 2024
3
## MAPR 2024
4
[![IEEE](https://a11ybadges.com/badge?logo=ieee)](https://ieeexplore.ieee.org/document/10660759/)
4
[![IEEE](https://a11ybadges.com/badge?logo=ieee)](https://ieeexplore.ieee.org/document/10660759/)
5
[![arXiv](https://a11ybadges.com/badge?logo=arxiv)](https://arxiv.org/abs/2409.05280)
5
[![arXiv](https://a11ybadges.com/badge?logo=arxiv)](https://arxiv.org/abs/2409.05280)
6
6
7
## Presentation
7
## Presentation
8
[![Papers with Code](https://img.shields.io/badge/Video-link-red)](https://www.facebook.com/share/v/2p6YfjM4RaPLVbJM/)
8
[![Papers with Code](https://img.shields.io/badge/Video-link-red)](https://www.facebook.com/share/v/2p6YfjM4RaPLVbJM/)
9
[![Papers with Code](https://img.shields.io/badge/Slide-link-green)](https://drive.google.com/file/d/11xyR8JpEheDu_ReHQwDMSGWLa5kjKcmQ/view?usp=sharing)
9
[![Papers with Code](https://img.shields.io/badge/Slide-link-green)](https://drive.google.com/file/d/11xyR8JpEheDu_ReHQwDMSGWLa5kjKcmQ/view?usp=sharing)
10
[![Papers with Code](https://img.shields.io/badge/Poster-link-brown)](https://drive.google.com/file/d/1Y3TW2CqJSxAwb3eiyyiyDMSdvlZkiiGr/view?usp=sharing)
10
[![Papers with Code](https://img.shields.io/badge/Poster-link-brown)](https://drive.google.com/file/d/1Y3TW2CqJSxAwb3eiyyiyDMSdvlZkiiGr/view?usp=sharing)
11
[![Papers with Code](https://img.shields.io/badge/Paper_with_code-link-blue)](https://paperswithcode.com/paper/rotcatt-transunet-novel-deep-neural-network)
11
[![Papers with Code](https://img.shields.io/badge/Paper_with_code-link-blue)](https://paperswithcode.com/paper/rotcatt-transunet-novel-deep-neural-network)
12
12
13
13
14
## Abstract
14
## Abstract
15
Cardiovascular disease is a major global health concern, contributing significantly to global mortality. Accurately segmenting cardiac medical imaging data is crucial for reducing fatality rates associated with these conditions. However, current state-of-the-art (SOTA) neural networks, including CNN-based and Transformer-based approaches, face challenges in capturing both inter-slice connections and intra-slice details, especially in datasets featuring intricate, long-range details along the z-axis like coronary arteries. Existing methods also struggle with differentiating non-cardiac components from the myocardium, resulting in segmentation inaccuracies and the "spraying" phenomenon. To address these issues, we introduce RotCAtt-TransUNet++, a novel architecture designed for robust segmentation of intricate cardiac structures. Our approach enhances global context mod-eling through multiscale feature aggregation and nested skip connections in the encoder. Transformer layers facilitate capturing intra-slice interactions, while a rotatory attention mechanism handles inter-slice connectivity. A channel-wise cross-attention gate integrates multiscale information and decoder features, effectively bridging semantic gaps. Experimental results across multiple datasets demonstrate superior performance over current methods, achieving near-perfect annotation of coronary arteries and myocardium. Ablation studies confirm that our rotatory attention mechanism significantly improves segmentation accuracy by transforming embedded vectorized patches in semantic dimensional space.
15
Cardiovascular disease is a major global health concern, contributing significantly to global mortality. Accurately segmenting cardiac medical imaging data is crucial for reducing fatality rates associated with these conditions. However, current state-of-the-art (SOTA) neural networks, including CNN-based and Transformer-based approaches, face challenges in capturing both inter-slice connections and intra-slice details, especially in datasets featuring intricate, long-range details along the z-axis like coronary arteries. Existing methods also struggle with differentiating non-cardiac components from the myocardium, resulting in segmentation inaccuracies and the "spraying" phenomenon. To address these issues, we introduce RotCAtt-TransUNet++, a novel architecture designed for robust segmentation of intricate cardiac structures. Our approach enhances global context mod-eling through multiscale feature aggregation and nested skip connections in the encoder. Transformer layers facilitate capturing intra-slice interactions, while a rotatory attention mechanism handles inter-slice connectivity. A channel-wise cross-attention gate integrates multiscale information and decoder features, effectively bridging semantic gaps. Experimental results across multiple datasets demonstrate superior performance over current methods, achieving near-perfect annotation of coronary arteries and myocardium. Ablation studies confirm that our rotatory attention mechanism significantly improves segmentation accuracy by transforming embedded vectorized patches in semantic dimensional space.
16
16
17
## Result
17
## Result
18
Benchmark 9 models across 4 medical datasets. Our model's lightweight nature is due to having only four transformer layers. The robustness of rotatory attention allows encoded vectorized patches to be effectively transformed in the semantic space, reducing the need for numerous transformer layers. Our network demonstrates the fastest convergence time when applied to cardiac data, thanks to robust long-range interslice connectivity.
18
Benchmark 9 models across 4 medical datasets. Our model's lightweight nature is due to having only four transformer layers. The robustness of rotatory attention allows encoded vectorized patches to be effectively transformed in the semantic space, reducing the need for numerous transformer layers. Our network demonstrates the fastest convergence time when applied to cardiac data, thanks to robust long-range interslice connectivity.
19
![results](publication/results.png)
19
20
20
21
22
**Comparison with state-of-the-arts**:
23
Our model achieves near-perfect accuracy and produces high-quality refined segmentation mask and 3D reconstruction.
24
|     |  |
25
| -------- | ------- |
26
| <img width="500px" src="publication/3D.png"> | <img width="300px" src="publication/comparison.png"> |
27
28
29
## Build for training
21
## Build for training
30
Install Pytorch in conda or use this docker image:
22
Install Pytorch in conda or use this docker image:
31
```
23
```
32
docker pull kylepaul/deeplearning:deployment
24
docker pull kylepaul/deeplearning:deployment
33
```
25
```
34
Setup the dataset and make sure to process the data into this structures. All dataset can be downloaded here.
26
Setup the dataset and make sure to process the data into this structures. All dataset can be downloaded here.
35
```
27
```
36
<dataset name>
28
<dataset name>
37
    ├── p_images
29
    ├── p_images
38
    |   ├── 0001_0001.npy
30
    |   ├── 0001_0001.npy
39
    │   ├── 0001_0002.npy
31
    │   ├── 0001_0002.npy
40
    │   ├── 0001_0003.npy
32
    │   ├── 0001_0003.npy
41
    │   ├── ...
33
    │   ├── ...
42
    |
34
    |
43
    └── p_labels
35
    └── p_labels
44
        ├── 0001_0001.npy
36
        ├── 0001_0001.npy
45
        ├── 0001_0002.npy
37
        ├── 0001_0002.npy
46
        ├── 0001_0003.npy
38
        ├── 0001_0003.npy
47
        ├── ...   
39
        ├── ...   
48
```
40
```
49
Then start training
41
Then start training
50
```
42
```
51
python train.py
43
python train.py
52
```
44
```
53
45
54
## Citation
46
## Citation
55
```
47
```
56
@INPROCEEDINGS{10660759,
48
@INPROCEEDINGS{10660759,
57
    author={Nguyen-Le, Quoc-Bao and Le, Tuan-Hy and Do, Anh-Triet and Trinh, Quoc-Huy},
49
    author={Nguyen-Le, Quoc-Bao and Le, Tuan-Hy and Do, Anh-Triet and Trinh, Quoc-Huy},
58
    booktitle={2024 International Conference on Multimedia Analysis and Pattern Recognition (MAPR)}, 
50
    booktitle={2024 International Conference on Multimedia Analysis and Pattern Recognition (MAPR)}, 
59
    title={RotCAtt-TransUNet++: Novel Deep Neural Network for Sophisticated Cardiac Segmentation}, 
51
    title={RotCAtt-TransUNet++: Novel Deep Neural Network for Sophisticated Cardiac Segmentation}, 
60
    year={2024},
52
    year={2024},
61
    volume={},
53
    volume={},
62
    number={},
54
    number={},
63
    pages={1-6},
55
    pages={1-6},
64
    keywords={Image segmentation;Attention mechanisms;Face recognition;Semantics;Spraying;Myocardium;Logic gates},
56
    keywords={Image segmentation;Attention mechanisms;Face recognition;Semantics;Spraying;Myocardium;Logic gates},
65
    doi={10.1109/MAPR63514.2024.10660759}}
57
    doi={10.1109/MAPR63514.2024.10660759}}
66
```
58
```