|
a |
|
b/documentation/calculate_test_metrics.md |
|
|
1 |
# How to calculate test metrics on the test set predicted masks? |
|
|
2 |
Once you have trained some models (as described in [trainddp.md](./trainddp.md)) and used them to perform inference (as described in [inference.md](./inference.md)) to generate predicted masks on the test images, you can proceed with the computation of test metrics. We compute three segmentation metrics: `Dice similarity coefficient (DSC)`, `false positive volume (FPV) in ml`, `false negative volume (FNV) in ml`. We also compute detection metrics such as `true positive (TP)`, `false positive (FP)`, and `false negative (FN)` lesion detections via three different criterion labeled as `Criterion1`, `Criterion2`, and `Criterion3`. These metrics have been defined in [metrics/metrics.py](./../metrics/metrics.py). |
|
|
3 |
|
|
|
4 |
## Step 1: Activate the required conda environment (`lymphoma_seg`) and navigate to `segmentation` folder |
|
|
5 |
First, activate the conda environment `lymphoma_seg` using (created as described in [conda_env.md](./conda_env.md)): |
|
|
6 |
|
|
|
7 |
``` |
|
|
8 |
conda activate lymphoma_seg |
|
|
9 |
cd segmentation |
|
|
10 |
``` |
|
|
11 |
|
|
|
12 |
## Step 2: Run the script to compute test metrics |
|
|
13 |
After this, run the following script in your terminal, |
|
|
14 |
``` |
|
|
15 |
python calculate_test_metrics.py --fold=0 --network-name='unet' --input-patch-size=192 |
|
|
16 |
``` |
|
|
17 |
|
|
|
18 |
Alternatively, modify the [segmentation/calculate_test_metrics.sh](./../segmentation/calculate_test_metrics.sh) for your use-case (which contains the same bash script as above) and run: |
|
|
19 |
|
|
|
20 |
``` |
|
|
21 |
bash calculate_test_metrics.sh |
|
|
22 |
``` |
|
|
23 |
|
|
|
24 |
The test metrics will be written to `LYMPHOMA_SEGMENTATION_FOLDER/results/test_metrics/fold{fold}/{network_name}/{experiment_code}/testmetrics.csv`, as described in [results_format.md](./results_format.md) file. The relevant directory structure may then look like: |
|
|
25 |
|
|
|
26 |
└───lymphoma.segmentation/ |
|
|
27 |
├── data |
|
|
28 |
└── results |
|
|
29 |
├── logs |
|
|
30 |
├── models |
|
|
31 |
├── predictions |
|
|
32 |
└── test_metrics |
|
|
33 |
├── fold0 |
|
|
34 |
│ └── unet |
|
|
35 |
│ └── unet_fold0_randcrop192 |
|
|
36 |
│ └── testmetrics.csv |
|
|
37 |
└── fold1 |
|
|
38 |
└── unet |
|
|
39 |
└── unet_fold1_randcrop192 |
|
|
40 |
└── testmetrics.csv |