This repository contains code and data for analyzing diversity in head and neck cancer clinical trials, specifically focusing on the inclusion of non-white participants in these studies.

## Project Overview

Head and neck cancer disproportionately affects certain racial and ethnic groups. This analysis aims to understand factors that contribute to higher diversity in clinical trials and identify patterns that could lead to more inclusive research.

### Key Components

1. **Diversity Metric**: The analysis uses a metric defined as the percentage of non-white participants in each study to measure diversity.

   - Score = (# non-white participants) / (# total participants) × 100

   - Total participants = # white participants + # non-white participants

2. **Comparative Analysis**: Studies are categorized into high-diversity (top 20%) and low-diversity (bottom 20%) groups based on this metric.

3. **Factor Identification**: Various factors are examined to understand what contributes to more diverse clinical trials.

## Analysis Methodology

The analysis followed these key steps:

1. **Data Collection**: Collected data on all head and neck cancer clinical trials from ClinicalTrials.gov

2. **Diversity Scoring**: Computed a diversity score for each trial based on participant demographics

3. **Stratification**: Identified the top 20th percentile and bottom 20th percentile of trials by diversity

4. **Feature Extraction**: Extracted key features from each clinical trial:

   - **Study Characteristics**: Start/end dates, institutional setting, number of participants, location, etc.

   - **Eligibility Criteria**: Detailed analysis of inclusion/exclusion criteria

5. **Comparative Analysis**: Compared the distribution of features between high-diversity and low-diversity trials

### Eligibility Features Analyzed

The study examined specific eligibility restrictions and their potential impact on diversity:

| Feature | Description |

|---------|-------------|

| age_restrict | 0 if the restriction is age>18, 1 for other restrictions (e.g., 18<age<75) |

| stage_size | Restrictions on the cancer stage and the size of the tumor |

| cancer_site | Restrictions on the cancer site |

| histological_type | Whether the study was limited to SCC (Squamous Cell Carcinoma) or any other type |

| performance_score | Restrictions on performance score (e.g., ECOG performance) |

| comorbidities | Restrictions on comorbidities |

| hx_of_tt | Restrictions on treatment history for cancer |

| lab_values | Restrictions on lab test values |

| pregnancy_or_contraception | Restrictions on pregnancy or particular contraceptives |

| misc | Other restrictions (e.g., smoking status, ethnicity requirements) |

| eligibility_score | Sum of all restriction scores above |

### General Features Analyzed

The analysis also included general study characteristics:

1. Study start date and end date

2. Single vs. multi-institutional study

3. Stringency in eligibility criteria (composite score)

4. Modality (Drug/Radiation/Biologic/Combination)

5. Number of participants

6. Geographic location

7. Male/female ratio

8. Trial type (Primary/Palliative/Recurrent/Metastatic)

## Repository Structure

```

├── README.md                        # Project documentation

├── src/                             # Source code directory

│   ├── data_processing.py           # Functions for data loading and preprocessing

│   ├── analysis.py                  # Functions for statistical analysis

│   ├── visualization.py             # Functions for creating visualizations

│   └── main.py                      # Main script that orchestrates the analysis

├── plots/                           # Generated visualizations

│   ├── box_plot_eligbility_score_diverse_vs_non_diverse.png

│   ├── box_plot_num_participants_top_vs_bottom.png

│   ├── distribution_age_restrict.png

│   ├── distribution_comorbidities.png

│   ├── distribution_histological_type.png

│   ├── distribution_hx_of_tt.png

│   ├── distribution_is_single_institution.png

│   ├── distribution_lab_values.png

│   ├── distribution_misc.png

│   ├── distribution_num_participants_top_vs_bottom_studies_strat_gender.png

│   ├── distribution_performance_score.png

│   ├── distribution_site.png

│   ├── distribution_stage_size.png

│   └── geo_distribution.png

├── top_20_studies.csv               # Dataset of top 20% diverse studies

├── bottom_20_studies.csv            # Dataset of bottom 20% diverse studies

├── Diversity in head and neck clinical trials - plots (2).pdf # PDF with plot descriptions

├── Analysis.ipynb                   # Jupyter notebook with initial analysis

└── Analysis top20 vs bottom20.ipynb # Jupyter notebook with comparative analysis

```

## Key Findings

### 1. Eligibility Criteria

The analysis of eligibility criteria revealed that more diverse studies tend to have fewer restrictive criteria:

![Eligibility Score Comparison](plots/box_plot_eligbility_score_diverse_vs_non_diverse.png)

*The above plot shows the distribution of eligibility scores for diverse vs. non-diverse studies. Higher scores indicate more restrictive eligibility criteria.*

### 2. Geographic Distribution

The geographic location of studies plays a significant role in diversity:

![Geographic Distribution](plots/geo_distribution.png)

*This map shows the locations of the top and bottom diverse studies, with color indicating the population diversity score of each location.*

### 3. Participant Demographics

Studies with higher diversity had different participant demographics:

![Participant Distribution by Gender](plots/distribution_num_participants_top_vs_bottom_studies_strat_gender.png)

*This plot shows the distribution of male and female participants in top vs. bottom diverse studies.*

### 4. Eligibility Restrictions

Specific eligibility criteria had different prevalence in diverse vs. non-diverse studies:

- **Age Restrictions**:

  ![Age Restrictions](plots/distribution_age_restrict.png)

  *This plot compares the prevalence of age restrictions beyond the standard adult age (18+) between high and low diversity studies.*

- **Histological Type Restrictions**:

  ![Histological Type Restrictions](plots/distribution_histological_type.png)

  *This plot compares the prevalence of restrictions on cancer histological type (e.g., SCC only) between high and low diversity studies.*

- **Performance Score Restrictions**:

  ![Performance Score Restrictions](plots/distribution_performance_score.png)

  *This plot compares the prevalence of ECOG or other performance score restrictions between high and low diversity studies.*

- **Comorbidity Restrictions**:

  ![Comorbidity Restrictions](plots/distribution_comorbidities.png)

  *This plot compares the prevalence of comorbidity restrictions between high and low diversity studies.*

- **Laboratory Value Restrictions**:

  ![Laboratory Value Restrictions](plots/distribution_lab_values.png)

  *This plot compares the prevalence of laboratory value restrictions between high and low diversity studies.*

- **Stage/Size Restrictions**:

  ![Stage/Size Restrictions](plots/distribution_stage_size.png)

  *This plot compares the prevalence of tumor stage or size restrictions between high and low diversity studies.*

- **Site Restrictions**:

  ![Site Restrictions](plots/distribution_site.png)

  *This plot compares the prevalence of cancer site restrictions between high and low diversity studies.*

- **History of Treatment Restrictions**:

  ![History of Treatment Restrictions](plots/distribution_hx_of_tt.png)

  *This plot compares the prevalence of previous treatment history restrictions between high and low diversity studies.*

- **Miscellaneous Restrictions**:

  ![Miscellaneous Restrictions](plots/distribution_misc.png)

  *This plot compares the prevalence of other restrictions (such as smoking status or ethnicity requirements) between high and low diversity studies.*

- **Institutional Setting**:

  ![Single Institution Distribution](plots/distribution_is_single_institution.png)

  *This plot shows the distribution of single-institution vs. multi-institution studies among diverse and non-diverse trials.*

## Data Source

The data for this analysis was extracted from [ClinicalTrials.gov](https://clinicaltrials.gov/), focusing on head and neck cancer clinical trials conducted in the United States. Only studies that reported race information were included in the analysis.

## Conclusions

The analysis identified several factors that are associated with more diverse head and neck cancer clinical trials:

1. **Less restrictive eligibility criteria**: Studies with fewer restrictions tend to have more diverse participation.

   - Specific criteria that appear to impact diversity include age restrictions, performance score requirements, and histological type restrictions.

2. **Geographic location**: Studies in areas with more diverse populations have higher diversity scores.

3. **Institutional setting**: Different types of institutions show varying levels of success in recruiting diverse participants.

4. **Study size**: There is a relationship between the number of participants and diversity.

These findings suggest potential strategies for improving diversity in future clinical trials, such as revisiting eligibility criteria, focusing on inclusive recruitment strategies, and considering geographic factors when planning trial sites.

## Running the Analysis

### Prerequisites

- Python 3.7+

- Required packages: pandas, numpy, plotly, scipy

### Usage

```bash

# Run the main analysis script

python src/main.py

```

Or explore the Jupyter notebooks for an interactive analysis experience:

```bash

jupyter notebook "Analysis.ipynb"

jupyter notebook "Analysis top20 vs bottom20.ipynb"

```

## License

This project is licensed under the MIT License - see the LICENSE file for details.

## Acknowledgements