Datasets
Models
Downloads: 1
2252 lines (2251 with data), 140.8 kB
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# RSNA Intracranial Hemorrhage Detection "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<b>Competition Overview</b><br/><br/>\n",
"Intracranial hemorrhage, bleeding that occurs inside the cranium, is a serious health problem requiring rapid and often intensive medical treatment. For example, intracranial hemorrhages account for approximately 10% of strokes in the U.S., where stroke is the fifth-leading cause of death. Identifying the location and type of any hemorrhage present is a critical step in treating the patient.\n",
"\n",
"Diagnosis requires an urgent procedure. When a patient shows acute neurological symptoms such as severe headache or loss of consciousness, highly trained specialists review medical images of the patient’s cranium to look for the presence, location and type of hemorrhage. The process is complicated and often time consuming."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<b>What am i predicting?</b><br/><br/>\n",
"In this competition our goal is to predict intracranial hemorrhage and its subtypes. Given an image the we need to predict probablity of each subtype. This indicates its a multilabel classification problem."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<b>Competition Evaluation Metric</b><br/><br/>\n",
"Evaluation metric is weighted multi-label logarithmic loss. So for given image we need to predict probality for each subtype. There is also an any label, which indicates that a hemorrhage of ANY kind exists in the image. The any label is weighted more highly than specific hemorrhage sub-types.\n",
"\n",
"<b>Note:</b>The weights for each subtype for calculating weighted multi-label logarithmic loss is **not** given as part of the competition. We will be using binary cross entropy loss as weights are not available"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<b>Dataset Description</b>\n",
"\n",
"The dataset is divided into two parts\n",
"\n",
"1. Train\n",
"2. Test\n",
"\n",
"**1. Train**\n",
"Number of rows: 40,45,548 records.\n",
"Number of columns: 2\n",
"\n",
"Columns:\n",
"\n",
"**Id**: An image Id. Each Id corresponds to a unique image, and will contain an underscore.\n",
"\n",
"Example: ID_28fbab7eb_epidural. So the Id consists of two parts one is image file id ID_28fbab7eb and the other is sub type name\n",
"\n",
"**Label**: The target label whether that sub-type of hemorrhage (or any hemorrhage in the case of any) exists in the indicated image. 1 --> Exists and 0 --> Doesn't exist.\n",
"\n",
"**2. Test**\n",
"Number of rows: 4,71,270 records.\n",
"\n",
"Columns:\n",
"\n",
"**Id**: An image Id. Each Id corresponds to a unique image, and will contain an underscore.\n",
"\n",
"Example: ID_28fbab7eb_epidural. So the Id consists of two parts one is image file id ID_28fbab7eb and the other is sub type name"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Using TensorFlow backend.\n"
]
}
],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import pydicom\n",
"import os\n",
"import glob\n",
"import random\n",
"import cv2\n",
"import tensorflow as tf\n",
"from math import ceil, floor\n",
"from tqdm import tqdm\n",
"from imgaug import augmenters as iaa\n",
"import matplotlib.pyplot as plt\n",
"from math import ceil, floor\n",
"import keras\n",
"import keras.backend as K\n",
"from keras.callbacks import Callback, ModelCheckpoint\n",
"from keras.layers import Dense, Flatten, Dropout\n",
"from keras.models import Model, load_model\n",
"from keras.utils import Sequence\n",
"from keras.losses import binary_crossentropy\n",
"from keras.optimizers import Adam"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"# Random Seed\n",
"SEED = 42\n",
"np.random.seed(SEED)\n",
"\n",
"# some constants\n",
"TEST_SIZE = 0.06\n",
"HEIGHT = 256\n",
"WIDTH = 256\n",
"TRAIN_BATCH_SIZE = 32\n",
"VALID_BATCH_SIZE = 64\n",
"\n",
"# Train and Test folders\n",
"input_folder = '../input/rsna-intracranial-hemorrhage-detection/'\n",
"path_train_img = input_folder + 'stage_1_train_images/'\n",
"path_test_img = input_folder + 'stage_1_test_images/'"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>ID</th>\n",
" <th>Label</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>ID_63eb1e259_epidural</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>ID_63eb1e259_intraparenchymal</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>ID_63eb1e259_intraventricular</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>ID_63eb1e259_subarachnoid</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>ID_63eb1e259_subdural</td>\n",
" <td>0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" ID Label\n",
"0 ID_63eb1e259_epidural 0\n",
"1 ID_63eb1e259_intraparenchymal 0\n",
"2 ID_63eb1e259_intraventricular 0\n",
"3 ID_63eb1e259_subarachnoid 0\n",
"4 ID_63eb1e259_subdural 0"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train_df = pd.read_csv(input_folder + 'stage_1_train.csv')\n",
"train_df.head()"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>ID</th>\n",
" <th>Label</th>\n",
" <th>sub_type</th>\n",
" <th>file_name</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>ID_63eb1e259_epidural</td>\n",
" <td>0</td>\n",
" <td>epidural</td>\n",
" <td>ID_63eb1e259.dcm</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>ID_63eb1e259_intraparenchymal</td>\n",
" <td>0</td>\n",
" <td>intraparenchymal</td>\n",
" <td>ID_63eb1e259.dcm</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>ID_63eb1e259_intraventricular</td>\n",
" <td>0</td>\n",
" <td>intraventricular</td>\n",
" <td>ID_63eb1e259.dcm</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>ID_63eb1e259_subarachnoid</td>\n",
" <td>0</td>\n",
" <td>subarachnoid</td>\n",
" <td>ID_63eb1e259.dcm</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>ID_63eb1e259_subdural</td>\n",
" <td>0</td>\n",
" <td>subdural</td>\n",
" <td>ID_63eb1e259.dcm</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" ID Label sub_type file_name\n",
"0 ID_63eb1e259_epidural 0 epidural ID_63eb1e259.dcm\n",
"1 ID_63eb1e259_intraparenchymal 0 intraparenchymal ID_63eb1e259.dcm\n",
"2 ID_63eb1e259_intraventricular 0 intraventricular ID_63eb1e259.dcm\n",
"3 ID_63eb1e259_subarachnoid 0 subarachnoid ID_63eb1e259.dcm\n",
"4 ID_63eb1e259_subdural 0 subdural ID_63eb1e259.dcm"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# extract subtype\n",
"train_df['sub_type'] = train_df['ID'].apply(lambda x: x.split('_')[-1])\n",
"# extract filename\n",
"train_df['file_name'] = train_df['ID'].apply(lambda x: '_'.join(x.split('_')[:2]) + '.dcm')\n",
"train_df.head()"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(4045572, 4)"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train_df.shape"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(4045548, 4)"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# remove duplicates\n",
"train_df.drop_duplicates(['Label', 'sub_type', 'file_name'], inplace=True)\n",
"train_df.shape"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of train images availabe: 674258\n"
]
}
],
"source": [
"print(\"Number of train images availabe:\", len(os.listdir(path_train_img)))"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th>sub_type</th>\n",
" <th>any</th>\n",
" <th>epidural</th>\n",
" <th>intraparenchymal</th>\n",
" <th>intraventricular</th>\n",
" <th>subarachnoid</th>\n",
" <th>subdural</th>\n",
" </tr>\n",
" <tr>\n",
" <th>file_name</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>ID_000039fa0.dcm</th>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>ID_00005679d.dcm</th>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>ID_00008ce3c.dcm</th>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>ID_0000950d7.dcm</th>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>ID_0000aee4b.dcm</th>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
"sub_type any epidural intraparenchymal intraventricular \\\n",
"file_name \n",
"ID_000039fa0.dcm 0 0 0 0 \n",
"ID_00005679d.dcm 0 0 0 0 \n",
"ID_00008ce3c.dcm 0 0 0 0 \n",
"ID_0000950d7.dcm 0 0 0 0 \n",
"ID_0000aee4b.dcm 0 0 0 0 \n",
"\n",
"sub_type subarachnoid subdural \n",
"file_name \n",
"ID_000039fa0.dcm 0 0 \n",
"ID_00005679d.dcm 0 0 \n",
"ID_00008ce3c.dcm 0 0 \n",
"ID_0000950d7.dcm 0 0 \n",
"ID_0000aee4b.dcm 0 0 "
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train_final_df = pd.pivot_table(train_df.drop(columns='ID'), index=\"file_name\", \\\n",
" columns=\"sub_type\", values=\"Label\")\n",
"train_final_df.head()"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(674258, 6)"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train_final_df.shape"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"# Invalid image ID_6431af929.dcm\n",
"train_final_df.drop('ID_6431af929.dcm', inplace=True)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting efficientnet\n",
" Downloading https://files.pythonhosted.org/packages/97/82/f3ae07316f0461417dc54affab6e86ab188a5a22f33176d35271628b96e0/efficientnet-1.0.0-py3-none-any.whl\n",
"Requirement already satisfied: keras-applications<=1.0.8,>=1.0.7 in /opt/conda/lib/python3.6/site-packages (from efficientnet) (1.0.8)\n",
"Requirement already satisfied: scikit-image in /opt/conda/lib/python3.6/site-packages (from efficientnet) (0.16.1)\n",
"Requirement already satisfied: numpy>=1.9.1 in /opt/conda/lib/python3.6/site-packages (from keras-applications<=1.0.8,>=1.0.7->efficientnet) (1.16.4)\n",
"Requirement already satisfied: h5py in /opt/conda/lib/python3.6/site-packages (from keras-applications<=1.0.8,>=1.0.7->efficientnet) (2.9.0)\n",
"Requirement already satisfied: PyWavelets>=0.4.0 in /opt/conda/lib/python3.6/site-packages (from scikit-image->efficientnet) (1.0.3)\n",
"Requirement already satisfied: scipy>=0.19.0 in /opt/conda/lib/python3.6/site-packages (from scikit-image->efficientnet) (1.2.1)\n",
"Requirement already satisfied: networkx>=2.0 in /opt/conda/lib/python3.6/site-packages (from scikit-image->efficientnet) (2.4)\n",
"Requirement already satisfied: imageio>=2.3.0 in /opt/conda/lib/python3.6/site-packages (from scikit-image->efficientnet) (2.6.0)\n",
"Requirement already satisfied: pillow>=4.3.0 in /opt/conda/lib/python3.6/site-packages (from scikit-image->efficientnet) (5.4.1)\n",
"Requirement already satisfied: matplotlib!=3.0.0,>=2.0.0 in /opt/conda/lib/python3.6/site-packages (from scikit-image->efficientnet) (3.0.3)\n",
"Requirement already satisfied: six in /opt/conda/lib/python3.6/site-packages (from h5py->keras-applications<=1.0.8,>=1.0.7->efficientnet) (1.12.0)\n",
"Requirement already satisfied: decorator>=4.3.0 in /opt/conda/lib/python3.6/site-packages (from networkx>=2.0->scikit-image->efficientnet) (4.4.0)\n",
"Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.6/site-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet) (0.10.0)\n",
"Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.6/site-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet) (1.1.0)\n",
"Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /opt/conda/lib/python3.6/site-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet) (2.4.2)\n",
"Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/lib/python3.6/site-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet) (2.8.0)\n",
"Requirement already satisfied: setuptools in /opt/conda/lib/python3.6/site-packages (from kiwisolver>=1.0.1->matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet) (41.4.0)\n",
"Installing collected packages: efficientnet\n",
"Successfully installed efficientnet-1.0.0\n",
"Collecting iterative-stratification\n",
" Downloading https://files.pythonhosted.org/packages/9d/79/9ba64c8c07b07b8b45d80725b2ebd7b7884701c1da34f70d4749f7b45f9a/iterative_stratification-0.1.6-py3-none-any.whl\n",
"Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.6/site-packages (from iterative-stratification) (0.21.3)\n",
"Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (from iterative-stratification) (1.2.1)\n",
"Requirement already satisfied: numpy in /opt/conda/lib/python3.6/site-packages (from iterative-stratification) (1.16.4)\n",
"Requirement already satisfied: joblib>=0.11 in /opt/conda/lib/python3.6/site-packages (from scikit-learn->iterative-stratification) (0.13.2)\n",
"Installing collected packages: iterative-stratification\n",
"Successfully installed iterative-stratification-0.1.6\n"
]
}
],
"source": [
"# Install Efficient Net as it is not part of Keras\n",
"!pip install efficientnet\n",
"!pip install iterative-stratification"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"import efficientnet.keras as efn \n",
"from iterstrat.ml_stratifiers import MultilabelStratifiedShuffleSplit"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"from IPython.display import HTML\n",
"\n",
"def create_download_link(title = \"Download CSV file\", filename = \"data.csv\"): \n",
" \"\"\"\n",
" Helper function to generate download link to files in kaggle kernel \n",
" \"\"\"\n",
" html = '<a href={filename}>{title}</a>'\n",
" html = html.format(title=title,filename=filename)\n",
" return HTML(html)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"def get_dicom_field_value(val):\n",
" \"\"\"\n",
" Helper function to get value of dicom field in dicom file\n",
" \"\"\"\n",
" if type(val) == pydicom.multival.MultiValue:\n",
" return int(val[0])\n",
" else:\n",
" return int(val)\n",
"\n",
"def get_windowing(data):\n",
" \"\"\"\n",
" Helper function to extract meta data features in dicom file\n",
" return: window center, window width, slope, intercept\n",
" \"\"\"\n",
" dicom_fields = [data.WindowCenter, data.WindowWidth, data.RescaleSlope, data.RescaleIntercept]\n",
" return [get_dicom_field_value(x) for x in dicom_fields]\n",
"\n",
"\n",
"def get_windowed_image(image, wc, ww, slope, intercept):\n",
" \"\"\"\n",
" Helper function to construct windowed image from meta data features\n",
" return: windowed image\n",
" \"\"\"\n",
" img = (image*slope +intercept)\n",
" img_min = wc - ww//2\n",
" img_max = wc + ww//2\n",
" img[img<img_min] = img_min\n",
" img[img>img_max] = img_max\n",
" return img \n",
"\n",
"\n",
"def _normalize(img):\n",
" if img.max() == img.min():\n",
" return np.zeros(img.shape)\n",
" return 2 * (img - img.min())/(img.max() - img.min()) - 1\n",
"\n",
"def _read(path, desired_size=(224, 224)):\n",
" \"\"\"\n",
" Helper function to generate windowed image \n",
" \"\"\"\n",
" # 1. read dicom file\n",
" dcm = pydicom.dcmread(path)\n",
" \n",
" # 2. Extract meta data features\n",
" # window center, window width, slope, intercept\n",
" window_params = get_windowing(dcm)\n",
"\n",
" try:\n",
" # 3. Generate windowed image\n",
" img = get_windowed_image(dcm.pixel_array, *window_params)\n",
" except:\n",
" img = np.zeros(desired_size)\n",
"\n",
" img = _normalize(img)\n",
"\n",
" if desired_size != (512, 512):\n",
" # resize image\n",
" img = cv2.resize(img, desired_size, interpolation = cv2.INTER_LINEAR)\n",
" return img[:,:,np.newaxis]"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(128, 128, 1)"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"_read(path_train_img + 'ID_ffff922b9.dcm', (128, 128)).shape"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.image.AxesImage at 0x7f821e7b95c0>"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAQUAAAD8CAYAAAB+fLH0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzsnXeYVdW5/z/r7NPmTC8wdBhgKKKIoIC9R0W9JmrUxMpNYqLG+Eu50XhTb0zx5t7Um6jYjS3G2GvsURERREFAeu8D08tpe/3+eNfeZw4zwwzMDFLW93nmOWf2XnvttffZe623fl+ltcbCwsLCQ+CzHoCFhcW+BTspWFhYZMFOChYWFlmwk4KFhUUW7KRgYWGRBTspWFhYZMFOChYWFlnotUlBKXWmUmqJUmq5Uuqm3jqPhYVFz0L1RvCSUsoBlgKnA+uBD4Avaa0X9fjJLCwsehTBXup3MrBca70SQCn1KHAe0O6kEFYRHSW3l4ZisbeRLpHfcuygbSxbWgyAbol/lkOyAOqprtJa9+msXW9NCgOBda3+Xw9Mad1AKXU1cDVAlBhT1Km9NBSLvY3aaVMBmPWb25l22kUApBct/SyHZAG8qh9f05V2vTUpqHa2ZekpWusZwAyAAlViEzAOADR9Qeb9bUfK/2cMmEDTF0RSiFnFcb9Bb00K64HBrf4fBGzspXNZ7COIPfk+ACOfbLvNYv9Bb3kfPgAqlVIVSqkwcAnwTC+dy8LCogfRK5KC1jqllPom8DLgAPdorRf2xrksLCx6Fr2lPqC1fgF4obf6t7Cw6B3YiEYLC4ss2EnBwsIiC3ZSsLCwyIKdFCwsLLJgJwULC4ss2EnBwsIiC3ZSsLCwyIKdFCwsLLJgJwULC4ss2EnBwsIiC70W5mxx8CGQnw+AW18vG5Ri5UOHA5Cf1wxA/281s+aSQQAM/PXMvT9Ii05hJQULC4ssWEnBottwRo0AoGlkCQD1g+WxKrvjPYZ/+SMA4v8cBoC7tYorL5OE2TvKTgdgxPdm7c3hWnSCXiFu3V0UqBJt6dj2fQTGj+EfLzwAwKFPXQ9A5TczJCoqEgEgedyhAGy6Js7gCz8BoPm8yQCsP02Rs9EB4OVr/huArww5bi+M3uJV/fhcrfWRnbWz6oOFhUUWrPpg0SmO+igNwC19HwXCAKw8/w7ZeT4Mf/LrAFReJ1JD8LW5APSNHoVTVgpAztOzpc3T8PJGUSlmxyN7ZfwWuwcrKVhYWGTBSgoWHeKCxVsBKHUadtlu5RdEarjocLELLX5mNAAD/nsmadNm3Q+PAWDwLTM5Y8AEAP605t2eHrJFD8BKChYWFlmw3geLLFRdfTRzf3rbXj3n9LXHs3FqfaftLlq8GYDHxvbr9jk3PnkIg69cD0C6rq7b/e0P6Kr3waoPBwkC+fkwQkpxuB8vlm15eaQmjgRg+ZdCAIwdvYbbawYC8I2iDXtlbPcOeduvClL5wDUADL/pPX9/+qSJAHyl8B4AHmPPJ4XgULkHZw5dzMsPjAFgSJGUuXOvySO9eNke932gwKoPFhYWWbCSwkECt74ePlrUZltzmbgYrz3uVQAuzP+YIcGYadHza8b0tceLZNABll0hqsvx74ubs6G/Q9nC5qw2KhJBx/esYG1qjZQ4Pb1gIb+ZPC9r37RNJ+5RnwcarKRgYWGRBWtoPMgR7C/6+af/3R+Arxw+k1AgBcC82iEAnFm6AIB+wVqOizYCEAuEO+37whWn8fgIkUAWJ5oA+H/DjvGDl7oKz4XpYemdR6Ei4uysvOLD3errYEavGxqVUoOBB4B+gAvM0Fr/QSlVAvwNGAasBi7SWlfv6Xks9gyBXGM8a2zcZbvUJrHol75WAcB9oSm4G0R96PeeLBiPLu4LgGps5n9Xr23TR+QtmVieqXwpa7s3IQAMD4khM3HmUcDuTQo7Y9XZd/rfz2DCLlp2DcF+5QCkNm/pdl8HArqjPqSA72qtxwJTgeuUUocANwGvaa0rgdfM/xYWFvsJ9lhS0FpvAjaZ7/VKqcXAQOA84CTT7H7gTeDGbo3SoktYf/MxlM9JABD655wuHeOUixSw4zCRCsJBF7dRAbD9EFkz6i+Tz4uHL6E8VAvAH5acDED+QwUErpNttS+JQbAwkNPmPBElksIfbv8TEO3yNY2+9xqGIe7JAbPyu3zc7sBKCNnoEUOjUmoYcATwPlBuJgxv4ujbwTFXK6XmKKXmJNkzS7KFhUXPo9suSaVUHvAP4P9preuUUl06Tms9A5gBYmjs7jgOZpy1sAaAP73iEqxP7tax7vYdAKRzhwGQSARJ9xUj3kVTJLNxSGQ7ALc9cC7Nh4o0UHG3/M7OG+/z6d1iuzri6f8HwLvn/i8ASeCMu74PgBYKBX795QcYH951LkVrLJl+G0xvu33UfRLkVMF7bXdadAvd8j4opULAc8DLWuvfmm1LgJO01puUUv2BN7XWo3fVj/U+7BkmGDf7P16fCsDImz5EJxO71YczTn6a1edLinPRMhfMI7HpNJkccleI6D/0sY3oRvEipLdIslSwfz+Wf1OMlKUL5MCmPiKADrtwBc0nthXNd9f70BU81ZjHbZUje7zfAwm9TrKiRCS4G1jsTQgGzwBXmu9XAk/v6TksLCz2PvZYUlBKHQe8DSxAXJIANyN2hceAIcBa4Ita6x276stKCruPwndKWfSsrPIDb90zVuRgxVBWXi55DsEWs9GF4PHyc9XVicFw8MOiZeZ+upXkgGIA0hHRB5r7hth8jDxDw58Q1SU0+1PpqqkJAkZvcL0kalj25ykAlMyTNWnOf/VMAtbWtLhfLx98bI/0d6Ch1+MUtNbvAB0ZEOwbbmGxn8LmPuxnCL0pkYcr7x3GwLv3UEIYLHUXEkNKyN0oq3xsm6zkdUOCxOcIK/PgObLya0fm/vrx5cQLZeWPF8q2ouVJ8taIzcFpESOk29SUOVkrCQFABYM+bVvq1En+9vkJEVX+Y5jYR5q+INLE23++w2/jRTYGDh9L/p/FpvHY8Nf8/RtT8jjHpx0FQOSFD7pyOyx2gs19sLCwyIKVFPYTDJqVB8Dch8XSX74HUoIX+tx4mEgbOWvraTxR7AaxbaZNSuO0iBTQVC6PR0up/J/Mh3RYJIuyj8WMFFtdQ7DFBBXNmt/pGHQq5X9f+UWROuYnWvh+5Qlmq3hPYk+KNDHttRNwDQ8ESL0I9+PFVN8kUkPF9K8CsOqsu5hgKOZrvy6ELbXTRNpoTUNv0TnspLAfYNkDE1k2W0T0yj/uucqgjVgfSMgLnSqKUrZAxPu85RKVWHVYKcl8efEbKozonyOfOcsjlH0q+4rmiqtR50YJzpSXVUUlUtFt8ayW7eOYj+XFf7nPDACOveEG8pLy4k5fsgaAS/IlXeaMARNgnvS//HeiWoTqFUN/LPEJo96RPk8892oe+rM4wWqqZAKNbpNJZ/MNx9DvD7ZEXVdh1QcLC4ssWElhH4ZzyCgABvStoeCbVQCkd3XALpBatx6nWNyJDQMl7blhiCJmDI2hgaICxMtcwjtkrQg2yuMRbJLP0sVJUjmyr3FMHwBir85HhaU/v7DsLrDy10dzX/FvABj50H8AMOLv7xEcJK7RS/KzA5taE6pUPij9v/TsQ5zx4+zsyOizs7mq/lvSr5a7FNooGaCLby7uBoHbwQcrKVhYWGTBSgr7MMb+dTkA8248gnTdym73lxorpCkNQ0x4iYbipaL/uyFZH7SjiZvch6JPZFufDyVXQaVd3KjYNtS7sqK7AMaGsPbHUtth0IlCeRY4dR11XxI7QO0F0kfwE8X0EacAMCKZyVt4fvbz7Y556Z3jfCIVPXehv92rPJWu2p7Z9mY24YoeL8SsJbPCfmj1zoQtFm1hJ4V9EA0XyYv0rgkeL3y1+1WZ3eOPYONx4mkIGSk/tsUlUSQveXOJiTxULqFamQz6vSUqi14vYjjpNKp1DALi0fDUh9hmUUWak9Knfmk41e9LX2WPGNKXkMuGG0xQnZFTQ8dtpyPilZWn3dOGSOWWqjHseEBUofxbZaJz3l2Q5dkAcOdLZGXDxUdzS5VMEP+5Us7zi+F2cugIVn2wsLDIgpUU9kFsOU8Ma2Uv7F4B1q3XHkPDUFmtR/5EUig99+CKi8OopLgiC5bJWpCOKJztJt5gmxxX8lGQ3M3GnOnKNt0skYo6lSJw+Fj5/qmoM25jIxjKNy+eob5ajJYVX/qYPKSdF6FYPSbAkOOF0u3lsc916bo80X/E374BwICmhcya8LjsfEQ+1qYauGzx5QAk/ir0aoUPioQ17D/f477oSQD88EsiPfyiS2c+OGElBQsLiyxYSWEfgueCHHGprPLO2Eqgczfk5hvEwFc3NkWwRmwDKlfIVzdfI9WVFn/hD/x4q+QEvPbJ0QCEG1yiq0wCa0geBeUWEt4h0kXNhDLp9/NCnlX+QZwdN4hU0Pe8tmxZQ/5gjI+t7A5q0jgZx1RZf0L18O2hr3RyRe1jxcW3d7hvSDCPe8f8FYDrv3KRbJwnWaTphUuo/LEwUscvkXyOZfdPpPJKywTdHuyksA9h2RViUR/5czHKdVbCbNWv5eV2TK2Ufv8KUPyxvOQNxwvhyMf/8RfTOsSt5fLSji+SSaRgTYqWocVZfW49IkSoQQyFKVMTxjNMPnL/H3edljxqGAAe+9bGUwqJbRH1xCNgqR+suPbdSwF48+Q/AvJCp7W0c9TuCa+bUuLVmNkygJsfEzamoS/KDVG5menUY7WedrmoICsfvLtHmKAPRFj1wcLCIgtWUtiH4BVVrb1EXJL5j7Z1Raqg/GQb/t9k0hHDbWM+tILF3xYj36ppMzo8z6HnS4HZjWcUsmaFqAY56436oMEVQYEhM6Rd6pChQMfkJZu+I5LH4MckbyFdXiT/P7ySdJVILk6ZpGPnj+rP2jzJkWjRGTqO3ZUQPPQPSp5Dn2Adsc3SX3CHSAXpT5e3aR98fS4gLNEjxhl+yoVL9ujcByqspGBhYZEFWzZuH0HqlEn+KubVYvDIUQFUSAKElv23GA4jgxuIhCRYp3ZdoTQqSLLytHs6Pdfpi88FYGzhZt7/gwQSlb0uq3x8VD+cNzo2wHkRigWPzPLHWv606PAbp2bnPgRiMZKTJWho+6EiHTQO0lwx7Q0Aflj2aZv+z/z0bLnOjXIPht8JrzxyLwBnjT4ekByLbdeIPeXDH2Wo3JYmRUI445/CKj36m5LK3W4xWqVY/oDYFEZePq/t/gMQvU7camFhcWDC2hT2EQRfn0vA8BG0lhA8rPqpUJdFh9QBkEo51MclbLmsQnTj2Uf8fZfnmBsXHoNfD/8HAN+5/psUPSd2jPhJIoGEtzR06AJ9eeNHnDEge9v6y0aSnto+V0Fi6liSBeIira0Uw4fOT1EREUaXxxpEwrn5g/O54BBZrZv+INmSBYPk0Vw9TXP2lHMAcOvX+333uc3kTfwoc76hQZGmfnb8kwA8Mvo02bFoeZsQaLQmtqBtJSsLOynsU+hIlXNKS0jlmOjCpQUABCvrGdtPiE5+NORZ0zJTCdpz1XmGOIBJEdk/5s5rARj63Ey2fEuMhLmbZCpoKi8hf1H2+b2IwjPPuxwh786g//92TF4SXbaFTVeY3IS4jN+pCfPLT84EoHmDjC1ns8NTq0UdGBCXl7fxOFEF3jj2z5xRKwVlBv0yMyl4mLZkGgAvjH6BGTXihr39QVFBmq6TmIRBL00i9pQUtqHVPQ6acIrVfxsPwLCLO2eOOhhg1QcLC4ssWElhH0K7BjGA0mJytsr8nThcVtCCWAsb6kX89iQAgLtrhU7k0Y0SvfjK2GfZGYPekIjF2sumMuCFTQBsPUmOaxgChYeKcdD9RAyBFU9dDcCoD2bv1vWk1q0nf53wK9YZRuiC1ZqWGhl3362iUlT/WwMsl4CtmhHiD71xgox7UDCPlrHNHZ/kPFOC7lP4yyLheRzyK5FenMrhch0FOaRPMEzQb2WMiv0fEpfrBdfJ5+PRYZ1SyR0MsJKChYVFFnqiwKwDzAE2aK3PUUpVAI8CJcCHwOVa690rcHgQQoXCHdaBXH5VX5Kloh8HNotxbEtjiFXT7mrT9u9XnA7AK0//tc2+kY9IiG/wWpE2+t/jsOQnEmjkNokuP+rrH/jlvuq+LO7H0ffIaqwR+wZAevsui375CNcbktiYrD+1IyEs9XBRRr2PRJKkmkSSKDtfCFqigUyh3L5ldR32n66TfUuTjRQ8IzaK9MliNMW4Vp2CAlRabCbNZ5maEC9+QLpayGF/8ep5AHz3wxd45pDSLl3XgYyeUB9uABYDBeb/W4Hfaa0fVUrdDnwF6Jm6YAcwAhWDSS9dkbXNi1dwEopUi1jxJx0l+RCra0va7eeldiYDD8cfK8xFn1QJxftTd83gqAe/A8BIE03pFBez/VxRHwpWiSjdUi4TUTQS6fJkABA/+yiSMXnZY5vks3h5Ejco3zecLBNFwT+LibQYQ6SSSeTdOkkGuzR/O5s3y8RVQMe44dAzqblJvpe9Kd6bTSaWod9jS3DN5BFslskhEI36qsKIx2Qydk51/YK7B3OUY7fUB6XUIOBs4C7zvwJOAUyyO/cDn+/OOSwsLPYuuisp/B74PmCqgVAK1GitPafwemBgN89xUGBnKQGg+QjJOUgUuKhSMULOe0fSq5dd0TXhq9YVI93FSy/khDLJBfjzBCm1dsx/f5cR94qLccflsqqWvrScogdEalh/s7gr+8wTUd6dNAbnY+lD5UhMhW5qRhvRXCdkxd1yvfQ18Mm1pKbIz1+4Rh6JnDcW+qnV0XHSf797PmTVzUcAUP+OuDDXJuTaP/rKW1TeuVOMQTtw6+tRKZFAUuvEdVl3vKR+3/79l/nxRf8OgPOvj2WsocyjH3hbjI+//fg0ZjwjktavRozv9JwHKrpTiv4cYKvWem7rze00bdf5rpS6Wik1Ryk1J0kHVncLC4u9ju5ICscC/6aUmgZEEZXv90CRUipopIVBwMb2DtZazwBmgOQ+dGMcBxycInHZbZ5imJPTGmeNrMxLpmckhOlrJRfg3iFvt+ljvQleOv61GwD4n2P/zt+3Sdj7BZfJqhw7xGX1tw8DYNDrsnqnt21j/Q9kBdeGyzW6Vfa50RB1Z0v76HaRHoKvf4jTV2pAqKjQxwWb5Odcf/4QkkaGzFsn23JHDsFJid1g6NPCxLzjwgmkTHBhuELyJ9KL5cCLZ3+NyNHiruzfSZGnin+IvSMwTKSNI4cK7dvkSIill0kflR8YqSbeNm5z+G9djn1KrksdJdepP1jQpt2Bjj2WFLTWP9BaD9JaDwMuAV7XWl8KvAFcaJpdCTzd7VFaWFjsNfRG8NKNwKNKqVuAecDdvXCOAxpbLjoEgFRMVtfIDkX+OrdNO09COPFqCS4KNaTo+4tVALy3QKz3r54lPPGnvfgdxnxXvA9rvi2ei+YhSfqbSu6BdySUOTB+DMYBQOEy+VI1ocCMRzHgQQn08dx5gfx84odKgNKmqSIp5G2QcYdPr6J+hZwrkJL1p/j+T2n+/GQANh4rosghk1fxZIXYpr2w7N+PHQZAVCX53fJ/6/ymAS++9CgAw5/4umxYLqv+Zeok5p4v9+FL930NAD1vYZvj9QcLmL5GsnVvevQh4OC0LfTIpKC1fhN403xfCUzuiX4PZLSXHt14oTAehxrlpQo2y4s05MUaXnzh4TZ9VLwgFZf7FUi7rROjrFo1DIAHPyd8hpf8TEqzjX12OWsekH2JuKgDFfc4hF6bI+MpkBc/WZxDwRqZDJI5YiIKN5jScs3anwy8aMH44GK2j5PJoHipiOR1FfKyP3bofYydKJxu126QmIcVv4YNFxoRfbOMe/m2MkpGSh9b0xJD8dVCiabMC0T5fWVD+zexFbz4CYCV598BwLj3hPZtRzxGwFDEbf+5nLv088FMubtWnJKL7xO26pN++iYAv+r0zAcebESjhYVFFmzuw2eFMkOYaiSFwPgxbD1S5mjHRPeVfyCr2pLvRNvtYuyNUlNh+Xcl4KbvxM1MKN0AwM8uFxdcaYsE7Qx+voFlb0oW4fAnZWUMVlXjmtXSiwx0Zi2iyEgNnuEwNVBWYWfVZpInipHSmScu1Eg4RN9mcRmmY/I41YwUSWFsOOaP9S8DDbXcRvCqQd24RfIRltX3YeocqdkwtkwyPx+ueMM/1nEMqasZV7qujqAxJq47fxAA87/3F3bG/KniXjxq7pf40WbJi/hg4mMAVN5yjU9/54yskH6Xr6L0E7k3HtlL7aWiGhU+1P0qXfsLrKRgYWGRBSspfEZQjdmZf9snFuMaF2CqXHTz7WPFJfnrqY+1Of6ZxhirrxljOhOdf/PCviy/SQJ36r8o0sX2c019x58fxvBn38vqoz0yFR2P43pGxFFiNwg0SVBS4+Rh5L4nEoInWbCwzg9O8R6mQcZ4ecavJnDGJ9LuOyUi1TxaX8wtC4UDIb5MVv5QnUKbg5tOkXOvSoodoV4H6Vcobsod54oBtvChWaTXi6d7/veeaecqBHWuhDGn3AALqoUd5qUikX6mT3udd34nAVLuasm3CA4ehN4q4128UkLBQ+c0m3N2eJoDDnZS+IyQWi0+9GA/KXEWL1Y4CXmBXaMtNE2QB/KivNo2x39n9sWkB4nYPuhlk1/w5Ptw5KEARGrklR/xa3mh9cKF7UeRtQOPpUjViyitXCO+98vLqvLcFfz5lc8BcI8xKhasdolPlPEqMyvpEJR+Iv/MLxsGgCsffNA8jB2NooZEWzlg2jAptYOLll4MwP8c+jhff3U6AP8bkPFcP+Q13u4jqpA2Klxq3Xr00YfLwQkZb9INc7DBqg8WFhZZsJLCZ4xtZ4qInsiHRKmsliFT+m34/WJoPL3/uT5Zys+2iQjtrIwy6q+ywnl5E2rSOPh4KQARk4btLa5bvnUMpZ9IOLnHGg0Z450qlE8dDUOdSZWOiPpSN0Hcp+GaFASMjuN2VsxOMPLb2Qa62sum+sHw/WZn+oiYCMlR4+Sa/lYrnJSbEwW4pj5EzrYkO+OXVWJkvbmsbVbjmvfFCFlQ0ULAuHdXrJNr+c68KwheJP0O/SRzTP0wL7TSSCJue5H7BzaspGBhYZEFKyl8BggOHQwpL9DH4xvQBFvk5+g3U3T5TcfL6j3wmmZOGimReGunyTxe+ZPZYAJ2vApNA/40p0OilgF/XcyOs2VVLR0qbja3OJ/NUyXPIs+Unw/XpHCWiVFw85el37z1si+6qYH4yeJGDL7WOg+uc+hjRFcvWNlM7XDJQ1h/urEtJBRjfisMzzUJMQQmTeLF2sYS6qrFplBQIPcn1Krfv8+QCMSbb17CcfPPB+Cd8U8AULJQrCj/teZcKg8XY+LKWeLKTEc18X7GdnKEFMHVHy0iWi3XevRYkb7mbRy0W9d5IMBOCp8FHAd3h9APxTYb85+CsvkiHq89S0TY5EAR99ercvrOE0t65Tc/AGDNfx1NvL9hS/qaZAq1NiR6EZO6yRRbLSqgdOZmAFLrxHLfNOlISj+R/Y0DxboZSWufuchLbMJEA1YdWUKiQL73q+s8YUgddRjOZvEmrJ8iE0H57CbShpk6vN3wTha5frrzxtUSDLutVBKiNtQXEo7JfYkXiNEvE/0Aff/PZEndDC8d6kV9yrU095H+P104mKlHiFq1JpExcrr9pN/Vn5eJceg8TVMfeSWWVUuS17WH/AuA58guxHsgw6oPFhYWWbCSwl6EX+xl7XoC+bISNhoKmr4futRWyM+RKDYGuCYRofvPbMBZtBqAxbfLSqpicQ75vkQves65QH4+LceIihCqk1WwpVzE8di6RvhYjHGBw4SopW6oQzxfpJL89cYwGQpQdZgc48oH9eLOxw1r34246jxJXEpeP5HAFmnYd65IANsmyGqsHQg1yHV6lIvrTo1RsFLa1UrOFsGmAIGYrP+lH8g1zxsmN6ZqUyE40j5vo3TiFBf7ORgeDnv/yyyYkp0f8vH3Jcrx8P++lsTh0m/hZDFkVi0qQ63NMec34xg8iKCpT7GtXvaVhwyhpJUULCwsDlZYSWEvouVECSyKvr2ILRdJNGKexDCxY4xDzhZDXlooK2Ll12RlT0+oZPmMYQCohMgFRTMjpDZLnoBXbNWJQ6JQVulQnZjjgoYQNQbUXiQkK8lcaRMv1n5GZMMA0de3HpcCDHN0o6yupaMlYGnbhiKCeYZcZYGs7DqUJmpIWdMmzmfs0ZK+3fDzgaz6oqw7sTUynkSJS6hRtumAnFu5iqZT5N70nWWIYZ+RKMZCt4b6E0WkCP3zfTlPO/d20KWroW3leQCa+ms+XCUGxtIScbeGawOETUyY9pZGrSlcINc68Fsyjie2TTI7u05Yu7/DSgoWFhZZsJLCXsSGk4wz7apKlFnVItWyyuZu0FSdIt6GMT8RPbbpJHGVbTw+SHqTWfG3yDze5/aZ/Ga1BAaND0vWYeWD1/i1EgOGXiB/rfTfPCCXdEi+950pq15xSYwtk41efaKskGptEcE6OYc7TDwTJTmidIeGpLlxxEsA3Foi9SBr3+hHyWKRHmoq5fo+3WSCnSZFUIb2LFFgQrgjrm9HcfMMNVqDw5bJIpWURYXOPd94PJb8R4xQVMZR8bwxcqTTbcKcA31KOWvalwHacE+kypIUfCj2nBFfXA3AokkOweflXOUvrjXjaGDDF8WAktws55o4oG39ygMddlLYC1j9cxHvh0+Rh2/JigFETKRcbIvJK0hoSt6WB3Hdb+RFbTBiebRPA+UPi7gee1JccDueG+VPBhXPSwxD4TpFyvjrUrnyEibzjDvxsCD9Z4pbUzvK//RYluKzpAiKKnNJG8ankeVVALw05vk213TaocJydNi2r1NXKy9c7XgxVqotMgj38CYwjEs0mRc65EJctnmqiNvo4LQYJuaIfAaqTZ2GcARt7lVgsCQ16Y1bcAZKwpLnLk2tXgtr248+DOclKF4q+96fL+njueWNvg83vU2uM5CXi2veiGBQJqzyiIxjW7s9H5iw6oOFhUUWrKTQi/Ai5RJ9ZNVZVy3iKmlFjll6St8Rt+K6Cwdz1XQRzZ/fLEa38yokMOj5O48nWmVSrSdL0NAHEzP/JiZ1AAAgAElEQVSVoEZfI3UL3MnjWH+KBAklik25NmNUDNcE2HRM1GyTVfvwE5dyTrFE7jlm2VzdUsoP+gr3Y5mT2+G1zU+IuH9C5XLeSouLM5wjK38ibapBFTRRbySEeJE8atHCOC0ml6EgVySX+nSUfGPw3HKa9FH4kARYVd4cYvmvJbgo2U8+A8tXoRpMgNJRIwDIaWhk5bdGmdFlisgCRGfmETKp6gVLRQpL9YWGwWYcx8v9Dr+1gESR3IcLhgqH4/oW85vRcem6Aw1WUrCwsMiClRR6AMGKoWw/VnTcwgczWYEbTpWVLbxdVp+WXBOmuyZI/0fF3bjkB+Ju+8G0J/jzshMBuGmMSAw3vnWRdDQuRV2lrLjvnP8n03uef56WzwkvwI6xIVr6yIobbJD53rMZtJSnCW+X1f3Kc18H2s8shDVAtoQwq0UknZfrD+OvCyV46rzR8wGIp4P07Su+vaqFEhqcO0JW1eraXM/uiYpKH47jEi4Qg2ptrXFrFqRp7itGypx8UxjIsxWsWoMzX3IwksZVm1sxlNSqNXKOSnHt5qwoZMC/TN7HV7OvSJ9czbpSWfEHvyYSw7qjA4w8UVynOxaJcTGUTPhVpgKGuGZUrgQ7baF9SrwDEXZS6AHocIgqyRMiXiAPcMMwTdp7EcrkQS94V16C/q9uZcnNMhmMPFys239YcjL5UWnnkaqccubvAfgkkc/IkLxoHgV6a2z7ingHkokggQ0mb8KLigyZWIAmh/hgeWn+o3QRAA/UlXNFgRjZbq8Rl8A3ijawyRSSOecWYYIuNwVbf/zSY4yetAmAASGJKHy25gifTn7Ya+IRqBkl526qDlOwRCaiOrMtUpokYOITEtXyoqlomsZBsi32nikjqzOZHPmmkEx0q6gbzZV9CJlJwSs2s+34cj9OYmfMn/wII1ZJxe3qUabc3RKIDRAVZfUwmUDzgX6zZOIpv1Du98QcmTjeZmL7nR+AsOqDhYVFFqyk0AOonlhGmbFtefH8eaOrUUYEDT7hFUQxov2MBoanTAbkqxJpd+Elb/GzPtkFSjxD30k5Lq3VhZ1x3GBJdX5l9ngCxisXLRUxuWWbSA45Gx36nSqr/DHzvgTApRWzecDYzxY1ibuv4s2zGfyybOs3V1yojeNl36VPXceLF/wvAKNCMrbnlYagqRORJ1JB80dyvZE0NA4296CsxR+vY+5L6Rxp31IWJF4s25oGmHs0WFKWU+vWU7BKjk1HpH2wMYUytHOFK6V9c2mA+C7SE3IrRPpKr5BGxUtcPh4l0tGA08TYG/z7MALvfJp13PZ0x/f9QIWVFCwsLLLQLUlBKVUE3AUcioSC/DuwBPgbMAxYDVykta7uoIv9GumTRM/M3Zwg+LYY3gpbRdp59QTW3CrLcdgExKx4aTiuCW5cfH2mXsEZAyZk9f/yxo92ef43DcXY9ris2ocetoatjbKyVdXIpzLuwdSEBtZsNfUbVor0cO9r0yibLxJL6C0p0a7uSvOv23ZV6S/bCHlr+UfcOk3GefsxsvL+/nEp8+aGIVUs9yNm6M1KYs2smiftBm0090oFSUyQylDBxTLu+kkinUSHlBHaJBGe6eWi3wcHDUQXSruECc7KqXKJntNxiNH8yY8AcNi71wJQsMYlUS9GiPMOlWt/etTpRAzXxF8WSZ2IOyb9deeuDnh0V334A/CS1vpCpVQYybu5GXhNa/1rpdRNwE1IfckDB4anMPyhxCqn6+raZUpedal4JPKj8rA2zBTrfKpQs/SK27LaDn/i61Ty/m4NQ9QKOGnEqwA8VF/Kj2fLC6lrjNXNDMxxXL44WnScJyNSH7FpRQFNn5MX7r2/zjG9ep+7j28UiRj+ja/KtVWnm5j09LflXHVi4Nsc0PQZJ/djXUDuR58PNclmmSXTRXJN28fKoxkYFaTPR4ZxyUwKqfUbwEQflxQKo1Pd8Bz+OOZRM5LW3EzZaDxMVJH+76SJrpUJbuYYiXVY/XnFMC33Rs2X+1czIdZOLwc29lh9UEoVACdgCshqrRNa6xrgPOB+0+x+4PPdHaSFhcXeQ3ckheFISPi9SqnDgbnADUC51noTgNZ6k1Kqb/eHuW/BqRS1QBtRsyMMv0OiBdc1yEqULJdlO39M2zTcym+2lRKO+IWIuvP+s21JtPZwaf52fhkTt2NzWub7MUPFuLiproBb+kqEpPfJ0V3q1se1G6byxnOiMnkRk8F6xbcvkIIsnqTgodiJ+cVevcKxx9/3PbaNECPojPPuBOBPk05jgAmo+Gi18EcGK6RNcayZmjqRuPq0M6bGQaIK1Q8JMDnSsYTg4akT5F5eWXgVOS+KFDB3tRh7oxuDuEbNyV8jv9VND1wFwGBmdtr3gYLuGBqDwETgNq31EUAjoip0CUqpq5VSc5RSc5LEuzEMCwuLnkR3JIX1wHqttbfEPY5MCluUUv2NlNAf2NrewVrrGcAMgAJV0tXiRZ85ggMH4Oa0LWHeHuqOGQZAygTDlb8vq+E7l/2tS+c68d9nA1Dx1NWs+vwMAL67SVbqJxdOYOVp97Q55oMp9wJw/LzLAFjxrkkFLnbhqC6d1sehs6SUe2KpBBRFtypM9jVhk14drtHM+KPYMe45VwKhZh/xd78PLxpyalT099nTf8sRr18HwJwmqXmRF4pz99BXABg773oAThsnxsuJeWu49MeS1v2n6+VanhuX8T2Ga2Vl/8X0R+kKvr/yAgB+dsgz3JQU9uexZdK/O1CxoXYYAH0/EkmldK78xl2rcnFgYI8lBa31ZmCdUmq02XQqsAh4BrjSbLsSeLpbI7SwsNir6K734XrgIeN5WAlMRyaax5RSXwHWAl/s5jn2KejCPNyPF3exsXwMfEdWHS/4BuDkfxcOhA0nyE9QQXbxV4DFkwyFO7M541rPXSnSRiUfmrLu2XijRVb1ukXCj+AZ4vOXOYy+9xoAlky/re2BrTD2DrFlxAyxS2GNfCZyM3RvOdtkHIE0BEy+QPrJMrmWzZJ8sOqsu5jbMgyATxOiIl5VsJWfT5F1osgRO0Oh08SDdWJL8Ahe02a9WhXvA/mykl9fLKHN129c47tvw6+IR+U3Kz7H5029h/bwbouMt+ohsR9s/G4xxw4Wb8bKerlXU0pXU3uWiHXuApFG9Io1u7xXByK6NSlorT8Cjmxn16nd6XefhEnQSS9a2qXm1VcdTZ+X5aFrnCAP/N23/87szSP8ktRvqHgpc8x/rhSR+YSdcm8uXHEaEwulmMlb43N2ed4f/v7fAQj0M5GETTLuZJ6wK3eGU674CoOSMoltnpJ9rmiNJhXz+BiNHpHQfr/RGnnx+r5pZqKz4LoiGbdXqOWq8U/whVwxfq4yBXHu3XQcjxu36tNHrQbwq0RHnBQrCsVNOiKUiS70YjhO+qpMrqmHgzwzQgyHlSFRYwYHZWI57JlvMXas+DDL5ktex21LT+Cew8VJdt12YWyauW04J5aLm/mDtMfNePDBRjRaWFhkweY+dBHBQYZYMOj4abvtwSNWKX5gNtq4LtfIItnuSjf875K9N/BNzQnR9iMYvVUU4OZde0GZ9YM/ADDpjzcAkDZSh3Y0eWbYR30oKdmHlm3izU/EJFR5j8jtdePCpHIMXZsRBgIm8LBuaIBwnUgg8RJDULLaJRHMbl87si0t2vcNt+P8RAvjwzKoceG217e92WSS5ta12tdx/sGbd4lb85dVo7nxwauATB2Hpv4iuRzyx40k7hZxJlUq525Y6ZAcb2pBRCSgafmCQTySI47PIaYSViB+8HnGrKRgYWGRBaX1Z+8NLFAleorat80Qy/8qRCYjL5/XScsMvDLvzf8Qo9Ub4zKOGM9QNmCWEALcO+RtLlt9EgDvrRAJo3KgeHNbE6d6XAfXr/k8XyoXl+UFeRmqsEtWnQLA+5+Kuy+01ej3WqpQAbQUy1pQvLgZDIlrw0AhccnZmqSp3NRoyJd9jlksEwWKiDE6erUjkgUQNSQyxYvFFrH1SFnt68YnWHXmXZ3fqFbX1R5fxJ5i6kcXypeHythm3LEeQWzZR5qf3iI5Hq/UScblk29MgX5ysSVFMp6iX8i1qJkf99i4Piu8qh+fq7VuzwaYBTsptELTF6YAEHsyE1244mF5eaMfy8Mx8NauRbYFBw8i1V8mg5efaptUM+ot8douPVGMXSNen47bKNqcSmULcE5xnHDEMB8vKPS3Bw6TdOCmWjEIhraECJgCqm5EftdQbaaEm+dN+P6NQoH+uxWnoR8UcTlSJxOGG1TsGCNidaLQ0MV7TnoXcrZlF35pGuiSsyWb5Sluwk7yV8LAS8XY+kxlK4vqTjh/+ek8MfKVDvd3Fyd98nmiP5A4iYZfGK7GS2tYfa2wNk2/WHLFH15xFK7RgS4bKRNuVInu9PAvzqLg4Vnsz+jqpGDVBwsLiyxYSYG2KcqtU5iDA8U1ltrQiYVvJzSfN5l4gay479/acVzAn2vEXXnn7edSV2no20xZtVSR/F+4MEjSSNUpU5MhNawFt96UhmuQ82iVcUGGjUbR3EfalyyCxv6yb9TZywA4oXQZt//jLABi5vLSOcqPwPQMdg1DRQTI2RrwjYmeGzJR5BLZEcgaW8GKTJv6Yea7MWmrikafxbnuI4kPuPzf3uCHZdnkJj2JMyumoLtiMAw4BE09iaqT5XdpPl+ySMvzG9j6nGzr97v9Mw/CSgoWFhZ7BOuSbIXpa4833+r9bbsrIXiIFzpUn7Xr3AjIBPcMvv4+bviXBNHouCzHJR/Kctw4SPsGMt/FmA6gTOWktOFVCDYG0EFZrZtNSqH3f93wAIb7lU/elSpJH+UNh6GyguZsNZF8IfzaB66p1oQnHajM95ZykWKCDQGS+dLeK1MfajbnHBbIuAdHSvZmYFOMqhwxagaHi35/RGx1p/epO+iSlADgpkmtkyCnoge8T9nljKwgb6KJ4jx8rDTvamTrfgYrKVhYWGTBSgrAUf8pOQEl97bNP9hdpE82fAMFiqvGdd1a/Xb9aIJVpvbBFlmOqw+VpTe20fELtKZLxAsR3hAmUSL7Pcq1QAI/QKnRVD+KbpXPYIumTmgdCNUbd2KhxjGSRNw4NUKNmfM3jJD+o5uNzSIAqTzjkWgxRWhDmTDnyHZTI9JkRjb3hemnvQnAqKiENj++9Ug+rRKKjcZ6kU5yVaLD+7I13cjlF0iAV3tenK6g6QtTsjxKe4L08lXkGeYnt1s97fuwkwI9Mxl42HKkPOiBBDy4VJzj7Rddycazzx5N3hb5Xlcpj13uennbVBpSA+TFcbaZtG0nMxmopHyWfqLZdJy8tEWL5AXVht65qVxhWOFoKTMvdjxAzkfizvQ4I5v7aUJ1ZpJpkj5KFstLnooqakZlRysqV5GOeBVfzPgHy2PlFseZUy3pzoea6s0Bpf3JwCscm0Zx8xahQftl+fys/vs6udSMESvrlBtl8r7mP//BVQUSw/GQSWa6NH97G47LSfPkPr795zuo+qMkX10++nQZ6uABpJcsx6ItrPpgYWGRBeuS7GHUXzwVgIZBAepHSuDLqvNmdHrcEbdc65eR91bcvA2y0tUOD/guPdcxRj2dyTWIVJsApSB+una41hgLg5nIQ29bfYVpU62IlxojpXFlukH8UvRRE5QUNnbXYJMmUivtt04ybshcjSnjQLBB+kgMNOqAVuSXyApdv0OCh5TjgiGV1TGRQKYcsoI5aySl+ZVj/w+AitCuIxtfMoVrfzdy7C7bAdS+MJJZEx7P2laVbmTqY98FYNQ9QjaeXti5RLc/w7okLSws9gjWptDDKH5XXIyFRfl8WlnQaft/NEib+grtG/hyN8pqXGdqHOogBGXBJWEqo4dqlC9ZeMQkiRzIX+vlJmS3DzZCS5mxFSQ9l6O4MSETeBRIgQ6Z/SaUuXGACZluUOwYn23AdBKKhKFlz19t2o8WSSG5Npd6LQMJhEUqcJuDqIDHOy+f4UCKdJM8ip97VOpXVk4Wi+kLo19o976dGRM34+/a3ZuNwmnLOQOxN3iBamVOLuk8U4fjEqmHMexHXejsIICdFHoYqfWG0XijQ7Bmcqftb1k8DQAnAaFGU526RF7UtEjIRLdDkyFN8TwHbghy18u26sNMQtIC5ccxuIZM0ctHcMPQPFDUmbzlQb8Pz3PgGIk/UZr2VRC/ZkQ8kxodqg343wHcoCZcY8brFXj9RJK8Ajkaqs0jpsWSGejfAo3mwky5uR3xXPIXmYNNv0s+EnVixPrpXHioJKHdWt42tdx7yXc2MnYFoe0ytlSuXKhTVEi6pna3+znQYNUHCwuLLFhJoZegJo6lYNz2Tts1LJFMytg25ec3eO7BkGTvkiiEgHE7eqtatEpRZ4rZegbBSL1L/eDsVduLKGzpo8lZLz+3J4FoJ5NNGa0yRsK+mui67PoJ8b6mjPxWh4CRKJImgzLdL47aISfz4hTccCbCMdyYMUgCpLdGfCboeFjOs66mqA1VnGtEe1qCbGgWHWjSXCGHmTvpMY76obgn7/vxbwGoe3EEBWetoCMEckWN8SSKlzd+xNKrJCfliA8uAaDqvEMovr/n3NP7K6ykYGFhkQUrKfQSWspzaO5CyL1jKrSnYmRyCAz9WcCQo4Zr8I2KfsHYnEyuQU6VHNcwIOAbGD03omcsVGnlGySTJjoy2KDQhmSlfpiJ629wfKOjJz2w1VCZ5Wifmi1VKF9CoTSYvIwmY5CMbDf/D8rE/gUbjS3EgXRU2mljU9BkuBv88Ubk4sLRJOsbRFJoahERp+LZr7HqFi/zVIKv3jv8Hz679X11EjH5yJgB/vndRrHUusd5toeMfaJmi9hAYgMUu6hmf9DATgq9hNjM5ThjjQ/9mI7b+UQmOcKjCBA21aQ9oxtK4gcAgualbO7rUrBC2rWI8RztZDwRUTNRVAtlJJEditRORNCBNDgmSSreT17CUFWQcI15gc0L6nkXgo3KV21Ui0wUyXSEfEO84nEieolaKpVRhTyrpdOSSaMOFYgukkgE/YnCbx2X/mNFjT7xSWGuJFCFZuVz8sLzgGw2Kw9etONVG+WztREy8E5bY+XIEZsBWNEygODgQQB+YtTBCKs+WFhYZMFKCr2EdHU1has7LzamSkXHCKyL+rkGXkShJ3JLQ/loHGqSlDY5OHFp19RP9mkHctdJw8aBJo7AqCcqDcl8WcnD1XKeRH4mGlElZVsgrnwVoaU0exxOs/JTsX11IOT4pCyeazJhkqvSJSmiq02uhjFoagfSOeakhnbO1QrXbPPyOJyYDKJ6YyF9KkUv2FglakQkDNXPGnbtcW1u6S5Rf8lU8y0jMRSGRQLRUZeao0VSyLOSgoWFhYWgW5KCUurbwFcRhXEBUjauP/AoUAJ8CFyute44N/YARu7azklW3GbzE0S1r1d7OQSeITHVyt7gSRPBZmg2EYpetGMyX5PMN/0aF59KZvoIGVegR8Qa2qF8qrWwoVRL52i0kQJ8OcUs7DoEEWN8bBxm0rYTipRxpXq2CA+hLSFfKvCuJR3TvrRRWCArdM2OXH+/Z2RN14oxQsXSbGsU66mbzKxhhauk4T+bpN3nYkk6wrI/T6HyOkmdLnzKSAi/zexftEVErYJFIZqk8h2R06RCVOjVuR32e6BijyUFpdRA4FvAkVrrQwEHuAS4Ffid1roSqAa+0hMDtbCw2Dvork0hCOQopZJADNgEnAJ82ey/H/gpsOuKpgcotNP5nHvsOCFRfXdBJeFt8nME0h7/mXy4YUgUeyu6sfrntqoNaQwDwWblByYFjS3B81okCjTR7V7ug+leQWxD9hgjOzSNRl33ApW8PtIhjdOcLQ1EqwI0l8vYvBBsb1yBhPJtFp4HQ6UlNBqgMEcGWZPIR+d63g+xQYSHNfjnqN4oRoqA8coEmyBUL5LCC7WHA/C52Bw6wrzzfs+UKsmI/PRrf2mzv2+BnKu+KZ/6U0TsGvCihDunOuz1wMUeTwpa6w1Kqf9BKks3A/8E5gI1WmvvXq4HBnZ7lPspUvmhTtucUCwFa9/Vo0j0kdsWrDWTg3m50lExAEImQjGZr9HmffZyE1Qa4mWmfkNT9gsabFJ+lGOoXl7KxkEZVSUokjwtZcoX5Z2EV5zWI2XJTAj+C9rYSjUw6o/nhpRzerqHOS6hSBvGqKoGUQuc/CRqvfhLk4Uy/nSdsV6mFZiYBdcPewj4KeGvPyT5JUtveItRoVzaQ2Egp93JwMP6bRKdkD4izeeGyyS9elVzh+0PdHRHfSgGzgMqgAFALnBWO03bJWxQSl2tlJqjlJqT5OCr12dhsa+iO+rDacAqrfU2AKXUE0iYTpFSKmikhUH4cWbZ0FrPAGaAkKwALP3LZMbeLEQX+3u2WiA3l+iG+k7bDQlJfkS4uIXUJglbdENedSeTN5DnEqky+QKt3IRejoFHh6YDyl/dfSuhv0JngpG8DMpAEsLmNicyhaf8duHt2fu0q4mbQCmPlCUVa+We9PItPLKVVjUhEmWeezZAXpGswomEycWoC0OBYYeu2ykJQmlImLUrmFlfQg2iA6VyRV/68i3f8w2vxSdKMNI745+gKxj9wx0A1E7sx+wBQh9XHlop15w8+Gzk3XFJrgWmKqViSikFnAosAt4ATBE/rgTahpxZWFjss+iOTeF9pdTjiNsxBcxDVv7ngUeVUreYbXd3tc9R186m83Cf/QM6HocVHZes9zAxIqtUKuHgRo0xzgTwePYBp1n5rj3PSOeGMiuyx7Ccimk/WMmDx2eSzsnUjmgYkgli8vR/7zNR7Porv1c7wpM6kvnalxA8F2kyH5Ieq3TKC2/2aksqPzvSC312oy6NpvZlaZlIUtubQ2gTNu2TwLa0CvWOmLyM2qC/LZVrXJbmgdFKmKgB6k2ORGfw3JmpVfI75a5aQ+4/ZF+g2NgZqg8+SaFb3get9U+An+y0eSXQObvIAQ6dSqFTnduu6115CWJ5cRpMvD+ewc68KCrl+Fb8UL0Rx0vTBJKZRCUQI56XV9CacxHk5fEIVzzjoxvRuK2MlN6nZ7j01RIzKTjNyo+K9FSbdI72JzFv8vDzHdxMvIQ2F+A0BXBKZOaqbciwOquER1Nvzm36cBoc0t5EmONdQIDGfoa0xbyzTiv1qHajBGs0ubIzFvDYX7Lx9TelyO8o2nou0tXV7R4DsP1rRwNQPwSG/ejAS7W2EY0WFhZZsLkPewE/3HoYALf0XdBm3wjDWlwca6ahWgyNykspVmY1LkwRaDRSgSEfia0L+mqDt0IHWzIruW7JXu0JgDaqRcZImIk78PIttEMmPsJb8b2wiQASooYfGoETVyS9FVx5PI8y7mSR28rQafIcIpqwY8ZoRJBkPFMCz+s3kC+GRJ0IEMw13zeKZBEv1eRUSbu0yfxM5KtM3IZZ6p5slGKxl+a3T3ZT+p4p0NtfIhpTmza3267NcXeKdFAKqCMk+UJ/bArkuvu/AmwlBQsLiyxYSaG3oBSYmhoP/0sIFW65sK2k4MEJuJT1FXKDHTUmCMdzK7rKN+yFakxEY6H2bQnaGOISGgLG2OcFNnkrb3SLon6MrLixlRkjnWcI9NqH6gJ+9KS/zSszl6+JbJPzJwpMm5DGMZRrfnl6Y3h0GgO+29QNG2NhIkDAWD+9YCSnIEm63ozJ2EncRvk/1KRItshjqjwuBzfDDeFfZzpzrYSk44HBju0Cae0SMUFcXZUQdgU99VAZx8yPu93XZw07KfQWWhXZ6feuebsv7KAtcHjJBj7YJgzGfpKUebhJBfyn3zMkBuLgmFkjbdiT3JiLYyaNVK73EmYiG50aw+JsjPOhBuW/VN5L5rTgJzh5qoXXxi1M4RqLvUcDH4hn4iW8RCfPC5EuSvlkLB5jlBt1/VuTMmpGuj6UUTO86EgTxZjKUwRCaXMNxkMR1aRyso2bicLMd49O/qQcPwSyDcbPupy+dd0T9bdedwy1o6WPym91r1blvgSrPlhYWGTBSgp7AfmrGjttUxxsojgqkX6bzErneJLClnCb6MV0TKONSO6YKEAvUhEy7M+eUbGpn+uToHhIxVqlVhvJQgcVOZulnUev5rkoSalW9GoC7bRyI8a8Fd3Ld8iQsng1HlST46sPQc+gWudkpI2db4xWuKZQjC9tBLX/3cutUCnlu1xPGbV0517aoM/dMXaMlovp91KnzdtF8tRaRv/U5GXsWRf7JKykYGFhkQUrKewNBFSnTS4qnMM/VkkacH6hSAwNpmS7W5wiVO1VdTJGOgeCno3AW40DWtyAtJdD0DZ4SaUzKdle+bh4n5QfceRTtRnVO9DQakXPMRvjDoEGOTa0TVbeZB8RP4LbM7YCN2rWHw3RkAR1Ve2QknlB3coeYdyUqi7kjyGQJ+3TZp9LoE2QViqmiRiSlzsHv0tnyF2wicZ+gztttysM+qXyRQQVMkVz0+a+7MeuSSspWFhYZMFKCr0JZUhIF60GIKkNkYhy2jQdGYrQJ09sD9VN2VzsgWbHpylzPS9BK3efZ3VXKeWHHHtuxBZT3SlUG/D5Djy3ZTJP+56CeH9jXEgrP6Mx0OJlaXpuS03Q0MF5K3+wVb/RKq/OpKzybiSTK0HCy6TUNDQb94dHFpsCbfgZvLBsT0zRClwjNXjjcSMu6UhG8gBxrUbGdp5Ze/OW8dLH9h2Uzi9q3cVuQ8/5ZI+P3ZdhJ4VehFMiSTWp0SKmXr5abvejFa+3aRtSDuOKNgHwRr3Ug3ONf95JqsyLaYyLNDtoMyl4cQKBVjSFnkHSM8gFkpmkqnhh2u8rXCXnyFkjL1681M3kRuRli8BOreMbJkPVxrgZybwWYVNDwkuWiJdpP8LSj4SMK1ImFsF7ycO1iniRicMwSV4tfYw7NKT96tQ+yYsO4BgKjniJuc6SOAumPExneP6B4wDo1zQTZ7Xc7/1X0O8dWPXBwsIiC1ZS6CUEolHS2yUtOrhNxN1Hj4MAABV9SURBVNTZc0bJznYkBYB+ZqltaRajledqRGl0IDswKNis0MFsLketaEXBLPDciV7kIkimIoCbyhjKvIKxys24GF0jMnjnDtVlVAVPmgjVBvy+E2I39MvOaQdfmvHo29yoi9tgjIhe9qOTCZDyIyyDGckoaIysrbkfPeIXb6y3HfMAXcGAOwybc24u6arOCwAfjLCSgoWFRRaspNBLSE0ZS+CteQCklwm1V9/ZUviUL7Z/zEd1po5hnSntblx9yQLX18lDJgApHdG+wdDLgYhtCvjkrN6q6pGuNPdP47R45CdmX2siVt8eoXySlRZjLwgb+4EnAQCkoxluAy9YyZMUgl54dMj1JRs/R8FV6IA5tpWU4tk7Ag1erLI5kaN9N6wfCBUAN2IMqqaWxak5u7YMnL74XDm0aZ1026cPNHYeVHYwwk4KvQRn5kKcfuUApDZvASBvo5B+bEo10D+Y1+aYy8olJfeDT0bIcV6UoaMJmVJvPgFKWBOuNkZErwBMMJMC7UX6OXHPCKl8QpJ0K1IWL2U5OUxiI2ILcnyaeE809+IUojsULcawFzFU866j/dgCj4zFe4kDLSorJgJkMvNyMNJF4lJRdQF//86kLCQCvgriEbskStO+SvPG9P8xd6/t/WyNwGlSBs6LJ0hv27bL9gczrPpgYWGRBSsp9BJ0MkHdMcMAiD0hkkLwnU8AOPaNb7Hy9HvaHPNvuVLU4Qd9RKxVRttoaojgGt5Bx5QjCDYrPzchb50pSV+WIU3xuQvNyhvZHsjQtrVSGzypwdlgCExKNKG6bGul06oojFd81pMGgo0ZHkbv3H6bYGYcftxEqJVnX2XaeVKJt0w5vmFSZzgijVs2p6yJYaVixG1P4toZY9+9nCFa0tYDI4WtOb14WafHHaywkoKFhUUWrKTQi4g9ITn2LecIj230udkAVNyv4PSOj3ts0l0AnPPKt2RD0PV16LQpnBSpzujrnp7fGt7KmzSreLBJZcrRGaOiG9ZoY+zz7BPJglZ97UTLlsrRfnCRZ3RMR3Um/t+TTrzVPtGqtJ1n3GzOrEOebUE7GekiaCQKLygpWNJCwNC3pZLS/vyRH7dLbdcRhnxxAU65iF1WQugcdlLYC8j7WOrhxE+eCEDw9bm82yIP+rHRtsLauLDEMuf3kRqH8YVF/r7WlOleirMX3dc42CXa6HEzyjaf1Tmi/XTqpGFNciOaoJfM5JVuVIpkfnZ0oR93EMz06yUkJYrSftSkMrNB6yhKZdQdbzJzQ9pnZfZCslUqE5WZ8CYlz9HguBTnG7Vq5ItARs3qDNPGnGC+1YF7IAYk9w6s+mBhYZEFKynsBaTWiTus5nOSA1H6BvzoG1cD8Pp9d3V43NEDVgPwasMYn2gkut6kJxe4PkOyt3qH6gJ+XIIXi+CrAJFMPoRXOyKdysQAJApaGR+j3kHy4RWRSRa6vhShvdyKpCJQIpZItUUkHC9RK1yjSBm6SU+aSRZAsN7EPZhSceGagD8Or3qNR/fmOC7fG/EK0HUJ4bDfXQvAgLqZAAQHD/J/A4vOYSUFCwuLLHQqKSil7gHOAbZqrQ8120qAvwHDgNXARVrralNT8g/ANKAJuEpr/WHvDH3/Q5+HJMJRlZXCK3M7bX/HIAlmGrFkDCrsBTLJvlB9gJxtsprWSawT4VpF0gtk8lbalkzugxdc5JeMj7pEN8kj4JVcixdljJQhr3CsMTSGdwRo6ZdNuaa0Im0klkBOdip3KpYZh2+LcDTKc4k6bY2a6Zibta8g1sIFeXV0BV9fL5WbBvxmZtZ23WAjF3cHXZEU7gPO3GnbTcBrWutK4DXzP0gp+krzdzVwW88M08LCYm+hU0lBa/0vpdSwnTafB5xkvt8PvAncaLY/oLXWwCylVJFSqr/WelNPDXhvY/nvpgKw4uLbATjrrC/hfrx4j/pyW0ThTx99CM4bkqFX8dJXAVh1Zse2haWn3E3la9IuUWwIUJKKtCnz7tVUQCtfd/fqKvreilZ1Ebzw4uD2UKZWg8lbyKo56VG7m6fEaclkaXoIJBROtcnqNBJAwsu4bMW14PEuKFeRyjMn8IlYM3YIj2NBGzuFZ0/oDHPjCVZPzS4GGxxsckn2gj1h5a0ipSy7XNbBMwZM6PVz9hb21NBY7r3oWutNSnmxdwwE1rVqt95s228nhRGPG8vdxfLx4ouPdPsHD73/KQHzwI76d1PcdGPH7R0VYOVpEgE5/PGvA/KSNffLxCCASVX2AgP9eg9eMZZWuQxVGf5DzzDpqQjxPmmiW0zKtKcGmBwM5QZ8dcQzVqIzEY9e7Yh0vmGjrnJ8VcLbFt4W9FWJdJFhQo4EMoxLxdLZ+KEbADpVHTalxJf646MvAHdL1r705q27PLYn4U0GBwJ62vvQHkNpuw5ipdTViIpBlFgPD8PCwmJPsaeTwhZPLVBK9Qe8KXk90JoidxAdrIFa6xnADIAC1U5I3j4C9e5HbbbtbkHSneE2NaHHiXXQqZWV8MgfXcOcn3e+2qy88A4AKp77GqmYzMGRqkxZeC//IFGanZ7sJBTJYq/evHyEdjgZV2F+Ji/Cd0l6JRta0b0lTaqB10alFEEToOQbJLc7bcajPL7HWCaXAa9CVEkSZVyRv5zyJACX5Hdc8g2k7BvAV0+8VLravMbfF6yQ/IbUqjVtD+xhLL1tsvkmz8nx14kkF2P/rRi1py7JZ4Arzfcrgadbbb9CCaYCtfuzPcHC4mCE0nrXi7RS6hHEqFgGbAF+AjwFPAYMAdYCX9Ra7zAuyf9DvBVNwHSt9ZzOBlGgSvQUdWo3LqP3UX2lGJJm/yqzmveEMSl1yiRAQp//bZEYH68rWrerQ3x49oVIlUdakDHYpYaIscCjPgvVOH5GpDZ6OzVhP1S6daZjosizQ2TbLALJjPHRI3iJl6WJrTeSQSR7fF5WY2u4OW6GLMXUi3QiaV4/7v8AGNKFrEeAM88WCUHPW+hv25uGRQ8vb8yWJPdlA+Or+vG5WusjO2vXFe/DlzrY1eYtNl6H6zof3v6H4vslZoBfZba1fqH3FN6x6ZMn8swhEtJx3cauTQqeKnHO0rMAqGnJ4S9jhNF4fFiSDW7cIg/pU88dDaWiKxQVSmRgdTpAIm7o2H1SFuUbDp1EtokoHZEoRcjkMuig9l9+b19LX9NXQqIgvXaARCwaVSGvSPSO9466h7xA55NBrSvtv3z8JehVC7P2OYeMwl29d6MWB83KjPmUK74CQIg9fxb2FdiIRgsLiyx0qj7sDewP6oMHfczh/PPx+7O29YjIqJRfvj44cAAAz3/wQvf7bYXZccmZdowF8cI3r8HZYayDbqaIjE+dZrIjEyVGzG8M+IZDj4exNUt0tMorZpvhVNR9RTpRntoRcDlyyFoAHq54o0vj/lO1GA6fP7yP9JtK+fucQ4Qh280JoecubHtwL6K16rAvqw0euqo+WEnBwsIiCzZLcjehZn5MkytKdywgkXwNLw0n78yV3etYawJRUdRTG8SLe/aUcwB4/v3nute3weRIdh35VWfczSG3SUZhvMQQmcQgaAKUPKkgUyRW+0SvKWMXcJqUH62YMnYGL/uRgCYvTySFgYVS0u2/Kp5qM45d4YzF5xD4roRb6tQif7szbjQA6VwTujm766Qr3cX+JiHsLqykYGFhkQVrU+gG9taK4ZSV8sL813qtf4CJcySOu3pjIQFTQcrPsCzwkiFaHWCEiJwNjl8zwnNlJgbIhrMPW8CP+0k1rL5O7m6NZ8ydIsFU3PoRblM2j0LqlEl+dazQPzv1ePcYVv7a5DdcIW7pyfO+SPHZ+w+9W1dtCnZS6AaqrpaHZO5PezZ2wYNncPTUCYCrl4qa0tV04j3BQ/WlANy55ngAttXLCx2PhwiYQi75uRIHsaMqn+9MfhWA64u7H0F45rkm/mBu2/iDxvH9AYi9u5R0TecVpnsa+1NMQnuwhkYLC4s9gpUUegC/Xf2eT7Y6fa2srhun1vdY/87YStL5xopnDGrJ0ybx+gN399g5PmtMGy+//85FX93jJlA9Ru5t6V3v7fVxeTgQjItWUrCwsNgjWJdkD+A7w472V5J7h7wNwInnXk302dk90n/rWgWpU01odXOaMwbJd3XEGACOvHv+btVD+Kzxk23jAJh1eAgQCUEF5ZFs+LxcW8FrSyl9Z9cZk72Jby3/1P9+9He/AUABsz6r4ewVWPWhh6Aikg300qpMymxvi5lVXxdDZ6hBfsPS9zbj5ouovX2C1IqoO1tISE4cttznfPws8UBdGY9MkhgDt1XVZzVJJoiWfsKtEXn+g70/uFYof09iIx4Y+i9A1MKeVAk/C1j1wcLCYo9gJYUeRtMXpgDw9p/v8LftNcPU5MPYNkky98J1phjrdokuTEcVeSY1GyU+/sU3FnHESHEjPjGya1yIu4PqdBPH3PU9AIb8bGab/cGhwsez+suDGfK8FIx153/apt3exLofHgPAomv/krV9fzUutoaVFCwsLPYIVlLoJSy7bxIrP5ftMvSpup7ce1RdziihfdM5YTYfVyzbEhmG58KVkpsQWSIBUrqxidS4CgCaBoobdOO5EqEYjKS4dKxEEL6/YxgAS+cOYdS9YghML1wi/Rr7io7HcUpLAIhPMH2Wh0hFRVLp84TkMnwWgUjtYe1Pj2Hx1QeehODBRjTuA1j798MAWHzsX7O27+tGq2C/cgB0sUlECotHIN4nRv0QSUD6/+2df2xV5RnHP19ahVE1UGTaiaHUNWNqJiXGUOUPMmeQjWmMZpORjA0TxmbUmWWOhky3/bFF2Sabc7KpjGQSJIJz2Ew75tyPGME5fzAmVIoss0QpbEgyTFwLz/543yP39La03N5z7m33fBJy73nPhfPtcw9Pn/d53/M8U58/BEDv2WfQVxfOKzZx7a0LlZjO3P1vbF8oGJOUt69GrnvtIADLJ+1/f2wsOYMEnz44jlMSHinkwEDJx4TZ3/kyAFPXVH658P+J/3xmDs+tXlM0PhYjhASPFBzHKQmPFCrAnW+8xBUTiv3x3FtCIrJu8+jtGVCtHFkc2v9tW1UcHTRtDnZvvmVs290jBcdxSsIjhUoxLmToO7oHLwk+86GQb5h+p+cbSmHPfSGX88b1xbmc+w5Pp/2iyXlLqii+JDnaGFdzUgeR9G94pSUvQaOLmin13Lw97Jr81MTi5U+3n08fHMcpkSEfnZa0FlgI9JjZxXFsFfBp4L/AXkJ7uHfiuTbgJuAYcKuZdWSkfWxx/FjRcljhk3p3nxOLfBS0623+wxcAaPpccRPcsc6R33wYgG2zNg36maatSwFoXvJSLprGCsOJFNYRekMWshW42Mw+BrwOtAFIuhC4Ebgo/p2fSqopm1rHcTJnOL0k/ySpsd/YbwsOtwE3xPfXAo+a2XvAPkldwGWAZ8pK4EBrKM46n1m8fXt4eu/Vr5/Ym79n3rrwpiB6ePrd8NzBN78bfkvWrx3dpt+/4nK2fuUeABpSzWeLo6OrFn0RgHF/fBmAZjxCKIVhJRqjU2hPpg/9zj0JbDSzRyT9BNhmZo/Ecw8DT5nZ4DEenmgsla57w9r73s8Wr72fjAs2hgpCjU/2jqg57kipbTgXgN13NJ7yz3DJPaEE/Lmrix/JdgambF2nT4aklUAfsD4ZGuBjA3odScuAZQATmDgSGY7jlJGSIwVJS4DlwJVm9m4cawMws+/F4w7gW2Z20hjWI4Uycll4MvPwXWFZ7oWWx8r2T7/ee5QH/zUXgOd7wqPQh49+gLPiEuDMyT0ALKx/FShPb4qmzV8a8zsN8yLTJUlJVwPfAK5JHEJkC3CjpPGSZgDNQHmqlzqOkwtDRgqSNgDzgLOBA8BdhNWG8SQleEMeYXn8/EpgKWFa8VUze2ooER4p5EuSi2i5tItNF/yuYjpmPLEMgDP31tLwA88NZI3vaHTKz7garDVMT96bHIqtnHa0j9o/7wDA+voqJs0ZGt/R6DhOSXgzGGf4HD+Gngv7AyYUDFc+1nTKiUcKjuOkcKfgOE4KdwqO46Rwp+A4Tgp3Co7jpHCn4DhOiqrYvCTpIHAUOFRpLYSdm67jBK4jzWjWMd3Mpg71oapwCgCSXhzObivX4TpcR7Y6fPrgOE4KdwqO46SoJqfw80oLiLiONK4jzZjXUTU5BcdxqoNqihQcx6kCqsIpSLpaUqekLkkrcrrm+ZKelbRL0t8l3RbH6yVtlbQnvubSW0xSjaSXJbXH4xmStkcdGyWdnoOGSZI2Sdod7dJaCXtIuj1+JzslbZA0IS97SForqUfSzoKxAW2gwI/jfbtD0uyMdayK380OSb+SNKngXFvU0Slp/kiuXXGnEPtC3A8sAC4EFsX+EVnTB3zNzD4KzAFujtddATxjZs3AM/E4D24DdhUc3w3cG3UcJjTYyZofAU+b2UzgkqgnV3tIOg+4Fbg01gStIfQSycse6yjuczKYDRYQSg42E4oQP5Cxjnz6rZhZRf8ArUBHwXEb0FYBHb8GrgI6gYY41gB05nDtaYSb7eNAO6Eq9iGgdiAbZaThLGAfMc9UMJ6rPYDzgDeBekK9j3Zgfp72ABqBnUPZAPgZsGigz2Who9+564D18X3q/wzQAbSWet2KRwqcuAkSuuNYbsRq1S3AduAcM3sLIL5+MAcJq4E7gOPxeArwjpkl9c3ysEkTcBD4RZzGPCSpjpztYWb7ge8D/wTeAo4AfyV/exQymA0qee8uBZL6p2XVUQ1OYdi9IjK5uHQGsJlQZHbkNclP/fpJn87CriyVsEktMBt4wMxaCNvO85o6vU+cr18LzAA+BNQRwvT+VMOyWUXu3ZH0WxkO1eAUuoHzC46nkWqElh2STiM4hPVm9ngcPiCpIZ5vAHoylnEFcI2kfwCPEqYQq4FJkpJyeXnYpBvoNrOkycImgpPI2x6fAPaZ2UEz6wUeBy4nf3sUMpgNcr93Y7+VhcBii3OFcuuoBqfwF6A5ZpdPJyRMtmR9UUkCHgZ2mdkPC05tAZbE90sIuYbMMLM2M5tmZo2En/33ZrYYeJYTPTrz0PE28Kakj8ShK4HXyNkehGnDHEkT43eU6MjVHv0YzAZbgM/HVYg5wJFkmpEFufVbyTJpdAoJlU8Ssql7gZU5XXMuIcTaQehW+krUMYWQ9NsTX+tztMM8QicuCHP8F4Au4DFgfA7XnwW8GG3yBDC5EvYAvg3sBnYCvyT0GMnFHsAGQi6jl/Ab+KbBbEAI2++P9+3fCCsmWeroIuQOkvt1TcHnV0YdncCCkVzbdzQ6jpOiGqYPjuNUEe4UHMdJ4U7BcZwU7hQcx0nhTsFxnBTuFBzHSeFOwXGcFO4UHMdJ8T/z+Hm69fOSnwAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plt.imshow(\n",
" _read(path_train_img + 'ID_ffff922b9.dcm', (128, 128))[:, :, 0]\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"# Augmentations\n",
"# Flip Left Right\n",
"# Cropping\n",
"sometimes = lambda aug: iaa.Sometimes(0.25, aug)\n",
"augmentation = iaa.Sequential([ \n",
" iaa.Fliplr(0.25),\n",
" sometimes(iaa.Crop(px=(0, 25), keep_size = True, \n",
" sample_independently = False)) \n",
" ], random_order = True)"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"# Train Data Generator\n",
"class TrainDataGenerator(keras.utils.Sequence):\n",
"\n",
" def __init__(self, dataset, labels, batch_size=16, img_size=(512, 512), img_dir = path_train_img, \\\n",
" augment = False, *args, **kwargs):\n",
" self.dataset = dataset\n",
" self.ids = dataset.index\n",
" self.labels = labels\n",
" self.batch_size = batch_size\n",
" self.img_size = img_size\n",
" self.img_dir = img_dir\n",
" self.augment = augment\n",
" self.on_epoch_end()\n",
"\n",
" def __len__(self):\n",
" return int(ceil(len(self.ids) / self.batch_size))\n",
"\n",
" def __getitem__(self, index):\n",
" indices = self.indices[index*self.batch_size:(index+1)*self.batch_size]\n",
" X, Y = self.__data_generation(indices)\n",
" return X, Y\n",
"\n",
" def augmentor(self, image):\n",
" augment_img = augmentation \n",
" image_aug = augment_img.augment_image(image)\n",
" return image_aug\n",
"\n",
" def on_epoch_end(self):\n",
" self.indices = np.arange(len(self.ids))\n",
" np.random.shuffle(self.indices)\n",
" \n",
" def __data_generation(self, indices):\n",
" X = np.empty((self.batch_size, *self.img_size, 3))\n",
" Y = np.empty((self.batch_size, 6), dtype=np.float32)\n",
" \n",
" for i, index in enumerate(indices):\n",
" ID = self.ids[index]\n",
" image = _read(self.img_dir + ID, self.img_size)\n",
" if self.augment:\n",
" X[i,] = self.augmentor(image)\n",
" else:\n",
" X[i,] = image \n",
" Y[i,] = self.labels.iloc[index].values \n",
" return X, Y\n",
" \n",
"class TestDataGenerator(keras.utils.Sequence):\n",
" def __init__(self, ids, labels, batch_size = 5, img_size = (512, 512), img_dir = path_test_img, \\\n",
" *args, **kwargs):\n",
" self.ids = ids\n",
" self.labels = labels\n",
" self.batch_size = batch_size\n",
" self.img_size = img_size\n",
" self.img_dir = img_dir\n",
" self.on_epoch_end()\n",
"\n",
" def __len__(self):\n",
" return int(ceil(len(self.ids) / self.batch_size))\n",
"\n",
" def __getitem__(self, index):\n",
" indices = self.indices[index*self.batch_size:(index+1)*self.batch_size]\n",
" list_IDs_temp = [self.ids[k] for k in indices]\n",
" X = self.__data_generation(list_IDs_temp)\n",
" return X\n",
"\n",
" def on_epoch_end(self):\n",
" self.indices = np.arange(len(self.ids))\n",
"\n",
" def __data_generation(self, list_IDs_temp):\n",
" X = np.empty((self.batch_size, *self.img_size, 3))\n",
" for i, ID in enumerate(list_IDs_temp):\n",
" image = _read(self.img_dir + ID, self.img_size)\n",
" X[i,] = image \n",
" return X"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As we have seen in EDA notebook that we have very few epidural subtypes so we need oversample this sub type"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train Shape: (677018, 6)\n"
]
}
],
"source": [
"# Oversampling\n",
"epidural_df = train_final_df[train_final_df.epidural == 1]\n",
"train_final_df = pd.concat([train_final_df, epidural_df])\n",
"print('Train Shape: {}'.format(train_final_df.shape))"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>ID</th>\n",
" <th>Label</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>ID_28fbab7eb_epidural</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>ID_28fbab7eb_intraparenchymal</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>ID_28fbab7eb_intraventricular</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>ID_28fbab7eb_subarachnoid</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>ID_28fbab7eb_subdural</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" ID Label\n",
"0 ID_28fbab7eb_epidural 0.5\n",
"1 ID_28fbab7eb_intraparenchymal 0.5\n",
"2 ID_28fbab7eb_intraventricular 0.5\n",
"3 ID_28fbab7eb_subarachnoid 0.5\n",
"4 ID_28fbab7eb_subdural 0.5"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# load test set\n",
"test_df = pd.read_csv(input_folder + 'stage_1_sample_submission.csv')\n",
"test_df.head()"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(78545, 6)"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# extract subtype\n",
"test_df['sub_type'] = test_df['ID'].apply(lambda x: x.split('_')[-1])\n",
"# extract filename\n",
"test_df['file_name'] = test_df['ID'].apply(lambda x: '_'.join(x.split('_')[:2]) + '.dcm')\n",
"\n",
"test_df = pd.pivot_table(test_df.drop(columns='ID'), index=\"file_name\", \\\n",
" columns=\"sub_type\", values=\"Label\")\n",
"test_df.head()\n",
"\n",
"test_df.shape"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th>sub_type</th>\n",
" <th>any</th>\n",
" <th>epidural</th>\n",
" <th>intraparenchymal</th>\n",
" <th>intraventricular</th>\n",
" <th>subarachnoid</th>\n",
" <th>subdural</th>\n",
" </tr>\n",
" <tr>\n",
" <th>file_name</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>ID_000012eaf.dcm</th>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>ID_0000ca2f6.dcm</th>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>ID_000259ccf.dcm</th>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>ID_0002d438a.dcm</th>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>ID_00032d440.dcm</th>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
"sub_type any epidural intraparenchymal intraventricular \\\n",
"file_name \n",
"ID_000012eaf.dcm 0.5 0.5 0.5 0.5 \n",
"ID_0000ca2f6.dcm 0.5 0.5 0.5 0.5 \n",
"ID_000259ccf.dcm 0.5 0.5 0.5 0.5 \n",
"ID_0002d438a.dcm 0.5 0.5 0.5 0.5 \n",
"ID_00032d440.dcm 0.5 0.5 0.5 0.5 \n",
"\n",
"sub_type subarachnoid subdural \n",
"file_name \n",
"ID_000012eaf.dcm 0.5 0.5 \n",
"ID_0000ca2f6.dcm 0.5 0.5 \n",
"ID_000259ccf.dcm 0.5 0.5 \n",
"ID_0002d438a.dcm 0.5 0.5 \n",
"ID_00032d440.dcm 0.5 0.5 "
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"test_df.head()"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Downloading data from https://github.com/Callidior/keras-applications/releases/download/efficientnet/efficientnet-b0_weights_tf_dim_ordering_tf_kernels_autoaugment_notop.h5\n",
"16809984/16804768 [==============================] - 1s 0us/step\n",
"Model: \"model_1\"\n",
"__________________________________________________________________________________________________\n",
"Layer (type) Output Shape Param # Connected to \n",
"==================================================================================================\n",
"input_1 (InputLayer) (None, 256, 256, 3) 0 \n",
"__________________________________________________________________________________________________\n",
"stem_conv (Conv2D) (None, 128, 128, 32) 864 input_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"stem_bn (BatchNormalization) (None, 128, 128, 32) 128 stem_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"stem_activation (Activation) (None, 128, 128, 32) 0 stem_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block1a_dwconv (DepthwiseConv2D (None, 128, 128, 32) 288 stem_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block1a_bn (BatchNormalization) (None, 128, 128, 32) 128 block1a_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block1a_activation (Activation) (None, 128, 128, 32) 0 block1a_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block1a_se_squeeze (GlobalAvera (None, 32) 0 block1a_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block1a_se_reshape (Reshape) (None, 1, 1, 32) 0 block1a_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block1a_se_reduce (Conv2D) (None, 1, 1, 8) 264 block1a_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block1a_se_expand (Conv2D) (None, 1, 1, 32) 288 block1a_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block1a_se_excite (Multiply) (None, 128, 128, 32) 0 block1a_activation[0][0] \n",
" block1a_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block1a_project_conv (Conv2D) (None, 128, 128, 16) 512 block1a_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block1a_project_bn (BatchNormal (None, 128, 128, 16) 64 block1a_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2a_expand_conv (Conv2D) (None, 128, 128, 96) 1536 block1a_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2a_expand_bn (BatchNormali (None, 128, 128, 96) 384 block2a_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2a_expand_activation (Acti (None, 128, 128, 96) 0 block2a_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2a_dwconv (DepthwiseConv2D (None, 64, 64, 96) 864 block2a_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2a_bn (BatchNormalization) (None, 64, 64, 96) 384 block2a_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2a_activation (Activation) (None, 64, 64, 96) 0 block2a_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2a_se_squeeze (GlobalAvera (None, 96) 0 block2a_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2a_se_reshape (Reshape) (None, 1, 1, 96) 0 block2a_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2a_se_reduce (Conv2D) (None, 1, 1, 4) 388 block2a_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2a_se_expand (Conv2D) (None, 1, 1, 96) 480 block2a_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2a_se_excite (Multiply) (None, 64, 64, 96) 0 block2a_activation[0][0] \n",
" block2a_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2a_project_conv (Conv2D) (None, 64, 64, 24) 2304 block2a_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2a_project_bn (BatchNormal (None, 64, 64, 24) 96 block2a_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_expand_conv (Conv2D) (None, 64, 64, 144) 3456 block2a_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_expand_bn (BatchNormali (None, 64, 64, 144) 576 block2b_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_expand_activation (Acti (None, 64, 64, 144) 0 block2b_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_dwconv (DepthwiseConv2D (None, 64, 64, 144) 1296 block2b_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_bn (BatchNormalization) (None, 64, 64, 144) 576 block2b_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_activation (Activation) (None, 64, 64, 144) 0 block2b_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_se_squeeze (GlobalAvera (None, 144) 0 block2b_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_se_reshape (Reshape) (None, 1, 1, 144) 0 block2b_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_se_reduce (Conv2D) (None, 1, 1, 6) 870 block2b_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_se_expand (Conv2D) (None, 1, 1, 144) 1008 block2b_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_se_excite (Multiply) (None, 64, 64, 144) 0 block2b_activation[0][0] \n",
" block2b_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_project_conv (Conv2D) (None, 64, 64, 24) 3456 block2b_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_project_bn (BatchNormal (None, 64, 64, 24) 96 block2b_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_drop (FixedDropout) (None, 64, 64, 24) 0 block2b_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block2b_add (Add) (None, 64, 64, 24) 0 block2b_drop[0][0] \n",
" block2a_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3a_expand_conv (Conv2D) (None, 64, 64, 144) 3456 block2b_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3a_expand_bn (BatchNormali (None, 64, 64, 144) 576 block3a_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3a_expand_activation (Acti (None, 64, 64, 144) 0 block3a_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3a_dwconv (DepthwiseConv2D (None, 32, 32, 144) 3600 block3a_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3a_bn (BatchNormalization) (None, 32, 32, 144) 576 block3a_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3a_activation (Activation) (None, 32, 32, 144) 0 block3a_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3a_se_squeeze (GlobalAvera (None, 144) 0 block3a_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3a_se_reshape (Reshape) (None, 1, 1, 144) 0 block3a_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3a_se_reduce (Conv2D) (None, 1, 1, 6) 870 block3a_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3a_se_expand (Conv2D) (None, 1, 1, 144) 1008 block3a_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3a_se_excite (Multiply) (None, 32, 32, 144) 0 block3a_activation[0][0] \n",
" block3a_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3a_project_conv (Conv2D) (None, 32, 32, 40) 5760 block3a_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3a_project_bn (BatchNormal (None, 32, 32, 40) 160 block3a_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_expand_conv (Conv2D) (None, 32, 32, 240) 9600 block3a_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_expand_bn (BatchNormali (None, 32, 32, 240) 960 block3b_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_expand_activation (Acti (None, 32, 32, 240) 0 block3b_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_dwconv (DepthwiseConv2D (None, 32, 32, 240) 6000 block3b_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_bn (BatchNormalization) (None, 32, 32, 240) 960 block3b_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_activation (Activation) (None, 32, 32, 240) 0 block3b_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_se_squeeze (GlobalAvera (None, 240) 0 block3b_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_se_reshape (Reshape) (None, 1, 1, 240) 0 block3b_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_se_reduce (Conv2D) (None, 1, 1, 10) 2410 block3b_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_se_expand (Conv2D) (None, 1, 1, 240) 2640 block3b_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_se_excite (Multiply) (None, 32, 32, 240) 0 block3b_activation[0][0] \n",
" block3b_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_project_conv (Conv2D) (None, 32, 32, 40) 9600 block3b_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_project_bn (BatchNormal (None, 32, 32, 40) 160 block3b_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_drop (FixedDropout) (None, 32, 32, 40) 0 block3b_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block3b_add (Add) (None, 32, 32, 40) 0 block3b_drop[0][0] \n",
" block3a_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4a_expand_conv (Conv2D) (None, 32, 32, 240) 9600 block3b_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4a_expand_bn (BatchNormali (None, 32, 32, 240) 960 block4a_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4a_expand_activation (Acti (None, 32, 32, 240) 0 block4a_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4a_dwconv (DepthwiseConv2D (None, 16, 16, 240) 2160 block4a_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4a_bn (BatchNormalization) (None, 16, 16, 240) 960 block4a_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4a_activation (Activation) (None, 16, 16, 240) 0 block4a_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4a_se_squeeze (GlobalAvera (None, 240) 0 block4a_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4a_se_reshape (Reshape) (None, 1, 1, 240) 0 block4a_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4a_se_reduce (Conv2D) (None, 1, 1, 10) 2410 block4a_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4a_se_expand (Conv2D) (None, 1, 1, 240) 2640 block4a_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4a_se_excite (Multiply) (None, 16, 16, 240) 0 block4a_activation[0][0] \n",
" block4a_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4a_project_conv (Conv2D) (None, 16, 16, 80) 19200 block4a_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4a_project_bn (BatchNormal (None, 16, 16, 80) 320 block4a_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_expand_conv (Conv2D) (None, 16, 16, 480) 38400 block4a_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_expand_bn (BatchNormali (None, 16, 16, 480) 1920 block4b_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_expand_activation (Acti (None, 16, 16, 480) 0 block4b_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_dwconv (DepthwiseConv2D (None, 16, 16, 480) 4320 block4b_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_bn (BatchNormalization) (None, 16, 16, 480) 1920 block4b_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_activation (Activation) (None, 16, 16, 480) 0 block4b_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_se_squeeze (GlobalAvera (None, 480) 0 block4b_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_se_reshape (Reshape) (None, 1, 1, 480) 0 block4b_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_se_reduce (Conv2D) (None, 1, 1, 20) 9620 block4b_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_se_expand (Conv2D) (None, 1, 1, 480) 10080 block4b_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_se_excite (Multiply) (None, 16, 16, 480) 0 block4b_activation[0][0] \n",
" block4b_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_project_conv (Conv2D) (None, 16, 16, 80) 38400 block4b_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_project_bn (BatchNormal (None, 16, 16, 80) 320 block4b_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_drop (FixedDropout) (None, 16, 16, 80) 0 block4b_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4b_add (Add) (None, 16, 16, 80) 0 block4b_drop[0][0] \n",
" block4a_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_expand_conv (Conv2D) (None, 16, 16, 480) 38400 block4b_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_expand_bn (BatchNormali (None, 16, 16, 480) 1920 block4c_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_expand_activation (Acti (None, 16, 16, 480) 0 block4c_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_dwconv (DepthwiseConv2D (None, 16, 16, 480) 4320 block4c_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_bn (BatchNormalization) (None, 16, 16, 480) 1920 block4c_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_activation (Activation) (None, 16, 16, 480) 0 block4c_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_se_squeeze (GlobalAvera (None, 480) 0 block4c_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_se_reshape (Reshape) (None, 1, 1, 480) 0 block4c_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_se_reduce (Conv2D) (None, 1, 1, 20) 9620 block4c_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_se_expand (Conv2D) (None, 1, 1, 480) 10080 block4c_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_se_excite (Multiply) (None, 16, 16, 480) 0 block4c_activation[0][0] \n",
" block4c_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_project_conv (Conv2D) (None, 16, 16, 80) 38400 block4c_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_project_bn (BatchNormal (None, 16, 16, 80) 320 block4c_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_drop (FixedDropout) (None, 16, 16, 80) 0 block4c_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block4c_add (Add) (None, 16, 16, 80) 0 block4c_drop[0][0] \n",
" block4b_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5a_expand_conv (Conv2D) (None, 16, 16, 480) 38400 block4c_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5a_expand_bn (BatchNormali (None, 16, 16, 480) 1920 block5a_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5a_expand_activation (Acti (None, 16, 16, 480) 0 block5a_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5a_dwconv (DepthwiseConv2D (None, 16, 16, 480) 12000 block5a_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5a_bn (BatchNormalization) (None, 16, 16, 480) 1920 block5a_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5a_activation (Activation) (None, 16, 16, 480) 0 block5a_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5a_se_squeeze (GlobalAvera (None, 480) 0 block5a_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5a_se_reshape (Reshape) (None, 1, 1, 480) 0 block5a_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5a_se_reduce (Conv2D) (None, 1, 1, 20) 9620 block5a_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5a_se_expand (Conv2D) (None, 1, 1, 480) 10080 block5a_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5a_se_excite (Multiply) (None, 16, 16, 480) 0 block5a_activation[0][0] \n",
" block5a_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5a_project_conv (Conv2D) (None, 16, 16, 112) 53760 block5a_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5a_project_bn (BatchNormal (None, 16, 16, 112) 448 block5a_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_expand_conv (Conv2D) (None, 16, 16, 672) 75264 block5a_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_expand_bn (BatchNormali (None, 16, 16, 672) 2688 block5b_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_expand_activation (Acti (None, 16, 16, 672) 0 block5b_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_dwconv (DepthwiseConv2D (None, 16, 16, 672) 16800 block5b_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_bn (BatchNormalization) (None, 16, 16, 672) 2688 block5b_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_activation (Activation) (None, 16, 16, 672) 0 block5b_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_se_squeeze (GlobalAvera (None, 672) 0 block5b_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_se_reshape (Reshape) (None, 1, 1, 672) 0 block5b_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_se_reduce (Conv2D) (None, 1, 1, 28) 18844 block5b_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_se_expand (Conv2D) (None, 1, 1, 672) 19488 block5b_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_se_excite (Multiply) (None, 16, 16, 672) 0 block5b_activation[0][0] \n",
" block5b_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_project_conv (Conv2D) (None, 16, 16, 112) 75264 block5b_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_project_bn (BatchNormal (None, 16, 16, 112) 448 block5b_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_drop (FixedDropout) (None, 16, 16, 112) 0 block5b_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5b_add (Add) (None, 16, 16, 112) 0 block5b_drop[0][0] \n",
" block5a_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_expand_conv (Conv2D) (None, 16, 16, 672) 75264 block5b_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_expand_bn (BatchNormali (None, 16, 16, 672) 2688 block5c_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_expand_activation (Acti (None, 16, 16, 672) 0 block5c_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_dwconv (DepthwiseConv2D (None, 16, 16, 672) 16800 block5c_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_bn (BatchNormalization) (None, 16, 16, 672) 2688 block5c_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_activation (Activation) (None, 16, 16, 672) 0 block5c_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_se_squeeze (GlobalAvera (None, 672) 0 block5c_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_se_reshape (Reshape) (None, 1, 1, 672) 0 block5c_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_se_reduce (Conv2D) (None, 1, 1, 28) 18844 block5c_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_se_expand (Conv2D) (None, 1, 1, 672) 19488 block5c_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_se_excite (Multiply) (None, 16, 16, 672) 0 block5c_activation[0][0] \n",
" block5c_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_project_conv (Conv2D) (None, 16, 16, 112) 75264 block5c_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_project_bn (BatchNormal (None, 16, 16, 112) 448 block5c_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_drop (FixedDropout) (None, 16, 16, 112) 0 block5c_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block5c_add (Add) (None, 16, 16, 112) 0 block5c_drop[0][0] \n",
" block5b_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6a_expand_conv (Conv2D) (None, 16, 16, 672) 75264 block5c_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6a_expand_bn (BatchNormali (None, 16, 16, 672) 2688 block6a_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6a_expand_activation (Acti (None, 16, 16, 672) 0 block6a_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6a_dwconv (DepthwiseConv2D (None, 8, 8, 672) 16800 block6a_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6a_bn (BatchNormalization) (None, 8, 8, 672) 2688 block6a_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6a_activation (Activation) (None, 8, 8, 672) 0 block6a_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6a_se_squeeze (GlobalAvera (None, 672) 0 block6a_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6a_se_reshape (Reshape) (None, 1, 1, 672) 0 block6a_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6a_se_reduce (Conv2D) (None, 1, 1, 28) 18844 block6a_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6a_se_expand (Conv2D) (None, 1, 1, 672) 19488 block6a_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6a_se_excite (Multiply) (None, 8, 8, 672) 0 block6a_activation[0][0] \n",
" block6a_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6a_project_conv (Conv2D) (None, 8, 8, 192) 129024 block6a_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6a_project_bn (BatchNormal (None, 8, 8, 192) 768 block6a_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_expand_conv (Conv2D) (None, 8, 8, 1152) 221184 block6a_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_expand_bn (BatchNormali (None, 8, 8, 1152) 4608 block6b_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_expand_activation (Acti (None, 8, 8, 1152) 0 block6b_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_dwconv (DepthwiseConv2D (None, 8, 8, 1152) 28800 block6b_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_bn (BatchNormalization) (None, 8, 8, 1152) 4608 block6b_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_activation (Activation) (None, 8, 8, 1152) 0 block6b_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_se_squeeze (GlobalAvera (None, 1152) 0 block6b_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_se_reshape (Reshape) (None, 1, 1, 1152) 0 block6b_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_se_reduce (Conv2D) (None, 1, 1, 48) 55344 block6b_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_se_expand (Conv2D) (None, 1, 1, 1152) 56448 block6b_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_se_excite (Multiply) (None, 8, 8, 1152) 0 block6b_activation[0][0] \n",
" block6b_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_project_conv (Conv2D) (None, 8, 8, 192) 221184 block6b_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_project_bn (BatchNormal (None, 8, 8, 192) 768 block6b_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_drop (FixedDropout) (None, 8, 8, 192) 0 block6b_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6b_add (Add) (None, 8, 8, 192) 0 block6b_drop[0][0] \n",
" block6a_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_expand_conv (Conv2D) (None, 8, 8, 1152) 221184 block6b_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_expand_bn (BatchNormali (None, 8, 8, 1152) 4608 block6c_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_expand_activation (Acti (None, 8, 8, 1152) 0 block6c_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_dwconv (DepthwiseConv2D (None, 8, 8, 1152) 28800 block6c_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_bn (BatchNormalization) (None, 8, 8, 1152) 4608 block6c_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_activation (Activation) (None, 8, 8, 1152) 0 block6c_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_se_squeeze (GlobalAvera (None, 1152) 0 block6c_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_se_reshape (Reshape) (None, 1, 1, 1152) 0 block6c_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_se_reduce (Conv2D) (None, 1, 1, 48) 55344 block6c_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_se_expand (Conv2D) (None, 1, 1, 1152) 56448 block6c_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_se_excite (Multiply) (None, 8, 8, 1152) 0 block6c_activation[0][0] \n",
" block6c_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_project_conv (Conv2D) (None, 8, 8, 192) 221184 block6c_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_project_bn (BatchNormal (None, 8, 8, 192) 768 block6c_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_drop (FixedDropout) (None, 8, 8, 192) 0 block6c_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6c_add (Add) (None, 8, 8, 192) 0 block6c_drop[0][0] \n",
" block6b_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_expand_conv (Conv2D) (None, 8, 8, 1152) 221184 block6c_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_expand_bn (BatchNormali (None, 8, 8, 1152) 4608 block6d_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_expand_activation (Acti (None, 8, 8, 1152) 0 block6d_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_dwconv (DepthwiseConv2D (None, 8, 8, 1152) 28800 block6d_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_bn (BatchNormalization) (None, 8, 8, 1152) 4608 block6d_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_activation (Activation) (None, 8, 8, 1152) 0 block6d_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_se_squeeze (GlobalAvera (None, 1152) 0 block6d_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_se_reshape (Reshape) (None, 1, 1, 1152) 0 block6d_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_se_reduce (Conv2D) (None, 1, 1, 48) 55344 block6d_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_se_expand (Conv2D) (None, 1, 1, 1152) 56448 block6d_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_se_excite (Multiply) (None, 8, 8, 1152) 0 block6d_activation[0][0] \n",
" block6d_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_project_conv (Conv2D) (None, 8, 8, 192) 221184 block6d_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_project_bn (BatchNormal (None, 8, 8, 192) 768 block6d_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_drop (FixedDropout) (None, 8, 8, 192) 0 block6d_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block6d_add (Add) (None, 8, 8, 192) 0 block6d_drop[0][0] \n",
" block6c_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block7a_expand_conv (Conv2D) (None, 8, 8, 1152) 221184 block6d_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block7a_expand_bn (BatchNormali (None, 8, 8, 1152) 4608 block7a_expand_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block7a_expand_activation (Acti (None, 8, 8, 1152) 0 block7a_expand_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block7a_dwconv (DepthwiseConv2D (None, 8, 8, 1152) 10368 block7a_expand_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block7a_bn (BatchNormalization) (None, 8, 8, 1152) 4608 block7a_dwconv[0][0] \n",
"__________________________________________________________________________________________________\n",
"block7a_activation (Activation) (None, 8, 8, 1152) 0 block7a_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"block7a_se_squeeze (GlobalAvera (None, 1152) 0 block7a_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"block7a_se_reshape (Reshape) (None, 1, 1, 1152) 0 block7a_se_squeeze[0][0] \n",
"__________________________________________________________________________________________________\n",
"block7a_se_reduce (Conv2D) (None, 1, 1, 48) 55344 block7a_se_reshape[0][0] \n",
"__________________________________________________________________________________________________\n",
"block7a_se_expand (Conv2D) (None, 1, 1, 1152) 56448 block7a_se_reduce[0][0] \n",
"__________________________________________________________________________________________________\n",
"block7a_se_excite (Multiply) (None, 8, 8, 1152) 0 block7a_activation[0][0] \n",
" block7a_se_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block7a_project_conv (Conv2D) (None, 8, 8, 320) 368640 block7a_se_excite[0][0] \n",
"__________________________________________________________________________________________________\n",
"block7a_project_bn (BatchNormal (None, 8, 8, 320) 1280 block7a_project_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"top_conv (Conv2D) (None, 8, 8, 1280) 409600 block7a_project_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"top_bn (BatchNormalization) (None, 8, 8, 1280) 5120 top_conv[0][0] \n",
"__________________________________________________________________________________________________\n",
"top_activation (Activation) (None, 8, 8, 1280) 0 top_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"avg_pool (GlobalAveragePooling2 (None, 1280) 0 top_activation[0][0] \n",
"__________________________________________________________________________________________________\n",
"dropout_1 (Dropout) (None, 1280) 0 avg_pool[0][0] \n",
"__________________________________________________________________________________________________\n",
"dense_1 (Dense) (None, 6) 7686 dropout_1[0][0] \n",
"==================================================================================================\n",
"Total params: 4,057,250\n",
"Trainable params: 4,015,234\n",
"Non-trainable params: 42,016\n",
"__________________________________________________________________________________________________\n"
]
}
],
"source": [
"base_model = efn.EfficientNetB0(weights = 'imagenet', include_top = False, \\\n",
" pooling = 'avg', input_shape = (HEIGHT, WIDTH, 3))\n",
"x = base_model.output\n",
"x = Dropout(0.125)(x)\n",
"output_layer = Dense(6, activation = 'sigmoid')(x)\n",
"model = Model(inputs=base_model.input, outputs=output_layer)\n",
"model.compile(optimizer = Adam(learning_rate = 0.0001), \n",
" loss = 'binary_crossentropy',\n",
" metrics = ['acc', tf.keras.metrics.AUC()])\n",
"model.summary()"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(636396, 40622)"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# https://github.com/trent-b/iterative-stratification\n",
"# Mutlilabel stratification\n",
"splits = MultilabelStratifiedShuffleSplit(n_splits = 2, test_size = TEST_SIZE, random_state = SEED)\n",
"file_names = train_final_df.index\n",
"labels = train_final_df.values\n",
"# Lets take only the first split\n",
"split = next(splits.split(file_names, labels))\n",
"train_idx = split[0]\n",
"valid_idx = split[1]\n",
"submission_predictions = []\n",
"len(train_idx), len(valid_idx)"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [],
"source": [
"# train data generator\n",
"data_generator_train = TrainDataGenerator(train_final_df.iloc[train_idx], \n",
" train_final_df.iloc[train_idx], \n",
" TRAIN_BATCH_SIZE, \n",
" (WIDTH, HEIGHT),\n",
" augment = True)\n",
"\n",
"# validation data generator\n",
"data_generator_val = TrainDataGenerator(train_final_df.iloc[valid_idx], \n",
" train_final_df.iloc[valid_idx], \n",
" VALID_BATCH_SIZE, \n",
" (WIDTH, HEIGHT),\n",
" augment = False)"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(19888, 635)"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(data_generator_train), len(data_generator_val)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Competition evaluation metric is evaluated based on weighted log loss but we haven't given weights for each subtype but as per discussion from this thread https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/109526#latest-630190 any has a wieght of 2 than other types below sample is taken from the discussion threas"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [],
"source": [
"from keras import backend as K\n",
"\n",
"def weighted_log_loss(y_true, y_pred):\n",
" \"\"\"\n",
" Can be used as the loss function in model.compile()\n",
" ---------------------------------------------------\n",
" \"\"\"\n",
" \n",
" class_weights = np.array([2., 1., 1., 1., 1., 1.])\n",
" \n",
" eps = K.epsilon()\n",
" \n",
" y_pred = K.clip(y_pred, eps, 1.0-eps)\n",
"\n",
" out = -( y_true * K.log( y_pred) * class_weights\n",
" + (1.0 - y_true) * K.log(1.0 - y_pred) * class_weights)\n",
" \n",
" return K.mean(out, axis=-1)\n",
"\n",
"\n",
"def _normalized_weighted_average(arr, weights=None):\n",
" \"\"\"\n",
" A simple Keras implementation that mimics that of \n",
" numpy.average(), specifically for this competition\n",
" \"\"\"\n",
" \n",
" if weights is not None:\n",
" scl = K.sum(weights)\n",
" weights = K.expand_dims(weights, axis=1)\n",
" return K.sum(K.dot(arr, weights), axis=1) / scl\n",
" return K.mean(arr, axis=1)\n",
"\n",
"\n",
"def weighted_loss(y_true, y_pred):\n",
" \"\"\"\n",
" Will be used as the metric in model.compile()\n",
" ---------------------------------------------\n",
" \n",
" Similar to the custom loss function 'weighted_log_loss()' above\n",
" but with normalized weights, which should be very similar \n",
" to the official competition metric:\n",
" https://www.kaggle.com/kambarakun/lb-probe-weights-n-of-positives-scoring\n",
" and hence:\n",
" sklearn.metrics.log_loss with sample weights\n",
" \"\"\"\n",
" \n",
" class_weights = K.variable([2., 1., 1., 1., 1., 1.])\n",
" \n",
" eps = K.epsilon()\n",
" \n",
" y_pred = K.clip(y_pred, eps, 1.0-eps)\n",
"\n",
" loss = -( y_true * K.log( y_pred)\n",
" + (1.0 - y_true) * K.log(1.0 - y_pred))\n",
" \n",
" loss_samples = _normalized_weighted_average(loss, class_weights)\n",
" \n",
" return K.mean(loss_samples)\n",
"\n",
"\n",
"def weighted_log_loss_metric(trues, preds):\n",
" \"\"\"\n",
" Will be used to calculate the log loss \n",
" of the validation set in PredictionCheckpoint()\n",
" ------------------------------------------\n",
" \"\"\"\n",
" class_weights = [2., 1., 1., 1., 1., 1.]\n",
" \n",
" epsilon = 1e-7\n",
" \n",
" preds = np.clip(preds, epsilon, 1-epsilon)\n",
" loss = trues * np.log(preds) + (1 - trues) * np.log(1 - preds)\n",
" loss_samples = np.average(loss, axis=1, weights=class_weights)\n",
"\n",
" return - loss_samples.mean()"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [],
"source": [
"filepath=\"model.h5\"\n",
"checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, \\\n",
" save_best_only=True, mode='min')\n",
"\n",
"callbacks_list = [checkpoint]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For a single epoch we are going to train only last 5 layers of Efficient. Since we have a large number of images around 600k so its better to train the all the layers on the whole train dataset but due its high computation resources required to train we only goin to train last five layers on whole dataset and for rest of epochs we only train on a sample of dataset but will train all the layers."
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [],
"source": [
"train = False"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [],
"source": [
"if train:\n",
" if not os.path.isfile('../input/orginal-087-eff/model.h5'):\n",
" for layer in model.layers[:-5]:\n",
" layer.trainable = False\n",
" model.compile(optimizer = Adam(learning_rate = 0.0001), \n",
" loss = 'binary_crossentropy',\n",
" metrics = ['acc'])\n",
"\n",
" model.fit_generator(generator = data_generator_train,\n",
" validation_data = data_generator_val,\n",
" epochs = 2,\n",
" callbacks = callbacks_list,\n",
" verbose = 1)"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {},
"outputs": [],
"source": [
"if train:\n",
" for base_layer in model.layers[:-1]:\n",
" base_layer.trainable = True\n",
"\n",
" model.load_weights('model.h5')\n",
"\n",
" model.compile(optimizer = Adam(learning_rate = 0.0004), \n",
" loss = 'binary_crossentropy',\n",
" metrics = ['acc'])\n",
" model.fit_generator(generator = data_generator_train,\n",
" validation_data = data_generator_val,\n",
" steps_per_epoch=len(data_generator_train)/6,\n",
" epochs = 10,\n",
" callbacks = callbacks_list,\n",
" verbose = 1)"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting gdown\n",
" Downloading https://files.pythonhosted.org/packages/b0/b4/a8e9d0b02bca6aa53087001abf064cc9992bda11bd6840875b8098d93573/gdown-3.8.3.tar.gz\n",
"Requirement already satisfied: filelock in /opt/conda/lib/python3.6/site-packages (from gdown) (3.0.12)\n",
"Requirement already satisfied: requests in /opt/conda/lib/python3.6/site-packages (from gdown) (2.22.0)\n",
"Requirement already satisfied: six in /opt/conda/lib/python3.6/site-packages (from gdown) (1.12.0)\n",
"Requirement already satisfied: tqdm in /opt/conda/lib/python3.6/site-packages (from gdown) (4.36.1)\n",
"Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.6/site-packages (from requests->gdown) (1.24.2)\n",
"Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.6/site-packages (from requests->gdown) (3.0.4)\n",
"Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.6/site-packages (from requests->gdown) (2.8)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.6/site-packages (from requests->gdown) (2019.9.11)\n",
"Building wheels for collected packages: gdown\n",
" Building wheel for gdown (setup.py) ... \u001b[?25ldone\n",
"\u001b[?25h Created wheel for gdown: filename=gdown-3.8.3-cp36-none-any.whl size=8850 sha256=ca7bf131547dd1503032ee6ec7567ff06fb7ddad8d44a32f00f874aadbd01a5e\n",
" Stored in directory: /tmp/.cache/pip/wheels/a7/9d/16/9e0bda9a327ff2cddaee8de48a27553fb1efce73133593d066\n",
"Successfully built gdown\n",
"Installing collected packages: gdown\n",
"Successfully installed gdown-3.8.3\n"
]
}
],
"source": [
"!pip install gdown"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Downloading...\n",
"From: https://drive.google.com/uc?id=1kZmMCCBOWSjCZjz2XWaouDIj5gFn2D-q\n",
"To: /kaggle/working/model (4).h5\n",
"49.2MB [00:03, 14.6MB/s]\n"
]
}
],
"source": [
"!gdown https://drive.google.com/uc?id=1kZmMCCBOWSjCZjz2XWaouDIj5gFn2D-q"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {},
"outputs": [],
"source": [
"!cp \"model (4).h5\" model.h5"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1228/1228 [==============================] - 856s 697ms/step\n"
]
},
{
"data": {
"text/plain": [
"(78592, 6)"
]
},
"execution_count": 37,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.load_weights('model.h5')\n",
"\n",
"preds = model.predict_generator(TestDataGenerator(test_df.index, None, VALID_BATCH_SIZE, \\\n",
" (WIDTH, HEIGHT), path_test_img), \n",
" verbose=1)\n",
"preds.shape"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {},
"outputs": [],
"source": [
"from tqdm import tqdm"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {},
"outputs": [],
"source": [
"cols = list(train_final_df.columns)"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████▉| 78545/78592 [00:01<00:00, 51807.96it/s]\n"
]
}
],
"source": [
"# We have preditions for each of the image\n",
"# We need to make 6 rows for each of file according to the subtype\n",
"ids = []\n",
"values = []\n",
"for i, j in tqdm(zip(preds, test_df.index.to_list()), total=preds.shape[0]):\n",
"# print(i, j)\n",
" # i=[any_prob, epidural_prob, intraparenchymal_prob, intraventricular_prob, subarachnoid_prob, subdural_prob]\n",
" # j = filename ==> ID_xyz.dcm\n",
" for k in range(i.shape[0]):\n",
" ids.append([j.replace('.dcm', '_' + cols[k])])\n",
" values.append(i[k]) "
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>0</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>ID_000012eaf_any</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>ID_000012eaf_epidural</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>ID_000012eaf_intraparenchymal</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>ID_000012eaf_intraventricular</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>ID_000012eaf_subarachnoid</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" 0\n",
"0 ID_000012eaf_any\n",
"1 ID_000012eaf_epidural\n",
"2 ID_000012eaf_intraparenchymal\n",
"3 ID_000012eaf_intraventricular\n",
"4 ID_000012eaf_subarachnoid"
]
},
"execution_count": 41,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = pd.DataFrame(data=ids)\n",
"df.head()"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>ID</th>\n",
" <th>Label</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>ID_28fbab7eb_epidural</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>ID_28fbab7eb_intraparenchymal</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>ID_28fbab7eb_intraventricular</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>ID_28fbab7eb_subarachnoid</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>ID_28fbab7eb_subdural</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" ID Label\n",
"0 ID_28fbab7eb_epidural 0.5\n",
"1 ID_28fbab7eb_intraparenchymal 0.5\n",
"2 ID_28fbab7eb_intraventricular 0.5\n",
"3 ID_28fbab7eb_subarachnoid 0.5\n",
"4 ID_28fbab7eb_subdural 0.5"
]
},
"execution_count": 42,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sample_df = pd.read_csv(input_folder + 'stage_1_sample_submission.csv')\n",
"sample_df.head()"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>ID</th>\n",
" <th>Label</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>ID_000012eaf_any</td>\n",
" <td>0.008506</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>ID_000012eaf_epidural</td>\n",
" <td>0.000114</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>ID_000012eaf_intraparenchymal</td>\n",
" <td>0.001682</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>ID_000012eaf_intraventricular</td>\n",
" <td>0.000329</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>ID_000012eaf_subarachnoid</td>\n",
" <td>0.000926</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" ID Label\n",
"0 ID_000012eaf_any 0.008506\n",
"1 ID_000012eaf_epidural 0.000114\n",
"2 ID_000012eaf_intraparenchymal 0.001682\n",
"3 ID_000012eaf_intraventricular 0.000329\n",
"4 ID_000012eaf_subarachnoid 0.000926"
]
},
"execution_count": 43,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['Label'] = values\n",
"df.columns = sample_df.columns\n",
"df.head()"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {},
"outputs": [],
"source": [
"df.to_csv('submission.csv', index=False)"
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<a href=submission.csv>Download CSV file</a>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"execution_count": 45,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"create_download_link(filename='submission.csv')"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.5"
}
},
"nbformat": 4,
"nbformat_minor": 1
}