Datasets
Models
Downloads: 1
Diff of
/notebooks/04-Efficient B0 Brain+Subdural Windowed Image.ipynb
[000000]
..
[d735e2]
Switch to side-by-side view
--- a +++ b/notebooks/04-Efficient B0 Brain+Subdural Windowed Image.ipynb @@ -0,0 +1,2244 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# RSNA Intracranial Hemorrhage Detection " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "<b>Competition Overview</b><br/><br/>\n", + "Intracranial hemorrhage, bleeding that occurs inside the cranium, is a serious health problem requiring rapid and often intensive medical treatment. For example, intracranial hemorrhages account for approximately 10% of strokes in the U.S., where stroke is the fifth-leading cause of death. Identifying the location and type of any hemorrhage present is a critical step in treating the patient.\n", + "\n", + "Diagnosis requires an urgent procedure. When a patient shows acute neurological symptoms such as severe headache or loss of consciousness, highly trained specialists review medical images of the patient’s cranium to look for the presence, location and type of hemorrhage. The process is complicated and often time consuming." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "<b>What am i predicting?</b><br/><br/>\n", + "In this competition our goal is to predict intracranial hemorrhage and its subtypes. Given an image the we need to predict probablity of each subtype. This indicates its a multilabel classification problem." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "<b>Competition Evaluation Metric</b><br/><br/>\n", + "Evaluation metric is weighted multi-label logarithmic loss. So for given image we need to predict probality for each subtype. There is also an any label, which indicates that a hemorrhage of ANY kind exists in the image. The any label is weighted more highly than specific hemorrhage sub-types.\n", + "\n", + "<b>Note:</b>The weights for each subtype for calculating weighted multi-label logarithmic loss is **not** given as part of the competition. We will be using binary cross entropy loss as weights are not available" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "<b>Dataset Description</b>\n", + "\n", + "The dataset is divided into two parts\n", + "\n", + "1. Train\n", + "2. Test\n", + "\n", + "**1. Train**\n", + "Number of rows: 40,45,548 records.\n", + "Number of columns: 2\n", + "\n", + "Columns:\n", + "\n", + "**Id**: An image Id. Each Id corresponds to a unique image, and will contain an underscore.\n", + "\n", + "Example: ID_28fbab7eb_epidural. So the Id consists of two parts one is image file id ID_28fbab7eb and the other is sub type name\n", + "\n", + "**Label**: The target label whether that sub-type of hemorrhage (or any hemorrhage in the case of any) exists in the indicated image. 1 --> Exists and 0 --> Doesn't exist.\n", + "\n", + "**2. Test**\n", + "Number of rows: 4,71,270 records.\n", + "\n", + "Columns:\n", + "\n", + "**Id**: An image Id. Each Id corresponds to a unique image, and will contain an underscore.\n", + "\n", + "Example: ID_28fbab7eb_epidural. So the Id consists of two parts one is image file id ID_28fbab7eb and the other is sub type name" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Using TensorFlow backend.\n" + ] + } + ], + "source": [ + "import numpy as np\n", + "import pandas as pd\n", + "import pydicom\n", + "import os\n", + "import glob\n", + "import random\n", + "import cv2\n", + "import tensorflow as tf\n", + "from math import ceil, floor\n", + "from tqdm import tqdm\n", + "from imgaug import augmenters as iaa\n", + "import matplotlib.pyplot as plt\n", + "from math import ceil, floor\n", + "import keras\n", + "import keras.backend as K\n", + "from keras.callbacks import Callback, ModelCheckpoint\n", + "from keras.layers import Dense, Flatten, Dropout\n", + "from keras.models import Model, load_model\n", + "from keras.utils import Sequence\n", + "from keras.losses import binary_crossentropy\n", + "from keras.optimizers import Adam" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "# Random Seed\n", + "SEED = 42\n", + "np.random.seed(SEED)\n", + "\n", + "# some constants\n", + "TEST_SIZE = 0.06\n", + "HEIGHT = 256\n", + "WIDTH = 256\n", + "TRAIN_BATCH_SIZE = 32\n", + "VALID_BATCH_SIZE = 64\n", + "\n", + "# Train and Test folders\n", + "input_folder = '../input/rsna-intracranial-hemorrhage-detection/'\n", + "path_train_img = input_folder + 'stage_1_train_images/'\n", + "path_test_img = input_folder + 'stage_1_test_images/'" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th></th>\n", + " <th>ID</th>\n", + " <th>Label</th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>0</th>\n", + " <td>ID_63eb1e259_epidural</td>\n", + " <td>0</td>\n", + " </tr>\n", + " <tr>\n", + " <th>1</th>\n", + " <td>ID_63eb1e259_intraparenchymal</td>\n", + " <td>0</td>\n", + " </tr>\n", + " <tr>\n", + " <th>2</th>\n", + " <td>ID_63eb1e259_intraventricular</td>\n", + " <td>0</td>\n", + " </tr>\n", + " <tr>\n", + " <th>3</th>\n", + " <td>ID_63eb1e259_subarachnoid</td>\n", + " <td>0</td>\n", + " </tr>\n", + " <tr>\n", + " <th>4</th>\n", + " <td>ID_63eb1e259_subdural</td>\n", + " <td>0</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + " ID Label\n", + "0 ID_63eb1e259_epidural 0\n", + "1 ID_63eb1e259_intraparenchymal 0\n", + "2 ID_63eb1e259_intraventricular 0\n", + "3 ID_63eb1e259_subarachnoid 0\n", + "4 ID_63eb1e259_subdural 0" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_df = pd.read_csv(input_folder + 'stage_1_train.csv')\n", + "train_df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th></th>\n", + " <th>ID</th>\n", + " <th>Label</th>\n", + " <th>sub_type</th>\n", + " <th>file_name</th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>0</th>\n", + " <td>ID_63eb1e259_epidural</td>\n", + " <td>0</td>\n", + " <td>epidural</td>\n", + " <td>ID_63eb1e259.dcm</td>\n", + " </tr>\n", + " <tr>\n", + " <th>1</th>\n", + " <td>ID_63eb1e259_intraparenchymal</td>\n", + " <td>0</td>\n", + " <td>intraparenchymal</td>\n", + " <td>ID_63eb1e259.dcm</td>\n", + " </tr>\n", + " <tr>\n", + " <th>2</th>\n", + " <td>ID_63eb1e259_intraventricular</td>\n", + " <td>0</td>\n", + " <td>intraventricular</td>\n", + " <td>ID_63eb1e259.dcm</td>\n", + " </tr>\n", + " <tr>\n", + " <th>3</th>\n", + " <td>ID_63eb1e259_subarachnoid</td>\n", + " <td>0</td>\n", + " <td>subarachnoid</td>\n", + " <td>ID_63eb1e259.dcm</td>\n", + " </tr>\n", + " <tr>\n", + " <th>4</th>\n", + " <td>ID_63eb1e259_subdural</td>\n", + " <td>0</td>\n", + " <td>subdural</td>\n", + " <td>ID_63eb1e259.dcm</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + " ID Label sub_type file_name\n", + "0 ID_63eb1e259_epidural 0 epidural ID_63eb1e259.dcm\n", + "1 ID_63eb1e259_intraparenchymal 0 intraparenchymal ID_63eb1e259.dcm\n", + "2 ID_63eb1e259_intraventricular 0 intraventricular ID_63eb1e259.dcm\n", + "3 ID_63eb1e259_subarachnoid 0 subarachnoid ID_63eb1e259.dcm\n", + "4 ID_63eb1e259_subdural 0 subdural ID_63eb1e259.dcm" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# extract subtype\n", + "train_df['sub_type'] = train_df['ID'].apply(lambda x: x.split('_')[-1])\n", + "# extract filename\n", + "train_df['file_name'] = train_df['ID'].apply(lambda x: '_'.join(x.split('_')[:2]) + '.dcm')\n", + "train_df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(4045572, 4)" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_df.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(4045548, 4)" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# remove duplicates\n", + "train_df.drop_duplicates(['Label', 'sub_type', 'file_name'], inplace=True)\n", + "train_df.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Number of train images availabe: 674258\n" + ] + } + ], + "source": [ + "print(\"Number of train images availabe:\", len(os.listdir(path_train_img)))" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th>sub_type</th>\n", + " <th>any</th>\n", + " <th>epidural</th>\n", + " <th>intraparenchymal</th>\n", + " <th>intraventricular</th>\n", + " <th>subarachnoid</th>\n", + " <th>subdural</th>\n", + " </tr>\n", + " <tr>\n", + " <th>file_name</th>\n", + " <th></th>\n", + " <th></th>\n", + " <th></th>\n", + " <th></th>\n", + " <th></th>\n", + " <th></th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>ID_000039fa0.dcm</th>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " </tr>\n", + " <tr>\n", + " <th>ID_00005679d.dcm</th>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " </tr>\n", + " <tr>\n", + " <th>ID_00008ce3c.dcm</th>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " </tr>\n", + " <tr>\n", + " <th>ID_0000950d7.dcm</th>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " </tr>\n", + " <tr>\n", + " <th>ID_0000aee4b.dcm</th>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " <td>0</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + "sub_type any epidural intraparenchymal intraventricular \\\n", + "file_name \n", + "ID_000039fa0.dcm 0 0 0 0 \n", + "ID_00005679d.dcm 0 0 0 0 \n", + "ID_00008ce3c.dcm 0 0 0 0 \n", + "ID_0000950d7.dcm 0 0 0 0 \n", + "ID_0000aee4b.dcm 0 0 0 0 \n", + "\n", + "sub_type subarachnoid subdural \n", + "file_name \n", + "ID_000039fa0.dcm 0 0 \n", + "ID_00005679d.dcm 0 0 \n", + "ID_00008ce3c.dcm 0 0 \n", + "ID_0000950d7.dcm 0 0 \n", + "ID_0000aee4b.dcm 0 0 " + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_final_df = pd.pivot_table(train_df.drop(columns='ID'), index=\"file_name\", \\\n", + " columns=\"sub_type\", values=\"Label\")\n", + "train_final_df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(674258, 6)" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "train_final_df.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "# Invalid image ID_6431af929.dcm\n", + "train_final_df.drop('ID_6431af929.dcm', inplace=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Collecting efficientnet\n", + " Downloading https://files.pythonhosted.org/packages/97/82/f3ae07316f0461417dc54affab6e86ab188a5a22f33176d35271628b96e0/efficientnet-1.0.0-py3-none-any.whl\n", + "Requirement already satisfied: keras-applications<=1.0.8,>=1.0.7 in /opt/conda/lib/python3.6/site-packages (from efficientnet) (1.0.8)\n", + "Requirement already satisfied: scikit-image in /opt/conda/lib/python3.6/site-packages (from efficientnet) (0.16.1)\n", + "Requirement already satisfied: h5py in /opt/conda/lib/python3.6/site-packages (from keras-applications<=1.0.8,>=1.0.7->efficientnet) (2.9.0)\n", + "Requirement already satisfied: numpy>=1.9.1 in /opt/conda/lib/python3.6/site-packages (from keras-applications<=1.0.8,>=1.0.7->efficientnet) (1.16.4)\n", + "Requirement already satisfied: pillow>=4.3.0 in /opt/conda/lib/python3.6/site-packages (from scikit-image->efficientnet) (5.4.1)\n", + "Requirement already satisfied: imageio>=2.3.0 in /opt/conda/lib/python3.6/site-packages (from scikit-image->efficientnet) (2.6.0)\n", + "Requirement already satisfied: PyWavelets>=0.4.0 in /opt/conda/lib/python3.6/site-packages (from scikit-image->efficientnet) (1.0.3)\n", + "Requirement already satisfied: scipy>=0.19.0 in /opt/conda/lib/python3.6/site-packages (from scikit-image->efficientnet) (1.2.1)\n", + "Requirement already satisfied: networkx>=2.0 in /opt/conda/lib/python3.6/site-packages (from scikit-image->efficientnet) (2.4)\n", + "Requirement already satisfied: matplotlib!=3.0.0,>=2.0.0 in /opt/conda/lib/python3.6/site-packages (from scikit-image->efficientnet) (3.0.3)\n", + "Requirement already satisfied: six in /opt/conda/lib/python3.6/site-packages (from h5py->keras-applications<=1.0.8,>=1.0.7->efficientnet) (1.12.0)\n", + "Requirement already satisfied: decorator>=4.3.0 in /opt/conda/lib/python3.6/site-packages (from networkx>=2.0->scikit-image->efficientnet) (4.4.0)\n", + "Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.6/site-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet) (0.10.0)\n", + "Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.6/site-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet) (1.1.0)\n", + "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /opt/conda/lib/python3.6/site-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet) (2.4.2)\n", + "Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/lib/python3.6/site-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet) (2.8.0)\n", + "Requirement already satisfied: setuptools in /opt/conda/lib/python3.6/site-packages (from kiwisolver>=1.0.1->matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet) (41.4.0)\n", + "Installing collected packages: efficientnet\n", + "Successfully installed efficientnet-1.0.0\n", + "Collecting iterative-stratification\n", + " Downloading https://files.pythonhosted.org/packages/9d/79/9ba64c8c07b07b8b45d80725b2ebd7b7884701c1da34f70d4749f7b45f9a/iterative_stratification-0.1.6-py3-none-any.whl\n", + "Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (from iterative-stratification) (1.2.1)\n", + "Requirement already satisfied: numpy in /opt/conda/lib/python3.6/site-packages (from iterative-stratification) (1.16.4)\n", + "Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.6/site-packages (from iterative-stratification) (0.21.3)\n", + "Requirement already satisfied: joblib>=0.11 in /opt/conda/lib/python3.6/site-packages (from scikit-learn->iterative-stratification) (0.13.2)\n", + "Installing collected packages: iterative-stratification\n", + "Successfully installed iterative-stratification-0.1.6\n" + ] + } + ], + "source": [ + "# Install Efficient Net as it is not part of Keras\n", + "!pip install efficientnet\n", + "!pip install iterative-stratification" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "import efficientnet.keras as efn \n", + "from iterstrat.ml_stratifiers import MultilabelStratifiedShuffleSplit" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [], + "source": [ + "from IPython.display import HTML\n", + "\n", + "def create_download_link(title = \"Download CSV file\", filename = \"data.csv\"): \n", + " \"\"\"\n", + " Helper function to generate download link to files in kaggle kernel \n", + " \"\"\"\n", + " html = '<a href={filename}>{title}</a>'\n", + " html = html.format(title=title,filename=filename)\n", + " return HTML(html)" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [], + "source": [ + "def get_corrected_bsb_window(dcm, window_center, window_width):\n", + " #------ Correct Dicom Image ------------#\n", + " if (dcm.BitsStored == 12) and (dcm.PixelRepresentation == 0) and (int(dcm.RescaleIntercept) > -100):\n", + " x = dcm.pixel_array + 1000\n", + " px_mode = 4096\n", + " x[x>=px_mode] = x[x>=px_mode] - px_mode\n", + " dcm.PixelData = x.tobytes()\n", + " dcm.RescaleIntercept = -1000\n", + " \n", + " #------ Windowing ----------------------#\n", + " img = dcm.pixel_array * dcm.RescaleSlope + dcm.RescaleIntercept\n", + " img_min = window_center - window_width // 2\n", + " img_max = window_center + window_width // 2\n", + " img = np.clip(img, img_min, img_max)\n", + " return img\n", + "\n", + "def get_rgb_image(img):\n", + " brain_img = get_corrected_bsb_window(img, 40, 80)\n", + " subdural_img = get_corrected_bsb_window(img, 80, 200)\n", + " soft_img = get_corrected_bsb_window(img, 40, 380)\n", + " \n", + " brain_img = (brain_img - 0) / 80\n", + " subdural_img = (subdural_img - (-20)) / 200\n", + " soft_img = (soft_img - (-150)) / 380\n", + " bsb_img = np.array([brain_img, subdural_img, soft_img]).transpose(1,2,0)\n", + " \n", + " return bsb_img\n", + "\n", + "def _read(path, desired_size=(WIDTH, HEIGHT)):\n", + "\n", + " dcm = pydicom.dcmread(path)\n", + " \n", + " try:\n", + " img = get_rgb_image(dcm)\n", + " except:\n", + " img = np.zeros(desired_size)\n", + " \n", + " \n", + " img = cv2.resize(img, desired_size[:2], interpolation=cv2.INTER_LINEAR)\n", + " \n", + " return img" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(128, 128, 3)" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "_read(path_train_img + 'ID_ffff922b9.dcm', (128, 128)).shape" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "<matplotlib.image.AxesImage at 0x7f6514ccc898>" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQUAAAD8CAYAAAB+fLH0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzsnXeYFtX1xz/z1u0N2F1YelFEmqKoKIKiokbF3hW7qZoYk2hiQWMsSYyJmvgLUaNGY429K1LEggIKgkjvy7K9t7fM749z7ry7sNQFpNzv8/DMu/PembkzvHPvud9zzvc4rutiYWFhYeD7vjtgYWGxe8EOChYWFq1gBwULC4tWsIOChYVFK9hBwcLCohXsoGBhYdEKdlCwsLBohZ02KDiOc6LjOAsdx1niOM5NO+s6FhYWOxbOzghechzHDywCjgfWAF8CF7iu++0Ov5iFhcUORWAnnXc4sMR13WUAjuM8B4wD2hwUHMexYZV7EdLSOwGw/37dmfv1NwBEYs3fZ5csBKWu63baUqOdNSgUAKtb/L0GOKxlA8dxrgGu2UnXt/heID+nYYeeDcCUSf+gILsHAIWVq763Xll4WLk1jXbWoOC0sa+VNeC67kRgIlhLYW/BqIGDABg9sicAjuMwasRYAAo/tYPCnoKdNSisAbq1+LsrULiTrmWxm2DqvK9abQGmfvre99Udi+3EzvI+fAn0cxynl+M4IeB84PWddC0LC4sdiJ1iKbiuG3Uc56fAe4AfeNx13fk741oWFhY7FjvFJbnNnbCcgoXFrsAs13UP2VIjG9FoYWHRCnZQsLCwaAU7KFhYWLSCHRQsLCxawQ4KFhYWrWAHBQsLi1awg4KFhUUr2EHBwsKiFeygYGFh0Qp2ULCwsGiFnZUlabEvwtdPtvF1uqML/zxvPAApyfJT+8lLn/KTg7IAuOfjp7SdjXLfnWAtBQsLi1awloJFO5FDOPVY+ZiWCUCfDikAfPvtQ1z7/NMAfHT1lQBU19Xw01Eny+dwVwD+/uH9QOMu7LPF5mCzJC22Gj06HMSK/94LwG//+gkA97zzIQnlvf1l00H0dZ4eux8X//dmAAZ0vxSAUwb04JvyBgDe/M3RAPjPOm0X9N4CmyVpYWGxPbCWgsUW8flt/wTgsPOHQEGG7EzuINuP5/K7+z8G4O6379EjRKJzYPcLmLdujeyKzNXv6nDfexkA97sSAHzXjwfiO/MWLATWUrCwsNh2WEvBog34AVj+x8cA6BmJye7Du8Gxo7VNsEV7qenw6t9eAOD9l14C4JHpryGavXDeERcA8Pxn9wPikqz/4FkAUo4fu4X+hHXbtI33YbEBrKVgYWGx7bCWwj6NTKBWP4cAuPig0/jPbRfLLsMfHKpBSa4DjdXyObmXHtfSYtg+TJkyh2OOGbqZFjkAfPfoIwD0v+q8dlwtCYCnL5nAxf9VgfHYp+043x6FrbIUbJzCvgLfYYTT+gDQVK0vNiH8WfLi/+4oqeR0crdMmudIRGIoO1maVVXKNrM/JOfv8K6NHj0EMzlNuPKHANzx+D+973t0lriG/ft11D1JbG9cQ1rqfgCM6pLOvb+VaMsePX4HwI9+MZHKmmXa0rwaptLhvrN0scsHCwuLVrDLh70evXWbBtTr5yTd+snrJiU+F4wfBkB2biqkK7Hn0+p/HSVCkVF9IbVfu3ozZcoXjB49fIvtRo8eDUBebTqfr5WlzcoHLgHAueA+4PN29WPVnG/pNvgA+aNJllBO6kkQM+VJTOVD88z2iohLSzRaWFhsO6ylsNeih24NBxAlQQqm69YBv+QrPHTGYAAu378TQbUQllbL7JjbMRWAcEqQtP21kvmIIbLN7rLxpQuluPGUSbMZfckZANR/LsFLqUck+IOtheOYWXsUALePO5YuuXIv1/7rt9t0rtYwz0Y5ExqB/vp5kW73qqCqrbIUtntQcBynG/AU8mTjwETXdf/mOE4O8DzQE1gBnOu6bsUWzmUHhR2GProdrFvz6KtIpCibwcFBlhXQp5+Y0jcP68KcMjGZ//1dKQC162ukeSQGbrkeO1u3ZUz+v4kAjL726tZdaS6DuJrdi6S+cN+TH2bJmie36Y4Sg4LkVLjuqhbf+fXT9ry86kEJ6rOKvLYd59ijsNOXD1Hgl67rHgAcDvzEcZwBwE3AJNd1+wGT9G8LC4s9BNvtknRddx2wTj/XOI6zACgAxgGjtdmTwBTgN+3qpcVW4Zojf8m/F4llECkp0r2lLVokCEaBC36JATi5p0QZhn0+ShujAJzQTZYWFx4jZOXIbplkhOTYN5eOAeC6KcsZ+6vJADR1KZDT7qd5EQXdIE335Utm5OwHztqme5pw2bXe58nvPpL4oq5KP2yrhZCq22RguXyMLN/Gc+zd2CFxCo7j9AQOAmYAeTpg4LruOsdxcjdxzDXANTvi+hYWFjsO7SYaHcdJA6YCf3Bd92XHcSpd181q8X2F67rZWziH5RS2C+Kqq7ztIQDunLGaB2fIejtauVDbaN4CARKzqpkLgsiMCRNOEddkfkqQ9Q0RAK4ZkAdAcpK0P/fthYzrKf+VN05aAkBDeSn3nX4oAFXNcq27zhkEgBN3ueixLwFIUfLyvosPIufa89t533DTcaLTcN8kE+S0WdrKQrBziUYAx3GCwJvAe67r/kX3LQRGq5XQGZjiuu7+WziPHRS2GVmU3nQ3AHfMWgvAQx98AazX780AENFtmITvPaRbHyQLMXn6UDHzp66rIa6/id8MFc/ClEKJgHx/1lqIm0FG/fdOCpceLed4Y6Ww+PtlSJzDH4/uxaiHX9b2Jq5gIK5rwopNolP7sfTjGfQ9+vAddr69FDuXaHSEEn4MWGAGBMXrwHj9PB7Y6yldC4u9Ce1xSR4FfAx8Q2Ja+i3CK7wAdAdWAee4rufH2tS5rKWw1RDzfe7PbueGz0QG7cOZM/S7ahL/FWa8N0uFEAmSTV2SgUzOHC7xDJXNQi7GXLh+oCwb1tZJSvQNH68AILK+FjKVrAzI+Tukh7miv8Qu3D9D+hOvMKZ8CWDyFSbrNsZtp94IwH8Xy89i8YK/4pGfZWWy7dBhi0+iFZbNgk+kn86l5+hO+7PaADs3Icp13ekk7NENMWZ7z2thYfH9wmZJ7jGQIKMvfnwbABe+v4R5C7/Q7xp0GyIRmJS2wXdJeP/djqZEpyczt1y4gVUavTioYyr/WSxuzDeXyEwe8cvYH+ySTm6K8BH5KXKd2UW1zCiW3IGgK3xDE9/oNVOQcBbzGeBQ7nzjfgDy8tQ9WTyf6tniQs08SQRXRo2S6MUpU6Z4T8AEMSVnnsg7158p7e64Qr7s3Y/aaZLhmJon++rWP4bFtsPmPlhYWLSCtRT2ECz5hYiiXvzRUgDmLZxNInOv5dhuGP0S3Zp1dQBjPQRyJfchWtnI4BxxSa5QSyESc6lqkhm/i/IHPdLFOshLDpKiXMLbK8TT4FbU84V6JJoqTYZhjm4rgbX62QROzUBCWuCGQyXMOPbeYjIvfUC/l3yEqVOnAuA4R9G3U1GrczRUNTPmbyIxf8dc0X743V9PIe1QCYN+aqxoJnxZcisA977zeyy2HjYhag/A/531e5Yq6fend9/XvfUkKB3jJkxm4/RoTWAKZkJUTHmng0QqupE4qWnywteV1AEwblgBecmyNMjVUm/ZIdlOXVfDx0WSB1GxWpOIgn5orNNrGZLTxEisJiFOYvqYzle/lwFg6C0SrzB6/x8zdZGUkFv22iQAep12lPTV6QII+Xj7TRIY27AI/vjyn/R8AwDo2204i285AYA/viPX/6JY+hVrquXVWf/AwqZOW1hYbAfs8mE3RkG2mNl9spP54SuGVDRSak0kUqCbdRslkddg2qk3OJIJjuQwFGTJkuGQTqletaYVGo3YOyPMyho5n8mBKG+S7fS1VSQH5fypubIUqVtXSsJi0RoP3pIlHzDyZhK/dtUxZzJUcyquP06Cr6YuKiaYJNWiep1mApBMgFUWxlJ49rEPAFhY/Dl/dO7T7+cBsGR1HWm/FUulUQ2WWJ0sif78gwG8OguLrYS1FCwsLFrBWgq7MT69+ioABj8xC+JGQLSli1F1DrygpGY2zm8waScdIV0si0M6Sfs4sFS1EgJ+mR+CjsP+SjC+vkqCkOZr+DJxl3q1FGJVS/S8Poy78YIREp5yzSAhC4/55/Mc3Pc4AG44TEjF11dU4Fz2xAb3As3f/p9+SqEl7rnjV9x8u4i5LioxGg5BCOnSuHmm7iulrswESGnAVFgEUx5fWIL79P8AcC6+UtsYYRWLDWGJxt0Qx+wn6sXfRiXacP2yBYheDSSWDDE21g10AI1B8JYRmqSa1YkT95OXpVKXCourGqnSJUJ+qpjrl/TryBolNf/zhSgo0WjIwigJwtBIw2eAIwPQ4P5S+OX+I6XfLi6PqVDL2ytlgAn5fQzUqtQhXXVce2AeZz14qp6up55XlaRxcZzWBu2UVz+k9PWvALjh7a8BWFVUAXynLUx7GZwuOfoo/jpSztusA1bn809nH4QlGi0sLLYddvmwG+LcISMAuG2W8fFXkog/MDO1S8LtKNbBsIHDGaYE4MSPvmnV/uYjelAfk6WFyXpMDfgpi4pVUFgj1sCLy8tZUakWiAlG9DItC8lJ7wtAeY0hN98GV6ySXunXAfCtnuv65z/x7imvQLIwj+6ewc91th5xiRCpjDiKxE8xRms4iZoQEyRWw83M5qzHJH/irFrNCp3+NW+9LKTjTzSWY+VSKXz7n2kfk6niMA+ddSAWm4e1FCwsLFrBcgq7EYwLcm2FzHSk/EC29V/ROjIRZM0thOHwIXLcid0yKVQ+4NGpsr4eOVjW99PuPZGF00R27Af/k8jDovpm6tap61KJRn9mErF6sQwyO8j5D1VicnlhOXefK3UfzvtrWyrKJg/O8A0dSc0Q9+NFh/QEYH1DhFfGScCRc85AadZ7MAkXZDvwiBSsnfGNREAe+ZQIvMTqpoEjFoL77g0A3P3aDH73j9vbf809CztfZGVHwQ4KgtuOl4i9Oz98R3a4mfpNMQmvgkkpTuaKY8SnX63E4ceFNawvEu3CUXliLk9Z/lTiAjMkdHjUj0Xi4uNVVTg62MQ11uDE3tmUKPmYE5YBqFiVmL5+4mycQ47eZP9TMyRluS4qS52j+3dibZUsRUzM49AOKYztKvf18xO0sEz/TrBKvQFnqHR8Wt9NXqcl3NmfAVBWXM0t94j34eUFEvpc3ST31lQ9HxNl2TtvHABLZ/8ap2Bz9Sv3Slii0cLCYtthicbdBj7u/OBBAPJ6inDV+hVf6XchEi46kb8cPWwAGSZmIC4zouvAr08V0/y+53+3wfkjUC0E4P+pe66oPsJkJR2/1loPcReSdSnxwZcrAEjPFPLSOWRcmz0/+WhJVZ705WIA0pIkMWnaN6sgYtym4oZ8tyyVXM2tIKKkYmYSHN5dPoeVUC3XOIiczVsMzsFHABCY+ByzKuRaJTVKjDaaOI4IhpRdtl6iIn9yYXe66nJtTcVXWCRgLQULC4tWsJzCboLeXU5hWeGb8kfoNNk2myhGP6JuBz8/QVSXh3VM9dKYvywRYq9zSojrXrql7QtESqBSCLhPbpXZsmd6mNGvyDWWrJJUayc9Fbd8nh60eoOTrGfU0IsAmPr1M9I+3IuPHhFX4TFXGJVmY9WMhWyxbA5VHmFQxxTuHyH3knWsVrNKDYHyF1MnSo7H/FKxXH49aQm1X4mwjNNL8yLixYw7XoKPXn3/Fb1WGfz9PQDu+UCsjN++ZqIdC/FqPHh9G8CDZws/ct1Lt7bxwPZKWE7BwsJi22E5hd0EywrnAcqGNy/WvSZUOY9LR0mp+OG54h5sjLpefYaj8iX0+QcPJaopbYRgJyLPiRbDIQMk9PmAP09h+WqpExHMluCiSEMjMHODg2W2d123RV1HwTVDR7awEAw0p6FDNslqARzdRfqYnxIiLUk4hebPxRJ5fM46TuwlwiwnvLYAgAM1FPqSoV1w+kkOCPFi7wqvffDqBpesB+UqbjpE7uW2D6VIbLQumuiTZlxCBa+uMAKzRrrOuFL3bdhBYbfBCkDj/9GcAxOb4MvyIvI+WCPE4JiCDIZoHEH/o3pKu1ST9wC8okuRM07xdgV/KsuSy8bcAcDy1fMYPkSIuu8q5KXJzEiidKXReZRBx3XlRRqQ13ujXv9zxlMb7fNQU8tx/SROoi4qLtWZJXW88J0sVQy5uaCygc/WywvZHBXy8Sc6cF15Qj9K6yX24qXPP9roEq/9TWITxl13Aqs0ceuH78pgcNux4vL8w1draVpjwjPNtsZTmHr8fIm5uOK59lSw3ntglw8WFhatYC2F3QY5JIg9k1eguQ3hVBZqPsI5vcXMzg77WVsnM/nBLTL+VjwpKcJT50gAz/gzWlyiWmbG9+ZL5mJen4P5Yp6Qj/36SQblIZ1SeatCzldd/TYAt54ilsWC4m0sxNq8kK9KRQ4uqGXjZhTXsbpWXKNL1UV6z/BuTNHIyqGau3HegZrd2T2T0UpSvtTGJU6/YQIA7uVDmbpE7uudL4QofSes1Qv9AcjSQreVxk1Zx6z5IgBznkZWXkE3NiZX9z1YS8HCwqIV2m0pOI7jR5ipta7rnuI4Ti/gOWTqmw1c4rpu8+bOYQFwBGCES1oWgIXLDutBnwyxGubp2r+0McpND1/c+hRNpQz4uYQw1xc/woa47pw/A3DvGJFG++GkJTx4huQEmOKwt77+AYZ469PrUgD+9rHmYnA4+DXMOvbWVtxTEdWNYs10UMJxZH46a5Uj8Kk7PC3oo0Il3x5SLQYModkYpa+pSuXNYS3Kz8eFP2h89VvunCFycL4cycKMlxvLpiPUyzHJeXK/DeunQ1yskzs/E7J19YRb6TbBFkLfEcuH64EFJNQ97gMecF33Ocdx/g+4Etj4F2rRCr7UFOJ15r9D1Yd8StJF4t5Le9X+Yo5/Xd8MXQ3xp+TZupXUV0zUfUmtv3vuTX6s5eAWa+Rfw91jueoZESl5bLL4+HF6UNCnMwArSiVjISVFz1WdBzGjWGTyBmLAOv1sUqxF8SgzfyApqgQ9V7Ug562v8ZYSl2sK9b1fFVKh9+fXwaBMcybS0sPMLzeeA+P58LNhinXy5Q9w6ciRACxZIcuIQQdKHsU3C1ZCXO+52eST9EKqGsLDWqn75sO60SlDyNiS6tfZV9Gu5YPjOF2BHwCP6t8OcCyJ5d+TwD4pcWNhsaeivZbCX4Ffk9AI6wBUuq5r/D5rgIJ2XmMvh+ol1hWRiEsQUze1o8xqnVOC9EqXnIAHlRic+NHdiVNUiglNz4PYaJx/VIjHz2cX0jNHLJBTtcT8yPs/ZvpcmSUL+ki04Npl61m7RJYxpx4+GID3l6k/PyMzIRLtUx3EeAwwFotYCocOFlfgl/PWQYGQfTO1rkSsqBSQ8y3sJt/NnreMC0aKWf/QfBFNqVcX5ivdMpkw1SyrTOBrdxLydIr4VzTFpFYEMXF1/kTJyiuuGEbaHR8C0FxhYh2SvGdVVz4XgBcX7MfKW6UcXcqvrKWwzXAc5xSg2HXdluLZbRWcbTOE2XGcaxzHmek4zoaRMhYWFt8j2mMpHAmc5jjOyciwm4FYDlmO4wTUWuiKBJ5vBNd1JwITYV/PfRih29V4XIIjuQEn9RNSLxJ3vSKuT75xk3fkl7+RqkeH3nGJ7mnGiJW4j4uF8MDbYkX8+JAC5muAUOdrJV8gLz+dM0fIrP6qztC4ZZw0XMLjzfq+SV2IBP2kdxceoEbFXCgrA0czG/2yNfzA8EFd6GwK0So/sTo5jXBA2r39lfw0evTpRpYGZ43IE6PzHa1A9eTnqzmopzyHaeX76X2mkiiLZ5QaHF75ZoX2Q6wvE+kZ7NOB60b0BODP75o6FNkkVLAlOOu6977lqlOED+nSQQrcFpZNZV/DdlsKruve7LpuV9d1ewLnAx+5rnsRMBk4W5uNB15rdy8tLCx2GXZG8NJvgOccx7kL+Aqw9cDbhAklNsE0HTHirP1VKt0oH62oaWZZmc6I6cqeN5Vy6H0/AiCti7jRfE1Rpl0sGgGvLpPKUBOOFz2CCR8s4Y43Rcz1tCMkO3F4p1QemCMcRbxKi7gmFxBVV+Hn68Sy6KNr/6ywn1lzTRUoo5MQxp8jLsyT1bKZWybeglsO6sI0rT3ZoBzB6sXzCBWIS/SK/uJJubJ/J4Yd1k3vT57B6WpFBH0OH601RIZxRZZj8jESloJL06w/APDjn/8HgA/WiAqVf/Iy/nTuIAAe/EQsheaaQhI1McQiaqhczDcz5f7m3ynPNPsn+56lsEMGBdd1pwBT9PMyYPiOOO9eDb+QeMQMDZNEh25C2Bmfvdm+M78It/bR1seHO/KH8VI6LTlJk4565fDpOnkJbx8lxVeG/eNzAGYvLeKZiw4FoEbFTW74aCn1paV6QjWlw0G+VFLQ5FuUqzxbVXMUr2BsWExzJzWZgwvEGz25UK59mCZtnXpEd049/mAAZjw5BYD3v3ybe0dIIdj5GnOxoKKRYVp3AnU/pmkUIxlhxiyUPv7HM/dXAAfoZ1W8Dh4MfWSw+8cbEwB49pJ7AaiOxLy4h+fPEjflGU+Uk3DbGldqDePfF1Jz9jGa1u2paJvaF3s/bESjhYVFK9jch+8LYTGhqdeMyJQ8Tu8r5rexEF5aKgTYHSfuT2LGSuCWZ2YAMP4oOdcl/Tqyf5bMfhl3i4hpLCZWwYrrR3PPVxJk9M8vdQlQ30zCDarEXWUjlZVCeFaaykxpOqPWNECWuiIrxTR3fT7mank5xy/nyvFpKvI150G5nPewCVdL+wlXw9siTFs6bQUAa+qaeeV1SZnur/0/YIgEUNEcI+Az1pRKttEMYfF0jzzwFwBM06VDS1xwuRCmrz41mzVKpJ4+Wiyoq1dW8K/JovaMT5YxxCPMWyv39dl7QtCeeMB5ALy7YDPZoHsZrKVgYWHRCtZS+L4QN+kgMiv3KMgipDNify0Vf3SBrKuvufL4jQ6veH8G44YLiejT9fKrKyp4+EMJxCnoJTPpHcOFwDvqhTmsWa1BTh5JmEZiPa0uSVIT+wK6hlcXY7hzJk1FRphE8woavqYpUScWgBdVx8Rx7meB1mLo/0MRYln/xJtMniRZjIaDWFffTMgn89NvD1ILwbg84y7dlG/I7S1BV8XL1kKTWCDTJt+jVzUyay2wXMjWSNz1MjILioQ8vWtED/41TWtPxlRcxcknEpHrTlNu5uzBYoW9u2Dj0++tsIPC94VGfbn8woAXpIU8IZKMkLD/52qadH63zI0Of+7RDxmq6kSPapr02tVzIVUGihot6HLV66rz2FBJwjBsWa26Sj8bNn8toJ6AqAamahRJJBoG15Sy+3qrbvO5TyX9uvyZSQC8vjrA2D5yX01axi7Z7+MdrfvQN1OWSQdrjke0qpEyXU7F4yacJQ3QJK14y8HAfC+D5Ce6TDpjYB53T5dlWkBXIof3zIZkjQsxpefcGATlurVKxsbi+14IjV0+WFhYtIK1FL4X9MToAe7XWyLo8pMD9FQfvSkFf990MdG7df+QU/8s7b5+WHLNphfV8NyXMvvFG9WtmNIX6mTmr64zrkaxMoYP6c9s9dtHy0yMRAiczMRnkPJxKolmSsmldBaXY31DlLYIz83hjv/8Rz+JxmR+n+FeLPyzS2Sd4bjQpOnUp/cUy6lIoy9jTVHMZF1fa5ZcdRiCdN5HUvlp4JldYMVs+bqnXOvRhbLEOLJfR4+8/ULdrc8uLef8oRIP8tx0k/kZoUCtr9SA1tTYDdTOdzWspWBhYdEK1lL4PhA6GGIyEw3NEzLv24pGTzNh0mKZQUf0lfXtaQ99SurTsob/2cFCtv3347ngF/fdyIMlWu/j2UuBer2I4Qrkv/iLuavI7y16CkVVsqZ3wkEO7Cmfv6sUtjBaH4EKIehGDJRgKqOFQHUjwQ6yL7LVOQGaJJshWZBFJXWU5InL8pfqdqyPxvnb+0KCmsAqdIZe3xBhhZa2D2mQVj0uhgP52z0vA/CvM49h0h+mADDmX2IpvLNKnsEXy8o9fuafCyRLMjPkZ1CO8BGvZ8i56qtrqFIr7bQemt1ZaiIm9x3YQeH7gBOCmPzYFqjgiQNM0h/xJfqyHNRRTFkf8PlKISbvfUcGh/OPGsygDvKj/t1rRuU4jpdUxYG61ZcsEKTISJpranEovzPzdEmRqfEBVa6LkyMvkBmkjDnZrUcWeSqjviB+IQB1FdNJyKcbr4a80JkdTqCqWo4eoS/lp8sqyFThleXqEeiWFsKQn9PVO3CAemAK6yN00MEgU69dSSrmp/voTIk1+Bcw5urWxW97Z8hS55Xl5VyiGpQmJTsSdxmoqeSnDJaB64XpM0lPlyWF8VYcvV9H9jXY5YOFhUUrWEthl0LIQpqqwJFHP0SJrReXlTOokywluqcJ6VemOQczl5USrZakoAmnSC5BdjjA9a8Zt6ApYtIDp5MQda4Sa2lKXtZWNEC9uiKThDgc1imV1cnSj9W6RPD7fRyieQdpQZkzDtF+pQX9RJT1M5GTwztd7S09nl0qy46ze0kfgj7HK2tvZugzB+by6Xrpx2glMMubopif4jO6dDpSlxhzy+sJahzGmkpjkeSQKAMnJOF/fv0Il/zxR7TEp+/9EoAjxt7P+RoteoW6Ot9cVcmXmo5e0WSI1Q74lGQt1kI7KUE/+xqspWBhYdEK1lLYhXA6iaCKW1rMgQf0BBIyZccVZPCd1nYo0Ai+H7/0MQDp2Xk8fJ4kntZFZMb913clEJWgpREHigxZSTRGJ113r2+QGdoQd7VAfi8hKTPCMvv1SAsxS6/fNVvW8NcOyDUcH6WqxHyCWg5fl9V75391ufATqQGfpzCdqgVvf6jVnQ55fi6/HyHis0Ykpmd6mFK1HkxOQyTmEsgXEnSpRiGOXaip3LhkdpVZPlZioi4rSeRsCDF46Z/e5ZLbpfitVykrV/iBQTkpTNXoScMzFNY3s0ZdnH6TWxFzWFskFtlhWn+icP2+V0rOWgoWFhatYC2FXQKRMLvsQJkNR+T1YXKhzEirdLaaW17PLweL1+HHr0puQCed6U7rQZjGAAAgAElEQVQ5INeTOTcaBPPmz6ZqoejXZDgy8/7oR896gT491UIwlkhtZhIBv8yIi1bIbLyoqIaj1SvwC5Vbn1FcS5Gupw/X2dKIvQzPTeWYPjJr91Te409zivhU61senC/t52iNyON65lCtHox8tTBSAz6PRzH7ihoinKMCLR+uEAugtExm9r+c2J8svf7lzxplvxwS+gY6r4WD5Bb8GIDiyqdpib4ZYZ7TjNOJR0uW5AV9OnCLhkEvWmTEXGOMOECsqTnlcg+9O6ayr8EOCrsAF44cD8A1BwjJ9VFhNTF9eRdpfQM3GudhTe99/nypqTBXX679spL4laYZF6+RKMZHb7+GjP2EuLz9jDsBGQCy9QXqqG48Y+73y0zi3UUmylHNZcfx+vGoFn3tnRH2zjFIXXZHjBsgjQ7vhhl1TvhAKmMvrmpkXYMMbGdq5egFugw6u3c2TXqBSiU+U4N+L8ejo/atpDHqLXMCSm66mpiUHvQn8g+CmpIdaQJHXa+aO0JTFSVNpqJ0a+QmB/lME5xe0MFhQHZyQlE4bgjMEEkaxWmS05LClmi0sLDYx2EthR0Go0D3hbcnJUvShftp2TMTENMcc1msFkLxapm5Th5UwPM/kJl/tpJbJ2gQ0zFvLKC6Ts3lNC3RPsEoOMOdr2rMf0Ym43SJYtyaplzbmrpmTtpfAnHMcuDGwfl0U4KxVt1+DQ0R+o1V1eQjRbqMsIqQRFdCmUZK6pJhbLdMmnUmNxaGcVt2SQmpuxE6K3naOSXoLXE6tmg/o1iWOb/Re75xmRR6verVuTx+hvTDnyIkYaxqPbhilfhzhASNFce4YOQQ2sI/vy0GdTtOUsJx/6xkDtalwaqO8lyaSkvoqs/tWJWYW9QY3fB0ez2spWBhYdEK1lJoFzRjMHA6eT1k5lq/VLMNiXBoHyGtlmvsvlnnf15cy+xvRJfgZ8eLiMeDI3vymgb/jNOQ3HuUCDulR5Y3c93/UMuisrI2D+dJ+8MKMjyrpFhdklGdlvtnJbFM+3H3yJ4AFBxcAKpfkKX6C0TjYPQb1EKoeEeqKzF9CS9qZuNpms0YibteSPLrGop9dGcRdV1T1+yt201dh7DP59WCMNmgXVKC9FNXoQmj9rQRmip5RYnRkAZaNdTlQlQslYPyhYydWVrPC5rX8F9a4/LhXb0M1OfmCW9zTf9O3DhYyNUPV4v101Ra7XEgrtIuKVlJ7Guwg8J2oa9u5QeJ38fZSrJNT5aIw8Nz08jQF8G8qI8qkfjFohJvMDhf2fyXlpThmOi5cVLC7eZ88ULU1zWTpAMF/fZv0Q8ZFB4/TvpTH43ztZKTPdNlcDLEWXlTlEPUXC7Qa0bnryegkX5xVUz2HdAJJouAyUGnS9rznGXyXfyuExl3iAwAnUz+c0kdL+hg9sg3cn/Ha1zDqtp6PlR59mO7yKDWJyPJZGR7npf0oI9hGjX5r++MJ8CQfzFml8o9NdRo6nR2KqhXJV8HmF59O5C6iejDayb+ku/G3grA/p2FrHx/TRXD9HkcpNf+cKWPKUvkXtOOFTXnAfug98EuHywsLFrBWgrbjGwS9XTF5M7vkslLWnzFmM4ndM3EBMr94lMp4urEZWb/8urhNKpb7vezZRlxz8VHMPRnZ7W+1EBxBaawCcwTgvFIjQ/4v2+LPQETYy4vUvfg12X1/O4gWc68OU3yBo7qlkmT9nudloZ76cnZ/OlriQdo1ui+zE5yT9e/t4i/GfekEpqxdxcRNmIs6r57fmlCG9HMxvupteTieuXonl4ss3Kv9LDX38HqBp3mV8soVkuhpi87GjHpNsUgRVKbP9Wsyu5pIXqkb1oApr/GQUxV1+RHhdWeRXOLpqN/OGct64ukT3X6IP0NEfY1WEvBwsKiFdplKTiOkwU8CgxEmKErgIXA84jm2ArgXNd1KzZxij0QYYxysC9birOWVDUSKxcX2ovLPpPtJ50geAQAT58rrrIUnelu+3Ityaog+s4bUreA3AIcx5QxU7LLNTJhLdGc+PiOSAwb3YOze+ewXmc2E9dvSsCd36cDCzQa0pRyu+erQj5Vko0ymd1vO3MYTR/8VPYNGKQX2vTcUTB6DKbk7flviK7DzX9837tfM3sn6b13DAe8bMrF2h+/A+epmOs72h9/nszi8dpkXCVIiUmuB6FcTNm9HCVvl9c0c4cpPdcGrnlYMiifO+S3AMwvjVCk5OrFakUEslOJFkuf3lIr5jzN3diX0N7lw9+Ad13XPdtxnBBi6f4WmOS67r2O49wE3ITUl9xLEAWUnKsw6kazgZUbtFvJWYdKIRHDvBvVny6pQR6dJMQXjgwwEyb8HZMGvFksl0hCeh0Iv5JIycENkjxUf/9bfK7KzobEM+x/0OdwuqospWt/phbW8LMjpMDK+X+5XBomGVXnbUfPU48F4Fnd8tUsbv6d1BcuqpdBMOBzOFVVjczy6oVl5VRqHIGRczdJWM3xOFNWynOJlejzbq4FxDOypFB+wvmdUjnxgoM20zutUalek8mLS73yeN+p0M0tw7tyxwx5Yq+qIM3pQ7tsyyPYK7DdywfHcTKAo9ECsq7rNrsytY0DntRmTwKnt7eTFhYWuw7tsRR6I7XG/u04zhBgFnA9kOe67joA13XXOY6T2/5u7k4IQUBj8KNGfmxDK0Hwvy+k2EhJo7q3NHpwdNdMz0IwuOOOX210/FlHnSvnmf5CYmevAzdqR7L421OuOoysG98E8KIMR6hYSWF9hKyLZDlzwYEDZbt2CRSo8EuRipakNEJGfuvz14iF8/kDr/Pn98RS6aEuz6L6CHddKeftNX5c6+MOGsY9b2uU4VsfAHDRfVMZpWTsg1oR+7KVlV5BG1OE5ch8aZMT9nsJWvNLTAxDNUajMU+f6cEdUwmNPnTjZ7MBrrtQ+tMtLcRtM4XkNcupueUNHpn5hUZYXv7q/C2ec29De4jGAHAw8Ijrugchuts3bf6QBBzHucZxnJmO48xsRx8sLCx2MNpjKawB1riuO0P/fgkZFNY7jtNZrYTOQHFbB7uuOxGYCOA4zp4jrh8cADqbEP1ss03T82XGzVKh0uc0S/GRp65po3XDRnt+9oOTALjrzOu45Zk7AJjyqjzubyfN5MeP3tL6gPyBXHT2CgBef01IyIfny+PvlhbijE6mxoP+txsrASC/V+JzufTzuesfBfACkBZVNXouz3U6exfWNXPqn6T6061TRZH5vMcTVk/9828BkHLeaQA8k9eBB+8VIrJSc0FSAj76a87D089L2TtDPHZKDzNvoqREz/zfdAAOPfsKjARdpZKFT58/GPwZbBpi1X3+oZSaP6N/J0+wxkjLDeuY4kVZfqEVq15evRU8z16G7bYUXNctAlY7jmNC7MYA3wKvA+N133jgtXb10MLCYpfCcdtRAcdxnKGISzIELAMuRwaaF5C64auAc1zXLd/CefYgS+FCiBixjymbadiXUMExAPh0Vk3VIJ/S4kcZ2OM4AM4+VKyJO/73X0Q8BGCzj8uD65oMvkR4b/HfJfL/1pdbr4W/Ka+nb4bMiE9NvYfN4ewjxVn0rQqlFursmR0OeHkLS3SWj8RdkvW+jNDrVf3FM/DrV25j+Z+eACBLvQvZt1zJ+vulrHueiqRGG2M0qgnym49XAHCLBlqlh3yknXKAfHn4KNk21OCkjNXeiifoxStHcfajV+o+de3WiVVAMJm69+cAcPCN7wAw84IhlFWL9bBa7+/A7GRPlGbsK1qDs9xYCq9s9pntIZjluu4hW2rULpek67pfA21dZEx7zrt7Qis/RwqBL7fYune/sSxbKqa701HM2sqpN+i3LvNXCbE3f9VH3jHrvhZNxvwhGjWoNRvefeRluuYJ8TboLNUhpBy+0SrSgw7wznHyPyV1u3+21nHQGIbc5CBBv8OmIeNyoNOVxDQp6Ng+Oa1arKlt9tKjTcxFfTROUH2L6+pk8PubJh39Guh1sbgnP7ztRQCOI07euSPlhO/Lc5w3Zx1DTxKD8wrVcjSp5WG/w2GmXsXh2pHkdFx3CgD7d5OlxQ1Tl3PSva8CkKrpz2iMxC0vfOMlcC3SOhfvLyvnrKGyZPlO+zunrN7LwfArURtrVbR2z5m72gMb0WhhYdEKNvdha2GiDQMORBo33S5dKictW7wMAuI2u+sIibQL9x/gNXNdce3dctkfAJg+638JC6FBudmg/Pec+KNzWhzXtuSYwcy3fwjA8WcISZipgUohv8OX6mZ76od/ASBvSGemq4vxLs2M3K9LhneMyVEweRqHdkr1IiZ76Gw8q6SOsFoK2pyR6k6kpgg052GERg02/PMFkq8V8RmuFHJzKMBUIStNrYuQHnfYDadCQVtRhXL9havlPr959AUue2IWAOV6joFaFu7Bjxdx4ngxM0IaHDVtXTVnaeq0yWZ9aXk5n6jATZKmaddVak0I+gKL2+jH3gdrKVhYWLRCu4jGHdaJ3ZpoHAzAQ2fLbP2zl54AlrbRzuQJGA9sfwwJ9thNQipecU+igpGj0+rkd6cBMHrsSL74+UMAfLlW1r1mfXv4NcNhqAQcMU2qQs146Wv6HCkzaMeTdbFd3sDHE98D4LNvpR8LNUsyjsuLmnNgJMcWFtZ4scYdVSiltKaRTkpIGsETI6qanxykUEvGG0m3vOQgKzXDcqFKnR2uFaKOGZDL3c/c3PoxVRRC9gahw+Xl8Oy78vknF7Kj8MyPHgDgtvcXM04tFcOxvLK8gpIfHQZAqSo33zWrkAPVujDh1ue8IpoWDZXFwLs7rG/fE7aKaLSDQguMGiXs9tSpiYrK/zz3LgA+WS0eh6c++8fWnSxwLiQJ0eXOvk329evrff3QX54D4Gc3iCn9j3P/4OkZmhJrLdOgTaGV15R0c4Ez9OVbqwTftxUN1Cmjn6aCI2uVWU/y+5inMvHfXCI5AtOWlzN+8jIASnVZ4PM5jFGVJ/NiNCrxGHNdz+uQrP05uEOKJztvkq9MGvQnRbU8fKrEQhx3n/EMtEhv1kK3X//8Xwy9XdPGO3bd+Fm2E5OvvZ9TX5DB9LWzRSn7uMc+44wjpIL208fK/8t7qyuJ6z2c1F2ebUwHzQP+M5u1yw3BPGuH93EXYasGBbt8sLCwaAVrKQAbPgMx7WVGIajKxpEX2BYkd7mabPXpr1389022W/qy5ARccMdbng6jIcqMAvJbqyrJTZJzZauQyeF5aazXaL4ytTCcFp9NWrARN3lndZWXe3GvphgfkJPMFR+JpfBNhczamUG/l0VpCq8elivLmIWVjR75GFL3ZtfUkKdBafr2iQqfBHyOd6yRhTsqP93TqjQFWu4+ri/db71qk8+ovXCcAcAC/atAt2tbtDDZlakQEpGXw3qJ9XDjYZJF2jk5yK9nSHr8p1+/o+0X7qwu7yxYS8HCwmLbYV2SLTBlypQWf6nbMdJm6sZmILNxTkqQO4YVbKEt9DlOLJLPF6/lz5+LbFutxuS/qFJpQ3JSqFWuwLgLI3HXq59g9pU0Rgn7ZJzfXy2EkM7QI/LSvGzDv2uwTqfkgDeTm0pVSQEfPdJk3Z8ZFKvDZDA6TsLt2F/zBUoaoh4haeo9GDLv0E6p3uch+TIDf1Ne7+WCjNJS9Jndt6DhUKHFZrPzN99uQ9SbyNAFLXaubaPhV4mPqmEzY6HkZ5yjxkAo5RiyO6rVmKqTbd0eZylsFaylYGFh0QrWUgCOHXs2AJPf/1+Lvd9tsN069OwsHoyc5CBjVD59s8iQPIHlFfVeBakFmnNwis6gc8rq6ZIqs7Fh9ueU1XsBRGpYUB+Je2pCwzrKzLywSv6ubo4xUnUMTABSl9SgJ7pakJqo62iuf5QGIX2nf4d9PjooH2AKxyYHfF7dxaXKLdTqd/2zkrhzuEqkKb+yrKiWJXo+Y7lkFtdBjeYYpG9gNXz6MX3HiQdoyYInZF/Hzpt4mBsgRcK0Rw06hanfvLl1x2wCzfWTWb9qcrvOsafADgpsOBi0D727SFRiXTTO9OXiPuy5Fcf97pNV3ss3WmskzFYV40jc9VSOvZwAn+OZ68Zl+NaqSn44QDRt3taU34Da+/tlJXnH9tE4hJpI3FuiGBfjgOxkj6Q0RWHfXy2p0+khH6PV5HfUyIzE4x4xaVIrhuiA1CMtzDK9Zj8dwAJOYlDyCsfisv7uZwHIaxHLAcCIkXTtIgPKqaMl5fwvv72SfheKoNf8ryRm5MCuYZzc1hqNkz+WZcOUuW/Acmnn9L4MgIy0ANW187RlKRYJ2OWDhYVFK1iX5A5Ghx7XAnBghxSPSLvzzQmbOUJM7qGDfk2WRgmaROg5Gml3VH66Z6KbjESXRHCTSf0N+hwvkc+IoJjj8pKDngVwhEq0raxtordaDcYNmhRwvGhFE5RUqiXoypuiXpDTmb1VBCU56AX8GAvgoA6pXh/NEmeFLi1Cfh+rNQLSKDGf1iObT1QS7fKrj5R2x43ezDODVZPElO9xnKmVsWnB8Ccm/Jvxt18mf1Rqu2ff5WeaXv7YZ0IYNtS9tNlr7gWwLkkLC4tth+UUdjDK1om7b1pRihcuvDmUviTFW0fkpfGt5il8ozO0mdHDPocSncm76czbUtugQcOis1NCzFQewsz2BRoAVdYYpY8WkzVh1BlBP6WNMrubkvWNMdcjH9MCYrPk58iavqwpyn6ZqkugXEF9NO7lUhix07Eqz/5lSZ1nRRjOoro5RsBYPSYQyudQqeTkz+6RYK5zX5Gw5DE3nApBdQ93T4jWdu9l8ic29xMWF+ZlE67nsgkiYe9ZxheOIe9d0aM442AR1v3vx5s51T4EOyjsaDRP1w9HeAVZNoeP3xQTti4a90x4Y3IbIZMVNc0ckNW6mnRywOcVkz2tpzD2r6+oJE2PCSvrZ6pOpwR8DO0gBOBkTVxK8jteHIOJjeibmURUy9u5uhap00HEh+PlWfh08RL2+1ir95ni5WcIyZkZ9nv1J8zgMDAnxRvgzOBT1RzjTa3tYGpBvKXVrb+96Vku0IK4Hf+gcQqxMPQWURa3WjQdnYy2PBIa30AqXvG9Wq0dkZnLEh3YTIQlgQMhuu+pN28Iu3ywsLBoBWsp7HCoeyslkzE9sjffFHhPy6Qtrmr0ZqxU1To0BF+XlKDndjRxAkurGxnVReIITDm44oYIB6k70GRJGtGSfplJzNKS7ul6/qDP8dqZWbN/dhJzyoyIjOMda/pYr5GVXVLEmhmQncwKJQ5NnILpfzTuUtiU0HcEWFjZ4BGeJvNzVW2TR4gadNIoydpI3FvifHLVYwCM+/cvOGfIFQD8+6U7AXjlz//mjBsvb/MZQ2dwpB9OuhSudd0ITzwmmZuv/P5lACb1HMX8JdZSsJaChYVFK1hLYSchLSNMdSS2xXYmNyA7HCBfZ8fGmKzhjUrymrpmz0JoUoshOxwgop9NluLAnGTP2ihSK8Os8yNxl3rlDUx0ZElDlIBP9o3Q6MWShig5mu1otBNM0FN2yE99TLMjlfdIDfi86EYTYGX6c1DHFAyvZ/iScErQC3Yy5GPchc7ap3S1XIwVkRnye5aFuc5d4+7mxTmPy4ldCaw6/ZeX4f7yMgDmzhIX45BDTF2LJZ6r9pCC4YmH30cyIY2l1TUnGWsn2EFhp6F2/Xre+k5Y+Ks30868XHWRBCtvmHhjUDuOQ8hn1JPl5RqQmcx0LbHWXROYgj7H8yyYuICTumV6fxsT3qApHqeuXq41MFte6CXVjR5BaiTbu6oHo6QxQqp6JMxg1hSLe4PGQB0UDLnZFHO9QckkUlU1x7w0ajMI1kfjXpKUCVgx5GaPtBBmaO2k6eOPflfCoGtFZ3LcX06WL1OTvSc2eJiSkLVCNDppiUSqmWs3VuI2Zewqm2O8F9LzNb+9Ubt9BXb5YGFh0QrWUthZcBcxu3jLZcx7aYLTrJI6TyDFiJUYeTZc13PVHZYrsQvzKxo87URT4yHs8zFL4xQGqqBKjc7okbjrpTivVksgPznozcyGQKyNxGjSmb63ti9V07+mOe7N8ob8S/b7vNRtI/3mJW+lhfhc6zgYQjPkd8jS+zPLpLjreIrKZl+W/v11WT3Ha9zDosp671x3q+DJuArpB6mm+GwLpOZttGvM/qM22pei18oK+cnsLPkTVSuT9NvNKHfvpbCWgoWFRSu0y1JwHOcXwFXIUvAbpGxcZ+A5pAbabOAS13W3HMWz16GBwsqNi8ZuiMpmmYXTQwkZtBLNIWhWIjEnKeDlPJjMxcqmKL0yxMowM3nnlKC37jYzeoOZecMB1ilhZ2bj0qYoh6vIyooa+S/KCgW882lGthd4lOR3vPRuc1xdNO5d0+RgmLlmQWUjmcoVmIzOnHDAcz8at+aKmiaPhzD3bKyOnKQAxQ2t+w0wWy2Q2reEGky7dhAbQyyL2yc8wh0TJPty0sLVG7X6VgVt31xV6aWmN3Q9BYBFa/b6fIiNsN2WguM4BcB1wCGu6w5E8njOB+4DHnBdtx+SpXLlps9iYWGxu6G9nEIASHYcJ4LEka4DjgWMeP+TwATgkXZeZw+EQwxni63O7CnZhi8uK/fyCcysaSTT0/x+unvZhjKDdkwKosS+F3Jc2RQjI5QIHYaExVCQEmSJFlQ1FZ/8PoevNKDJYFVtM0M6CB9hcirMzJ4c8FHheUZk37LqJk+araihdSZnfTTuWRnGyxKNuyQH5GdnuJNvK+OeK3WG5k8crtyJ44igDOBdu6wxSlStmcUqeX8QMRL5pWLNEBULYMLYwTR9JjU473n3aTaEEZgpb4rxqyHiqbj61ekbtdtXsN2Dguu6ax3H+TNSWboBeB8RxK90E+WQ15CQz90KBIHI9nZpN0M6oaQtP948LT7iLnM95WVjOhsXZWbIT43GGBjyMS856ImaGPddJO7SR5cUKQF5gcwLWtYY9WIASjSGYWiHFEqUMDSDSO+MMHo6aiPSLtdEFzbFvLgDs+wpaYzSRwexDD2/cUNWNEe9WAujp1IXidMjTfaZVOtOSQEvj6OLKjSt06IzEdf1+m0GSRc8RZfrtOzd+yPeI3mw1tXQSE+Cmg9x+AjueXcEm8J3moh2bu9sL4nNbSzZZPu9He1ZPmQD44BeQBck6+SkNpq2qZXgOM41juPMdBxn5vb2wcLCYsejPcuH44DlruuWADiO8zIwAshyHCeg1kJXoLCtg13XnQhM1GNdgN+ffQu3vvKENIgtb0fXdgd0pLlqy+6soAb8dEsNe7UXTKSfIQZzk4MsrTZSamIJlDREvRnZBBn5HSeR0ahWhMl0rI3GvBk3SWfZpljcu0Z+SsKllxSQ74saxHow6ddRn+ORm+VaE6JjUoKYTNF+mHyLrqkhr6Sc6TcklhJmeVJUH/ECmYxuoymr5wfqlWA0SyFHOu9dH+Cg37zjyczdMEiWAGMmai6Ek8Pm8NPXJU27b6cchqpAjMxx+yba45JcBRzuOE6KI/+DY4BvgcnA2dpmPPBa+7poYWGxK9EeTmGG4zgvIW7HKCKePxF4C3jOcZy7dN9jW3vOW1+6fXu7sxsiBM31W2yV1lXWsHUtZnJDBJpZubI56rn2DOmXFHC8GdrMljlhv8cNGJi1W0444AU7DVcSb3Vtszf7mlyD7mlh77x99frG6shPCXqchuEqcpMDXph1s87oTbqNuS7Z2u+odiQj6Pc4k346s1c0RT3OxPTDuF59TiKwaY0e53MA3dfsib8mnocJE2eN8gLd2rYUKj+QkOdY4yQAFq6u4KynJFCKwNGyjbZU8+6u21rdlrM3ol3eB9d1bwc2fJOXAcPbaL6PwY9HmsZ1cPClbNxMX65OyUFP6MRE85kEoOZYHFeJdWNe90lPojGqiUrK4jdEExGH5uU1QiaRuOvFAJgXOjXoIzmS+N5sayOti9SaUnEVTYnCL+bFzg4FE94MbWfyI2Jx1xN7MSZpRVPUi+IsVpIz2uKaJnfDLKFKGyPeZxM5GXOhk5Kypn1DNO4NKF+XiQfjrNUqstJtf9rCP/5uyr+ZFPcKQEVYom/oPrOsygdk2XPcQKlAnpudxX8/fka/X9PmNfZE2IhGCwuLVrC5DzsNLiYmcNbjMusMu+q8jZupRFrHpICXk2Bmd58jxxekhjwJM+O2/LKkzpu1TVm3yuaYt686IvsyDAnpc7ylhyECs8OBVqnbIFaEiRv0Mhz176DP8dykxmqvjsS8GAoTu9BQJ2folhbyXJiGLEwL+rzitGq4UBOJee5Gc968FhmUJm18rrot+2SEvSxQYyXlJge8vhmLpWiaFM/NH7FxvgPA49/p8iLQS7bRZW20Mi7yRCTkh/OW6Kc+kDZaPtZ+rvuWsKfDWgoWFhatYC2FnYY4plrpS/+T+PxhbVVb79EDgIDzlVcUdnVLQg0h08wsb8i2gtQQ2brGNmv/uBv0RFjMbGkIvoUVDYxVbYXp62r1O9ebaU37wvpmuqsL0q/XNLkHeclBT7ats7owkwM+LxgqqBaAyR8oa4p663yzrYvGvapVBp1TEpZQY1Tuz7g8yxqjVCk/Ylypsbjr8QsmmjMSdz2rxFg46RoY1ibmL/QCt4hub6HYKoJ6Kzm5knuxvnjPtxTsoLDT8CXQG4DnFkjl6nvaatZVUnX3y0xinsYpGJPeRAM6Tpy6iElKSkilV+spDHGYEw6wqs6EQct/rSHwgj5nI/GU4oaINxiY5UNNc5xczRo2HgDTpksL74Mh/2oiCYIv3CL+AaB7WoiqJkOWSv+zQn5vqWDaFTVEPOl1400wqtS5yUHvJTdCM+khv+eR8LfwjJh+mvapV5228fNWTLz3FSINZmmwrZXFBScPPZ8D84Wk/NO7D27XOXZH2OWDhYVFK1hLYaehEhMVV1K9uXgFMcPDQZ/n00/2G5ehzIbfFTeQ5EUvyr7sUMCb8Qs1KjESdz1S0LjqTBLRgdnJrKlrnVeSEw54yw1jWYR8jlcubpCa3+AfRmoAABl5SURBVCaGoDnuetaLQbiFBFxOWJYdWaFEaTsTHRn2J9SlDSHougmLZUNS00QfxF3XW0o0az2KsN/nJY0Zi6Ux7noFa0doSbs2XcCKX3ywhEFa1u+L4u3L7L9kYAEXvvKN/lW1XefYHWEtBQsLi1awlsJOQwqggh1RM/+ZXIikjVrn7NeRjz6ScundNEKwSDMFu6cnXHAGwYDfIyQNCRnwOV6ewvr61laBz3GI6brdEH2RuOuVoWsZxRjbwD1oZujihohX08EIrVZHYtTpsSb1e39NpV5W3eRxBUZAJu4mSMdva8QiibkiJSefxSowQVpxElaMScN2nLjntjWWUHbI75GwB//ugg0fbwLVUtavvqyGksyN/x+2BVe9Phc3ZubV03VrIkrfaOOIPQPWUrCwsGgFaynsNByMsQjiNSLFzhwNjhkyYOPm3TK9gB1TI8Gsqyuaot7a36zXS5sSWZKGgW+KuTSoBLzRKti/hUaDya0wOQodkoKeB8DwB02xhCaD4RJMv/yO43kfTC5GYV3zRrkJpj/pQb8nRutvEb5coftMlmRTLO7pMxi3o7GtAo7jaSu0zI8w1oZ5SB2TAhzRU8OVMzZdmWvmi1PlQzTCmqL28QB11c8CRhw2Tbd7frizHRR2GlLwlgkZQni998wUAMa2NSh0zvBeRkMcmmIyDdE4ucnyX2UGgqrmmPdimBe1LpLQMDRmvnHxNcZcMlWVqSBVrpMW9Hs1G0zF6J7pIa/obKfk1j+PtXUR70VeqcuZjJDfe4GLNliy9MnweQOKcZvWRGI06ABXpfe3pq7Zi4o0yxiTLJXk93luWJPmHXddL0Xc5FH0Sg9z7RO/Zku48eHP9FMlkZo2pT62Ees32O75sMsHCwuLVrCWwk5DKp6bqkEe84ffSNbe2Laa9+5KRkjEPkx25Lq6hOCICcwxy4iKppg3oxtC0Ock8iCMAW6Clzq2kIaraBFQZFx7plZDzE0ELRliz8slqE8sQczMv7Yu4p3bnMOQkEGf47kMzT1lhvyeWnWSzwQlOR75adyyIc8t62OlBl2ZazZE456uorEYLrjw8LaeagtIu6lfG/GeLHBnbOGYfRPWUrCwsGgFaynsLGRlQqUSjBFRpHt2kYbEEiGRp68I57JcJdcMYWfyAbqkBL1ZcnWLUGVTByEnbLIIm7zgJsPD1TRLm4E5yZ7IitFOqGpRANcEILlugqMw5zLaCR1a1KI05KJLYgbvnJxQRQYJMjKkpiEfY3EIBE1ehvSjIDVElp7b5FkYiyfoc7zwacMtBBzHC+wqVBIy54zj2Bye/MNT5k51m8kmlAL3edhBYWeh8jPwa4GS2DoAik1xmBlfw2GHbnBAkG79OgLgWy6y5cYsD/gcT+vQEyHx+1ilA0RDSoLZNyXZTO5AfVSOa4jGNxJxqYvGvRfziE7Cnr+6osJbItS3UIkGWFrTRB8l9kzcRNDntFq+QCKKsaY55iVJmQrZaUHHG9hMjERhfbM3eJiYC2PC1kZipAVaezf6ZITx69Lj+UcuZmtw2S3P6qeQbj/equP2Rdjlg4WFRStYS2GnYSE5nUXco3xNGQCRcsnG++sf3+Xn/9vQUoCsy44HoK8WT90/S/YXN0Q9098sB6piMQZqbMGsEnEn9s1M8rISzcxsCLwVtU1eXYbaaGLZYJYepu5Cz/SwF+NgYMjKZL/PE0tJbuEONecw2Z3GbZni9xF15frmnMkBnxd/YSjRZL/PO8aQmuZcGSG/V8Y+T9O1+2YmkWvUoQe2VS6uNSbe+A9A/g9IlVR16vYeF+KOhrUULCwsWsExsenfaye07sPeB3XRdR4PQOU6iXZL7diZ2pLHN3lU9LEXAPjTG6IkHPY5LNM1vIkGXF3X7JWlL/QiCRMZiwY5yjGUNcXQpb4nvJIS8HkRhKYcXeeUIBUaXZi/QSn6zFAiQtG4HaNx1+MSyjfQX0gP+j3Xoul35xSzpk/oL0TjbsLVqtfuqfkfA7KTE65ItVLG9ulAr7t/uMnn56FUrAOn0zHg7yf7Yi9v+bi9F7Nc1z1kS43s8mGnQqtHVy0CIJAjoit1pWuo+GAaANnHH73RUYFLxwDQd6r41N9cVel9Z/QKo67LvHIx+Y2vfljHVMo0QtGkVZe18CQYE90UfkkPJlSTShsN65/4vsTTcjRFZHykxFoLqvTJCHtEpBnZjaBKYyxOVbOWlFNyM+x3PKKzUds1Efd0I81AZJKygj7H68+oAbnSjxsu3eiZtQUn/3z9lATx+GbbWiRglw8WFhatYC2FXYD6erEK+nQbDMDS8lqGjH8CgFWFG1sKBDsAMDBXRVoao16U4SytEp2fEvRmfuOyK6xv9iIHTaSicRemBf2eq9GkVTeF/Z54i5mNHRyPkDTnqFZyMy3F7+UymBiGxljc02Q0hVpNfsaaumYvtsEQpZ1TghQ3RL3PoEVp9BgjwGJiEoI+h6P27yT7ttJCGHf4DfJBXcEED4CILVS2tbCWgoWFRSts0VJwHOdx4BSg2HXdgbovB3ge6AmsAM51XbdCa0r+DTgZCR27zHXd2Tun67saJgIxstlWm8PShRow4/Ri9TqtXlSs9QRyu23U/oBbzgBg0tX/9lyAwRYKy0uqhXwcmS+l19fWNXszvifw2iL3wUQJGsGTjKCP+RUyu5soxK6pQY+sNASjybhcVdvMAdlJrfrRHHO97EzDFRhfY0444PU7PZTof43hC0yYIwkOIcvjHqR9h6QAqddvWoC1JT7789MAvD7DBCZ11JNX0p7/t30NW2MpPAGcuMG+m4BJruv2Aybp3yCl6Pvpv2uAR3ZMNy0sLHYVtmgpuK47zXGcnhvsHgeM1s9PAlOA3+j+p1zxc37uOE6W4zidXdddt6M6vKtx+y0PADDh9z8HoHfHoSwvm7OdZ9Pjsg+C8hUA3HLtvwG465XbNm6eVQDAT28/mQcnvA3g6Q40RONeqLHRVYi5rscppKToTKtr+kjc9bIeTXjxsuomL0fCrO+j8URYs3EnGqugKRb3PAsm67A+GmdVqQRPGS9IlxTDOyTmHMNnxFyXTupBMedKCjhe/UkTim0Coo7s1wFCmxZNMaiY8gkjfvW0/qVCrE5v7eoLWzy+vbh5/G8AuPuJO+XSTninX3NnYaviFHRQeLPF8qHSdd2sFt9XuK6b7TjOm8C9rutO1/2TgN+4rjtzC+ffbeMUhnU5GICZa2fJjvJ1OB26tPOsAyBwoHyMSnq1677FZsfoYtFv/PX4JwGJVFxb31qFuKo5llAsMrqNumTomJSoOm1e8ribEC4xZn6/zLBHGCYGEfmBr6+PeEVYTHHYuJuIeDRJTQdrKbzF1Y1ezIIpd7eoqtEjKbulJnIfTM6DqWB9VL7EYBz88I8wWpdtwV0sasq+AXe0KOqSqlvzE31vk8fvKCTeI1lyOU5w042/P3wvcQpOG/vafOEdx7kGWWJYWFjsRtjeQWG9WRY4jtOZRImdNUBLxqwrm8hPdV13IjARdm9LYVbhBjxpTmeCSRI/H2lcuZ1n/ZZg6hA5R5XMiGMG/pxJ8x7e9CG5fQD444vXA3D3RQ97S4MlmnKdEfJ7uRFmmWHcirXRuLcvoBmGy2savTwII/dWG4l7Zr+Z/ExgUWMsTq5f2hk3aHM8TkWT1mBQctBEX+aEA56gS8tCtoZfNBGKPdPCXom68YfIkinzpgv1xjdhJayV8my+QXfJ39GVgJaJ8/eU7S6IXrz9ut+3+nv0gWN2+jV3NrbXJfk6MF4/jwdea7H/UkdwOFC1J/MJFhb7IrbIKTiO8yxCKnZE1ClvB14FXgC6A6uAc1zXLVeX5MOIt6IeuHxLfIJeY7e1FAxGHXkyAFOmvwWVUsLcyc5t93mdDpcD4JYVMvMNIauGnXLMlg+sKuXX50v9wqXqmnRIBA4dagKfNFBobV0zmRqubArIrq2PML9cNB5ykuS7muaE1Jk5l5ntG2LxFrJwMsv3zUhilhKNnvCK/m/mJm9siGaFAh6BacjH9KCPK047AIDARWdu+d5L1xPufSMAzTWmoKsP/F3lY8xUbVqw5XO1E4n3x9Sk2K1Df7aKU7AJUdsI140C8mMe2UPSn6ev2uK4txmIyZuacx515YvlGrH35avNlD0DoFmqR3/4i0f17xjHHCuMu39YXwC+e0bUi38/eRkj8mSpYgrGLKlu8qpIp+iSYXl1k/dyb4gkv88TSDFxBwOykr14CSOCYmTlG2JxzxOR7MnROx4JaiIhz/vhCBgxcvP3CrBCiESn/23QVKE7Vbgm3B2adaXqTtnyuXYAJr80idFnHQvA6J6yHJy6cu4uufZ2YqsGhd16WLOwsNj1sJbCNmJo5/34qtC4vtRsd9pXfkxwKKagSChJCp82Nby6A86riNTh/u8DAFyVhfvr+0tYoTJvhpBc3xDBZF8babceSlCWNcW8HIzK5kSkpPkJLVWCcWC2WD9hv+NZDYZIDDoOI9TdeMDvz5YD0zfv4v1u4ovS/tondU8ZnrsxLMVk/eEYsernt+ZJ7ADItV23AirFYnGyc3bRtdsFaylYWFhsO6ylsB1wV6lDpVs+AK/dO5HTb752B5z5VN2KfkJKisyodXVv74Bzt4GmWs48ViLwemjQUF0k5rkPm9R6MDN/3E2IoJh8h6rmmJcbYURWhqsIbNDneMVmTaDS0IM6k3zNWdqBzQX4CD8x+Ud/59gn1S3cYLzbcQiJdRFIVqGWqme27d7bAbdJVbpDaS2sxKZNtt+NYC0FCwuLbYe1FNqBls9OEkcBKtpuvE04RberZBMswG02oSA7J3z2tcv+CMCc8novWMlYDF1UQi2O6/EHxjU5u6zOk1LrqtbAEA1zPrlPDgNO3l8OOGaEXmkLHpX6agDOH3s3AM9PXwiU6pfCtfhy8kgLqdZD0RPbdqPtwG8vvRmAPzwpfXvxgb9z7g0/3WXX3wGwLsmdjZNOugKAt99+DBDXnuMk74AzqwEXUJ99tAyQWICFH/4FgP3GHLkDrtMGKotY96935VpLROOwUMvX1UZiHmFoalIsqWpi3DHiBu17mUbzpedvx4U1KjPzKgBqqjWlnIAnOuPvkAlArGQhxHZ13YZsXLe81R7HaSuqf7eGXT5YWFhsO6ylsANQs2g1af0kmm7KExJvf8zlZ23ukG1D8tmEQmKBNFfJDNo7vydL59wj3+duz8y8GyFWj5N8gXyOGJFaIT59mV3o3VlEZJZ8Z8jEHbFE2za4bgwzhzpOmu6t2+X9aCespWBhYbHtsJbCDsKGz3F0v1FMXTJtB15BchnCnc6V60XiNGsORmaWiJBMmngpw845YQdec+fik6fF1XrUJXdiQsdBCNucbpItWb7uO4hO/R56J1g+VwLVeg7aj9HHSubm1MnPbu6Q3RmWaNy1EMVh1y329jiOagSakmU7FD05cIDENRhvQdGKCtC8hR7dhJS7/lDJZB988iDGXGTSendEBOb2YeFbn9H/VEn8wi3RvV0hTZZAqSZeosgsFRp3bQcVkx8WFafRP7kIgCkfTuGY47ciUW33hl0+WFhYbDuspbCDMWqkaNxOmfaOt2/nWgxgTO7sDsfRNV9m3BIt6FqkuQ1JAf//t3f2sVlWVwD/nVBaBIIVEKxUPlxwfIlgHYGxhDJh2E5xMqOQJpBhRlhMxtgSFfsHLwluEJkDFgU/pkzH1xQVhCkyaEnQUVazrSvQDhA2GVDGUGQyCYyzP+594X36RUv73velnF9y8/Cc+7y9h9On5z33Pvc5hy9r3FuV8dQ68wsGMKBwCADf/c4IJ7wxt/XUqtjL9NkrAHhl204vPMvFZCiZLroqGNaHD6tdmH7q1DutN/4V8PBDLhfnmrUuNyc1/jX5m1r+mnwaYJGCYRjNxyKFJPHcrPn8YHFxRJZ/u9twtL3ywwAa+EKunVwEkJWRw6De7n2BL3yi1XMXlCO+jP3Z0z6KOK9kdnGLmnf1cGsP4/NdcdZ2t+Vw3629Adj9qXvTct+mD1i4zSUz+fLzM37s+K7LMyBuB+P1Xd1iaOcuWReTq1Qd8AuxF5OipJapk3/Mr1f/3J+5SEsks+EPXH3YQmOqWfvYAgAeWvh4RF763nbGFuSnQKPGiN/8OZDRDwDp4KYiesH9EbfrmEWfbi70//iQ292XcV0m1/kK1Gf9vZTlFztPn6qB/+zzP/eAP0YzUKcD5Tt2AZA3+mvEU9eL1J9o5irHpg+GYTQfixQCMGaEmzaUlu2o0zfxWy7RyDtb1gXV6VpnzKBxlO7eEhWePIl065YahcJgkYJhGM3HIoUUULPnY3oM7FdLquTf7jYXba8sCa9UG2fMwPEAlO7Z4CWXNnDFZsQAmPfivMBaBcciBcMwmo9FCinDbWjSeN2CztfXuaJ4qkvx9tPXXgimVVti7gz3SDj2/Pw6fWVlFYwceUdolVKNPZK8uuiOnvBZorvVzQxc+vYHAIx94NteciqQXlcJ7XtzsMRlp+o7elid7tJ33Y7KsYWjgqqVZtj0wTCMK0BVG23Ay7gCspUJsqeBKqACeAvITuibA+wHqoEJl/v5/jNqrW4rWfW+lqx6XxtiyaJXdcmiVxU6+pZ6ncO0G3Xd8vW6bvn6Bm2jqvrUgqX61IKlaaBv2rTypvw9NiVSWIGrDZnIFmCIqg4F/oZzBIjIIGAyMNh/5jlpo1vDDKPN0sRv8r4kRAq1+h4AViZECXMS+jYDoyxSaHkrKpykRYWTVPWsb/VzqKRcD5WUa+HQcVo4dFzK9W5pmznufr1QWaUXKqsajQr02DEdkZunI3LzUq5zGrcmRQp1ywI3n+lAvF5XL2BnQt9hLzNayMrfudyPKyXromzuzO8DEFv2Sy/Jok9+HgCb/lJrtx7AZ8fc9YvXAPCbF3/LgSN/SJLGl6d9h8EAPDl7CrFZU52w5y2X/+C/j1M0xT2ZWbWlFUvrGQAtcwoiUgycB+JpcurLea0NfHYGMKMl4xuG0fo06ZGkiPQFNqrqkATZNGAmcLeqnvGyOQCq+jN/vhmIqWqjX0f2SLL1GNxzOACxmNud9+DM+xq7/DL4VGhHatzxwEl2HnHFWipOulenT/z3HNm+BsRXurgoJq+7e/W6e+9suM0lUiGjuQVYXYq5WOwl5s170svCZ3FuYyTvkaSI3AM8DkyMOwTPBmCyiGSJSD+gP7DrSsYwDCM1XDZSEJHVQD5uC14NMBe3oJjFpfxiO1V1pr++GLfOcB74kaq+W/tn1jOGRQpBcMlP5j46DYCheQ8z6Xv5vq81lpeayGkXecSecZWo/v7711mxY1O48a9dbEej0Vp08sde3NTDZVC6uaNzMCfP/o9DR+Pl1OLZmaPl1Yy0wXY0GobRfCxSMIxrB4sUDMNoPuYUDMOIYE7BMIwI5hQMw4hgTsEwjAjmFAzDiBBwG1ujnAC+8MdU0x3TIxHTI8rVrEefplyUFvsUAESkvCnPUE0P08P0SK4eNn0wDCOCOQXDMCKkk1NIl+IGpkcU0yNKm9cjbdYUDMNID9IpUjAMIw1IC6cgIveISLWI7BeRJwKNeYuIlIjIXhHZLSKzvLyriGwRkX3+eEMgfdqJyJ9EZKM/7yciZV6PtSKSGUCHbBF5Q0SqvF1GpcIeIjLb/04qRWS1iHQIZQ8ReVlEjotIZYKsXhuIY6m/bytE5M4k6/G0/91UiMhbIpKd0DfH61EtIhNaMnbKnYKvC/EsUAAMAqb4+hHJ5jzwE1UdCIwEHvXjPgFsVdX+wFZ/HoJZwN6E84XAL7wenwKPBNBhCfCeqg4A7vD6BLWHiPQCfgjc5XOCtsPVEglljxXUrXPSkA0KcCkH++OSEC9Lsh5h6q00JQ98MhswCticcB6pHRFQj/XAeFxlqxwvywGqA4ydi7vZvglsxGXFPgFk1GejJOnQBTiIX2dKkAe1B64kwCdAV9zmuo3AhJD2oFadk4ZsADwPTKnvumToUauvxfVWGmopjxS4dBPECV4rwmerHg6UAT1V9SiAP/YIoMJi4DHggj/vBnymquf9eQib3IrLp/aKn8a8JCKdCGwPVf0nsAj4B3AUV0n3I8LbI5GGbJDKe3c6EM9/2qp6pINTaHKtiKQMLtIZWIdLMvt5qHETxr8XOK6qHyWK67k02TbJAO4ElqnqcNy281BTp4v4+fr9QD/gZlyCyIJ6Lk2Hx2YpuXdbUm+lKaSDUzgMJJYFygWOhBhYRNrjHMJKVX3Ti2tEJMf35+CK6yaT0cBEETkErMFNIRYD2SISfzclhE0OA4dVtcyfv4FzEqHtMQ44qKr/UtVzwJvA1wlvj0QaskHwe9fXW7kXKFI/V2htPdLBKfwR6O9XlzNxCyYbkj2oiAjwK2Cvqj6T0LUBmOb/PQ231pA0VHWOquaqal/c/32bqhYBJcCDAfU4BnwiIl/1oruBPQS2B27aMFJEOvrfUVyPoPaoRUM22ABM9U8hRgKn4tOMZBCs3koyF42asaBSiFtNPQAUBxrzG7gQqwL4s2+FuPn8VmCfP3YNaId8XCUucHP8XcB+4HUgK8D4w4Byb5O3gRtSYQ9gHlAFVAKv4WqMBLEHsBq3lnEO9w38SEM2wIXtz/r79q+4JybJ1GM/bu0gfr8uT7i+2OtRDRS0ZGzb0WgYRoR0mD4YhpFGmFMwDCOCOQXDMCKYUzAMI4I5BcMwIphTMAwjgjkFwzAimFMwDCPC/wEk8ojMFEt+jgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "<Figure size 432x288 with 1 Axes>" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plt.imshow(\n", + " _read(path_train_img + 'ID_ffff922b9.dcm', (128, 128))\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [], + "source": [ + "# Augmentations\n", + "# Flip Left Right\n", + "# Cropping\n", + "sometimes = lambda aug: iaa.Sometimes(0.25, aug)\n", + "augmentation = iaa.Sequential([ \n", + " iaa.Fliplr(0.25),\n", + " sometimes(iaa.Crop(px=(0, 25), keep_size = True, \n", + " sample_independently = False)) \n", + " ], random_order = True)" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [], + "source": [ + "# Train Data Generator\n", + "class TrainDataGenerator(keras.utils.Sequence):\n", + "\n", + " def __init__(self, dataset, labels, batch_size=16, img_size=(512, 512), img_dir = path_train_img, \\\n", + " augment = False, *args, **kwargs):\n", + " self.dataset = dataset\n", + " self.ids = dataset.index\n", + " self.labels = labels\n", + " self.batch_size = batch_size\n", + " self.img_size = img_size\n", + " self.img_dir = img_dir\n", + " self.augment = augment\n", + " self.on_epoch_end()\n", + "\n", + " def __len__(self):\n", + " return int(ceil(len(self.ids) / self.batch_size))\n", + "\n", + " def __getitem__(self, index):\n", + " indices = self.indices[index*self.batch_size:(index+1)*self.batch_size]\n", + " X, Y = self.__data_generation(indices)\n", + " return X, Y\n", + "\n", + " def augmentor(self, image):\n", + " augment_img = augmentation \n", + " image_aug = augment_img.augment_image(image)\n", + " return image_aug\n", + "\n", + " def on_epoch_end(self):\n", + " self.indices = np.arange(len(self.ids))\n", + " np.random.shuffle(self.indices)\n", + " \n", + " def __data_generation(self, indices):\n", + " X = np.empty((self.batch_size, *self.img_size, 3))\n", + " Y = np.empty((self.batch_size, 6), dtype=np.float32)\n", + " \n", + " for i, index in enumerate(indices):\n", + " ID = self.ids[index]\n", + " image = _read(self.img_dir + ID, self.img_size)\n", + " if self.augment:\n", + " X[i,] = self.augmentor(image)\n", + " else:\n", + " X[i,] = image \n", + " Y[i,] = self.labels.iloc[index].values \n", + " return X, Y\n", + " \n", + "class TestDataGenerator(keras.utils.Sequence):\n", + " def __init__(self, ids, labels, batch_size = 5, img_size = (512, 512), img_dir = path_test_img, \\\n", + " *args, **kwargs):\n", + " self.ids = ids\n", + " self.labels = labels\n", + " self.batch_size = batch_size\n", + " self.img_size = img_size\n", + " self.img_dir = img_dir\n", + " self.on_epoch_end()\n", + "\n", + " def __len__(self):\n", + " return int(ceil(len(self.ids) / self.batch_size))\n", + "\n", + " def __getitem__(self, index):\n", + " indices = self.indices[index*self.batch_size:(index+1)*self.batch_size]\n", + " list_IDs_temp = [self.ids[k] for k in indices]\n", + " X = self.__data_generation(list_IDs_temp)\n", + " return X\n", + "\n", + " def on_epoch_end(self):\n", + " self.indices = np.arange(len(self.ids))\n", + "\n", + " def __data_generation(self, list_IDs_temp):\n", + " X = np.empty((self.batch_size, *self.img_size, 3))\n", + " for i, ID in enumerate(list_IDs_temp):\n", + " image = _read(self.img_dir + ID, self.img_size)\n", + " X[i,] = image \n", + " return X" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As we have seen in EDA notebook that we have very few epidural subtypes so we need oversample this sub type" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Train Shape: (677018, 6)\n" + ] + } + ], + "source": [ + "# Oversampling\n", + "epidural_df = train_final_df[train_final_df.epidural == 1]\n", + "train_final_df = pd.concat([train_final_df, epidural_df])\n", + "print('Train Shape: {}'.format(train_final_df.shape))" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th></th>\n", + " <th>ID</th>\n", + " <th>Label</th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>0</th>\n", + " <td>ID_28fbab7eb_epidural</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " <tr>\n", + " <th>1</th>\n", + " <td>ID_28fbab7eb_intraparenchymal</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " <tr>\n", + " <th>2</th>\n", + " <td>ID_28fbab7eb_intraventricular</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " <tr>\n", + " <th>3</th>\n", + " <td>ID_28fbab7eb_subarachnoid</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " <tr>\n", + " <th>4</th>\n", + " <td>ID_28fbab7eb_subdural</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + " ID Label\n", + "0 ID_28fbab7eb_epidural 0.5\n", + "1 ID_28fbab7eb_intraparenchymal 0.5\n", + "2 ID_28fbab7eb_intraventricular 0.5\n", + "3 ID_28fbab7eb_subarachnoid 0.5\n", + "4 ID_28fbab7eb_subdural 0.5" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# load test set\n", + "test_df = pd.read_csv(input_folder + 'stage_1_sample_submission.csv')\n", + "test_df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(78545, 6)" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# extract subtype\n", + "test_df['sub_type'] = test_df['ID'].apply(lambda x: x.split('_')[-1])\n", + "# extract filename\n", + "test_df['file_name'] = test_df['ID'].apply(lambda x: '_'.join(x.split('_')[:2]) + '.dcm')\n", + "\n", + "test_df = pd.pivot_table(test_df.drop(columns='ID'), index=\"file_name\", \\\n", + " columns=\"sub_type\", values=\"Label\")\n", + "test_df.head()\n", + "\n", + "test_df.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th>sub_type</th>\n", + " <th>any</th>\n", + " <th>epidural</th>\n", + " <th>intraparenchymal</th>\n", + " <th>intraventricular</th>\n", + " <th>subarachnoid</th>\n", + " <th>subdural</th>\n", + " </tr>\n", + " <tr>\n", + " <th>file_name</th>\n", + " <th></th>\n", + " <th></th>\n", + " <th></th>\n", + " <th></th>\n", + " <th></th>\n", + " <th></th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>ID_000012eaf.dcm</th>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " <tr>\n", + " <th>ID_0000ca2f6.dcm</th>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " <tr>\n", + " <th>ID_000259ccf.dcm</th>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " <tr>\n", + " <th>ID_0002d438a.dcm</th>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " <tr>\n", + " <th>ID_00032d440.dcm</th>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + "sub_type any epidural intraparenchymal intraventricular \\\n", + "file_name \n", + "ID_000012eaf.dcm 0.5 0.5 0.5 0.5 \n", + "ID_0000ca2f6.dcm 0.5 0.5 0.5 0.5 \n", + "ID_000259ccf.dcm 0.5 0.5 0.5 0.5 \n", + "ID_0002d438a.dcm 0.5 0.5 0.5 0.5 \n", + "ID_00032d440.dcm 0.5 0.5 0.5 0.5 \n", + "\n", + "sub_type subarachnoid subdural \n", + "file_name \n", + "ID_000012eaf.dcm 0.5 0.5 \n", + "ID_0000ca2f6.dcm 0.5 0.5 \n", + "ID_000259ccf.dcm 0.5 0.5 \n", + "ID_0002d438a.dcm 0.5 0.5 \n", + "ID_00032d440.dcm 0.5 0.5 " + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "test_df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Downloading data from https://github.com/Callidior/keras-applications/releases/download/efficientnet/efficientnet-b0_weights_tf_dim_ordering_tf_kernels_autoaugment_notop.h5\n", + "16809984/16804768 [==============================] - 1s 0us/step\n", + "Model: \"model_1\"\n", + "__________________________________________________________________________________________________\n", + "Layer (type) Output Shape Param # Connected to \n", + "==================================================================================================\n", + "input_1 (InputLayer) (None, 256, 256, 3) 0 \n", + "__________________________________________________________________________________________________\n", + "stem_conv (Conv2D) (None, 128, 128, 32) 864 input_1[0][0] \n", + "__________________________________________________________________________________________________\n", + "stem_bn (BatchNormalization) (None, 128, 128, 32) 128 stem_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "stem_activation (Activation) (None, 128, 128, 32) 0 stem_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block1a_dwconv (DepthwiseConv2D (None, 128, 128, 32) 288 stem_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block1a_bn (BatchNormalization) (None, 128, 128, 32) 128 block1a_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block1a_activation (Activation) (None, 128, 128, 32) 0 block1a_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block1a_se_squeeze (GlobalAvera (None, 32) 0 block1a_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block1a_se_reshape (Reshape) (None, 1, 1, 32) 0 block1a_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block1a_se_reduce (Conv2D) (None, 1, 1, 8) 264 block1a_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block1a_se_expand (Conv2D) (None, 1, 1, 32) 288 block1a_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block1a_se_excite (Multiply) (None, 128, 128, 32) 0 block1a_activation[0][0] \n", + " block1a_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block1a_project_conv (Conv2D) (None, 128, 128, 16) 512 block1a_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block1a_project_bn (BatchNormal (None, 128, 128, 16) 64 block1a_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2a_expand_conv (Conv2D) (None, 128, 128, 96) 1536 block1a_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2a_expand_bn (BatchNormali (None, 128, 128, 96) 384 block2a_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2a_expand_activation (Acti (None, 128, 128, 96) 0 block2a_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2a_dwconv (DepthwiseConv2D (None, 64, 64, 96) 864 block2a_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2a_bn (BatchNormalization) (None, 64, 64, 96) 384 block2a_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2a_activation (Activation) (None, 64, 64, 96) 0 block2a_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2a_se_squeeze (GlobalAvera (None, 96) 0 block2a_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2a_se_reshape (Reshape) (None, 1, 1, 96) 0 block2a_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2a_se_reduce (Conv2D) (None, 1, 1, 4) 388 block2a_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2a_se_expand (Conv2D) (None, 1, 1, 96) 480 block2a_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2a_se_excite (Multiply) (None, 64, 64, 96) 0 block2a_activation[0][0] \n", + " block2a_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2a_project_conv (Conv2D) (None, 64, 64, 24) 2304 block2a_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2a_project_bn (BatchNormal (None, 64, 64, 24) 96 block2a_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_expand_conv (Conv2D) (None, 64, 64, 144) 3456 block2a_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_expand_bn (BatchNormali (None, 64, 64, 144) 576 block2b_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_expand_activation (Acti (None, 64, 64, 144) 0 block2b_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_dwconv (DepthwiseConv2D (None, 64, 64, 144) 1296 block2b_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_bn (BatchNormalization) (None, 64, 64, 144) 576 block2b_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_activation (Activation) (None, 64, 64, 144) 0 block2b_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_se_squeeze (GlobalAvera (None, 144) 0 block2b_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_se_reshape (Reshape) (None, 1, 1, 144) 0 block2b_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_se_reduce (Conv2D) (None, 1, 1, 6) 870 block2b_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_se_expand (Conv2D) (None, 1, 1, 144) 1008 block2b_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_se_excite (Multiply) (None, 64, 64, 144) 0 block2b_activation[0][0] \n", + " block2b_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_project_conv (Conv2D) (None, 64, 64, 24) 3456 block2b_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_project_bn (BatchNormal (None, 64, 64, 24) 96 block2b_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_drop (FixedDropout) (None, 64, 64, 24) 0 block2b_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block2b_add (Add) (None, 64, 64, 24) 0 block2b_drop[0][0] \n", + " block2a_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3a_expand_conv (Conv2D) (None, 64, 64, 144) 3456 block2b_add[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3a_expand_bn (BatchNormali (None, 64, 64, 144) 576 block3a_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3a_expand_activation (Acti (None, 64, 64, 144) 0 block3a_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3a_dwconv (DepthwiseConv2D (None, 32, 32, 144) 3600 block3a_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3a_bn (BatchNormalization) (None, 32, 32, 144) 576 block3a_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3a_activation (Activation) (None, 32, 32, 144) 0 block3a_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3a_se_squeeze (GlobalAvera (None, 144) 0 block3a_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3a_se_reshape (Reshape) (None, 1, 1, 144) 0 block3a_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3a_se_reduce (Conv2D) (None, 1, 1, 6) 870 block3a_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3a_se_expand (Conv2D) (None, 1, 1, 144) 1008 block3a_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3a_se_excite (Multiply) (None, 32, 32, 144) 0 block3a_activation[0][0] \n", + " block3a_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3a_project_conv (Conv2D) (None, 32, 32, 40) 5760 block3a_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3a_project_bn (BatchNormal (None, 32, 32, 40) 160 block3a_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_expand_conv (Conv2D) (None, 32, 32, 240) 9600 block3a_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_expand_bn (BatchNormali (None, 32, 32, 240) 960 block3b_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_expand_activation (Acti (None, 32, 32, 240) 0 block3b_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_dwconv (DepthwiseConv2D (None, 32, 32, 240) 6000 block3b_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_bn (BatchNormalization) (None, 32, 32, 240) 960 block3b_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_activation (Activation) (None, 32, 32, 240) 0 block3b_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_se_squeeze (GlobalAvera (None, 240) 0 block3b_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_se_reshape (Reshape) (None, 1, 1, 240) 0 block3b_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_se_reduce (Conv2D) (None, 1, 1, 10) 2410 block3b_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_se_expand (Conv2D) (None, 1, 1, 240) 2640 block3b_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_se_excite (Multiply) (None, 32, 32, 240) 0 block3b_activation[0][0] \n", + " block3b_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_project_conv (Conv2D) (None, 32, 32, 40) 9600 block3b_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_project_bn (BatchNormal (None, 32, 32, 40) 160 block3b_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_drop (FixedDropout) (None, 32, 32, 40) 0 block3b_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block3b_add (Add) (None, 32, 32, 40) 0 block3b_drop[0][0] \n", + " block3a_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4a_expand_conv (Conv2D) (None, 32, 32, 240) 9600 block3b_add[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4a_expand_bn (BatchNormali (None, 32, 32, 240) 960 block4a_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4a_expand_activation (Acti (None, 32, 32, 240) 0 block4a_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4a_dwconv (DepthwiseConv2D (None, 16, 16, 240) 2160 block4a_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4a_bn (BatchNormalization) (None, 16, 16, 240) 960 block4a_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4a_activation (Activation) (None, 16, 16, 240) 0 block4a_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4a_se_squeeze (GlobalAvera (None, 240) 0 block4a_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4a_se_reshape (Reshape) (None, 1, 1, 240) 0 block4a_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4a_se_reduce (Conv2D) (None, 1, 1, 10) 2410 block4a_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4a_se_expand (Conv2D) (None, 1, 1, 240) 2640 block4a_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4a_se_excite (Multiply) (None, 16, 16, 240) 0 block4a_activation[0][0] \n", + " block4a_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4a_project_conv (Conv2D) (None, 16, 16, 80) 19200 block4a_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4a_project_bn (BatchNormal (None, 16, 16, 80) 320 block4a_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_expand_conv (Conv2D) (None, 16, 16, 480) 38400 block4a_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_expand_bn (BatchNormali (None, 16, 16, 480) 1920 block4b_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_expand_activation (Acti (None, 16, 16, 480) 0 block4b_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_dwconv (DepthwiseConv2D (None, 16, 16, 480) 4320 block4b_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_bn (BatchNormalization) (None, 16, 16, 480) 1920 block4b_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_activation (Activation) (None, 16, 16, 480) 0 block4b_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_se_squeeze (GlobalAvera (None, 480) 0 block4b_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_se_reshape (Reshape) (None, 1, 1, 480) 0 block4b_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_se_reduce (Conv2D) (None, 1, 1, 20) 9620 block4b_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_se_expand (Conv2D) (None, 1, 1, 480) 10080 block4b_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_se_excite (Multiply) (None, 16, 16, 480) 0 block4b_activation[0][0] \n", + " block4b_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_project_conv (Conv2D) (None, 16, 16, 80) 38400 block4b_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_project_bn (BatchNormal (None, 16, 16, 80) 320 block4b_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_drop (FixedDropout) (None, 16, 16, 80) 0 block4b_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4b_add (Add) (None, 16, 16, 80) 0 block4b_drop[0][0] \n", + " block4a_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_expand_conv (Conv2D) (None, 16, 16, 480) 38400 block4b_add[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_expand_bn (BatchNormali (None, 16, 16, 480) 1920 block4c_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_expand_activation (Acti (None, 16, 16, 480) 0 block4c_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_dwconv (DepthwiseConv2D (None, 16, 16, 480) 4320 block4c_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_bn (BatchNormalization) (None, 16, 16, 480) 1920 block4c_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_activation (Activation) (None, 16, 16, 480) 0 block4c_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_se_squeeze (GlobalAvera (None, 480) 0 block4c_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_se_reshape (Reshape) (None, 1, 1, 480) 0 block4c_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_se_reduce (Conv2D) (None, 1, 1, 20) 9620 block4c_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_se_expand (Conv2D) (None, 1, 1, 480) 10080 block4c_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_se_excite (Multiply) (None, 16, 16, 480) 0 block4c_activation[0][0] \n", + " block4c_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_project_conv (Conv2D) (None, 16, 16, 80) 38400 block4c_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_project_bn (BatchNormal (None, 16, 16, 80) 320 block4c_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_drop (FixedDropout) (None, 16, 16, 80) 0 block4c_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block4c_add (Add) (None, 16, 16, 80) 0 block4c_drop[0][0] \n", + " block4b_add[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5a_expand_conv (Conv2D) (None, 16, 16, 480) 38400 block4c_add[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5a_expand_bn (BatchNormali (None, 16, 16, 480) 1920 block5a_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5a_expand_activation (Acti (None, 16, 16, 480) 0 block5a_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5a_dwconv (DepthwiseConv2D (None, 16, 16, 480) 12000 block5a_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5a_bn (BatchNormalization) (None, 16, 16, 480) 1920 block5a_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5a_activation (Activation) (None, 16, 16, 480) 0 block5a_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5a_se_squeeze (GlobalAvera (None, 480) 0 block5a_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5a_se_reshape (Reshape) (None, 1, 1, 480) 0 block5a_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5a_se_reduce (Conv2D) (None, 1, 1, 20) 9620 block5a_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5a_se_expand (Conv2D) (None, 1, 1, 480) 10080 block5a_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5a_se_excite (Multiply) (None, 16, 16, 480) 0 block5a_activation[0][0] \n", + " block5a_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5a_project_conv (Conv2D) (None, 16, 16, 112) 53760 block5a_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5a_project_bn (BatchNormal (None, 16, 16, 112) 448 block5a_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_expand_conv (Conv2D) (None, 16, 16, 672) 75264 block5a_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_expand_bn (BatchNormali (None, 16, 16, 672) 2688 block5b_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_expand_activation (Acti (None, 16, 16, 672) 0 block5b_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_dwconv (DepthwiseConv2D (None, 16, 16, 672) 16800 block5b_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_bn (BatchNormalization) (None, 16, 16, 672) 2688 block5b_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_activation (Activation) (None, 16, 16, 672) 0 block5b_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_se_squeeze (GlobalAvera (None, 672) 0 block5b_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_se_reshape (Reshape) (None, 1, 1, 672) 0 block5b_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_se_reduce (Conv2D) (None, 1, 1, 28) 18844 block5b_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_se_expand (Conv2D) (None, 1, 1, 672) 19488 block5b_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_se_excite (Multiply) (None, 16, 16, 672) 0 block5b_activation[0][0] \n", + " block5b_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_project_conv (Conv2D) (None, 16, 16, 112) 75264 block5b_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_project_bn (BatchNormal (None, 16, 16, 112) 448 block5b_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_drop (FixedDropout) (None, 16, 16, 112) 0 block5b_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5b_add (Add) (None, 16, 16, 112) 0 block5b_drop[0][0] \n", + " block5a_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_expand_conv (Conv2D) (None, 16, 16, 672) 75264 block5b_add[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_expand_bn (BatchNormali (None, 16, 16, 672) 2688 block5c_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_expand_activation (Acti (None, 16, 16, 672) 0 block5c_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_dwconv (DepthwiseConv2D (None, 16, 16, 672) 16800 block5c_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_bn (BatchNormalization) (None, 16, 16, 672) 2688 block5c_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_activation (Activation) (None, 16, 16, 672) 0 block5c_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_se_squeeze (GlobalAvera (None, 672) 0 block5c_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_se_reshape (Reshape) (None, 1, 1, 672) 0 block5c_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_se_reduce (Conv2D) (None, 1, 1, 28) 18844 block5c_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_se_expand (Conv2D) (None, 1, 1, 672) 19488 block5c_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_se_excite (Multiply) (None, 16, 16, 672) 0 block5c_activation[0][0] \n", + " block5c_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_project_conv (Conv2D) (None, 16, 16, 112) 75264 block5c_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_project_bn (BatchNormal (None, 16, 16, 112) 448 block5c_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_drop (FixedDropout) (None, 16, 16, 112) 0 block5c_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block5c_add (Add) (None, 16, 16, 112) 0 block5c_drop[0][0] \n", + " block5b_add[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6a_expand_conv (Conv2D) (None, 16, 16, 672) 75264 block5c_add[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6a_expand_bn (BatchNormali (None, 16, 16, 672) 2688 block6a_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6a_expand_activation (Acti (None, 16, 16, 672) 0 block6a_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6a_dwconv (DepthwiseConv2D (None, 8, 8, 672) 16800 block6a_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6a_bn (BatchNormalization) (None, 8, 8, 672) 2688 block6a_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6a_activation (Activation) (None, 8, 8, 672) 0 block6a_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6a_se_squeeze (GlobalAvera (None, 672) 0 block6a_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6a_se_reshape (Reshape) (None, 1, 1, 672) 0 block6a_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6a_se_reduce (Conv2D) (None, 1, 1, 28) 18844 block6a_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6a_se_expand (Conv2D) (None, 1, 1, 672) 19488 block6a_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6a_se_excite (Multiply) (None, 8, 8, 672) 0 block6a_activation[0][0] \n", + " block6a_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6a_project_conv (Conv2D) (None, 8, 8, 192) 129024 block6a_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6a_project_bn (BatchNormal (None, 8, 8, 192) 768 block6a_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_expand_conv (Conv2D) (None, 8, 8, 1152) 221184 block6a_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_expand_bn (BatchNormali (None, 8, 8, 1152) 4608 block6b_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_expand_activation (Acti (None, 8, 8, 1152) 0 block6b_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_dwconv (DepthwiseConv2D (None, 8, 8, 1152) 28800 block6b_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_bn (BatchNormalization) (None, 8, 8, 1152) 4608 block6b_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_activation (Activation) (None, 8, 8, 1152) 0 block6b_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_se_squeeze (GlobalAvera (None, 1152) 0 block6b_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_se_reshape (Reshape) (None, 1, 1, 1152) 0 block6b_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_se_reduce (Conv2D) (None, 1, 1, 48) 55344 block6b_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_se_expand (Conv2D) (None, 1, 1, 1152) 56448 block6b_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_se_excite (Multiply) (None, 8, 8, 1152) 0 block6b_activation[0][0] \n", + " block6b_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_project_conv (Conv2D) (None, 8, 8, 192) 221184 block6b_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_project_bn (BatchNormal (None, 8, 8, 192) 768 block6b_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_drop (FixedDropout) (None, 8, 8, 192) 0 block6b_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6b_add (Add) (None, 8, 8, 192) 0 block6b_drop[0][0] \n", + " block6a_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_expand_conv (Conv2D) (None, 8, 8, 1152) 221184 block6b_add[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_expand_bn (BatchNormali (None, 8, 8, 1152) 4608 block6c_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_expand_activation (Acti (None, 8, 8, 1152) 0 block6c_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_dwconv (DepthwiseConv2D (None, 8, 8, 1152) 28800 block6c_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_bn (BatchNormalization) (None, 8, 8, 1152) 4608 block6c_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_activation (Activation) (None, 8, 8, 1152) 0 block6c_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_se_squeeze (GlobalAvera (None, 1152) 0 block6c_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_se_reshape (Reshape) (None, 1, 1, 1152) 0 block6c_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_se_reduce (Conv2D) (None, 1, 1, 48) 55344 block6c_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_se_expand (Conv2D) (None, 1, 1, 1152) 56448 block6c_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_se_excite (Multiply) (None, 8, 8, 1152) 0 block6c_activation[0][0] \n", + " block6c_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_project_conv (Conv2D) (None, 8, 8, 192) 221184 block6c_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_project_bn (BatchNormal (None, 8, 8, 192) 768 block6c_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_drop (FixedDropout) (None, 8, 8, 192) 0 block6c_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6c_add (Add) (None, 8, 8, 192) 0 block6c_drop[0][0] \n", + " block6b_add[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_expand_conv (Conv2D) (None, 8, 8, 1152) 221184 block6c_add[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_expand_bn (BatchNormali (None, 8, 8, 1152) 4608 block6d_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_expand_activation (Acti (None, 8, 8, 1152) 0 block6d_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_dwconv (DepthwiseConv2D (None, 8, 8, 1152) 28800 block6d_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_bn (BatchNormalization) (None, 8, 8, 1152) 4608 block6d_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_activation (Activation) (None, 8, 8, 1152) 0 block6d_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_se_squeeze (GlobalAvera (None, 1152) 0 block6d_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_se_reshape (Reshape) (None, 1, 1, 1152) 0 block6d_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_se_reduce (Conv2D) (None, 1, 1, 48) 55344 block6d_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_se_expand (Conv2D) (None, 1, 1, 1152) 56448 block6d_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_se_excite (Multiply) (None, 8, 8, 1152) 0 block6d_activation[0][0] \n", + " block6d_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_project_conv (Conv2D) (None, 8, 8, 192) 221184 block6d_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_project_bn (BatchNormal (None, 8, 8, 192) 768 block6d_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_drop (FixedDropout) (None, 8, 8, 192) 0 block6d_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block6d_add (Add) (None, 8, 8, 192) 0 block6d_drop[0][0] \n", + " block6c_add[0][0] \n", + "__________________________________________________________________________________________________\n", + "block7a_expand_conv (Conv2D) (None, 8, 8, 1152) 221184 block6d_add[0][0] \n", + "__________________________________________________________________________________________________\n", + "block7a_expand_bn (BatchNormali (None, 8, 8, 1152) 4608 block7a_expand_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block7a_expand_activation (Acti (None, 8, 8, 1152) 0 block7a_expand_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block7a_dwconv (DepthwiseConv2D (None, 8, 8, 1152) 10368 block7a_expand_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block7a_bn (BatchNormalization) (None, 8, 8, 1152) 4608 block7a_dwconv[0][0] \n", + "__________________________________________________________________________________________________\n", + "block7a_activation (Activation) (None, 8, 8, 1152) 0 block7a_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "block7a_se_squeeze (GlobalAvera (None, 1152) 0 block7a_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "block7a_se_reshape (Reshape) (None, 1, 1, 1152) 0 block7a_se_squeeze[0][0] \n", + "__________________________________________________________________________________________________\n", + "block7a_se_reduce (Conv2D) (None, 1, 1, 48) 55344 block7a_se_reshape[0][0] \n", + "__________________________________________________________________________________________________\n", + "block7a_se_expand (Conv2D) (None, 1, 1, 1152) 56448 block7a_se_reduce[0][0] \n", + "__________________________________________________________________________________________________\n", + "block7a_se_excite (Multiply) (None, 8, 8, 1152) 0 block7a_activation[0][0] \n", + " block7a_se_expand[0][0] \n", + "__________________________________________________________________________________________________\n", + "block7a_project_conv (Conv2D) (None, 8, 8, 320) 368640 block7a_se_excite[0][0] \n", + "__________________________________________________________________________________________________\n", + "block7a_project_bn (BatchNormal (None, 8, 8, 320) 1280 block7a_project_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "top_conv (Conv2D) (None, 8, 8, 1280) 409600 block7a_project_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "top_bn (BatchNormalization) (None, 8, 8, 1280) 5120 top_conv[0][0] \n", + "__________________________________________________________________________________________________\n", + "top_activation (Activation) (None, 8, 8, 1280) 0 top_bn[0][0] \n", + "__________________________________________________________________________________________________\n", + "avg_pool (GlobalAveragePooling2 (None, 1280) 0 top_activation[0][0] \n", + "__________________________________________________________________________________________________\n", + "dropout_1 (Dropout) (None, 1280) 0 avg_pool[0][0] \n", + "__________________________________________________________________________________________________\n", + "dense_1 (Dense) (None, 6) 7686 dropout_1[0][0] \n", + "==================================================================================================\n", + "Total params: 4,057,250\n", + "Trainable params: 4,015,234\n", + "Non-trainable params: 42,016\n", + "__________________________________________________________________________________________________\n" + ] + } + ], + "source": [ + "base_model = efn.EfficientNetB0(weights = 'imagenet', include_top = False, \\\n", + " pooling = 'avg', input_shape = (HEIGHT, WIDTH, 3))\n", + "x = base_model.output\n", + "x = Dropout(0.125)(x)\n", + "output_layer = Dense(6, activation = 'sigmoid')(x)\n", + "model = Model(inputs=base_model.input, outputs=output_layer)\n", + "model.compile(optimizer = Adam(learning_rate = 0.0001), \n", + " loss = 'binary_crossentropy',\n", + " metrics = ['acc', tf.keras.metrics.AUC()])\n", + "model.summary()" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(636396, 40622)" + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# https://github.com/trent-b/iterative-stratification\n", + "# Mutlilabel stratification\n", + "splits = MultilabelStratifiedShuffleSplit(n_splits = 2, test_size = TEST_SIZE, random_state = SEED)\n", + "file_names = train_final_df.index\n", + "labels = train_final_df.values\n", + "# Lets take only the first split\n", + "split = next(splits.split(file_names, labels))\n", + "train_idx = split[0]\n", + "valid_idx = split[1]\n", + "submission_predictions = []\n", + "len(train_idx), len(valid_idx)" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [], + "source": [ + "# train data generator\n", + "data_generator_train = TrainDataGenerator(train_final_df.iloc[train_idx], \n", + " train_final_df.iloc[train_idx], \n", + " TRAIN_BATCH_SIZE, \n", + " (WIDTH, HEIGHT),\n", + " augment = True)\n", + "\n", + "# validation data generator\n", + "data_generator_val = TrainDataGenerator(train_final_df.iloc[valid_idx], \n", + " train_final_df.iloc[valid_idx], \n", + " VALID_BATCH_SIZE, \n", + " (WIDTH, HEIGHT),\n", + " augment = False)" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(19888, 635)" + ] + }, + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "len(data_generator_train), len(data_generator_val)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Competition evaluation metric is evaluated based on weighted log loss but we haven't given weights for each subtype but as per discussion from this thread https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/109526#latest-630190 any has a wieght of 2 than other types below sample is taken from the discussion threas" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [], + "source": [ + "from keras import backend as K\n", + "\n", + "def weighted_log_loss(y_true, y_pred):\n", + " \"\"\"\n", + " Can be used as the loss function in model.compile()\n", + " ---------------------------------------------------\n", + " \"\"\"\n", + " \n", + " class_weights = np.array([2., 1., 1., 1., 1., 1.])\n", + " \n", + " eps = K.epsilon()\n", + " \n", + " y_pred = K.clip(y_pred, eps, 1.0-eps)\n", + "\n", + " out = -( y_true * K.log( y_pred) * class_weights\n", + " + (1.0 - y_true) * K.log(1.0 - y_pred) * class_weights)\n", + " \n", + " return K.mean(out, axis=-1)\n", + "\n", + "\n", + "def _normalized_weighted_average(arr, weights=None):\n", + " \"\"\"\n", + " A simple Keras implementation that mimics that of \n", + " numpy.average(), specifically for this competition\n", + " \"\"\"\n", + " \n", + " if weights is not None:\n", + " scl = K.sum(weights)\n", + " weights = K.expand_dims(weights, axis=1)\n", + " return K.sum(K.dot(arr, weights), axis=1) / scl\n", + " return K.mean(arr, axis=1)\n", + "\n", + "\n", + "def weighted_loss(y_true, y_pred):\n", + " \"\"\"\n", + " Will be used as the metric in model.compile()\n", + " ---------------------------------------------\n", + " \n", + " Similar to the custom loss function 'weighted_log_loss()' above\n", + " but with normalized weights, which should be very similar \n", + " to the official competition metric:\n", + " https://www.kaggle.com/kambarakun/lb-probe-weights-n-of-positives-scoring\n", + " and hence:\n", + " sklearn.metrics.log_loss with sample weights\n", + " \"\"\"\n", + " \n", + " class_weights = K.variable([2., 1., 1., 1., 1., 1.])\n", + " \n", + " eps = K.epsilon()\n", + " \n", + " y_pred = K.clip(y_pred, eps, 1.0-eps)\n", + "\n", + " loss = -( y_true * K.log( y_pred)\n", + " + (1.0 - y_true) * K.log(1.0 - y_pred))\n", + " \n", + " loss_samples = _normalized_weighted_average(loss, class_weights)\n", + " \n", + " return K.mean(loss_samples)\n", + "\n", + "\n", + "def weighted_log_loss_metric(trues, preds):\n", + " \"\"\"\n", + " Will be used to calculate the log loss \n", + " of the validation set in PredictionCheckpoint()\n", + " ------------------------------------------\n", + " \"\"\"\n", + " class_weights = [2., 1., 1., 1., 1., 1.]\n", + " \n", + " epsilon = 1e-7\n", + " \n", + " preds = np.clip(preds, epsilon, 1-epsilon)\n", + " loss = trues * np.log(preds) + (1 - trues) * np.log(1 - preds)\n", + " loss_samples = np.average(loss, axis=1, weights=class_weights)\n", + "\n", + " return - loss_samples.mean()" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [], + "source": [ + "filepath=\"model.h5\"\n", + "checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, \\\n", + " save_best_only=True, mode='min')\n", + "\n", + "callbacks_list = [checkpoint]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For a single epoch we are going to train only last 5 layers of Efficient. Since we have a large number of images around 600k so its better to train the all the layers on the whole train dataset but due its high computation resources required to train we only goin to train last five layers on whole dataset and for rest of epochs we only train on a sample of dataset but will train all the layers." + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "False" + ] + }, + "execution_count": 30, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "os.path.isfile('../input/orginal-087-eff/model.h5')" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [], + "source": [ + "train=False\n", + "\n", + "if train:\n", + " if not os.path.isfile('../input/orginal-087-eff/model.h5'):\n", + " for layer in model.layers[:-5]:\n", + " layer.trainable = False\n", + " model.compile(optimizer = Adam(learning_rate = 0.0001), \n", + " loss = 'binary_crossentropy',\n", + " metrics = ['acc'])\n", + "\n", + " model.fit_generator(generator = data_generator_train,\n", + " validation_data = data_generator_val,\n", + " epochs = 1,\n", + " callbacks = callbacks_list,\n", + " verbose = 1)" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [], + "source": [ + "if train:\n", + " for base_layer in model.layers[:-1]:\n", + " base_layer.trainable = True\n", + "\n", + " model.load_weights('model.h5')\n", + "\n", + " model.compile(optimizer = Adam(learning_rate = 0.0004), \n", + " loss = 'binary_crossentropy',\n", + " metrics = ['acc'])\n", + " model.fit_generator(generator = data_generator_train,\n", + " validation_data = data_generator_val,\n", + " steps_per_epoch=len(data_generator_train)/6,\n", + " epochs = 10,\n", + " callbacks = callbacks_list,\n", + " verbose = 1)" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Collecting gdown\n", + " Downloading https://files.pythonhosted.org/packages/b0/b4/a8e9d0b02bca6aa53087001abf064cc9992bda11bd6840875b8098d93573/gdown-3.8.3.tar.gz\n", + "Requirement already satisfied: filelock in /opt/conda/lib/python3.6/site-packages (from gdown) (3.0.12)\n", + "Requirement already satisfied: requests in /opt/conda/lib/python3.6/site-packages (from gdown) (2.22.0)\n", + "Requirement already satisfied: six in /opt/conda/lib/python3.6/site-packages (from gdown) (1.12.0)\n", + "Requirement already satisfied: tqdm in /opt/conda/lib/python3.6/site-packages (from gdown) (4.36.1)\n", + "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.6/site-packages (from requests->gdown) (2019.9.11)\n", + "Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.6/site-packages (from requests->gdown) (2.8)\n", + "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.6/site-packages (from requests->gdown) (1.24.2)\n", + "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.6/site-packages (from requests->gdown) (3.0.4)\n", + "Building wheels for collected packages: gdown\n", + " Building wheel for gdown (setup.py) ... \u001b[?25ldone\n", + "\u001b[?25h Created wheel for gdown: filename=gdown-3.8.3-cp36-none-any.whl size=8850 sha256=14a9f4491a378ff8119e1ca63a536cd63c0f8e914ac97133e046e690719cf217\n", + " Stored in directory: /tmp/.cache/pip/wheels/a7/9d/16/9e0bda9a327ff2cddaee8de48a27553fb1efce73133593d066\n", + "Successfully built gdown\n", + "Installing collected packages: gdown\n", + "Successfully installed gdown-3.8.3\n" + ] + } + ], + "source": [ + "!pip install gdown" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Downloading...\n", + "From: https://drive.google.com/uc?id=1OMWQjtnVkMKLQ3jG4RpUQR2IyimO-W46\n", + "To: /kaggle/working/model (2).h5\n", + "49.2MB [00:00, 135MB/s] \n" + ] + } + ], + "source": [ + "!gdown \"https://drive.google.com/uc?id=1OMWQjtnVkMKLQ3jG4RpUQR2IyimO-W46\"" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": {}, + "outputs": [], + "source": [ + "!cp \"model (2).h5\" model.h5" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1228/1228 [==============================] - 1048s 854ms/step\n" + ] + }, + { + "data": { + "text/plain": [ + "(78592, 6)" + ] + }, + "execution_count": 37, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model.load_weights('model.h5')\n", + "\n", + "preds = model.predict_generator(TestDataGenerator(test_df.index, None, VALID_BATCH_SIZE, \\\n", + " (WIDTH, HEIGHT), path_test_img), \n", + " verbose=1)\n", + "preds.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [], + "source": [ + "from tqdm import tqdm" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [], + "source": [ + "cols = list(train_final_df.columns)" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|█████████▉| 78545/78592 [00:01<00:00, 48430.17it/s]\n" + ] + } + ], + "source": [ + "# We have preditions for each of the image\n", + "# We need to make 6 rows for each of file according to the subtype\n", + "ids = []\n", + "values = []\n", + "for i, j in tqdm(zip(preds, test_df.index.to_list()), total=preds.shape[0]):\n", + "# print(i, j)\n", + " # i=[any_prob, epidural_prob, intraparenchymal_prob, intraventricular_prob, subarachnoid_prob, subdural_prob]\n", + " # j = filename ==> ID_xyz.dcm\n", + " for k in range(i.shape[0]):\n", + " ids.append([j.replace('.dcm', '_' + cols[k])])\n", + " values.append(i[k]) " + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th></th>\n", + " <th>0</th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>0</th>\n", + " <td>ID_000012eaf_any</td>\n", + " </tr>\n", + " <tr>\n", + " <th>1</th>\n", + " <td>ID_000012eaf_epidural</td>\n", + " </tr>\n", + " <tr>\n", + " <th>2</th>\n", + " <td>ID_000012eaf_intraparenchymal</td>\n", + " </tr>\n", + " <tr>\n", + " <th>3</th>\n", + " <td>ID_000012eaf_intraventricular</td>\n", + " </tr>\n", + " <tr>\n", + " <th>4</th>\n", + " <td>ID_000012eaf_subarachnoid</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + " 0\n", + "0 ID_000012eaf_any\n", + "1 ID_000012eaf_epidural\n", + "2 ID_000012eaf_intraparenchymal\n", + "3 ID_000012eaf_intraventricular\n", + "4 ID_000012eaf_subarachnoid" + ] + }, + "execution_count": 41, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df = pd.DataFrame(data=ids)\n", + "df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th></th>\n", + " <th>ID</th>\n", + " <th>Label</th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>0</th>\n", + " <td>ID_28fbab7eb_epidural</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " <tr>\n", + " <th>1</th>\n", + " <td>ID_28fbab7eb_intraparenchymal</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " <tr>\n", + " <th>2</th>\n", + " <td>ID_28fbab7eb_intraventricular</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " <tr>\n", + " <th>3</th>\n", + " <td>ID_28fbab7eb_subarachnoid</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " <tr>\n", + " <th>4</th>\n", + " <td>ID_28fbab7eb_subdural</td>\n", + " <td>0.5</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + " ID Label\n", + "0 ID_28fbab7eb_epidural 0.5\n", + "1 ID_28fbab7eb_intraparenchymal 0.5\n", + "2 ID_28fbab7eb_intraventricular 0.5\n", + "3 ID_28fbab7eb_subarachnoid 0.5\n", + "4 ID_28fbab7eb_subdural 0.5" + ] + }, + "execution_count": 42, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sample_df = pd.read_csv(input_folder + 'stage_1_sample_submission.csv')\n", + "sample_df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th></th>\n", + " <th>ID</th>\n", + " <th>Label</th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>0</th>\n", + " <td>ID_000012eaf_any</td>\n", + " <td>0.029101</td>\n", + " </tr>\n", + " <tr>\n", + " <th>1</th>\n", + " <td>ID_000012eaf_epidural</td>\n", + " <td>0.001475</td>\n", + " </tr>\n", + " <tr>\n", + " <th>2</th>\n", + " <td>ID_000012eaf_intraparenchymal</td>\n", + " <td>0.001740</td>\n", + " </tr>\n", + " <tr>\n", + " <th>3</th>\n", + " <td>ID_000012eaf_intraventricular</td>\n", + " <td>0.001194</td>\n", + " </tr>\n", + " <tr>\n", + " <th>4</th>\n", + " <td>ID_000012eaf_subarachnoid</td>\n", + " <td>0.001531</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + " ID Label\n", + "0 ID_000012eaf_any 0.029101\n", + "1 ID_000012eaf_epidural 0.001475\n", + "2 ID_000012eaf_intraparenchymal 0.001740\n", + "3 ID_000012eaf_intraventricular 0.001194\n", + "4 ID_000012eaf_subarachnoid 0.001531" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df['Label'] = values\n", + "df.columns = sample_df.columns\n", + "df.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": {}, + "outputs": [], + "source": [ + "df.to_csv('submission.csv', index=False)" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<a href=submission.csv>Download CSV file</a>" + ], + "text/plain": [ + "<IPython.core.display.HTML object>" + ] + }, + "execution_count": 45, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "create_download_link(filename='submission.csv')" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.5" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +}