Datasets
Models
Downloads: 1
[de45a9]:
/
DL_Genomics_v8_resnet-fastai.ipynb
1470 lines (1469 with data), 119.8 kB
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Deep learning in genomics - Custom ResNet model with PyTorch and fastai"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook is based on the [jupyter notebook](https://nbviewer.jupyter.org/github/abidlabs/deep-learning-genomics-primer/blob/master/A_Primer_on_Deep_Learning_in_Genomics_Public.ipynb) from the publication [\"A primer on deep learning in genomics\"](https://www.nature.com/articles/s41588-018-0295-5) but uses the [fastai](https://www.fast.ai) library based on [PyTorch](https://pytorch.org)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Notebook setup"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"%reload_ext autoreload\n",
"%autoreload 2\n",
"%matplotlib inline"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"from fastai import *\n",
"from fastai.vision import *"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.preprocessing import LabelEncoder, OneHotEncoder"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'1.0.38.dev0'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# fastai version\n",
"__version__"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Data setup"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Loading data from the web, generating dthe ataframe, and saving it to disk is carried out in [Basic model with PyTorch jupyter notebook](https://nbviewer.jupyter.org/github/MicPie/genomics/blob/master/DL_Genomics_v8_basic-pytorch.ipynb)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Data frame setup"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"seq_df = pd.read_csv('seq_df.csv', index_col=0)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"# add column NoTarget which is not(Target)\n",
"seq_df['NotTarget'] = seq_df['Target'].apply(lambda x: int(not(bool(x))))"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Sequences</th>\n",
" <th>Target</th>\n",
" <th>NotTarget</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>CCGAGGGCTATGGTTTGGAAGTTAGAACCCTGGGGCTTCTCGCGGA...</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>GAGTTTATATGGCGCGAGCCTAGTGGTTTTTGTACTTGTTTGTCGC...</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>GATCAGTAGGGAAACAAACAGAGGGCCCAGCCACATCTAGCAGGTA...</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>GTCCACGACCGAACTCCCACCTTGACCGCAGAGGTACCACCAGAGC...</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>GGCGACCGAACTCCAACTAGAACCTGCATAACTGGCCTGGGAGATA...</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Sequences Target NotTarget\n",
"0 CCGAGGGCTATGGTTTGGAAGTTAGAACCCTGGGGCTTCTCGCGGA... 0 1\n",
"1 GAGTTTATATGGCGCGAGCCTAGTGGTTTTTGTACTTGTTTGTCGC... 0 1\n",
"2 GATCAGTAGGGAAACAAACAGAGGGCCCAGCCACATCTAGCAGGTA... 0 1\n",
"3 GTCCACGACCGAACTCCCACCTTGACCGCAGAGGTACCACCAGAGC... 1 0\n",
"4 GGCGACCGAACTCCAACTAGAACCTGCATAACTGGCCTGGGAGATA... 1 0"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"seq_df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## fastai data object"
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"### Data encoding test (incorporated into \"open_seq_image\" function in the next section)"
]
},
{
"cell_type": "code",
"execution_count": 86,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"# setup class instance to encode the four different bases to integer values (1D)\n",
"int_enc = LabelEncoder()"
]
},
{
"cell_type": "code",
"execution_count": 87,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"# setup one hot encoder to encode integer encoded classes (1D) to one hot encoded array (4D)\n",
"one_hot_enc = OneHotEncoder(categories=[range(4)])"
]
},
{
"cell_type": "code",
"execution_count": 88,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"seq_enc = []\n",
"\n",
"for s in seq:\n",
" enc = int_enc.fit_transform(list(s)) # bases (ACGT) to int (0,1,2,3)\n",
" enc = np.array(enc).reshape(-1,1) # reshape to get rank 2 array (from rank 1 array)\n",
" enc = one_hot_enc.fit_transform(enc) # encoded integer encoded bases to sparse matrix (sparse matrix dtype)\n",
" seq_enc.append(enc.toarray()) # export sparse matrix to np array"
]
},
{
"cell_type": "code",
"execution_count": 89,
"metadata": {
"hidden": true
},
"outputs": [
{
"data": {
"text/plain": [
"2000"
]
},
"execution_count": 89,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(seq_enc)"
]
},
{
"cell_type": "code",
"execution_count": 462,
"metadata": {
"hidden": true
},
"outputs": [
{
"data": {
"text/plain": [
"(array([[0., 0., 0., 1., ..., 0., 1., 0., 0.],\n",
" [1., 1., 0., 0., ..., 1., 0., 1., 1.],\n",
" [0., 0., 1., 0., ..., 0., 0., 0., 0.],\n",
" [0., 0., 0., 0., ..., 0., 0., 0., 0.]]), (4, 50))"
]
},
"execution_count": 462,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"seq_enc[0].T, seq_enc[0].T.shape"
]
},
{
"cell_type": "code",
"execution_count": 311,
"metadata": {
"hidden": true
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAADIAAAAECAMAAAD7wpwzAAADAFBMVEUAAAABAQECAgIDAwMEBAQFBQUGBgYHBwcICAgJCQkKCgoLCwsMDAwNDQ0ODg4PDw8QEBARERESEhITExMUFBQVFRUWFhYXFxcYGBgZGRkaGhobGxscHBwdHR0eHh4fHx8gICAhISEiIiIjIyMkJCQlJSUmJiYnJycoKCgpKSkqKiorKyssLCwtLS0uLi4vLy8wMDAxMTEyMjIzMzM0NDQ1NTU2NjY3Nzc4ODg5OTk6Ojo7Ozs8PDw9PT0+Pj4/Pz9AQEBBQUFCQkJDQ0NERERFRUVGRkZHR0dISEhJSUlKSkpLS0tMTExNTU1OTk5PT09QUFBRUVFSUlJTU1NUVFRVVVVWVlZXV1dYWFhZWVlaWlpbW1tcXFxdXV1eXl5fX19gYGBhYWFiYmJjY2NkZGRlZWVmZmZnZ2doaGhpaWlqampra2tsbGxtbW1ubm5vb29wcHBxcXFycnJzc3N0dHR1dXV2dnZ3d3d4eHh5eXl6enp7e3t8fHx9fX1+fn5/f3+AgICBgYGCgoKDg4OEhISFhYWGhoaHh4eIiIiJiYmKioqLi4uMjIyNjY2Ojo6Pj4+QkJCRkZGSkpKTk5OUlJSVlZWWlpaXl5eYmJiZmZmampqbm5ucnJydnZ2enp6fn5+goKChoaGioqKjo6OkpKSlpaWmpqanp6eoqKipqamqqqqrq6usrKytra2urq6vr6+wsLCxsbGysrKzs7O0tLS1tbW2tra3t7e4uLi5ubm6urq7u7u8vLy9vb2+vr6/v7/AwMDBwcHCwsLDw8PExMTFxcXGxsbHx8fIyMjJycnKysrLy8vMzMzNzc3Ozs7Pz8/Q0NDR0dHS0tLT09PU1NTV1dXW1tbX19fY2NjZ2dna2trb29vc3Nzd3d3e3t7f39/g4ODh4eHi4uLj4+Pk5OTl5eXm5ubn5+fo6Ojp6enq6urr6+vs7Ozt7e3u7u7v7+/w8PDx8fHy8vLz8/P09PT19fX29vb39/f4+Pj5+fn6+vr7+/v8/Pz9/f3+/v7////isF19AAAAO0lEQVR4nGWNSQoAMAgD8/9PT6loIq0HMauShOR9DwphIkO5eALbEI1xp5DUoxYnNtf3t1rci9mGy30A2+0xz9q2+b0AAAAASUVORK5CYII=\n",
"text/plain": [
"<PIL.Image.Image image mode=P size=50x4 at 0x1A20462668>"
]
},
"execution_count": 311,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"PIL.Image.fromarray(seq_enc[0].T.astype('uint8')*255).convert('P')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Setup custom fastai data object"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"# open sequence image function\n",
"def open_seq_image(seq:str, cls:type=Image)->Image:\n",
" \"Return `Image` object created from sequence string `seq`.\"\n",
" \n",
" int_enc = LabelEncoder() # setup class instance to encode the four different bases to integer values (1D)\n",
" one_hot_enc = OneHotEncoder(categories=[range(4)]) # setup one hot encoder to encode integer encoded classes (1D) to one hot encoded array (4D)\n",
" \n",
" enc = int_enc.fit_transform(list(seq)) # bases (ACGT) to int (0,1,2,3)\n",
" enc = np.array(enc).reshape(-1,1) # reshape to get rank 2 array (from rank 1 array)\n",
" enc = one_hot_enc.fit_transform(enc) # encoded integer encoded bases to sparse matrix (sparse matrix dtype)\n",
" enc = enc.toarray().T # export sparse matrix to np array\n",
" \n",
" # https://stackoverflow.com/questions/22902040/convert-black-and-white-array-into-an-image-in-python\n",
" x = PIL.Image.fromarray(enc.astype('uint8')).convert('P')\n",
" x = pil2tensor(x,np.float32)\n",
" \n",
" # optional functions not needed\n",
" #x = x.view(4,-1) # remove first dimension\n",
" #x = x.expand(3, 4, 50) # expand to 3 channel image\n",
" \n",
" return cls(x)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"image/jpeg": "/9j/4AAQSkZJRgABAQEAZABkAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgICAgUEBAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/2wBDAQICAgICAgUDAwUKBwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgr/wAARCAAEADIDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD80/8AhrrUr/8A0rxT8B/h1q95B/pGltNotzaWGm6iflfUYdIsrmDSvPkjjsopUeze3nTTrfzYZGadpvuj4Jf8EwP2bf2gf2wL39kD4j6n4uuYLDwj4g1u38e/8JEz68IdG8WXnhax0rMqvZ/YlsbG3faLUTCVcJMkAWBSivm/HbGYzgrIq2IyKrLDTjhcTUThKSanTdBQkuZztbnleKXLK95xqOMXD9sPnT9kmOD9rf8A4Wbb6rcav4Ls/gx8Ita+Inwp0/wV4s1VYfCuqaf5LmKyF/dXRhguriVbqcg+eZreIxTQqHR+R+CXxp8V/tC+Mr34X/FXTdI1fwnpnhHxBrWlaFqWlx3s2nWmjaPearZaJaapd+bqtnpYkso4TbwXkbeTLOFkWSZ5SUV9lXwmGhmfENJRVsJDCOj3pSq0HOpKEvjUpz9+cudylJuTbbk2Hrn7DnwE+DH7cXg3xd4t1X4aaR8PovBvi7wX4fsNJ8DWguIbqPxRrA0fUbm4l1v+0Ll50tdptsTLHazJ58UayvI7/On/AA2R480r/ibfDvwF4R8F+IW+SXxP4N0uXT7mWE/vXiaCOb7H/wAfuL+OUW4mtLmK3+ySW0VpaQwFFHCuFoZnx5n2W4xOrQw06KpQnKUoxU6VKUk+ab51KUm2qjqpt+SSA/4ba+Mn/QmfCL/xH/wf/wDKqiiiv0n/AFT4Y/6A6f3S/wDlgH//2Q==\n",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAADIAAAAECAYAAADMHGwBAAAABHNCSVQICAgIfAhkiAAAAHNJREFUKJFjdGEM+c8ABTufXWBwlzJgQAe4xNHVMDAwoKiD6cMmRwlANhdmJuPf5yr/cVlCjAdIdQAMELIP3bHobkEPJEZYjKBLYLMMV6gji+GykJDHcHkOm9uwqWNETlqEDEF2LD7LiUlK+JIxTC8pKQIA7jhk1J8wofAAAAAASUVORK5CYII=\n",
"text/plain": [
"Image (1, 4, 50)"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# test open sequence image function\n",
"open_seq_image('CCGAGGGCTATGGTTTGGAAGTTAGAACCCTGGGGCTTCTCGCGGACACC')"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"class SeqItemList(ImageItemList):\n",
" \"Sequence Item List\"\n",
" _bunch,_square_show = ImageDataBunch,True\n",
" def __post_init__(self):\n",
" super().__post_init__()\n",
" self.sizes={}\n",
" \n",
" def open(self, seq): return open_seq_image(seq)\n",
" \n",
" def get(self, i):\n",
" seq = self.items[i][0]\n",
" res = self.open(seq)\n",
" return res\n",
" \n",
" @classmethod\n",
" def import_from_df(cls, df:DataFrame, cols:IntsOrStrs=0, **kwargs)->'ItemList':\n",
" \"Get the sequences in `col` of `df` and return cls and df.\"\n",
" return cls(items=df[cols].values, xtra=df)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"bs = 64"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"data = (SeqItemList.import_from_df(seq_df, ['Sequences'])\n",
" .random_split_by_pct(valid_pct=0.25)\n",
" #.split_by_idxs(range(1500), range(1500,2000))\n",
" .label_from_df(['Target', 'NotTarget'])\n",
" .databunch(bs=bs))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Verify data object"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Check data object"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"ImageDataBunch;\n",
"\n",
"Train: LabelList\n",
"y: MultiCategoryList (1500 items)\n",
"[MultiCategory NotTarget, MultiCategory NotTarget, MultiCategory Target, MultiCategory Target, MultiCategory NotTarget]...\n",
"Path: .\n",
"x: SeqItemList (1500 items)\n",
"[Image (1, 4, 50), Image (1, 4, 50), Image (1, 4, 50), Image (1, 4, 50), Image (1, 4, 50)]...\n",
"Path: .;\n",
"\n",
"Valid: LabelList\n",
"y: MultiCategoryList (500 items)\n",
"[MultiCategory Target, MultiCategory NotTarget, MultiCategory NotTarget, MultiCategory NotTarget, MultiCategory Target]...\n",
"Path: .\n",
"x: SeqItemList (500 items)\n",
"[Image (1, 4, 50), Image (1, 4, 50), Image (1, 4, 50), Image (1, 4, 50), Image (1, 4, 50)]...\n",
"Path: .;\n",
"\n",
"Test: None"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"data"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(2, ['Target', 'NotTarget'])"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# check classes\n",
"data.c, data.classes"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"64"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"data.train_dl.batch_size"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Check data points"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"data": {
"image/jpeg": "/9j/4AAQSkZJRgABAQEAZABkAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgICAgUEBAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/2wBDAQICAgICAgUDAwUKBwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgr/wAARCAAEADIDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD4C+Nvxp8V/s9eMrL4bfCrTdI0vw5P4R8P+JtP0y00uOCbTdU1PR7PWIrqK/h2X8k+n3d7J9huJ7maeCENA0ksM91HOfBL45fET46eMr34fa/caRpeh2/hHxB4pu/D+ieGNPj0e+1rRtHvNUtL9tJkgk06CdjYWlnM9tbQme0SWJ9xuLl5iivpv7Jyz/UX+0XRi6/subnavLm57czbbvK32mua/vcylq/2w679kmSf9t7/AIWbY/tCW+kX2lfCr4Ra18QfD2i6J4T0rSIZLvTfJ8vTGksbWKe30uU3E8k1paS26vNPLcApPI8zeRf8NrfGeT/iY3K6QuuTfNqfi3TbA6ZrGqSR/vbOe7vLB4Jbie3vcagJnYyXF3HBJeNdi2tkhKKMlynLMZxhnOArUYyo4f6p7KFvdh7XD1KlXlV7fvJxjOd+ZSlFNq6uB1/xJ+NPiv4I+Dfh/wCJPgbpukeFtN+IfhGbxHfeFk0uPVbDSbtdY1LSLiOwbVvtdzbwXlrpdut5EZmW8Rnhn8y28u3TkP8AhtH4rP8A6be+HPCOpah/Dd+IvC8Gr20Gfkf7Ppt+J9Nst0EOnWo+zWsXlW2kWUMXlIJhMUV7OQcPZHjssVbEYeM5udVOTTcmo168I80uZSlaMIxvKTbUVzOTu5B1/wC0B4y+BP7PXx58bfALRv2Kvh1r1n4H8Xal4ftdc8Qa14o+36jHZXUlstzc/ZdZgg8+QRh38mGKPezbI0XCgooo4cyHLsfw7gsViPaSqVKNGcn7fEaynRpTk9MQlrKUnoktdElZIP/Z\n",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAADIAAAAECAYAAADMHGwBAAAABHNCSVQICAgIfAhkiAAAAHRJREFUKJFjdGEM+c+ABHY+u8DgLmWAwYbxGRgYUMSw6SMkDhPDZj42u3GpRwaMMI8gOxLdEHTHIxsMA8h8bA7HZjY2gK4Wn3nIcox/n6v8JyYkCTkAl6MIhTi+WMZmFq4YZnRhDPlPDUeiyxHjOGLNI0YfAGvwc9QnBcXaAAAAAElFTkSuQmCC\n",
"text/plain": [
"Image (1, 4, 50)"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"i = 2\n",
"data.x[i]"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"MultiCategory Target"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"data.y[i]"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"image/jpeg": "/9j/4AAQSkZJRgABAQEAZABkAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgICAgUEBAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/2wBDAQICAgICAgUDAwUKBwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgr/wAARCAAEADIDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD4w/ZJl1L9tn/hZumfHvxFq+o6H8MPhFrXxHs/Dy6zctDrutaf5LH+0Z5pJLuaC4mu9QuJI454jFNqd89q1qbqXcfbvFcf7eX/AA7us/Her2/gKX4u/wDCtdR8maMPqHh9NU/s+3imtCh05p7cvdXsVwtosov764u2ZpmV0KK5MZy0uMOIsBBJUMJgo1qMLR5aVV4erUdSEeWym5xhLmd3eENUoRUf2w674o/Aj4U/C/wb8UviP/wiv/CSeIfhH4u8T+HfDeq+ML6fUftVtoGseHNI01ryCR/s8+y28RYMPlLbN/Y2mp5Ii+2xXvkf7P8A8ZNZ+P3x58E/s6SeDPCPhDw94/8AF2m+FPELeDfCVnb3Muh3d1HbpZNJPHN532fznnjuJxLdPcpb3FxNcS2Vm9uUUcKc2acD5pmGMbqVqFOUqcpOT5GsBCsnGPMoK1Zuol7NpSs0lZJByH/DZHjyw/e+FvAXhHTZ/wDUvPc6XLq/2iwHyx6Zcpq013He2UMUOmxwW9ysqW39kWksHlTm5muOv0X40+K/E/wG8SfHHxfpuka/F4S8XaJ4T8P+EvE+lx6tYWPh/VLXVbiXS4Li983UbaCEaPbRWrQ3kctqk100MiS3EkpKK/Ss74eyTB0KEqNCKlKtQg3rzcs6jjJczk5K8dLxcXbRNLQDkP8Aht747T/vtZ034daveP8ANdat4g+DHhfUb+9kP3p7m7utOknup3OWeaZ3kkdmZ2ZmJJRRXsf6p8Mf9AdP7mvwVRJeiSS2SSSSD//Z\n",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAADIAAAAECAYAAADMHGwBAAAABHNCSVQICAgIfAhkiAAAAHNJREFUKJFj/Ptc5b+7lAHDzmcXGJBpdIAsz8DAgKIGlx5cAF09MXx0O9HVMrowhvwn5CBs4sQ4Hpvn8QUYshi+AEOX2/nsAsQjxHoAn+PRHYjNwfgcj8uDxNjJwMCAGiPICmA+JmQQesgQ0oPLIcQCXHoAnhaF1FMzixMAAAAASUVORK5CYII=\n",
"text/plain": [
"Image (1, 4, 50)"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"i = 3\n",
"data.x[i]"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"MultiCategory Target"
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"data.y[i]"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAETCAYAAAA79nyeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADwxJREFUeJzt3WuM7HV9x/HPF08ajCjgpRDwkmgsRh9UaxoTRUxqqPUClVC8P/DBMdSEtmKtMa3FaLRpWlOtwYREHzSpjVFSa4xojLaxNqkXitBokyKlXqgHVBQOpSKC59cH8ycOyyJzZmdnhu95vZLN2Z3/zP+yu/M77/nPb2ZrjBEAgE6O2/QOAACsmsABANoROABAOwIHAGhH4AAA7QgcAKAdgQMAtCNwuI+qun3u40hV3TH39avXvC/HV9Woqseuc7vA/jPWsJ8ObHoH2D5jjBPu+byqvpXk4Bjjc8usq6oOjDHuXtW+AX0Ya9hPzuBw1KrqOVX15ao6XFWHquo9VXVgWnbPo6DXV9X1Sb4+Xf7iqrquqm6tqvdW1Zeq6jVz67ywqq6tqh9V1RVVdfq06AvTv9dOj+peutaDBTbGWMNeCByWcVeSi5I8Mslzk5yT5OCO67wkyTOTPKOqTk3ykSQXJ3lMkkPTsiRJVb0iyRum9ZyS5OokH5oWnzX9e8YY44Qxxsf344CArWSsYWkCh6M2xvjKGOPKMcbPxhjXJ/lgkuftuNq7xhi3jjHuSHJukivHGJ8cY9yV5N1Jbpm77oVJ3jnG+Ma0/O1JzqyqU9ZwOMCWMtawFwKHo1ZVT62qT1fV96rqtiSXJHn0jqvdMPf5afNfjzGOJPnu3PInJLlsOqV8a5IfJLk7icl+cAwz1rAXAodlfCDJV5M8aYzxiCTvSFI7rjP/Z+pvzNwAUlXHJTl9bvkNSV47xjhp7uOhY4yrdqwHOLYYa1iawGEZD09yeIxxe1U9LcnrHuD6n0jyrKp60TRB8I1JTp5bflmSt1bVGUlSVSdX1flJMsa4M8nhJE9c9UEAW89Yw9IEDsu4OMnBqro9yfszm9R3v8YYNyZ5ZZL3Jbk5s0dYX0ty57T8w0kuTfKx6TT0NUnOnlvFJUkun04rn7viYwG2l7GGpdUYzsqxXtMjq5uSnDPG+OKm9wfoyVhzbHMGh7WoqhdW1YlVdXyStyX5cZKrNrxbQDPGGu4hcFiXs5J8M8n3kzw/yXljjJ9udpeAhow1JPEUFQDQkDM4AEA7AgcAaGdjf038yE1PXui5sRec9vT7XPaZQ9csdL3d7LztordbZF33Zy/b2IS9fH/3so3dLPtz3e22XX9ei1r0+Be12/fps0cu3/kmbBt39nEXbO3z8Ov4nVz1/XmRfV7HeLGOsXs3y253U+PqJn531nGsi4w1zuAAAO0IHACgHYEDALSzsZeJr/p58VU+V7ibdTyPud/7ssq5S/d3203MrVn0tts0j2i/v0+bmkdkDs7PbeL3by/3q93s91hjXtz2HP8qf2brGN/NwQEAjkkCBwBoR+AAAO0IHACgna2fZLyOibHrXtd+WHbi17Yf1yZsy5tl3d9td7PImxquY5Lrcadet3WTjFf9pqKL2sT9aJt+dzdhW/Z3U29M2OFns+j1TDIGAI5JAgcAaEfgAADtCBwAoJ2tn2Tc1bZMnt6LTe3HJra7Ld/zbfdgfifj/f4Zd7jP78WmJkB3+Gvqe9F1gvIiL2hwBgcAaEfgAADtCBwAoB2BAwC0s7FJxgAA+8UZHACgHYEDALQjcACAdgQOANCOwAEA2hE4AEA7AgcAaEfgAADtCBwAoB2BAwC0I3AAgHYEDgDQjsABANoROABAOwIHAGhH4AAA7QgcAKAdgQMAtCNwAIB2BA4A0I7AAQDaETgAQDsCBwBoR+AAAO0IHACgHYEDALQjcACAdgQOANCOwAEA2hE4AEA7AgcAaEfgAADtCBwAoB2BAwC0I3AAgHYEDgDQjsABANoROABAOwIHAGhH4AAA7QgcAKAdgQMAtCNwAIB2BA4A0I7AAQDaETgAQDsCBwBoR+AAAO0IHACgHYEDALQjcACAdgQOANCOwAEA2hE4AEA7AgcAaEfgAADtCBwAoB2BAwC0I3AAgHYEDgDQjsABANoROABAOwIHAGhH4AAA7QgcAKAdgQMAtCNwAIB2BA4A0I7AAQDaETgAQDsCBwBoR+AAAO0IHACgHYEDALQjcACAdgQOANCOwAEA2hE4AEA7AgcAaEfgAADtCBwAoB2BAwC0I3AAgHYEDgDQjsABANoROABAOwIHAGhH4AAA7QgcAKAdgQMAtCNwAIB2BA4A0I7AAQDaETgAQDsCBwBoR+AAAO0IHACgHYEDALQjcACAdgQOANCOwAEA2hE4AEA7AgcAaEfgAADtCBwAoB2BAwC0I3AAgHYEDgDQjsABANoROABAOwIHAGhH4AAA7QgcAKAdgQMAtCNwAIB2BA4A0I7AAQDaETgAQDsCBwBoR+AAAO0IHACgHYEDALQjcACAdgQOANCOwAEA2hE4AEA7AgcAaEfgAADtCBwAoB2BAwC0I3AAgHYEDgDQjsABANoROABAOwIHAGhH4AAA7QgcAKAdgQMAtCNwAIB2BA4A0I7AAQDaETgAQDsCBwBoR+AAAO0IHACgHYEDALQjcACAdgQOANCOwAEA2hE4AEA7AgcAaEfgAADtCBwAoB2BAwC0I3AAgHYEDgDQjsABANoROABAOwIHAGhH4AAA7QgcAKAdgQMAtCNwAIB2BA4A0I7AAQDaETgAQDsCBwBoR+AAAO0IHACgHYEDALQjcACAdgQOANCOwAEA2hE4AEA7Aof7qKrb5z6OVNUdc1+/es37cnxVjap67Dq3Cyyvqr5VVd+rqofNXXawqj6/wG0/X1UHp88fv2M8GlX1f3NfP3cfD2O3fXtKVd29zm2yvAOb3gG2zxjjhHs+r6pvJTk4xvjcMuuqqgNjDAMCHHsOJPmDJH+27ArGGN9JMj8ejSS/Osb4r2XWZzw6tjiDw1GrqudU1Zer6nBVHaqq91TVgWnZPWdcXl9V1yf5+nT5i6vquqq6tareW1VfqqrXzK3zwqq6tqp+VFVXVNXp06IvTP9eOz1ie+laDxZY1l8meVNVnbRzQVU9u6qunMaQK6vq2dPl70ry3CSXTvf3Sx9oI1V1XlX9e1XdVlXfrqo/nlv2lKq6u6peV1U3JPnUdPnBqvpOVf2gqt5cVTdV1ZnTsodU1Z9W1X9X1c1V9Xdzx/CFJA+ZO4P0jL1+k9g/Aodl3JXkoiSPzGwwOifJwR3XeUmSZyZ5RlWdmuQjSS5O8pgkh6ZlSZKqekWSN0zrOSXJ1Uk+NC0+a/r3jDHGCWOMj+/HAQEr929JPp/kTfMXVtUjk1yR5H1JHpXkr5JcUVWPGmP8SZJ/SXLRdH+/aIHt3JbkVUlOSnJeZlH1W3PLH5LkWUnOSPLbVfX0aZsvS/LY6ePRc9f/oyS/meTMadldSd4zLTsryc+mfTthjHH1It8INkPgcNTGGF8ZY1w5xvjZGOP6JB9M8rwdV3vXGOPWMcYdSc5NcuUY45NjjLuSvDvJLXPXvTDJO8cY35iWvz3JmVV1yhoOB9g/lyT5vap6zNxlL05y3Rjjb8cYd48xPpzkPzN7gHPUxhj/OMb4jzHGkTHGV5N8NPcdjy4ZY/x4Go9eluTvxxhfGmPcmeStuff/hRcmecsY49AY4yeZjUcvr6paZv/YHIHDUauqp1bVp6dJhLdlNog9esfVbpj7/LT5r8cYR5J8d275E5JcNj19dWuSHyS5O7NHT8CD1Bjj60k+meQtcxefluTbO6767SSnZwnTU+b/PD3ddDjJa3Pv8ejIGOPQju3Pj0e3JTk8rauSPC7Jp+bGo6sz+7/yUcvsH5sjcFjGB5J8NcmTxhiPSPKOJDsf3Yy5z2/MXKxU1XG592B2Q5LXjjFOmvt46Bjjqh3rAR583pbkdfn5ff5QZg9q5j0+P3/Qc7T3+Y9m9hT448YYJyb5m9x7PNq5vp3j0SOSnJgkY4wx7cdv7BiPjh9j3LzEvrFBAodlPDzJ4THG7VX1tMwGr1/kE0meVVUvmiYjvzHJyXPLL0vy1qo6I0mq6uSqOj9JplPIh5M8cdUHAey/6RVPH0ny+9NFn0ryK1X1qqo6UFUvT/LUzM70JMn3suD9fTrjckKSH44xfjJNVr7gAW720STnV9WvV9UvZfYA7cjc8suS/HlVPW7axi9X1T1Pn30/s0nGj19k/9gsgcMyLk5ysKpuT/L+zAav+zXGuDHJKzObVHhzZo+evpbkzmn5h5NcmuRj01Ne1yQ5e24VlyS5fDplfO6KjwXYf+9I8rAkGWP8MLMXIfxhkh8meXOSl0xnSJLkr5P8TlXdUlXv+0Urnc64/G6Sd1fV/07ruvwBbnN1ZhOJ/yGzszU3ZvYg6s7pKn+R5HNJ/mla578m+bXptrdMy6+axqOnL/wdYO1q9vsB6zOdxbkpyTljjC9uen+AY1dVnZzkR0lOmx6M0YQzOKxFVb2wqk6squMze07+x0mu2vBuAcegqjq3qh5aVSdk9pLxL4ubfgQO63JWkm9m9hz285OcN8b46WZ3CThGXZDZWeT/yWzy81r/BA3r4SkqAKAdZ3AAgHY29sc2j9z05PucOnrBafs/If0zh655wG3uvM79Xa+r3Y5/Uav+fq7yZ7HocW3Tz3ovP4tFrPpYP3vk8q17t9dFx5ptud8vuh+b2t9FtrvN+3Y011t2u3vZ5jq+d6sckze1b4uMNc7gAADtCBwAoB2BAwC0I3AAgHY29jLxs4+7YKUb3pYJgova70lue13ffu/Hqice72aVk5EXPYZV7se22+34jzv1uq2bZLyXsWZbJmOu+n61LePFoh5s+7tNtuVFHnux7FjjDA4A0I7AAQDaETgAQDsCBwBop80k42Vt0wSsTby75jZx/NsxoW83q3x30XXzgob9nzS/ifvatr/j737b5n3bzar31zsZAwDHJIEDALQjcACAdgQOANDOVk0yXnSS2272+906N/Xutqtc37H+zrt7sc3vvLwXq963B8sk4028QGCbfw/WYdXj+162u9/vAr2pd57exItmVjk5fdFtJiYZAwDHKIEDALQjcACAdgQOANDOxiYZAwDsF2dwAIB2BA4A0I7AAQDaETgAQDsCBwBoR+AAAO0IHACgHYEDALQjcACAdgQOANCOwAEA2hE4AEA7AgcAaEfgAADtCBwAoB2BAwC0I3AAgHYEDgDQjsABANoROABAOwIHAGhH4AAA7QgcAKCd/wc6xV0unK5QSAAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 576x576 with 4 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"data.show_batch(rows=2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Model setup"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Custom fastai ResNet model"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"learn_dummy = create_cnn(data, models.resnet18)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"BCEWithLogitsLoss()"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"learn_dummy.loss_func.func"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"#learn_dummy.model[0]"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Sequential(\n",
" (0): AdaptiveConcatPool2d(\n",
" (ap): AdaptiveAvgPool2d(output_size=1)\n",
" (mp): AdaptiveMaxPool2d(output_size=1)\n",
" )\n",
" (1): Lambda()\n",
" (2): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
" (3): Dropout(p=0.25)\n",
" (4): Linear(in_features=1024, out_features=512, bias=True)\n",
" (5): ReLU(inplace)\n",
" (6): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
" (7): Dropout(p=0.5)\n",
" (8): Linear(in_features=512, out_features=2, bias=True)\n",
")"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"learn_dummy.model[1]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Setup custom input stage"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"# define function to create 3 channel image from 1 channel image.\n",
"def ExpandInput(): return Lambda(lambda x: x.expand(-1, 3, 4, 50))"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"EI = ExpandInput()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Test ExpandInput layer"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([64, 1, 4, 50])"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tt = torch.rand((64,1,4,50)); tt.shape"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([64, 3, 4, 50])"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tt.expand(-1, 3, 4, 50).shape"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"data": {
"image/jpeg": "/9j/4AAQSkZJRgABAQEAZABkAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgICAgUEBAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/2wBDAQICAgICAgUDAwUKBwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgr/wAARCAAEADIDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwDnviXJ4M8PfFfQvhDffCLwjqWn+AvG3xP8MaXdSaItlPfWXhuw8N6/owvRYG3ju/Jubx43jkQw3EUUJuIppoxPX1l+2v8ABDwH+zl8Yvh1ofwXgvdCn8W+Itd8K6hqtrfyGeLR7fQovELW0asTCTPPZ2lrLM8bzNZ2sMKyJ5auCigDzv8Aae+MHjz4W/Df4j+LZr+z8R3Pw7+G/hnWvDX9vaTbK0F/rT+Io9Ru1mtI4J4rqWa0t7z7XDJHdLcq8gmAmmWR97+zB4O+JH7VPwX+Amj+M/FXg3Q/iF8G/Fl7qV14M1s2mpWZtLRNMhihv3WS5kj8uFJmFzJOzztNIzH7ROJCigDxzxh+0f8AF/wXa/sx6rceLrjX7zxuPB1x4g1DxRK9/eXEmq3Wsyy5upGNwyQxyXdtDE0jRpb6heRlH89jXX/8Eofj94s/bL8P/GL4XfGnSrS603wfpUOih0vL2WbV/wC0dU1m+kvbt7m4lzdx3OkQypLD5WWnm8xZf3PlFFAG38P/AI93yeAtES8+Efw6uphpFsJbqbwHYq8zeUuXYRxqoJPJCqq88ADiiiigD//Z\n",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAADIAAAAECAYAAADMHGwBAAAABHNCSVQICAgIfAhkiAAAAstJREFUKJEFwU8oswEcB/DvuylPmSfNiGVTiOI6FzWxi7KLspY1mpVdVnoeJRntoNwUicRzsFL+tDZao205SE9abezJwZo8KHGYhSzbaKvf+/n8a2trI6PRiM7OTiSTSYiiCEEQMD09DZPJhJqaGvT19SEQCGBmZgajo6O4ublBPp/H9fU1HA4HhoaGEA6HIcsyzs7OkEgk0NraCp7nsb29DbVajVKphEwmA5VKhe7ublitVvj9fjgcDuRyOdTX12NgYACJRAIGgwHhcBiiKMJsNoPneSwtLSGTySCXy2FiYgIulwssy+Lr6wstLS34x3Eczc3NQalUIhKJ4Pv7G4Ig4OHhAZIk4fT0FIVCAVNTU7Barfj5+YHP50MoFIJWq0WlUoHb7UY6ncbv7y/i8TicTicUCgUGBwchSRKen5/h8Xiws7ODsbExMAyDv78/RCIRDA8P4+7uDpubm9Dr9ZAkCYVCAclkEhzHYXZ2FizLwu124+TkBDqdDhzHIRgM4uLiApOTk9BoNFAolUro9XoQEcxmM9bX13F0dIR4PI7b21vYbDY0NDQgm83C4/Hg4OAA6XQau7u7KJVKuLy8xP7+Pvb29sDzPCwWC8bHx/H5+Qmv1wtZlvHy8gKlUonFxUUwDIP7+3vY7XZUV1eDZVn09PRArVbj+PgYq6urKJfLYBgG7+/vaGxshMlkgk6nw/z8PKLRKLLZLIxGIwwGAxYWFiDLMvD4+Egul4vq6uqoUqmQSqWi/v5+isViZLPZ6OPjg/L5PAUCASqXy7S2tka1tbUkiiIJgkA+n4+urq7o/Pycmpub6fX1lYLBIDmdTmpvb6eNjQ06PDykYrFIW1tbpNVqyev1UrFYpKenJ+ro6KBQKESpVIpGRkZoZWWFmpqaqKuri/x+P1ksFtJoNNTb20tVVVX09vZGsViMUqkURaNRstvttLy8TP8B0X5r1+LxSH4AAAAASUVORK5CYII=\n",
"text/plain": [
"Image (3, 4, 50)"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Image(EI(tt)[0])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Insert custom input stage"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
"net_custom_resnet = nn.Sequential(ExpandInput(), learn_dummy.model) # insert ExpandInput layer at the beginning of the network"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([64, 3, 4, 50])"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# run dummy data through custom input stage to test it\n",
"net_custom_resnet[0](tt).shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Model training"
]
},
{
"cell_type": "markdown",
"metadata": {
"heading_collapsed": true
},
"source": [
"## Tensorboard logger"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {
"hidden": true
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"# From https://gist.github.com/gyglim/1f8dfb1b5c82627ae3efcfbbadb9f514\n",
"\"\"\"Simple example on how to log scalars and images to tensorboard without tensor ops.\n",
"License: Copyleft\n",
"\"\"\"\n",
"#__author__ = \"Michael Gygli\"\n",
"\n",
"#import tensorflow as tf\n",
"#from StringIO import StringIO\n",
"#import matplotlib.pyplot as plt\n",
"#import numpy as np\n",
"\n",
"class Logger(object):\n",
" \"\"\"Logging in tensorboard without tensorflow ops.\"\"\"\n",
"\n",
" def __init__(self, log_dir):\n",
" \"\"\"Creates a summary writer logging to log_dir.\"\"\"\n",
" self.writer = tf.summary.FileWriter(log_dir)\n",
"\n",
" def log_scalar(self, tag, value, step):\n",
" \"\"\"Log a scalar variable.\n",
" Parameter\n",
" ----------\n",
" tag : basestring\n",
" Name of the scalar\n",
" value\n",
" step : int\n",
" training iteration\n",
" \"\"\"\n",
" summary = tf.Summary(value=[tf.Summary.Value(tag=tag,\n",
" simple_value=value)])\n",
" self.writer.add_summary(summary, step)\n",
"\n",
" def log_images(self, tag, images, step):\n",
" \"\"\"Logs a list of images.\"\"\"\n",
"\n",
" im_summaries = []\n",
" for nr, img in enumerate(images):\n",
" # Write the image to a string\n",
" s = StringIO()\n",
" plt.imsave(s, img, format='png')\n",
"\n",
" # Create an Image object\n",
" img_sum = tf.Summary.Image(encoded_image_string=s.getvalue(),\n",
" height=img.shape[0],\n",
" width=img.shape[1])\n",
" # Create a Summary value\n",
" im_summaries.append(tf.Summary.Value(tag='%s/%d' % (tag, nr),\n",
" image=img_sum))\n",
"\n",
" # Create and write Summary\n",
" summary = tf.Summary(value=im_summaries)\n",
" self.writer.add_summary(summary, step)\n",
" \n",
"\n",
" def log_histogram(self, tag, values, step, bins=1000):\n",
" \"\"\"Logs the histogram of a list/vector of values.\"\"\"\n",
" # Convert to a numpy array\n",
" values = np.array(values)\n",
" \n",
" # Create histogram using numpy \n",
" counts, bin_edges = np.histogram(values, bins=bins)\n",
"\n",
" # Fill fields of histogram proto\n",
" hist = tf.HistogramProto()\n",
" hist.min = float(np.min(values))\n",
" hist.max = float(np.max(values))\n",
" hist.num = int(np.prod(values.shape))\n",
" hist.sum = float(np.sum(values))\n",
" hist.sum_squares = float(np.sum(values**2))\n",
"\n",
" # Requires equal number as bins, where the first goes from -DBL_MAX to bin_edges[1]\n",
" # See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/framework/summary.proto#L30\n",
" # Thus, we drop the start of the first bin\n",
" bin_edges = bin_edges[1:]\n",
"\n",
" # Add bin edges and counts\n",
" for edge in bin_edges:\n",
" hist.bucket_limit.append(edge)\n",
" for c in counts:\n",
" hist.bucket.append(c)\n",
"\n",
" # Create and write Summary\n",
" summary = tf.Summary(value=[tf.Summary.Value(tag=tag, histo=hist)])\n",
" self.writer.add_summary(summary, step)\n",
" self.writer.flush()\n",
" \n",
"\"A `Callback` that saves tracked metrics into a log file for Tensorboard.\"\n",
"# Based on https://gist.github.com/gyglim/1f8dfb1b5c82627ae3efcfbbadb9f514\n",
"# and devforfu: https://nbviewer.jupyter.org/gist/devforfu/ea0b3fcfe194dad323c3762492b05cae\n",
"# Contribution from MicPie\n",
"\n",
"#from ..torch_core import *\n",
"#from ..basic_data import DataBunch\n",
"#from ..callback import *\n",
"#from ..basic_train import Learner, LearnerCallback\n",
"#import tensorflow as tf\n",
"\n",
"__all__ = ['TBLogger']\n",
"\n",
"@dataclass\n",
"class TBLogger(LearnerCallback):\n",
" \"A `LearnerCallback` that saves history of metrics while training `learn` into log files for Tensorboard.\"\n",
" \n",
" log_dir:str = 'logs'\n",
" log_name:str = 'data'\n",
" log_scalar:bool = True # log scalar values for Tensorboard scalar summary\n",
" log_hist:bool = True # log values and gradients of the parameters for Tensorboard histogram summary\n",
" log_img:bool = False # log values for Tensorboard image summary\n",
"\n",
" def __post_init__(self): \n",
" super().__post_init__()\n",
" #def __init__(self):\n",
" # super().__init__()\n",
" self.path = self.learn.path\n",
" (self.path/self.log_dir).mkdir(parents=True, exist_ok=True) # setup logs directory\n",
" self.Log = Logger(str(self.path/self.log_dir/self.log_name))\n",
" self.epoch = 0\n",
" self.batch = 0\n",
" self.log_grads = {}\n",
" \n",
" def on_backward_end(self, **kwargs:Any):\n",
" self.batch = self.batch+1\n",
" #print('\\nBatch: ',self.batch)\n",
" \n",
" if self.log_hist:\n",
" for tag, value in learn.model.named_parameters():\n",
" tag_grad = tag.replace('.', '/')+'/grad'\n",
" \n",
" if tag_grad in self.log_grads:\n",
" #self.log_grads[tag_grad] += value.grad.data.cpu().detach().numpy()\n",
" self.log_grads[tag_grad] = self.log_grads[tag_grad] + value.grad.data.cpu().detach().numpy() # gradients are summed up from every batch\n",
" #print('if')\n",
" else:\n",
" self.log_grads[tag_grad] = value.grad.data.cpu().detach().numpy()\n",
" #print('else')\n",
" \n",
" #print(tag_grad, self.log_grads[tag_grad].sum())\n",
" return self.log_grads\n",
" \n",
" #def on_step_end(self, **kwards:Any):\n",
" #print('Step end: ', self.log_grads)\n",
"\n",
" def on_epoch_end(self, epoch:int, smooth_loss:Tensor, last_metrics:MetricsList, **kwargs:Any) -> bool:\n",
" last_metrics = ifnone(last_metrics, [])\n",
" tr_info = {name: stat for name, stat in zip(self.learn.recorder.names, [epoch, smooth_loss] + last_metrics)}\n",
" self.epoch = tr_info['epoch']\n",
" self.batch = 0 # reset batch count\n",
" #print('\\nEpoch: ',self.epoch)\n",
" \n",
" if self.log_scalar:\n",
" for tag, value in tr_info.items():\n",
" if tag == 'epoch': continue\n",
" self.Log.log_scalar(tag, value, self.epoch+1)\n",
" \n",
" if self.log_hist:\n",
" for tag, value in learn.model.named_parameters():\n",
" \n",
" tag = tag.replace('.', '/')\n",
" self.Log.log_histogram(tag, value.data.cpu().numpy(), self.epoch+1)\n",
" \n",
" tag_grad = tag.replace('.', '/')+'/grad'\n",
" self.Log.log_histogram(tag_grad, self.log_grads[tag_grad], self.epoch+1)\n",
" #print(tag_grad, self.log_grads[tag_grad].sum())\n",
" \n",
" #if self.log_img:\n",
" # for tag, value in learn.model.named_parameters():\n",
" # \n",
" # tag = tag.replace('.', '/')\n",
" # self.Log.log_images(tag, value.data.cpu().numpy(), self.epoch+1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Train custom fastai ResNet with fastai "
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [],
"source": [
"learn_resnet = Learner(data, net_custom_resnet, metrics=accuracy_thresh)"
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {},
"outputs": [],
"source": [
"#[p.shape for p in net_custom_resnet.parameters()]"
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"======================================================================\n",
"Layer (type) Output Shape Param # Trainable \n",
"======================================================================\n",
"Lambda [64, 3, 4, 50] 0 False \n",
"______________________________________________________________________\n",
"Conv2d [64, 64, 2, 25] 9408 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 64, 2, 25] 128 True \n",
"______________________________________________________________________\n",
"ReLU [64, 64, 2, 25] 0 False \n",
"______________________________________________________________________\n",
"MaxPool2d [64, 64, 1, 13] 0 False \n",
"______________________________________________________________________\n",
"Conv2d [64, 64, 1, 13] 36864 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 64, 1, 13] 128 True \n",
"______________________________________________________________________\n",
"ReLU [64, 64, 1, 13] 0 False \n",
"______________________________________________________________________\n",
"Conv2d [64, 64, 1, 13] 36864 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 64, 1, 13] 128 True \n",
"______________________________________________________________________\n",
"Conv2d [64, 64, 1, 13] 36864 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 64, 1, 13] 128 True \n",
"______________________________________________________________________\n",
"ReLU [64, 64, 1, 13] 0 False \n",
"______________________________________________________________________\n",
"Conv2d [64, 64, 1, 13] 36864 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 64, 1, 13] 128 True \n",
"______________________________________________________________________\n",
"Conv2d [64, 128, 1, 7] 73728 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 128, 1, 7] 256 True \n",
"______________________________________________________________________\n",
"ReLU [64, 128, 1, 7] 0 False \n",
"______________________________________________________________________\n",
"Conv2d [64, 128, 1, 7] 147456 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 128, 1, 7] 256 True \n",
"______________________________________________________________________\n",
"Conv2d [64, 128, 1, 7] 8192 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 128, 1, 7] 256 True \n",
"______________________________________________________________________\n",
"Conv2d [64, 128, 1, 7] 147456 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 128, 1, 7] 256 True \n",
"______________________________________________________________________\n",
"ReLU [64, 128, 1, 7] 0 False \n",
"______________________________________________________________________\n",
"Conv2d [64, 128, 1, 7] 147456 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 128, 1, 7] 256 True \n",
"______________________________________________________________________\n",
"Conv2d [64, 256, 1, 4] 294912 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 256, 1, 4] 512 True \n",
"______________________________________________________________________\n",
"ReLU [64, 256, 1, 4] 0 False \n",
"______________________________________________________________________\n",
"Conv2d [64, 256, 1, 4] 589824 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 256, 1, 4] 512 True \n",
"______________________________________________________________________\n",
"Conv2d [64, 256, 1, 4] 32768 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 256, 1, 4] 512 True \n",
"______________________________________________________________________\n",
"Conv2d [64, 256, 1, 4] 589824 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 256, 1, 4] 512 True \n",
"______________________________________________________________________\n",
"ReLU [64, 256, 1, 4] 0 False \n",
"______________________________________________________________________\n",
"Conv2d [64, 256, 1, 4] 589824 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 256, 1, 4] 512 True \n",
"______________________________________________________________________\n",
"Conv2d [64, 512, 1, 2] 1179648 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 512, 1, 2] 1024 True \n",
"______________________________________________________________________\n",
"ReLU [64, 512, 1, 2] 0 False \n",
"______________________________________________________________________\n",
"Conv2d [64, 512, 1, 2] 2359296 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 512, 1, 2] 1024 True \n",
"______________________________________________________________________\n",
"Conv2d [64, 512, 1, 2] 131072 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 512, 1, 2] 1024 True \n",
"______________________________________________________________________\n",
"Conv2d [64, 512, 1, 2] 2359296 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 512, 1, 2] 1024 True \n",
"______________________________________________________________________\n",
"ReLU [64, 512, 1, 2] 0 False \n",
"______________________________________________________________________\n",
"Conv2d [64, 512, 1, 2] 2359296 False \n",
"______________________________________________________________________\n",
"BatchNorm2d [64, 512, 1, 2] 1024 True \n",
"______________________________________________________________________\n",
"AdaptiveAvgPool2d [64, 512, 1, 1] 0 False \n",
"______________________________________________________________________\n",
"AdaptiveMaxPool2d [64, 512, 1, 1] 0 False \n",
"______________________________________________________________________\n",
"Lambda [64, 1024] 0 False \n",
"______________________________________________________________________\n",
"BatchNorm1d [64, 1024] 2048 True \n",
"______________________________________________________________________\n",
"Dropout [64, 1024] 0 False \n",
"______________________________________________________________________\n",
"Linear [64, 512] 524800 True \n",
"______________________________________________________________________\n",
"ReLU [64, 512] 0 False \n",
"______________________________________________________________________\n",
"BatchNorm1d [64, 512] 1024 True \n",
"______________________________________________________________________\n",
"Dropout [64, 512] 0 False \n",
"______________________________________________________________________\n",
"Linear [64, 2] 1026 True \n",
"______________________________________________________________________\n",
"\n",
"Total params: 11705410\n",
"Total trainable params: 538498\n",
"Total non-trainable params: 11166912\n"
]
}
],
"source": [
"learn_resnet.summary()"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEKCAYAAAAvlUMdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VNX9//HXJzshC0FCWMISdsKqBAqCijvu2rrA112/tS5ov7a2tb9u1m62atVWrVqr1rriWlxxBVzYwr7vSMKSBAKEJGQ/vz9mwAAJ2eZmMsn7+XjMw5k75977OU6Yz5xzzz3HnHOIiIgEWliwAxARkdZJCUZERDyhBCMiIp5QghEREU8owYiIiCeUYERExBNKMCIi4gklGBER8YQSjIiIeCIi2AEESqdOnVzv3r2DHYaISEhZuHDhLudcshfHbjUJpnfv3mRmZgY7DBGRkGJm33h1bHWRiYiIJ5RgRETEE0owIiLiCSUYERHxhBKMiIh4QglGREQ8oQQjIiKeUIKRRvto5U427yoKdhgi0kIpwUij5O4v4eYXFvKH91YFOxQRaaGUYKRRpi/ZTpWD2et2se9AebDDEZEWSAlGGuWtxdtIio2krLKKT1blBDscEWmBlGCkwdbl7Gfl9gKmntaf7h3a8d7yHcEOSURaoFYz2aU0n7cWbyM8zLhoZDdyCkp49qvN7CsuJzE2MtihiUgL4mkLxswmmdlaM9tgZnfX8H5PM/vczBab2TIzO7faez/377fWzM72Ms7WIHd/CTv3lXh+nqoqx38Xb+Pk/p3oFBfNecO6Ul7pmLFqp+fnFpHQ4lmCMbNw4DHgHCAdmGJm6UcU+yUwzTl3PDAZeNy/b7r/9RBgEvC4/3hSg1nr8jjjwVmc/uBM3l22vVHHcM6xNGsvOQUlOOdqLTd382627yvhkhNSARiemkiPju14b5m6yUTkcF52kY0BNjjnNgGY2SvARUD1ca0OSPA/TwQOfjteBLzinCsFNpvZBv/x5ngYb8hxzvHErE3cP2MNA1LiiY0KZ+pLi1n4zR5+fs5goiLq//vh/eU7ue2lRQAkxUYyICWe9G4J3HRyH7omtjtU7q1F24iLjuDMwSkAmBnnDevG019sYk9RGUntowJbSREJWV52kXUHsqq9zvZvq+4e4CozywbeB25vwL6Y2U1mlmlmmXl5eYGKOyQUl1Uw9eXF/PnDNZwzrCtv3noir9w0juvH9+bZr7Yw5Z9zG9Rl9u+vt9CjYzvuuSCdSUO7UFZZxYvztnLB379i4Td7ACgpr+SDFTuZNLQL7aK+bVCeP7wrFVWOGSvVTSYi3/IywVgN247se5kCPOecSwXOBf5jZmH13Bfn3FPOuQznXEZysicrfjZZVn4xv3hrOZMens3anfsDckznHDc8t4APlu/g7nMG8eiU44mNiiAqIozfXDCER//neFbvKODcv33BB/UY4bVmZwHzt+RzzdjeXDc+jT99dzhv3Tqe926fQPvocKY8NZfXF2bz8aocCksr+O7xh+f6Id0S6H1crEaTichhvEww2UCPaq9T+bYL7KAbgWkAzrk5QAzQqZ77tmgb8wr58bSlTHxgJq9lZrOzoITJT81h5fZ9TT72/M35zN2Uz6/OT+fmU/pidng+Pn94N6ZPnUD3Du245cVF3PnqkmPeDPnC3G+Ijgjj0lGph23vnxLP27eOJ6N3Ene9tpR7pq+ka2IMY/scd1g5M+O84V35euNudheWNrl+ItI6eJlgFgD9zSzNzKLwXbSffkSZrcDpAGY2GF+CyfOXm2xm0WaWBvQH5nsYa0C9uSibM/46i/eWb+facb2Z/dNTefvW8bSLDOd//jmPpVl7m3T8f36xiY7to5gypmetZfp1juPNW0/kh6f3Z/rS7Ux6eDZfrt91VLn9JeW8tWgbF4zoVuP1k6T2Ufz7hjFcO64Xu4vKuPj47oSFHd3APH94NyqrHO+rFSMifp4lGOdcBTAVmAGsxjdabKWZ3WtmF/qL/Rj4vpktBV4GrnM+K/G1bFYBHwK3OecqvYo1kJxz/GPmRgZ3SeDLn53Gry9Ip0tiDL07tefVH4wjoV0EVz09j4Xf5Dfq+BtyC/lkdS7XjOtFTOSxB9ZFhodx55kDePOWE4mNCufaZ+cflWTeXryNorJKrh7b65jH+e1FQ3n7tvH83xn9aywzqEs8g7rEc887q/jJa0vZuru44ZUTkVbFjjUkNZRkZGS4zMzMYIfB4q17uOTxr/nTd4fV2MLYvvcAVz49j5yCEqb9YBxDuyc26Ph3v7GMtxZv4+u7T+O4uOh677e/pJxL/zGHHfsO8PZt4+mTHIdzjrMfnk1MZDjTp05oUBw1ydtfyuMzN/DivK1UVjm+d0J37ji9P6lJsU0+toh4w8wWOucyvDi2pooJsGmZWbSLDOf84V1rfL9bh3a8etNYEttF8oP/LCS/qKzex87dX8Kbi7Zx6ajUBiUXgPiYSJ6+NoOI8DD+99+Z7CsuZ/7mfNblFHLVMVovDZEcH81vLhjCFz89lavH9uLtJdu56NGvNKW/SBulBBNAxWUVvLN0B+cN70p8TO3TpnROiOHJq0eRV1jK1JcWUVFZVa/j/2fON5RXVXHjhLRGxdejYyxPXDWKrD3F3PbSIp77eguJ7SK5YHi3Rh2vNikJMdxz4RDev+MkHHD1v+aRW+D9LAMi0rIowQTQ+8t3UlhaweUZPeosOzy1A3+8ZBhfb9zNfR+sqbN8cVkF/5n7DWcOTqFPclyjYxyT1pE/XDyMLzfs4oMVO7lsVOph97QEUr/OcTx73Wjyi8q45pn5mtZfpI3RZJcBNG1BFmmd2jO6d1K9yl86KpUV2/bx9JebGZaayJnpKSzeupf5m/NZmr2XzvHRjOjRgRGpHZi3OZ+9xeX84JQ+TY7z8tE92JhXyL/nbOHKAHWP1WZEjw48efUobnhuAd9/PpPnbxhT5+AEEWkddJE/QDblFXLag7P46aSB3DqxX733K6+s4sqn57F46x6cg4oqR5j5fv3n7i9lb/G3v/pP6NmBN28dH7CYC0sriItunt8Y05du54evLGZCv0784eJh9DxOF/5FWgIvL/KrBRMgry3MJjzMuPSE1LoLVxMZHsbjV57Ab99ZRY+kdoxO68ioXkkkxETinCMr/wBLs/eyakcB5w2reeBAYzVXcgG4cEQ3ikor+O07Kzn9rzO58ju9uP20fg0erCAioUMtmACoqKzixPs+Y1j3RP513eigxBAqcgpKePiT9UzLzCImIozrx6dx0chu9Oscd9SMBCLiPbVgWrhZ6/LI3V/K5aPrvrjf1qUkxPCn7w7jxglp3D9jDY9+voFHP99An07tOXNIChcM79bge4NEpGVSggmAaZlZdIqL4rRBnYMdSsjo1zmOJ6/OYOe+Ej5encNHK3fyry828+SsTTz6P8dzfoCHTotI81OCaaJdhaV8ujqXGyakERmuUd8N1SUxhqvH9uLqsb3YV1zOjf9ewI+nLaVnx1iGp3YIdngi0gT6RmyitxZto6LKcXlGwy7uy9ESYyN54upRdIqL5vvPZzbLEtAi4h0lmCZwzjEtM4sTenagX+f4YIfTKnSKi+bpazPYX1LBTf/J5EBZSMxxKiI1UIJpgiVZe1mfW1ivO/el/gZ3TeCRycezfNs+7nptKRtyCykpV6IRCTW6BtME0zKzaRcZznm1TGwpjXdmego/PXsQf/5wzaGVMjvHR9OzYyyjeidxSv9kRvVOIjpCswKItFRKMI3km9hye50TW0rj3TKxLxP6dWJD3n6y8g+QlV/M5l1Fh0abtYsMZ1zf47j5lL6MSesY7HBF5AhKMHUoKa/krcXb6NUxlhP7dTq0/YMGTGwpjTcsNZFhqYffF1NYWsHcjbuZvT6Pj1bmMPmpOdx2aj/uOL2/RvKJtCBKMLUoq6jitYVZPPrZBnb4RzPddHIf7jprIFERYUzLzKL3cbH1nthSAicuOoIz0lM4Iz2Fn04axD3TV/L3zzbw5YZdPHLF8ZrnTKSFUIKpwXvLdnDfh6vJyj/ACT078KfvDuOT1Tk8NXsT8zbt5kdnDWTe5nx+cvZATW8SZHHRETxw2QhOHpDML95azrl/+4IrRvdgeGoiQ7olkNYpjvAwfUYiwaC5yI6wekcB5/7tCwZ3SeAnkwYycUDyoSTywfId/OyNZRSUVBBmMOfnp5OSENPkc0pgZO8p5pdvr+Drjbspq/At4tYuMpxBXeMZ1CWegSnxDOySwMAu8XRsHxXkaEVaBs1F1oz+8uEa4qMjeOn736FD7OFfQucM68qw1ETufmM5PTq2U3JpYVKTYnnu+jGUV1axMa+QFdsKWLFtH6t3FPDBip28PD/rUNnj2kfRt3Mc/TrHMapnEheM6EZUxNHXb7btPcC8Tbs5f3jN74OvO7W29yqrHC/P30qf5Pac2LdTjWVEWiu1YKqZs3E3U/45l7vPGcTNp/QNUGTSEjjnyN1fypqd+1mfs58NuYWszy1kQ24h+w6U071DO6ae1o9LR6USGR5GVn4x/5i1kdcysyivdIzs0YHHrjyB7h3aHTpmRWUVT8zayN8+3cApA5P51Xnph13/ycov5s5Xl5D5zR4ArjuxN3efM0gLrkmL4mULRgnGzznHxY9/TW5BCZ/fNVFfAm2Ec45Z6/J46JP1LM3aS2pSO0b1SuK9ZTsIM+Py0akM657I795dTUS48dDlIzl1UGfW5eznrteWsix7H+P7HcfirXupqHLcfHIfbpnYj/eX7+A301diwK8uSGf1jgKe/WoLfZPb8/AVxx81Mk4kWJRg6qGpCeb95Tu49cVF/OXS4Rp63AY555i5Lo+HP1nP6h0FTBndg5sn9qVroq/FsnlXEbe+uIjVOwqYNKQLn63JJS4mgt9fPJRzh3Vlx74D3PfBGv67ZDvx0RHsL61gTO+OPHj5CHp09LVqvly/i7teW8quwlLuPmcQN05I0yARCTolmHpoSoIpr6zirIdmExlufPDDkzXqqA1zzuEchNXwN1BSXsk901fyyoIszh3WhXsvGkqnI1bknL85n0c+XceEfsncdHKfo/6W9hWX87M3lvHhyp3cdHIffn7OICUZCSolmHpoSoJ5Ye43/PLtFTx9TQZnpKcEODJpbXIKSpo0wKOqynHPOyt5fs43XDYqlT99dxgRukFUgkSjyDxUVFrBw5+sZ3TvJE4frAXDpG5NHT0YFmb89sIhJMVG8cin69l7oJy/Tzle1/2k1WnzP5v2l1QwrHsCd58zWF0V0mzMjDvPHMA9F6Tz8aocbnhuAcVlFcEOSySg2nyC6ZIYw7PXj2FUL035Is3vuvFp/PXyEczdtJsbn8tUkpFWpc0nGJFg++4Jqfz18pHM26wkI62LpwnGzCaZ2Voz22Bmd9fw/kNmtsT/WGdme6u99xczW2lmq83sb6b+K2nFLj6+u5KMtDqeJRgzCwceA84B0oEpZpZevYxz7k7n3Ejn3Ejg78Cb/n1PBMYDw4GhwGjgFK9iFWkJjkwyWi5aQp2XLZgxwAbn3CbnXBnwCnDRMcpPAV72P3dADBAFRAORQI6HsYq0CAeTzNzNu7n5hYWUVijJSOjyMsF0B7Kqvc72bzuKmfUC0oDPAJxzc4DPgR3+xwzn3GoPYxVpMS4+vjv3fXcYs9blccfLi6morAp2SCKN4mWCqemaSW13dU4GXnfOVQKYWT9gMJCKLymdZmYnH3UCs5vMLNPMMvPy8gIUtkjwXTG6J7+5IJ0ZK3P48WtLqaxqHTdES9viZYLJBqpP6pUKbK+l7GS+7R4DuASY65wrdM4VAh8AY4/cyTn3lHMuwzmXkZycHKCwRVqG68en8ZOzB/LfJdv55dvLaS2zbkjb4WWCWQD0N7M0M4vCl0SmH1nIzAYCScCcapu3AqeYWYSZReK7wK8uMmlzbju1H7dO7MvL87N4dUFW3TuItCCeJRjnXAUwFZiBLzlMc86tNLN7zezCakWnAK+4w3+evQ5sBJYDS4Glzrl3vIpVpCW766yBjO93HL99ZxUb8wqDHY5IvWmyS5EQsHNfCZMemU1qUjvevGV8rStoijSUl5Nd6q9UJAR0SYzhz98bzoptBTz40dpghyNSL0owIiHi7CFdmDKmJ0/O3sRXG3YFOxyROinBiISQX50/mL7J7bnz1SWs2LYv2OGIHJMSjEgIiY2K4O9TTsABFz32FQ/MWKu7/aXFavMLjomEmvRuCXx858n87t3VPPr5Bmas3MnvLh5KaUUVmVvyWbAln9U79jOuz3HcdEofTuippSgkODSKTCSEfb4ml//31nJ27CsBIDzMGNItgX6d4/h0dS77DpQzuncS3z+pD2cMTiEsTJOSy+G8HEWmBCMS4gpKynlv2Q56doxlZI8OtI/2dUwUlVYwLTOLp7/YzLa9BxiRmshvLxrKyB4dghyxtCRKMPWgBCNSs4rKKv67ZDv3fbiGXYWlXJHRg59OGkTH9lHBDk1aAN0HIyKNFhEexvdGpfLZj0/hfyek8frCbE59YCafr8kNdmjSyinBiLQR8TGR/OK8dD744Ukkx0fzm+krNUuzeEoJRqSN6Z8Sz51nDGBrfjGfrtY6fuIdJRiRNujsISl079COf325OdihSCumBCPSBkWEh3Htib2YtzlfMwKIZ5RgRNqoK0b3JDYqnGe+UitGvKEEI9JGJbaL5PKMHryzdDu5+0uCHY60QkowIm3YdSf2pqLK8cKcb4IdirRCSjAibVjvTu05fVAKL8zbSkm5Js2UwFKCEWnjbpyQRn5RGf9dsi3YoUgrowQj0saN7dORwV0TeParLbSWqaOkZVCCEWnjzIzrTuzFmp37WbBlT7DDkVZECUZEuHBEdxLbRfL8nC3BDkVaESUYEaFdVDiXjUrlwxU7yS3QkGUJDCUYEQHgqrG9qKhyvDR/a7BDkVZCCUZEAN+Q5YkDk3lp3lbKK6uCHY60AkowInLINeN6kbu/lBkrdwY7FGkFlGBE5JBTBnSmR8d2PK87+yUAlGBE5JDwMOOq7/Ri/uZ81uwsCHY4EuKUYETkMJdn9CA6IozHPt9Yr+ljikor+NMHqxn7x095f/mOZohQQoUSjIgcJql9FFeN7cU7S7cz4c+f89jnG9hXXH5UOeccHyzfwRl/ncWTszYRHmbc+uIi/vrxOqq0FLMA1lqmhsjIyHCZmZnBDkOkVXDOMWfjbp6YvYnZ6/KIjQrnrPQUEttFEhMZTnRkOEuy9jJ7XR6DusTzh0uGMqRbIr98ewWvL8zm7CEpPHj5SOKiI4JdFamDmS10zmV4cmwvE4yZTQIeAcKBp51z9x3x/kPAqf6XsUBn51wH/3s9gaeBHoADznXObantXEowIt5Ytb2Af36xiTkbd1NSUUlJeSUl5VXEx0Rw5xkDuGZcLyLCfZ0hzjme+WoLf3hvFf07x/PC/36H5PjoINdAjiUkE4yZhQPrgDOBbGABMMU5t6qW8rcDxzvnbvC/ngn8wTn3sZnFAVXOueLazqcEI9J8nHM4B2FhVuP7X67fxQ3/XsBpAzvzxNWjmjk6aQgvE4yX12DGABucc5ucc2XAK8BFxyg/BXgZwMzSgQjn3McAzrnCYyUXEWleZlZrcgGY0L8Td54xgA9X7uQDXfhvs7xMMN2BrGqvs/3bjmJmvYA04DP/pgHAXjN708wWm9n9/hbRkfvdZGaZZpaZl5cX4PBFpCm+f1IaQ7sn8Kv/rmRvcVmww5Eg8DLB1PTzprb+uMnA6865g2MiI4CTgLuA0UAf4LqjDubcU865DOdcRnJyctMjFpGAiQgP48/fG86e4jJ+/97qYIcjQeBlgsnGd4H+oFRgey1lJ+PvHqu272J/91oF8DZwgidRiohnhnRL5OZT+vD6wmxmr1MvQ1vjZYJZAPQ3szQzi8KXRKYfWcjMBgJJwJwj9k0ys4PNktOAGgcHiEjLdvtp/emT3J6fv7mcotKKYIcjzcizBONveUwFZgCrgWnOuZVmdq+ZXVit6BTgFVdtOJu/q+wu4FMzW46vu+2fXsUqIt6JiQznL98bzra9B3jmy82NPo5zjgrN8hxS6jVM2cz6AtnOuVIzmwgMB553zu31OL560zBlkZbtmmfms3ZnAV/+7DQiwxv229Y5x+0vL2Zdzn7eunU87XUDZ8C0hGHKbwCVZtYP+Be+EV8veRGQiLRO153Yi5yCUj5c0fClAKZlZvHush2syynkj+9rwECoqG+CqfJ3eV0CPOycuxPo6l1YItLaTBzQmV7HxfLc11satN83u4v47TurOLHvcdw4IY0X521l5tpcb4KUgKpvgik3synAtcC7/m2R3oQkIq1RWJhx7bjeLPxmD8uz9x32XlWV457pK7l/xhoOlH07g3NFZRV3vrqE8DDjgctG8JOzB9K/cxw/e2NZjRNwSstS3wRzPTAO39Qtm80sDXjBu7BEpDW6NCOV2Kjwo1oxD32yjue+3sJjn2/krIdnHRrS/MSsjSzaupffXzyUbh3aERMZzkNXjGR3YRm/nr4iCDWQhqhXgnHOrXLO3eGce9nMkoD4IyeuFBGpS0JMJJeOSuWdpdvZVVgKwAfLd/D3zzZweUYqr9w0lsiwMK55Zj7ffz6Thz9Zz/nDu3LhiG6HjjG0eyK3n9af/y7ZznvLNA1NS1avBGNmM80swcw6AkuBZ83sr96GJiKt0TXjelNWWcUr87eydud+fvzaUkb26MDvLh7K2D7H8f4PT+KO0/szc20uneKi+f3FQzE7fGKQW0/ty/DURH713xX1WhRNgqO+XWSJzrkC4LvAs865UcAZ3oUlIq1Vv85xnNS/E/+Z+w03/SeT9tERPHn1KKIjfNMNxkSG86MzB/DZjyfy5q0n0iE26qhjRIaH8bNJg8gvKmPGyoaPSpPmUd8EE2FmXYHL+fYiv4hIo1w/vjc5BaVs33uAJ64aRUpCzFFlenSMpVuHdrUeY1yf4+jZMZaX52/1MlRpgvommHvx3ZG/0Tm3wMz6AOu9C0tEWrOJAzpz6ahU/nr5SEb1SmrUMcLCjCtG92Dupnw27yoKcISho7Sikpa6MnF9L/K/5pwb7py7xf96k3Pue96GJiKtVZh/2PEF1S7eN8alo1IJDzNeXZBVd+FW6pYXFnHx418HO4wa1fcif6qZvWVmuWaWY2ZvmFmq18GJiBxLSkIMpw7szOsLsylvo/OU7S4qIyGmZU6dU98usmfxzYTcDd+iYe/4t4mIBNXk0T3YVVjKp6vb5t39+UWlHNf+6IEQLUF9E0yyc+5Z51yF//EcoBW+RCToJg5MJiUhmlcXtM2L/fmFZSSFeILZZWZXmVm4/3EVsNvLwERE6iMiPIzLRvVg1ro8tu89EOxwmlVJeSVFZZUh34K5Ad8Q5Z3ADuBSfNPHiIgE3RWje1Dl4LXM7GCH0qz2FJcB0LF9dJAjqVl9R5Ftdc5d6JxLds51ds5djO+mSxGRoOvRMZYJ/ToxLTOrTd3Zv7vwYIIJ7RZMTX4UsChERJroxpPS2Lb3ALe9uIiyirYxouzbFkzrSzBWdxERkeZx6sDO/P7ioXy6JpfbX17UJoYt5xe13gTTMm8dFZE266qxvfjNBenMWJnDna8uoaKVJ5mDXWQt9SL/Me/OMbP91JxIDKh9kiARkSC5fnwa5ZVV/PH9NUSFh3H/ZSMID2udHS57issIM0hs1zLXfzxmgnHOxTdXICIigXLTyX0pr3TcP2MtGNx/aetMMruLykiKjSKshdatZc4vICLSRLed2o+qKseDH68DR6tsyeQXlrXY6y+gBCMirdjtp/fHDB74aB1VzvHg5SNbVZLJL1KCEREJmqmn9cfMuH/GWhzw4GUjiAhvyvimliO/uIz+neOCHUatlGBEpNW77dR+hJnx5w/XMLbPcUwZ0zPYIQVES2/BtI40LiJSh5tP6cOQbgk88+XmFrtAV0NUVjn2FJe12CHKoAQjIm2EmXHD+DTW5xby5YZdwQ6nyfYdKMc5WuxMyqAEIyJtyPkjutIpLppnv9oS7FCaLL+oFGi5d/GDEoyItCHREeFcNbYnn63JZVNeYbDDaZJv7+JvmTMpg8cJxswmmdlaM9tgZnfX8P5DZrbE/1hnZnuPeD/BzLaZ2aNexikibceV3+lFVHgY//56S7BDaZKDE10mtW+Zd/GDhwnGzMKBx4BzgHRgipmlVy/jnLvTOTfSOTcS+Dvw5hGH+R0wy6sYRaTtSY6P5oIR3XhtYTb7DpQHO5xG213UtlswY4ANzrlNzrky4BXgomOUnwK8fPCFmY0CUoCPPIxRRNqg68f3priskmkLsoIdSqPlF7bhFgzQHaj+6WX7tx3FzHoBacBn/tdhwIPATzyMT0TaqKHdExmT1pHnvt4SsjMu7y4qIz46guiI8GCHUisvE0xN8zHUNvh8MvC6c+7gUnS3Au87547588LMbjKzTDPLzMvLa0KoItLW3DC+N9v2HuDxmRupqgq9+2L2FJe16CHK4G2CyQZ6VHudCmyvpexkqnWPAeOAqWa2BXgAuMbM7jtyJ+fcU865DOdcRnJycmCiFpE24cz0Lpw9JIW/fryOa5+dT05BSbBDapCWfhc/eJtgFgD9zSzNzKLwJZHpRxYys4FAEjDn4Dbn3JXOuZ7Oud7AXcDzzrmjRqGJiDRWeJjxxFWj+MMlQ1mwJZ9JD8/mwxU7gx1Wve0ubNl38YOHCcY5VwFMBWYAq4FpzrmVZnavmV1YregU4BXXGuZuEJGQYmZc+Z1evHfHSaQmxXLzCwt5YtbGYIdVL6HQRebpZJfOufeB94/Y9usjXt9TxzGeA54LcGgiIof0TY7jjVtO5P9eXcwDM9YyoV8nhnZPDHZYtXLOsbuoDbdgRERCSVREGH+8ZBgd20fxo2lLKCmvrHunICkqq6SsoqpNX4MREQkpHWKj+Mulw1mXU8hDH68Ldji12uO/yVIJRkQkhEwc2Jn/+U5PnvpiE/M35wc7nBrtVoIREQlNvzh3MD2SYrnrtaUUlVYEO5yjhMJMyqAEIyJylPbRETxw2Qiy9hRz+8uLW9ycZflFvnha8jxkoAQjIlKjMWkduffCIcxel8d5f/uCJVmBUL2gAAARnUlEQVR7696pmRxswbTkechACUZEpFZXj+vNtJvH4Rxc9sTXPP3Fphax3PLuojKiwsOIi/b0TpMmU4IRETmGE3om8f4dJzFxYGd+/95qvv985qFRXMGSX+ibJsaspikfWw4lGBGROiTGRvLU1aP49fnpzFqXxzmPfMG8TbuDFs+e4pY/DxkowYiI1IuZccOENN68ZTwxkWFM+edcHvlkPZVBmIl5dwhMdAlKMCIiDTIsNZF37ziJi0Z256FP1jH1pUXNHkMozKQMHs9FJiLSGsVFR/DQFSNJSYjhiVkbycovpkfH2GY7f6gkGLVgREQa6crv9ATg3WU7mu2cZRVV7C+pUIIREWnNenSMZWSPDry7rLa1FANvT3FoTBMDSjAiIk1ywYhurNxewKa8wmY5X75/iHRLn6oflGBERJrkvGFdMWu+brL8EJnoEpRgRESapEtiDKN7deSdpc3TTRYqMymDEoyISJNdMKIr63MLWbtzv+fnyi8MjZmUQQlGRKTJJg3tSpjRLBf784vLMfMtjtbSKcGIiDRRcnw0J/btxDtLt3s+GWZ+USkd2kUSHtay5yEDJRgRkYA4f3hXtuwuZuX2Ak/PEyo3WYISjIhIQEwa2oWIMPP8Yn9+UVmLX2jsIE0VIyISAB1iozipfyemZWYRHmaMSevIqF5JxMcEdlGw3YVl9E2OC+gxvaIWjIhIgPz4rIH0Oq49T87exHXPLmDEbz/iyqfnUlRaEbBz5O4vpXOCWjAiIm3K0O6JvH3beIpKK1i8dS9fbMjjyVmbeHHeN9x0ct8mH7+kvJJ9B8pJSYgJQLTeUwtGRCTA2kdHMKF/J35+zmAm9OvEP7/YTEl5ZZOPm1NQAqAEIyIicNup/cjbX8q0zKwmHyunwHeTZUqIdJEpwYiIeGhsn45k9EriyVmbKKuoatKx1IIREZFDzIzbTuvHtr0HeHvxtiYdSwlGREQOM3FAMkO7J/D4zA1UVDa+FZNTUEJMZBgJMaExPsvTBGNmk8xsrZltMLO7a3j/ITNb4n+sM7O9/u0jzWyOma00s2VmdoWXcYqIeMnMmHpqf7bsLua95Y2f1j+noJSUhBjMWv40MeBhgjGzcOAx4BwgHZhiZunVyzjn7nTOjXTOjQT+Drzpf6sYuMY5NwSYBDxsZh28ilVExGtnpacwICWOxz7fQFVV4+YryykoISU+NLrHwNsWzBhgg3Nuk3OuDHgFuOgY5acALwM459Y559b7n28HcoFkD2MVEfFUWJjxg5P7si6nkMVZexp1jJyCElISlWAAugPVx+Vl+7cdxcx6AWnAZzW8NwaIAjZ6EKOISLMZ368TACu2NXxCTOecr4ssPjSGKIO3CaamTsLa2oWTgdedc4fdiWRmXYH/ANc75466MmZmN5lZppll5uXlNTlgEREvpSRE07F9FKsaMePy/tIKDpRXhswIMvA2wWQDPaq9TgVqm2Z0Mv7usYPMLAF4D/ilc25uTTs5555yzmU45zKSk9WDJiItm5mR3jWB1TsbnmBy9vmHKKuLDIAFQH8zSzOzKHxJZPqRhcxsIJAEzKm2LQp4C3jeOfeahzGKiDSrwV3jWbNzf4OHKx+6i19dZOCcqwCmAjOA1cA059xKM7vXzC6sVnQK8Io7fBm4y4GTgeuqDWMe6VWsIiLNJb1bAmUVVWzaVdSg/ULtJkvweDZl59z7wPtHbPv1Ea/vqWG/F4AXvIxNRCQY0rsmArBqewEDUuLrvV/O/tBLMLqTX0SkGfVJbk9URBirdjTsOkzOvhISYiJoFxXuUWSBpwQjItKMIsPDGJgS3+CRZAfv4g8lSjAiIs0svWsCq3YUcPil52PL2V+iBCMiIseW3i2B/KKyQyPD6iNnnxKMiIjUIb1bAgCrduyrV/mqKkfu/tKQWWjsICUYEZFmNqiLb/TY6h3761U+v7iMiiqnFoyIiBxbfEwkPTvG1vtC/859oTdEGZRgRESC4uCF/vrIPXQPjLrIRESkDundEtiyu4jC0oo6yx6aJkYtGBERqUt61wScg7X1mPgyp6AEM0gOoXnIQAlGRCQoDo0kq8d1mJyCEo5rH01keGh9ZYdWtCIirUTXxBg6xEbW6zqM7y7+0Gq9gBKMiEhQHFwbpr4tmFC7/gJKMCIiQTO4a0K91oZRghERkQZJ75pAaUUVW3bXvjZMeWUVuwrL1EUmIiL1N7S7b22YxVv31lomb39oDlEGJRgRkaDp3zmOTnHRzFqXV2uZnf6VLLsowYiISH2FhRmnDEjmi/W7qKyqeer+XH+C6awuMhERaYiJA5PZd6CcJVk1d5OF6l38oAQjIhJUJ/XvRJjBrLW5Nb6/s6CEyHCjY2xUM0fWdEowIiJB1CE2iuN7JjGzluswOQUldI6PISzMmjmyplOCEREJsokDklmWvY9dhUevcJlbUBqS119ACUZEJOgmDuwMwOwaWjE5BSWkxIfe9RdQghERCboh3RLoFBd11HDl3YWlZO0ppmsHJRgREWmEsDDj5P7JzF6Xd9hw5d9MX0lllWPKmJ5BjK7xlGBERFqAUwYms6e4nGXZvuHKH67YybvLdnDHaf0ZkBIf5OgaRwlGRKQFOLl/MmEGM9fmsa+4nF/9dwXpXRO4eWLfYIfWaBHBDkBERCCpfRQjenRg5ro8tu09QH5RGc9eNzrkFhmrLnQjFxFpZSYO6MzSrL28vjCbW07pe2gyzFDlaYIxs0lmttbMNpjZ3TW8/5CZLfE/1pnZ3mrvXWtm6/2Pa72MU0SkJZg4MBmAfp3juP30fkGOpuk86yIzs3DgMeBMIBtYYGbTnXOrDpZxzt1ZrfztwPH+5x2B3wAZgAMW+vfd41W8IiLBNqx7Ijef0peLRnYjOiI82OE0mZctmDHABufcJudcGfAKcNExyk8BXvY/Pxv42DmX708qHwOTPIxVRCTowsKMu88ZxOCuCcEOJSC8TDDdgaxqr7P9245iZr2ANOCzhu4rIiItk5cJpqaZ2Wpe8AAmA6875yobsq+Z3WRmmWaWmZdX+4I9IiLS/LxMMNlAj2qvU4HttZSdzLfdY/Xe1zn3lHMuwzmXkZyc3MRwRUQkkLxMMAuA/maWZmZR+JLI9CMLmdlAIAmYU23zDOAsM0sysyTgLP82EREJEZ6NInPOVZjZVHyJIRx4xjm30szuBTKdcweTzRTgFeecq7Zvvpn9Dl+SArjXOZfvVawiIhJ4Vu17PaRlZGS4zMzMYIchIhJSzGyhcy7Di2PrTn4REfGEEoyIiHii1XSRmVke8M0RmxOBfQ3cVtfzTsCuRoZZ07kbUqY+9WmuutQVa11lGlqXI18ffF59mz6b+sVaVxl9NsH9DjhWOS/q0t45580wXOdcq30ATzV0W13P8Q1QCFg8DSlTn/o0V12aWp+G1uUYdai+TZ+NPpsW/dnUpy6B/Gy8/jur69Hau8jeacS2+jwPZDwNKVOf+jRXXep7nNrKNLQuR75+p5YyjaXP5tjb9dk033fAscq1pLrUqdV0kTUXM8t0Ho24aG6tqS7QuurTmuoCras+qkv9tfYWjBeeCnYAAdSa6gKtqz6tqS7QuuqjutSTWjAiIuIJtWBERMQTbTrBmNkzZpZrZisase8oM1vuX63zb2Zm1d673b+S50oz+0tgo641noDXxczuMbNt1VYdPTfwkdcakyefjf/9u8zMmVmnwEV8zHi8+Gx+Z2bL/J/LR2bWLfCR1xiPF3W538zW+Ovzlpl1CHzktcbkRX0u8//brzIzz6/VNKUOtRyvxtWE6/p3VSMvh6i19AdwMnACsKIR+84HxuFbWuAD4Bz/9lOBT4Bo/+vOIVyXe4C7Wstn43+vB7758b4BOoVqXYCEamXuAJ4I4bqcBUT4n/8Z+HMo/50Bg4GBwEwgo6XWwR9f7yO2dQQ2+f+b5H+edKz6HuvRplswzrnZwGGTaJpZXzP70MwWmtkXZjboyP3MrCu+f+BznO///PPAxf63bwHuc86V+s+R620tfDyqS9B4WJ+HgJ9S+9pEAedFXZxzBdWKtqeZ6uNRXT5yzlX4i87FtzxHs/CoPqudc2ubI37/+RpVh1rUuJpwY78n2nSCqcVTwO3OuVHAXcDjNZTpjm/NmoOqr7g5ADjJzOaZ2SwzG+1ptMfW1LoATPV3XTxjvqUTgqlJ9TGzC4FtzrmlXgdaD03+bMzsD2aWBVwJ/NrDWOsSiL+zg27A9+s4mAJZn2CpTx1qUttqwo2qr2fT9YciM4sDTgReq9a9GF1T0Rq2HfwFGYGvaTkWGA1MM7M+/qzfbAJUl38Av/O//h3wIL4vgGbX1PqYWSzwC3zdMUEVoM8G59wvgF+Y2c+BqcBvAhxqnQJVF/+xfgFUAC8GMsaGCGR9guVYdTCz64Ef+rf1A943szJgs3PuEmqvV6PqqwRzuDBgr3NuZPWNZhYOLPS/nI7vi7d6M776ipvZwJv+hDLfzKrwzffT3Gs6N7kuzrmcavv9E3jXy4Dr0NT69AXSgKX+f3SpwCIzG+Oc2+lx7EcKxN9ZdS8B7xGEBEOA6uK/mHw+cHpz/xg7QqA/m2CosQ4AzrlngWcBzGwmcJ1zbku1ItnAxGqvU/Fdq8mmMfX1+gJUS38Aval2cQz4GrjM/9yAEbXstwBfK+XgBa9z/dtvxrdAGvi6y7Lw328UgnXpWq3MnfgWhgvZz+aIMltopov8Hn02/auVuR14PYTrMglYBSQ359+X139nNNNF/sbWgdov8m/G1wuT5H/esT71rTGuYHygLeUBvAzsAMrxZegb8f3K/RBY6v+j/3Ut+2YAK4CNwKN8e9NqFPCC/71FwGkhXJf/AMuBZfh+tXVtjrp4VZ8jymyh+UaRefHZvOHfvgzfvFLdQ7guG/D9EFvifzTLiDgP63OJ/1ilQA4woyXWgRoSjH/7Df7PZANwfV31PdZDd/KLiIgnNIpMREQ8oQQjIiKeUIIRERFPKMGIiIgnlGBERMQTSjDSqplZYTOf72kzSw/QsSrNN1vyCjN7p65Zhs2sg5ndGohziwSChilLq2Zmhc65uAAeL8J9OzGjp6rHbmb/BtY55/5wjPK9gXedc0ObIz6RuqgFI22OmSWb2RtmtsD/GO/fPsbMvjazxf7/DvRvv87MXjOzd4CPzGyimc00s9fNt47JiwfXxvBvz/A/L/RPSLnUzOaaWYp/e1//6wVmdm89W1lz+HbSzjgz+9TMFplvfY6L/GXuA/r6Wz33+8v+xH+eZWb22wD+bxSpkxKMtEWPAA8550YD3wOe9m9fA5zsnDse3+zEf6y2zzjgWufcaf7XxwP/B6QDfYDxNZynPTDXOTcCmA18v9r5H/Gfv875nPzzYJ2ObzYFgBLgEufcCfjWH3rQn+DuBjY650Y6535iZmcB/YExwEhglJmdXNf5RAJFk11KW3QGkF5tptkEM4sHEoF/m1l/fDPFRlbb52PnXPU1N+Y757IBzGwJvrmgvjziPGV8O0HoQuBM//NxfLuWxkvAA7XE2a7asRfiW5sDfHNB/dGfLKrwtWxSatj/LP9jsf91HL6EM7uW84kElBKMtEVhwDjn3IHqG83s78DnzrlL/NczZlZ7u+iIY5RWe15Jzf+Wyt23FzlrK3MsB5xzI80sEV+iug34G771X5KBUc65cjPbAsTUsL8Bf3LOPdnA84oEhLrIpC36CN/6KQCY2cFpzROBbf7n13l4/rn4uuYAJtdV2Dm3D9+yyHeZWSS+OHP9yeVUoJe/6H4gvtquM4Ab/OuDYGbdzaxzgOogUiclGGntYs0su9rjR/i+rDP8F75X4VtiAeAvwJ/M7Csg3MOY/g/4kZnNB7oC++rawTm3GN/MuJPxLciVYWaZ+Foza/xldgNf+Yc13++c+whfF9wcM1sOvM7hCUjEUxqmLNLM/KtrHnDOOTObDExxzl1U134ioUbXYESa3yjgUf/Ir70EaRlqEa+pBSMiIp7QNRgREfGEEoyIiHhCCUZERDyhBCMiIp5QghEREU8owYiIiCf+P3XN3BUkPa75AAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"learn_resnet.lr_find()\n",
"learn_resnet.recorder.plot()"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" <div>\n",
" <style>\n",
" /* Turns off some styling */\n",
" progress {\n",
" /* gets rid of default border in Firefox and Opera. */\n",
" border: none;\n",
" /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
" background-size: auto;\n",
" }\n",
" .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
" background: #F44336;\n",
" }\n",
" </style>\n",
" <progress value='2' class='' max='10', style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
" 20.00% [2/10 03:32<14:11]\n",
" </div>\n",
" \n",
"<table style='width:300px; margin-bottom:10px'>\n",
" <tr>\n",
" <th>epoch</th>\n",
" <th>train_loss</th>\n",
" <th>valid_loss</th>\n",
" <th>accuracy_thresh</th>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <th>0.777768</th>\n",
" <th>0.793338</th>\n",
" <th>0.515000</th>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <th>0.692816</th>\n",
" <th>0.765466</th>\n",
" <th>0.633000</th>\n",
" </tr>\n",
"</table>\n",
"\n",
"\n",
" <div>\n",
" <style>\n",
" /* Turns off some styling */\n",
" progress {\n",
" /* gets rid of default border in Firefox and Opera. */\n",
" border: none;\n",
" /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
" background-size: auto;\n",
" }\n",
" .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
" background: #F44336;\n",
" }\n",
" </style>\n",
" <progress value='0' class='progress-bar-interrupted' max='23', style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
" Interrupted\n",
" </div>\n",
" "
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Traceback (most recent call last):\n",
"Traceback (most recent call last):\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/queues.py\", line 240, in _feed\n",
" send_bytes(obj)\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/connection.py\", line 200, in send_bytes\n",
" self._send_bytes(m[offset:offset + size])\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/connection.py\", line 404, in _send_bytes\n",
" self._send(header + buf)\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/connection.py\", line 368, in _send\n",
" n = write(self._handle, buf)\n",
"BrokenPipeError: [Errno 32] Broken pipe\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/queues.py\", line 240, in _feed\n",
" send_bytes(obj)\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/connection.py\", line 200, in send_bytes\n",
" self._send_bytes(m[offset:offset + size])\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/connection.py\", line 404, in _send_bytes\n",
" self._send(header + buf)\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/connection.py\", line 368, in _send\n",
" n = write(self._handle, buf)\n",
"BrokenPipeError: [Errno 32] Broken pipe\n",
"Traceback (most recent call last):\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/queues.py\", line 240, in _feed\n",
" send_bytes(obj)\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/connection.py\", line 200, in send_bytes\n",
" self._send_bytes(m[offset:offset + size])\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/connection.py\", line 404, in _send_bytes\n",
" self._send(header + buf)\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/connection.py\", line 368, in _send\n",
" n = write(self._handle, buf)\n",
"BrokenPipeError: [Errno 32] Broken pipe\n",
"Traceback (most recent call last):\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/queues.py\", line 240, in _feed\n",
" send_bytes(obj)\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/connection.py\", line 200, in send_bytes\n",
" self._send_bytes(m[offset:offset + size])\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/connection.py\", line 404, in _send_bytes\n",
" self._send(header + buf)\n",
" File \"/Users/MMP/anaconda3/lib/python3.6/multiprocessing/connection.py\", line 368, in _send\n",
" n = write(self._handle, buf)\n",
"BrokenPipeError: [Errno 32] Broken pipe\n"
]
},
{
"ename": "KeyboardInterrupt",
"evalue": "",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-25-d4b78c83e318>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mlearn_resnet\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfit_one_cycle\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m10\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmax_lr\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1e-2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;32m~/Downloads/fastai/fastai/train.py\u001b[0m in \u001b[0;36mfit_one_cycle\u001b[0;34m(learn, cyc_len, max_lr, moms, div_factor, pct_start, wd, callbacks, **kwargs)\u001b[0m\n\u001b[1;32m 19\u001b[0m callbacks.append(OneCycleScheduler(learn, max_lr, moms=moms, div_factor=div_factor,\n\u001b[1;32m 20\u001b[0m pct_start=pct_start, **kwargs))\n\u001b[0;32m---> 21\u001b[0;31m \u001b[0mlearn\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcyc_len\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmax_lr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mwd\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mwd\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcallbacks\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcallbacks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 22\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 23\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mlr_find\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlearn\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mLearner\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstart_lr\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mFloats\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1e-7\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend_lr\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mFloats\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m10\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_it\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mint\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m100\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstop_div\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mbool\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mAny\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m~/Downloads/fastai/fastai/basic_train.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, epochs, lr, wd, callbacks)\u001b[0m\n\u001b[1;32m 164\u001b[0m \u001b[0mcallbacks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mcb\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mcb\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcallback_fns\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mlistify\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcallbacks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 165\u001b[0m fit(epochs, self.model, self.loss_func, opt=self.opt, data=self.data, metrics=self.metrics,\n\u001b[0;32m--> 166\u001b[0;31m callbacks=self.callbacks+callbacks)\n\u001b[0m\u001b[1;32m 167\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 168\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mcreate_opt\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlr\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mFloats\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mwd\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mFloats\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0.\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m->\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m~/Downloads/fastai/fastai/basic_train.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(epochs, model, loss_func, opt, data, callbacks, metrics)\u001b[0m\n\u001b[1;32m 82\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mxb\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0myb\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mprogress_bar\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrain_dl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparent\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpbar\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 83\u001b[0m \u001b[0mxb\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0myb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcb_handler\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mon_batch_begin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mxb\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0myb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 84\u001b[0;31m \u001b[0mloss\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mloss_batch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mxb\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0myb\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mloss_func\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mopt\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcb_handler\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 85\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcb_handler\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mon_batch_end\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mloss\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 86\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m~/Downloads/fastai/fastai/basic_train.py\u001b[0m in \u001b[0;36mloss_batch\u001b[0;34m(model, xb, yb, loss_func, opt, cb_handler)\u001b[0m\n\u001b[1;32m 24\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mopt\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 25\u001b[0m \u001b[0mloss\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcb_handler\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mon_backward_begin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mloss\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 26\u001b[0;31m \u001b[0mloss\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 27\u001b[0m \u001b[0mcb_handler\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mon_backward_end\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 28\u001b[0m \u001b[0mopt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstep\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/torch/tensor.py\u001b[0m in \u001b[0;36mbackward\u001b[0;34m(self, gradient, retain_graph, create_graph)\u001b[0m\n\u001b[1;32m 100\u001b[0m \u001b[0mproducts\u001b[0m\u001b[0;34m.\u001b[0m \u001b[0mDefaults\u001b[0m \u001b[0mto\u001b[0m\u001b[0;31m \u001b[0m\u001b[0;31m`\u001b[0m\u001b[0;31m`\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;31m`\u001b[0m\u001b[0;31m`\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 101\u001b[0m \"\"\"\n\u001b[0;32m--> 102\u001b[0;31m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mautograd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgradient\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mretain_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 103\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 104\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mregister_hook\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhook\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/torch/autograd/__init__.py\u001b[0m in \u001b[0;36mbackward\u001b[0;34m(tensors, grad_tensors, retain_graph, create_graph, grad_variables)\u001b[0m\n\u001b[1;32m 88\u001b[0m Variable._execution_engine.run_backward(\n\u001b[1;32m 89\u001b[0m \u001b[0mtensors\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgrad_tensors\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mretain_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 90\u001b[0;31m allow_unreachable=True) # allow_unreachable flag\n\u001b[0m\u001b[1;32m 91\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 92\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mKeyboardInterrupt\u001b[0m: "
]
}
],
"source": [
"learn_resnet.fit_one_cycle(10, max_lr=1e-2)"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd8VFX6+PHPkw6pkISaQAKEEiCEEDpSBBUsoIhAUBRUsKDu2lbW3VW/urqu7ir6A10FAQtFREFEEBsWQErA0FvoIZSEkoSSfn5/3CEETEhIZjIpz/v1mldy75x75rnDkGfOPfecI8YYlFJKKQAXZweglFKq8tCkoJRSqoAmBaWUUgU0KSillCqgSUEppVQBTQpKKaUKaFJQSilVQJOCUkqpApoUlFJKFXBzdgBXKygoyISFhTk7DKWUqlLWr1+faowJLqlclUsKYWFhxMfHOzsMpZSqUkTkQGnK6eUjpZRSBTQpKKWUKqBJQSmlVIEq16eglKo+cnJySEpKIjMz09mhVBteXl6EhITg7u5epuM1KSilnCYpKQlfX1/CwsIQEWeHU+UZYzhx4gRJSUmEh4eXqQ69fKSUcprMzEwCAwM1IdiJiBAYGFiulpcmBaWUU2lCsK/yvp+aFKqzg6th60JnR6GUqkI0KVRHaYdh/r0w/Qb47B5ImOPsiJSqdE6cOEF0dDTR0dE0aNCAxo0bF2xnZ2eXqo6xY8eyc+dOB0dasbSjuTrJyYTfJsOv/4X8POjzDBz8DRY9CnWaQtMezo5QqUojMDCQhIQEAF544QV8fHx46qmnLiljjMEYg4tL0d+fZ8yY4fA4K5pDWwoiMlBEdopIoohMLOL5N0UkwfbYJSKnHRlPtWUM7FwK73SDH1+C5tfCI2uh37Mw/CMrIcy9E07udXakSlV6iYmJtGvXjgcffJCYmBiOHDnC+PHjiY2NpW3btrz44osFZXv16kVCQgK5ubkEBAQwceJEOnToQPfu3Tl+/LgTz6LsHNZSEBFXYApwHZAErBORRcaYbRfKGGMeL1T+UaCjo+KptlIT4ZuJkPgdBLWC0QuspHBBrTowah5M6w+zR8B930GtAOfFq1Qx/u+rrWxLTrdrnZGN/Hj+lrZXfdy2bduYMWMG//vf/wB49dVXqVu3Lrm5ufTr149hw4YRGRl5yTFpaWn06dOHV199lSeeeILp06czceIfvgtXeo5sKXQBEo0xe40x2cBcYMgVyscBevG7tLIy4LvnrNbBoTVwwyvw0MpLE8IFgc1hxCdwcp/Vx5CXU/HxKlWFNG/enM6dOxdsz5kzh5iYGGJiYti+fTvbtm37wzG1atVi0KBBAHTq1In9+/dXVLh25cg+hcbAoULbSUDXogqKSFMgHPjRgfFUD8bApnlWQjhzFKLvgv7PgW/9Kx8X1gtumQRfToAlT8PNb4LeCqgqkbJ8o3cUb2/vgt93797NW2+9xdq1awkICOCuu+4qchyAh4dHwe+urq7k5uZWSKz25siWQlF/cUwxZUcC840xeUVWJDJeROJFJD4lJcVuAVY5yQnWHUULxoNfI7j/B7h1SskJ4YKOd0HPP8P6GbD6XcfGqlQ1kZ6ejq+vL35+fhw5coRly5Y5OySHcmRLIQkILbQdAiQXU3YkMKG4iowx7wPvA8TGxhaXWKqvsyfgxxdh/YdQOxAGT4boO6GYOyKuqP/zcCIRlj0LdZtBq4H2j1epaiQmJobIyEjatWtHs2bN6Nmzp7NDcigxxjF/Y0XEDdgF9AcOA+uAUcaYrZeVawUsA8JNKYKJjY01NWaRnbxciJ8Oy/8JWWeg6wPWbabl7SjOPgszBsGJPXDvMmjQzj7xKnWVtm/fTps2bZwdRrVT1PsqIuuNMbElHeuwy0fGmFzgEaw/+NuBecaYrSLyoogMLlQ0DphbmoRQo+xfAe/1hqVPQ8NoeGgVDPyXfe4c8vCGuLng6WvdkZRxrPx1KqWqBYcOXjPGLAGWXLbvucu2X3BkDFVOWhJ8+w/Y+gX4N4HhH0ObW+zfKezXyEoMMwbB3DgY8zW417Lvayilqhyd5qKyyMmEX16HyZ1h5xLoMxEmrIHIwY67S6hRNAydCoc3wIIHIT/fMa+jlKoydJoLZ7swGnnZX+HUfmgzGK7/pzUKuSK0uRmu+z/rFtefWsK1f6uY11VKVUqaFJwpdbdtNPL3ttHIC6F5v4qPo8djkLoLfnkNAltAhxEVH4NSqlLQpOAMmenWH+DV74J7bbjhX9BlHLiWbfm8chOBm96EUwdg0SNWK6VJN+fEopRyKu1TqEj5+bBxLkyOhVX/DzqMhEfXQ/eHnZcQLnDzsCbP8w+FuaOsKTGUqub69u37h8FokyZN4uGHHy72GB8fHwCSk5MZNmxYsfWWdOv8pEmTOHfuXMH2jTfeyOnTzp8TVJNCRUn+3TYa+QHwD4H7f4QhU8CnnrMju6h2XWvyvPw861bVzDRnR6SUQ8XFxTF37txL9s2dO5e4uLgSj23UqBHz588v82tfnhSWLFlCQIDzJ6vUpOBo50/Dosfg/X5wap+VCO77HkI6OTuyogW1gBEfw8k98NkYawCdUtXUsGHDWLx4MVlZWQDs37+f5ORkoqOj6d+/PzExMbRv354vv/zyD8fu37+fdu2sgZ/nz59n5MiRREVFMWLECM6fP19Q7qGHHiqYdvv5558H4O233yY5OZl+/frRr5/VjxgWFkZqaioAb7zxBu3ataNdu3ZMmjSp4PXatGnDuHHjaNu2Lddff/0lr2Mv2qfgaK7usPcn6PYw9H0GvPydHVHJwntbE+YtehS+eQZu/I9Onqccb+lEOLrZvnU2aA+DXi326cDAQLp06cI333zDkCFDmDt3LiNGjKBWrVosWLAAPz8/UlNT6datG4MHDy52/eN3332X2rVrs2nTJjZt2kRMTEzBcy+//DJ169YlLy+P/v37s2nTJh577DHeeOMNli9fTlBQ0CV1rV+/nhkzZrBmzRqMMXTt2pU+ffpQp04ddu/ezZw5c5g6dSrDhw/n888/56677rLPe2WjLYVSSs/MYU/Kmas/0MMbJqyFga9UjYRwQczd1l1J66bB2vedHY1SDlP4EtKFS0fGGJ599lmioqIYMGAAhw8f5tix4kf+//LLLwV/nKOiooiKiip4bt68ecTExNCxY0e2bt1a5LTbha1YsYLbbrsNb29vfHx8GDp0KL/++isA4eHhREdHA46bnltbCldgjGHL4XR2HcvgtWU7OJaeRb9WwdT19qRZsDcD2zWgWZA3q/eeZMryRE6ezWZMjzDuiA259BuFu5fzTqI8BrxgzY/0zUSoEw4tr3d2RKo6u8I3eke69dZbeeKJJ9iwYQPnz58nJiaGmTNnkpKSwvr163F3dycsLKzI6bILK6oVsW/fPv7zn/+wbt066tSpw5gxY0qs50oz/nh6ehb87urq6pDLR9pSuIL//byXWyav4MnPNuLt6caDfZqz69gZVu1J5fVlO+n/35/p95+fiJu6mp3HMjDAXz7fxAuLtpKZk8fHqw8w+oM17C1LC6MycHGF26dC/XYw/144trXkY5SqYnx8fOjbty/33ntvQQdzWloa9erVw93dneXLl3PgwIEr1tG7d29mzZoFwJYtW9i0aRNgTbvt7e2Nv78/x44dY+nSpQXH+Pr6kpGRUWRdCxcu5Ny5c5w9e5YFCxZwzTXX2Ot0S6QthSIkHs/g03WHmPrrPm5q35CH+jYnor4Pnm6uTBzUGoBj6Zl8Fn+IX3alMrZnOCM6h+Lp5sLLX29n2op9fPib9SFydxVGvL+aqXfHEh3q/DsLrtqFyfOm9YfZI2HcD5Xrjiml7CAuLo6hQ4cWXEa68847ueWWW4iNjSU6OprWrVtf8fiHHnqIsWPHEhUVRXR0NF26dAGgQ4cOdOzYkbZt2/5h2u3x48czaNAgGjZsyPLlywv2x8TEMGbMmII67r//fjp27FhhK7k5bOpsR3H01Nm7jmVw65SVZOXmc31kfd4cEY2Xu2upjzfGMH99EsczsmjT0JfQOrUZ/cFajqZncntMCH8Z2Ir6flXwclLy7zB9kDXN9j1f6eR5yi506mzHKM/U2dpSKORsVi4PfLweb083vnuiJ40Drv4Pn4hwR2zoJfu+e6I3k5cnMmPFfr7enMwtUY3IMwYfTzfu6BRK+5Aq0AHdqCMMfR/mjbaW9Lz9A70jSalqSPsUCvl8QxL7Us/y1ojoMiWE4vh6ufPXQW347oneDOnQmK82JfPLrlTmxR9i8JQVfLWxuAXpKpnIwdbKbVs+h5+c0ymolHKsGt1SSDuXw93T1/DK0PZENvTj498OEBXiT48WQSUfXAZNA73597Ao/jW0PS4uQnpmDvfPjOeJeQkkHj9DaN3aHDxxltqebgzrFEKQj2fJlVa0Xo9by3n+/Ko1eV7UHc6OSFVxxphi7/9XV6+8XQI1OikkJJ1mY1IaX2w4TEZkLruPn+G1YVElH1hOLi7WfwA/L3em3hPLn+f+zts/7qbwv+XctQf5+L6uhNat7fB4rooI3DzJmub7ywkQ0ASadHV2VKqK8vLy4sSJEwQGBmpisANjDCdOnMDLq+z9ljU6Kew5bt0q+tPO4xxNz8S/ljuDOzSq0Bj8a7kzY2wXkk+f51x2LuFBPiQcOsXYGeu46e1f+cfNkdweE1KQSCoFNw8Y8QlMvdaaPG/cjxW3/oOqVkJCQkhKSiIlJcXZoVQbXl5ehISElPn4Gn330d8XbuaT1QcBcHUR7u0Zxt9uirRL3eW1N+UMf5m/ifgDp2gW7E12bj6pZ7Jo5F+LqffE0jzYx9khWutBTOsPvo3gvmVVa8S2UjVMae8+qtEdzXtTzhLk4wFAXr7hzq6V59tus2Af5j3QnbdGRlO3tgdRIf6M7taU9MwcRry3unIMiAuKsKbbPrHbGtymk+cpVeXV6JZC11e+p2eLIBIOniYsyJvpYzrbpV5HSjyewYj3VuPt6cbtMSEcOHGWAZH1ubZ1vasaT2FX62fCV3+CLg/Aja85Jwal1BVVinEKIjIQeAtwBaYZY/5wH6OIDAdeAAyw0RgzypExXXAmK5dj6Vk0D/bh2Rvb4OlWNRpNLer5Mu2eWEa+v5o3v9+Ffy13vvj9MD6ebozpEcaIzqGsTEzl1o6NKy5JdBpjXUr6bbLVeugyrmJeVylldw5LCiLiCkwBrgOSgHUissgYs61QmQjgr0BPY8wpEamw+RP2pZwFoHmwd+W89fMKOjapw1eP9sLVRQgL9Oa3PSeYu+4gk5cnMnl5IgD7Tpzlr4MqcKTodS9ak+ct/Ys1eV7EgIp7baWU3Tjy63EXINEYs9cYkw3MBYZcVmYcMMUYcwrAGHPcgfFwJiuXG9/6lfUHThVMg10pOmzLoGV9X5oH++DqIvSKCGLyqBim3R3LY/0juKl9Qz74dR/bj6Sz42g6Ez/fxOHT9p9N8RIurnD7NKjXFuaPhePbHft6SimHcOTlo8bAoULbScDlN7S3BBCRlViXmF4wxnxzeUUiMh4YD9CkSZMyB7Qv5SzbjqQzb90hanu64u4qNAmsZOMAymFAZH0GRNbn5NlsVu1JZciUlbgIZObks27/ST57sAd1vT0cF4CnD4yaa92qOns4jPsJvAMd93pKKbtzZEuhqBvrL+/VdgMigL5AHDBNRP4wlagx5n1jTKwxJjY4OLjMAaWesZbc+2HHcb7edIS+rerh6eakzlkHquvtwaJHejGsUwi9WgQzZVQMh06d528L7LyqVVH8QyBuDpw+BOumOv71lFJ25ciWQhJQeGa4EODySX6SgNXGmBxgn4jsxEoS6xwRUIotKVxIDkOiK3agWkUKrVubV25rX7C9N+UM//1uF6v3nqBbMwd/e2/cyVrSM2E29P4LuFSNTnyllGNbCuuACBEJFxEPYCSw6LIyC4F+ACIShHU5aa+jArqQDFwEvD1c6d+6vqNeqtIZ17sZjQNq8cSnCcxac6DgvXCY6FFw+gAc/M2xr6OUsiuHJQVjTC7wCLAM2A7MM8ZsFZEXRWSwrdgy4ISIbAOWA08bY044KqbUjGy8PVy5Nboxo7uHUcuj+l06Ko6Xuytvx3WkjrcHf1uwhdh/fs+rS3eUe/KsYrW5BTx8YONsx9SvlHKIGjV47bE5v7Mx6TQ/P93PzlFVHRfWnZ6xah9fbDjM6G5NGdE5lHaNHTBFxcIJsO1LeGoXeFSfDn2lqiKd5qIIqWeyqtyYBHsTEdqH+POfYR2I6xLKx6sPcPP/W8G8dYdKPvhqRcdBdgbsWGz/upVSDlEDk4IDb8msQlxchH8NjWL93wdwTUQQzy7YzMyV+8jMybPfizTpAQFNIWGW/epUSjlUDUsK2TW+pXC5QB9P3rkzhk5N6/DCV9u45f+tID0zxz6Vu7hAhzjY+zOkJdmnTqWUQ9WYpJCbl8+pc5oUiuLr5c7c8d2Ydncs+1LP8ujs3zl1Nts+lXcYCRjY9Kl96lNKOVSNSQonz2VjDAT5alIoiogwILI+L93ajp93pdDj1R/5MuFw+SuuG25dRkqYDVXspgalaqIakxRSM6xvvsHap3BFcV2asOzPvWnX2I+nP9vEhoOnyl9p9ChrXeck+0x5rpRynJqTFGyDtQL18lGJWjXw5f3RsTTw92L0tDXMX59UvvEMkUPArZaOWVCqCqhxSUH7FEqnjrcHc8d3o11jf576bCOPzvmdjLJ2QHv5WYPZtnwOOZn2DVQpZVc1MCno5aPSahRQi9njuvH0Da1YuuUorywpx3TY0aMgMw12LrFfgEopu3PoymuVSf829Qny8cTHs8acsl24uggT+rUg9UwWH67az9ie4bSs73v1FYX3Br/GsHEOtBtq/0CVUnZRY1oKzYN9GBoTgkhRM3qrkjx2bQTenm7cO3MdU3/Ze/V9DC6uEDUCEn+AjKOOCVIpVW41Jimo8qnj7cGUUTHU9/Pi5SXb+XrzkauvJHoUmDzYNM/+ASql7EKTgiq13i2DmfdAd1o38OW1b3aSlXuVU2IERUBIZ+sSko5ZUKpS0qSgroqri/DXG9tw8OQ5/jJ/09XPldQhDo5vgyMbHROgUqpcNCmoq9anZTBPXd+SLxOSaf2Pb+j92nKOppXyVtN2Q8HV02otKKUqHU0KqkweuTaCj+/rwqPXtuB4RibPLthcus7nWnWg1SCrXyHXTvMrKaXsRu/PVGV2TUQw10QE41/LnX9+vZ2/frGZVg18ScnI4qnrW+HiUsydXtF3wraFsPtbaHNzxQatlLoiTQqq3Mb2DOdYeiYfrNhHvq2x0KqBL0OiGxd9QPNrwae+dQlJk4JSlYpePlLl5uoi/O2mSH58si/fPt6byIZ+/PfbXWTn5hdzgBu0vwN2fQNnUys2WKXUFWlSUHYTFuRNy/q+/GVgKw6ePMfUX/cWXzh6FOTnwub5FRegUqpEDk0KIjJQRHaKSKKITCzi+TEikiIiCbbH/Y6MR1WMPi2DGdSuAW99v5vdxzKKLlS/LTTsoDOnKlXJOCwpiIgrMAUYBEQCcSISWUTRT40x0bbHNEfFoyqOiPDSre3w8XLjT3MTih/L0GGUNV7h2NaKDVApVSxHthS6AInGmL3GmGxgLjDEga+nKpEgH09eHxbFtiPp/N9X24ou1H4YuLhZq7IppSoFRyaFxsChQttJtn2Xu11ENonIfBEJdWA8qoL1b1Of8b2bMWftQX4vagU37yBoOdAas5CXW/EBKqX+wJFJoaib1C8f3fQVEGaMiQK+Bz4ssiKR8SISLyLxKSkpdg5TOdJj/SMI9Pbg9WU7iy7QIQ7OHoc9P1ZsYEqpIjkyKSQBhb/5hwDJhQsYY04YY7Jsm1OBTkVVZIx53xgTa4yJDQ4OdkiwyjF8PN2Y0K8Fq/ac4JstRcysGnE91KoLCbMqPjil1B84MimsAyJEJFxEPICRwKLCBUSkYaHNwUA5lvZSldWd3ZoQFeLPk/M2suvyu5HcPCBquLUi2/kiLjEppSqUw5KCMSYXeARYhvXHfp4xZquIvCgig23FHhORrSKyEXgMGOOoeJTzeLq58t7oTtTycOPPcxPIzbtsUFuHOMjLhi1fOCdApVQBueoVtJwsNjbWxMfHOzsMVQZLNh/h4VkbeO7mSO7tFX7xCWPg3R7g4Q33f++8AJWqxkRkvTEmtqRyOqJZVZhB7RrQu2Uwr36zgw9W7Ls4q6qI1VpIWgepu50bpFI1nCYFVWFEhEkjoukdEcRLi7cxf33SxSejhoO46pgFpZxMk4KqUHW9PZh6dyxRIf689cNuci70L/g2gBb9YdOnkH+Vq7kppexGk4KqcCLC4wNaknTqPB+s2HfxiQ5xkH4Y9v3svOCUquE0KSin6NsqmH6tgnl16Q7+tmAz+fkGWt0IXv6QoEt1KuUsmhSUU4gIU++OZXzvZsxac5BJP+wGdy9odzts/woy050dolI1kiYF5TRuri78dVBr7ugUwts/7Obj1QesmVNzz8O2L50dnlI1kiYF5VQiwj9va8eANvX4x8ItzE2uB4Et9C4kpZxEk4JyOk83V969qxO9WgTx0tfbSW89HA6ugpP7Sj5YKWVXmhRUpeDu6sIrt7UnN9/wclIUBoGNc50dllI1jiYFVWk0CazN49e15NOd+aQEd7OW6szPL/lApZTdaFJQlcq4a5rRqWkd3kiJhdMHrctISqkKo0lBVSquLsKbw6P5Sbpyllpkxn/i7JCUqlE0KahKp0lgbSbf05Ol+V0xWxdy/oyOWVCqomhSUJVSbFhdQvvdTy1znk9mTiY7V/sWlKoImhRUpdW1z01k1AqhzbHFPDxrA5k5OlGeUo6mSUFVXi4u+HYdTU/XbWzbvpXh7/3G0bRMZ0elVLWmSUFVbh1GIhg+jN3LnuNnuHv6GtLO5zg7KqWqLU0KqnKrEwZNexGR/BVTR3diX+pZHp61/o/rPCul7EKTgqr8ouPg5B56eO3j5VvbszLxBG98t8vZUSlVLZUqKYhIcxHxtP3eV0QeE5EAx4amlE3kEHCvDQmzGd45lLguobzz0x5eWryNrFztfFbKnkrbUvgcyBORFsAHQDhQ4jSWIjJQRHaKSKKITLxCuWEiYkQktpTxqJrE0xfa3AJbvoCc87wwuC13d2/KByv2cfcHa0nP1D4GpeyltEkh3xiTC9wGTDLGPA40vNIBIuIKTAEGAZFAnIhEFlHOF3gMWHM1gasaJnoUZKXBziV4urny4pB2vDUymvUHTjH0nVVsOHjK2REqVS2UNinkiEgccA+w2LbPvYRjugCJxpi9xphsYC4wpIhyLwGvAXqvoSpeWG/wC7lkqc4h0Y2ZObYLZ7NyGfrOKoa+s5JtyTr6WanyKG1SGAt0B142xuwTkXCgpElpGgOHCm0n2fYVEJGOQKgxZjFKXYmLC3QYAXt+gPQjBbt7RQTx7eO9mTioNQdPnmfC7A2cz9Z+BqXKqlRJwRizzRjzmDFmjojUAXyNMa+WcJgUVVXBkyIuwJvAkyW9voiMF5F4EYlPSUkpTciqOuoQByYfNs+7ZLevlzsP9mnO23HR7Es9y7+/2eGkAJWq+kp799FPIuInInWBjcAMEXmjhMOSgNBC2yFAcqFtX6Ad8JOI7Ae6AYuK6mw2xrxvjIk1xsQGBweXJmRVHQVFQEgX6xKSMX94ukfzIMb0CGPmqv2sSkx1QoBKVX2lvXzkb4xJB4YCM4wxnYABJRyzDogQkXAR8QBGAosuPGmMSTPGBBljwowxYcBqYLAxJv6qz0LVHNFxkLIdjiQU+fQzA1sTHuTN0/M36ZQYSpVBaZOCm4g0BIZzsaP5imx3Kz0CLAO2A/OMMVtF5EURGVymaJVqOxRcPSGh6Duia3m4MmlENGnnc7j93VXsSz1bwQEqVbWVNim8iPXHfY8xZp2INAN2l3SQMWaJMaalMaa5MeZl277njDGLiijbV1sJqkS1AqD1TbB5PuRmF1mkQ2gAc8d343xOHndPX0NKRlYFB6lU1VXajubPjDFRxpiHbNt7jTG3OzY0pYoRPQrOn4Tdy4ot0q6xP9PHdCY1I5vRH6zhWLpeSlKqNErb0RwiIgtE5LiIHBORz0UkxNHBKVWkZv3Ap36xl5AuiA4NYOrdsRw6eY7b312liUGpUijt5aMZWJ3EjbDGGnxl26dUxXN1g6jhsPtbOHvlu4x6RQQxe1w3Tp3N5r4P13EuO7eCglSqaiptUgg2xswwxuTaHjMBvTdUOU+HUZCfC5s/K7loaACTR8WwLTmd577cWgHBKVV1lTYppIrIXSLianvcBZxwZGBKXVH9SGgYXeIlpAv6ta7HhH4tmL8+iSWbj5R8gFI1VGmTwr1Yt6MeBY4Aw7CmvlDKeaJHwdFNcHRLqYo/1j+CDqEBPP5pAr/s0pHxShWltHcfHTTGDDbGBBtj6hljbsUayKaU87QbBi7usHFOyWUBd1cXZozpTLNgH+7/KJ6fdh53cIBKVT3lWXntCbtFoVRZeAdCyxtg0zzIK92aCnW9PZh9f1daBPsw/qP1LNfEoNQlypMUiprwTqmKFT0Kzh6HxB9KfUgdbw9mj+tKRH0fHvh4vV5KUqqQ8iSFP85IplRFa3Ed1A6EjaXrcL4goLYHs+7vSrMgbx6d87vOk6SUzRWTgohkiEh6EY8MrDELSjmXmwe0Hw47l8K5k1d1aEBtD965M4bs3Hye/CyB/Hz9nqPUFZOCMcbXGONXxMPXGONWUUEqdUXRcZCXDVu/uOpDmwX78NwtkaxMPMH0lfscEJxSVUt5Lh8pVTk0iIJ6bUs9ZuFyIzuHcl1kfV77ZidLNh/BFLFWg1I1hSYFVfWJWB3Oh9dDyq4yHC78+/YoWtTz4eFZG3j80wTy9FKSqqE0KajqIWo4iOtVdzhfUNfbg0WP9ORP/SNYmJDMP77coi0GVSNpUlDVg089aDEANs6F/LwyVeHm6sLj17XkwT7Nmb3mIP/5dqedg1Sq8tOkoKqP6DjIOAJ7fypXNc8MbEVcl1CmLN/DlOWJ9olNqSpC7yBS1UfLQeAVAGv+B+F9rCm2y0BE+Oet7cnMyef1ZTtJP5/DUze0wt1Vv0Op6k8/5ar6cPdq0ZL6AAAa2UlEQVSCa56w1lmYNQzOnypzVa4uwn/u6MCdXZvw3i97GTx5JT/uOKb9DKra06Sgqpeef4LBk2H/Cpg2AFLLfvnH1UV4+bb2/O+uTpzJyuHemfHcM2Mdh06es2PASlUumhRU9RMzGu5ZZLUUpl0Le5aXq7qB7Rrw45N9ef6WSDYcOMXgyStYu+/qRk8rVVU4NCmIyEAR2SkiiSIysYjnHxSRzSKSICIrRCTSkfGoGqRpDxj3I/g1hk9uh7VTy1Wdu6sLY3uG89WjvahT24O7p69h+5F0OwWrVOXhsKQgIq7AFGAQEAnEFfFHf7Yxpr0xJhp4DXjDUfGoGqhOGNy7DCKugyVPwddPlnqK7eKEB3kz94Fu+Hm58+An6zl5Nts+sSpVSTiypdAFSDTG7DXGZANzgSGFCxhjCn/V8kZnXlX25uUHI2dDj8dg3TSr1VCODmiAer5evHNnDEfTMhn+3m8cPn3eTsEq5XyOTAqNgUOFtpNs+y4hIhNEZA9WS+GxoioSkfEiEi8i8SkpOve9ukournD9SzDkHTiwCqb2h9Td5aoyNqwuH97bhaNpmQx88xfmr0+yU7BKOZcjk0JRi/D8oSVgjJlijGkOPAP8vaiKjDHvG2NijTGxwcHBdg5T1Rgd74QxiyEzDab1hz0/lqu6bs0CWfxoLyIb+fHUZxv5amOynQJVynkcmRSSgNBC2yHAlf7XzAVudWA8SkGTbrYO6BD4ZBiseR/KMfYgLMibj+7rQuewOjz52UbWHyjfpSmlnM2RSWEdECEi4SLiAYwEFhUuICIRhTZvAsrXpleqNOo0hfuWQcT1sPRp+PqJcnVAe7q58t7oWBr4eTH+o3gdx6CqNIclBWNMLvAIsAzYDswzxmwVkRdFZLCt2CMislVEEoAngHscFY9Sl/D0hZGzoOefIX46fDL0qlduK6yutwfTx3QmJy+f+z5cR3pm+e5yUspZpKoN24+NjTXx8fHODkNVJwlz4KvHwD8E4j6F4JZlrmpVYip3T19L9+aBzBjTGTedL0lVEiKy3hgTW1I5/cQqFR0H9yyGrAxraozEH8pcVY8WQbx8Wzt+3Z3K84u26lxJqsrRpKAUQJOuVgd0QKg1md7q/5W5A3pE5yY82Kc5s9YcZPDklTryWVUpmhSUuiCgiTUCuuUg+OYZWPznMndAPzOwFf+9owNH0zMZO2MdxzMy7RysUo6hSUGpwjx9YMQn0OtxWD8TPr6tTB3QIsLtnUL4cGwX0s7nMO6j9ZzJyrV/vErZmSYFpS7n4gIDXoDb3oNDa2DqtZBStqU5Ixv58dbIaLYcTmPM9LWs3XdS+xlUpaZJQanidBgJY76G7DNWB/Tu78tUzfVtGzBpRDRbktMY/t5vPP5pAnn5mhhU5aRJQakrCe0C45ZDQFOYfQesfrdMHdC3dGhE/N+v47H+ESxMSGbCrA2kndOxDKry0aSgVEkCQuHeb6DVjfDNRGtMQ+7VT5nt4+nGE9e15G83tuH77ce4ftLP7D6W4YCAlSo7TQpKlYanDwz/GK55EjZ8ZHVAnz1RpqrG9W7Ggod7YgzETV1D4vEzdg5WqbLTpKBUabm4QP/nYOhUSFpnLfV5fEeZqmof4s/scd0AGDV1NXtTNDGoykGTglJXK2q4rQP6nNUBvevbMlXTop4Pc8Z1JS/fMPy91WxKOm3nQJW6epoUlCqL0M4wfjnUDYM5I+C3KWXqgI6o78unD3TH082FEe+tZvmO4/aPVamroElBqbLyD7FGQLe+CZY9C4seLVMHdIt6PiyY0IPm9by5/6N4pv26V8cyKKfRpKBUeXh4wx0fQe+n4fePrdtWM69+rqN6vl58Or47/VvX459fb2fcR+s5fe7qE4xS5aVJQanycnGBa/9urQG9fwXMGATpR666Gm9PN94b3Ynnbo7k513HuWHSL7zzU6JOj6EqlCYFpeyl450w6lM4td/qgD6+/aqrEBHu7RXO/Ad70CzIh9e+2cnNb//KhoO6zKeqGJoUlLKnFgOsO5Pyc2D6DVbLoQw6hAYwZ3w35o7vRlZuPkPfWcWE2Rt0RTflcJoUlLK3RtFw33fgU98a5LblizJX1a1ZIN8+3ps/9Y9g2ZajDHt3FUfSztsxWKUupUlBKUeo09S6M6lxJ5g/FlZNLnNVvl7uPH5dSz66twtHTmcS9/5qjqbp+gzKMTQpKOUotevC6IXQZjB8+zdYOhHy88pcXY8WQXx4XxdSz2Qz7H+rSDyu8yYp+3NoUhCRgSKyU0QSRWRiEc8/ISLbRGSTiPwgIk0dGY9SFc7dC+74ELo+BGvehc/GQE7Zv+XHNKnDrPu7kpmTz5DJK3n/lz1k5+bbL15V4zksKYiIKzAFGAREAnEiEnlZsd+BWGNMFDAfeM1R8SjlNC4uMOhVuOEV2L4IPr61TKu5XdAhNICFE3rQtVkgryzZwcC3fuGrjclk5Za9FaLUBY5sKXQBEo0xe40x2cBcYEjhAsaY5caYc7bN1UCIA+NRyrm6T4BhM+DweuvOpFMHylxVSJ3aTB/TmeljYjEGHp3zO31f/4kth9PsGLCqiRyZFBoDhwptJ9n2Fec+YKkD41HK+doNtfoZzhyDD66DIxvLVd21revz/RN9mDm2My4iDH/vN50/SZWLI5OCFLGvyAldROQuIBZ4vZjnx4tIvIjEp6Sk2DFEpZwgrCfc+y24esCMGyGxbMt8XuDqIvRtVY8FD/egWbA39324jo9Xl70Vomo2RyaFJCC00HYIkHx5IREZAPwNGGyMySqqImPM+8aYWGNMbHBwsEOCVapC1WttjWWoEw6zhsPvs8pfpZ81f1LfVvX4x8ItvPz1Nl0LWl01RyaFdUCEiISLiAcwElhUuICIdATew0oI2uZVNYtfQxi7BMKvgS8fhp9fK9P024V5e7rx/uhO3N29KVN/3ceYGWs5dVYn1lOl57CkYIzJBR4BlgHbgXnGmK0i8qKIDLYVex3wAT4TkQQRWVRMdUpVT15+MOoz6BAHy1+Gr/4EeeWbAM/N1YUXh7Tj1aHtWbP3JLdMXsHWZO2AVqUjVW3e9tjYWBMfH+/sMJSyL2Pgx3/Cr/+BiOutu5Q8fcpd7e8HT/HQJxs4eS6bF25pS1yXUESK6u5T1Z2IrDfGxJZUTkc0K1UZiED/f8DNb1odzx/eDGfKf0W1Y5M6LH6sF13D6/Lsgs38aW4CGw6eIl/7GlQxNCkoVZnE3gsjZ8PxHdYtq6mJ5a4yyMeTD8d24cnrWvL15iMMfWcVw9/7TedPUkXSy0dKVUZJ62H2cDD51hoNoV3sUu3Js9ks3XKEl7/ejpuL8PTA1tzZpQkuLnpJqbrTy0dKVWUhneC+b6FWAHx4C2xfbJdq63p7cGfXpnz1aC/aNvLnHwu3cN+H6/QOJVVAk4JSlVVgc2ssQ/22MG80rJ1qt6qbB/swe1xXXhrSlpWJJ7jp7V+J31/2+ZhU9aFJQanKzDsI7lkMETfAkqfgu+ch3z6zoooIo7uHMf+h7ri4CMP+9xsPz1pP4vEzdqlfVU2aFJSq7Dxqw4hPrE7olZNgwXjILXLwf5lEhQSw9E/X8Kf+Efy8M4Xr3/yZv8zfSPJpXeGtJtKOZqWqCmNgxRvww4sQdg2MnAVe/nZ9iRNnspiyfA+frD4AAmN6hPFw3+YE1Paw6+uoilfajmZNCkpVNRvnwpcTIKgV3PkZ+F9p8uGySTp1jje/280Xvyfh4+nGQ32bM7ZHOLU8XO3+WqpiaFJQqjrbsxw+HW1Nk3HnfKh/+fpV9rHjaDqvf7OTH3Ycp05td0Z0bsJDfZrjX9vdIa+nHEeTglLV3dHNMOsOyD4H170A4X2gbjNrdLSdrdt/kukr9rFs61ECanvwzMBW3NEpVMc3VCGaFJSqCU4fgjkj4dgWa9s7GJp0gyY9rJ8NosDVzW4vty05necXbWHd/lN0CA3gmRtaEejjSYt6PrhqgqjUNCkoVVPk50PqLjj4Gxxcbf08bVtkx90bQmKhqS1JNI4t90R7xhgWJhzmlSU7SMmw7oJqUc+Hp65vyQ1tG+iEe5WUJgWlarL05EuTxNEtgAFxhYZRF1sSTbqBT70yvURGZg4rE1NJO5/D1F/3kXj8DB1C/HnulrZ0alrHvuejyk2TglLqosw0OLTuYqI4HA+5tgnx6jaHJt2haXfrZxn6JXLz8vni98P899udHEvPYmhMYyYObE09Py8HnIwqC00KSqni5WbDkYRLWxPnT1nPlaNf4mxWLpOXJ/LBr/vwcHNhQr8WjO7eFB9P+/VrqLLRpKCUKr38fDixGw6ssku/xL7Us7y0eBs/7jiOr6cbcV2bcE+PMAK9PVi1J5V2jf2p56utiIqkSUEpVT7pyRcTRJH9Et0hpDPUCYOAplC77h8uOyUcOs0HK/axZPMRAAJquXPibDauLkJDfy/8a7nTv019RnYOpVFArYo/xxpEk4JSyr4y0yBpHRwool8CrBZFQCgENAF/28+AUAhoyhEJZkbCWfadOMfQjo3ZdDiNY2mZJJ0+T/z+k7iIcHNUQ+6/phntGtt36g5l0aSglHKs3GxI2QGnD0LaIetn4Ufm6UvLu3mBf4gtWVxIHE057hrM7J0wLeEcZ7INPZoHMq53M/q2DNbbW+1Ik4JSyrky023JwpYw0gonjUNwLvWS4sbFnXTP+uzKrMPenEAyvRvRuk1bOraPwiMwDHwb2XUg3lUxxloFLz8XEHCrehMEljYpOPQdFpGBwFuAKzDNGPPqZc/3BiYBUcBIY8x8R8ajlKpAXn7g1dZaJKgo2WchLakgUcjpg/inHaLTqYO0S91KrayfIAHrARhxRfwaX7ws5eZp/ZHOzyv6p7l8/9WUKbR9oUxhrp7g6Vvo4XfZduH9PsWXda/tkGlJysNhSUFEXIEpwHVAErBORBYZY7YVKnYQGAM85ag4lFKVlIc3BLeyHoW4ALUAk3Oe+E1b+HH1Ok4l7yHM9QQ9XM7SMvs0Xvt+hfwccHEDF1er89vFrdDD1fawbbu6W5evLnne7dIyf6inmDL5+ZCdAVlnICvj4iP98KXbeaVY80JcrpxUPC7bbtIdgls65J/jAke2FLoAicaYvQAiMhcYAhQkBWPMfttz9llKSilVbYh7LTp36kznTp3ZfiSdab/u478bD5Obb+gYGsBtHRtza8fG+HpV0hlbc7NsiSP90mSRlWFLKpc/bOXOn7JaTwVlC62Ed/OkKp0UGgOHCm0nAV0d+HpKqWqqTUM//ju8A0/f0Ip58YdYsvkI//hyK/9auoMh0Y25q1sT2jaqZHctuXlaD+/A8tWTn2clhqwMq0XhYI5MCkVdKCtTr7aIjAfGAzRp0qQ8MSmlqrAG/l481j+CR69twcakNGatPsCC35OYs/YgUSH+3NEphMEdGlev9R5cXK0V9uy8yl5xHHb3kYh0B14wxtxg2/4rgDHmX0WUnQksLk1Hs959pJQqLO1cDgt+T+LT+CS2H0nHw82FgW0bMKJzKN2bBeqaDzaV4e6jdUCEiIQDh4GRwCgHvp5Sqgbyr+3OmJ7hjOkZzpbDaXwWf4iFCcks2phMSJ1a3NEplGGxITTWEdOl4tBxCiJyI9Ytp67AdGPMyyLyIhBvjFkkIp2BBUAdIBM4aowp5v41i7YUlFIlyczJY9nWo8yLP8TKxBOIQOemdRnUvgF9Wgbj4+lGsK9njRocp4PXlFIKOHTyHJ9vSGLp5qPsPJZRsL9xQC2ub1ufhv5eLN1ylLBAb/q2CqZPy2ACale9wWkl0aSglFKX2ZtyhvgDpziblcvKxFR+2Z1Kdm4+bRr6cTTtPKfO5eDqInQNr8sNbRswILJ+tbnspElBKaVKcDYrl+MZWYQHeZOXb9iYdJrvtx1j2daj7Ek5C1hLjfZpabUguoTXxcvd1clRl40mBaWUKofE42f4aedxft6Vwpq9J8nOy8fL3YXuzQIZEFmf69rUr1Iry2lSUEopOzmXncuavSf5eVcKP+w4xqGT5wFoWd+H9o0DiArxp0fzQFrU86m0ndeaFJRSygGMMew8lsH3246x4eBpNiWdJvVMNgAN/LzoFRHENRFB9GoRRKCPp5OjvagyjFNQSqlqR0Ro3cCP1g2sKSeMMSSdOs+KxFRW7E7lu23HmL8+CYB6vp6EBXrTtVlduoYHUsfbHT8vd0Lq1Kq8LQptKSillP3k5Rs2H05jZWIqB0+cY8exDLYcTiMv/+Lf2iAfD6JDA2jg70V9Xy8iG/kR2ciPBn5eDksW2lJQSikncHURokMDiA4NKNh3JiuX3w+e4lx2Hsczsvj94Ck2HjrN+gOnOH0+hwvfzet6exDZ0I+2jfzoEBpAh9AAGvk7LlEURVsKSinlRGeyctlxJJ2tyelsS05n65E0dh7NICfP+tvs4+lG44BafHx/F+r5lv1uJ20pKKVUFeDj6UZsWF1iw+oW7MvKzWPHkQw2JZ1mT8pZkk6dx79Wxcz8qklBKaUqGU8314LLRxXNpcJfUSmlVKWlSUEppVQBTQpKKaUKaFJQSilVQJOCUkqpApoUlFJKFdCkoJRSqoAmBaWUUgWq3DQXIpICHCjj4UFAqh3Dqar0fbhI3wuLvg+W6vw+NDXGBJdUqMolhfIQkfjSzP1R3en7cJG+FxZ9Hyz6PujlI6WUUoVoUlBKKVWgpiWF950dQCWh78NF+l5Y9H2w1Pj3oUb1KSillLqymtZSUEopdQU1JimIyEAR2SkiiSIy0dnxVCQR2S8im0UkQUTibfvqish3IrLb9rOOs+O0NxGZLiLHRWRLoX1FnrdY3rZ9PjaJSIzzIrevYt6HF0TksO0zkSAiNxZ67q+292GniNzgnKjtT0RCRWS5iGwXka0i8ifb/hr3mbiSGpEURMQVmAIMAiKBOBGJdG5UFa6fMSa60O12E4EfjDERwA+27epmJjDwsn3FnfcgIML2GA+8W0ExVoSZ/PF9AHjT9pmINsYsAbD9vxgJtLUd847t/091kAs8aYxpA3QDJtjOtyZ+JopVI5IC0AVINMbsNcZkA3OBIU6OydmGAB/afv8QuNWJsTiEMeYX4ORlu4s77yHAR8ayGggQkYYVE6ljFfM+FGcIMNcYk2WM2QckYv3/qfKMMUeMMRtsv2cA24HG1MDPxJXUlKTQGDhUaDvJtq+mMMC3IrJeRMbb9tU3xhwB6z8LUM9p0VWs4s67Jn5GHrFdFple6PJhjXgfRCQM6AisQT8Tl6gpSUGK2FeTbrvqaYyJwWoOTxCR3s4OqBKqaZ+Rd4HmQDRwBPivbX+1fx9ExAf4HPizMSb9SkWL2Fet3oui1JSkkASEFtoOAZKdFEuFM8Yk234eBxZgXQ44dqEpbPt53HkRVqjizrtGfUaMMceMMXnGmHxgKhcvEVXr90FE3LESwixjzBe23fqZKKSmJIV1QISIhIuIB1ZH2iInx1QhRMRbRHwv/A5cD2zBOv97bMXuAb50ToQVrrjzXgTcbbvjpBuQduGSQnV02bXx27A+E2C9DyNFxFNEwrE6WddWdHyOICICfABsN8a8Uegp/UwU4ubsACqCMSZXRB4BlgGuwHRjzFYnh1VR6gMLrP8PuAGzjTHfiMg6YJ6I3AccBO5wYowOISJzgL5AkIgkAc8Dr1L0eS8BbsTqWD0HjK3wgB2kmPehr4hEY10O2Q88AGCM2Soi84BtWHfrTDDG5DkjbgfoCYwGNotIgm3fs9TAz8SV6IhmpZRSBWrK5SOllFKloElBKaVUAU0KSimlCmhSUEopVUCTglJKqQKaFFSlIyJ5tpk7N4rIBhHpUUL5ABF5uBT1/iQiNXr93cuJyEwRGebsOFTloUlBVUbnbTN3dgD+CvyrhPIBQIlJwVlEpEaMB1LVgyYFVdn5AafAmrNGRH6wtR42i8iFmW5fBZrbWhev28r+xVZmo4i8Wqi+O0RkrYjsEpFrbGVdReR1EVlnmyDuAdv+hiLyi63eLRfKFybWWhX/ttW5VkRa2PbPFJE3RGQ58G/bnP0LbfWvFpGoQuc0wxbrJhG53bb/ehH5zXaun9nm60FEXhWRbbay/7Htu8MW30YR+aWEcxIRmWyr42tqzkSIqrSMMfrQR6V6AHlAArADSAM62fa7AX6234OwRpoKEAZsKXT8IGAVUNu2Xdf28yfgv7bfbwS+t/0+Hvi77XdPIB4IB54E/mbb7wr4FhHr/kJl7gYW236fCSwGXG3b/w943vb7tUCC7fd/A5MK1VfHdm6/AN62fc8AzwF1gZ1cHHQaYPu5GWh82b7izmko8J3tfBoBp4Fhzv4310fleWizVlVG540x0QAi0h34SETaYSWAV2yzvOZjTWNcv4jjBwAzjDHnAIwxhdcSuDAJ2nqsZALWfFBRha6t+2PN+bMOmG6bRG2hMSaBos0p9PPNQvs/MxeniOgF3G6L50cRCRQRf1usIy8cYIw5JSI3Yy0GtdI2PYkH8BuQDmQC02zf8hfbDlsJzLRNT3Hh/Io7p97AHFtcySLyYzHnpGooTQqqUjPG/CYiQUAw1rf7YKyWQ46I7Ae8ijhMKH6K4yzbzzwufv4FeNQYs+wPFVkJ6CbgYxF53RjzUVFhFvP72ctiKuq4omIV4DtjTFwR8XQB+mMlkkeAa40xD4pIV1ucCbY5jYo8J7GW3dS5bVSxtE9BVWoi0hrrUscJrG+7x20JoR/Q1FYsA/AtdNi3wL0iUttWR90SXmYZ8JCtRYCItBRrdtmmttebijW7ZnFr9I4o9PO3Ysr8Atxpq78vkGqsufy/xfrjfuF86wCrgZ6F+idq22LyAfyNtXTmn7HWQkBEmhtj1hhjngNSsaZ7LvKcbHGMtPU5NAT6lfDeqBpGWwqqMqpVaBZLAe4xxuSJyCzgKxGJ52KfA8aYEyKyUqyF6ZcaY562fVuOF5FsrNkun73C603DupS0QazrNSlYSzL2BZ4WkRzgDFafQVE8RWQN1pesP3y7t3kBmCEim7Bm3LwwVfM/gSm22POA/zPGfCEiY4A5IuJpK/d3rOT3pYh42d6Xx23PvS4iEbZ9PwAbgU3FnNMCrD6NzcAu4OcrvC+qBtJZUpUqB9slrFhjTKqzY1HKHvTykVJKqQLaUlBKKVVAWwpKKaUKaFJQSilVQJOCUkqpApoUlFJKFdCkoJRSqoAmBaWUUgX+P9TpfGM+qedSAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"learn_resnet.recorder.plot_losses()"
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VPW9//HXJzt7gESQJRCWsCmKBtRS9yKItmj1tujVq9ZK763orbZaXKq9VC3d9FfrUpfidluptV5FUdEquNSNYBElbCGChLBDWLPOfH5/zNAOIcAgSc4k834+HnnMnHO+J7xzmLxzcnLmHHN3REQkOaQEHUBERJqPSl9EJImo9EVEkohKX0Qkiaj0RUSSiEpfRCSJqPRFRJKISl9EJImo9EVEkkha0AHqy8nJ8b59+wYdQ0SkRZk/f/4md8892LiEK/2+fftSVFQUdAwRkRbFzFbFM06Hd0REkohKX0Qkiaj0RUSSiEpfRCSJqPRFRJKISl9EJImo9EVEkkjCnacvItJSuTt1YaeqNkR1XZiq2hBVtWGq66KPMfP/tXzP8zC5HTK5+IS8Js2o0heRJlNTF2b99irWVFSyZmsl5RWVlG+rZE1FFeUVlWzcUU2KQWpKCumpRmqKkZZipKWmkJYSnY55HhkTmY6Ma2jaSIvOS0010lNSGvy8sZ8Lg+r6BR193Kug6yLFveexoQIPH8Ztx0fkZav0RSQxuTvbKmtZU1FJebTEyysqKYs+lldUsmFHNV6vBHPaZ9IzO4uBR7TnK/27AlAbckLhMHVhpy7khMJOXThMXSiy57xnuro2TF04RCjs1IbChKLLasNhQtGxkc8Rjs73f445FCkGWempZKalkJWe+s/nmdHHTm3SyeqQSWZ6KllpKWSmp5CVllpvnRQy01LJjD5mpafstXyfz52WQlpq0x9xV+mLSINqQ2HWbYuU+Zpoia+JKffyikp21YT2WicjLYWe2W3omd2GUwty6ZHdhh7R6R7ZbTiyUxZZ6anN/rXsOewSivmhsGe6NhTGnb2KOj3VMLNmz9kcVPoiScjd2V5Z988yjxxyieyxr9m6m/KKKtbvqGpgLz2DHtlt6J/bnpMH5tIjO4tendv8s9y7tstIyLI0ixzOCeDnTcJR6Yu0YKGws72ylm0xHxXRxz3zK3bXxCyvY3tlLVt317C7/l56ago9srPokd2Gkwfm7LWHvmd+EHvp0rhU+iIBC4edHdV1MSVdv8Rr9ir22OU7quoO+Lmz0iPHn7PbZNCpTTo9s9sw9MiOZLdN58hOWXsVe9d2GaSkJN5eujQulb5IM9m8s5rfvVnCio079yr17ZW1BzzjIyM1hY5t0unUJo3sthl065hFQbcOdGqTvtdHdtuY6ejzzDTtmcveVPoiTczdmfXpWm5/YRHbq2oZ2qMTndtm0Ldru73KumObdLLrlXZ2mwyy0lMS8ji5tEwqfZEmtGFHFT95/jNmL1rP8F6d+OOFJzC4e8egY0kSU+mLNAF357mP1zD1pWIqa0NMOXsw3/1qfrOchy1yICp9kUa2dlslNz/3KXOWbuT4Pp355YXD6Z/bPuhYIkCcpW9m44DfAqnAo+4+rd7yPsB0IBfYAlzi7mXRZSHg0+jQL9z9G42UXSShuDsz5q3mrlmLqQs7t399KP9xUl9SdUaMJJCDlr6ZpQL3A2OAMmCemc109+KYYb8GnnT3J8zsDODnwKXRZZXufmwj5xZJKKu37GbKcwv5e8lmTurXlV9cMJy8rm2DjiWyj3j29EcBJe5eCmBmM4AJQGzpDwWuiz6fAzzfmCFFElU47Dz5/kp+8epSUlOMO88/iotG5ul8d0lY8ZR+T2B1zHQZcEK9MZ8AFxA5BHQ+0MHMurr7ZiDLzIqAOmCau+sHgrQKpRt38uO/LmTeyq2cUpDLz795ND2z2wQdS+SA4in9hnZZ6r+V5EfAfWZ2OfA2sIZIyQPkuXu5mfUD3jSzT919xV7/gNkkYBJAXl7TXlZU5HCFws4f3i3lN68tIzMthV//2zFccFxPnUsvLUI8pV8G9I6Z7gWUxw5w93LgmwBm1h64wN23xSzD3UvNbC4wAlhRb/2HgYcBCgsLD+Nq1CJNa9n6Hdzw7EI+WV3BmKHduPO8oziiY1bQsUTiFk/pzwMGmlk+kT34icDFsQPMLAfY4u5h4CYiZ/JgZp2B3e5eHR0zGvhlI+YXaRa1oTAPvbWCe98ooX1WGr+7aATnDj9Se/fS4hy09N29zswmA7OJnLI53d0XmdlUoMjdZwKnAT83MydyeOfq6OpDgIfMLEzkfrzT6p31I5LwFpVv44a/LKR47XbOHX4k//ONYXRtnxl0LJEvxbz+BbMDVlhY6EVFRUHHEKG6LsT9b5bwwNwVZLfN4I7zjmLcUd2DjiXSIDOb7+6FBxund+SKNGDB6gpufPYTlq3fyQXH9eIn5w4hu21G0LFEDptKXyRGVW2Ie15fxiPvlNKtYxaPXTGS0wcdEXQskUaj0heJmrdyCzc+u5DPN+3i4hPyuOnswXTISg86lkijUulL0ttVXcevZi/lifdX0jO7DX/87gmMHpATdCyRJqHSl6T2XskmfvzcQsq2VnLZSX25Yewg2mXq20JaL726JSltr6rl5y8v4emPvqBfTjue+d5JjOzbJehYIk1OpS9JZ87SDdz83Kes317F907tx3VfKyArXfeSleSg0pekUbG7hqkvFfPcx2so6NaeBy8ZzbG9s4OOJdKsVPrS6oXDkRuTT32pmK27arj2jAFcfcYAMtO0dy/JR6UvrVYo7Ly0sJz755SwbP1OhvXoyONXjGRYj05BRxMJjEpfWp3aUJgXFpTzwJwSSjftoqBbe+69aATnHH2kbl0oSU+lL61GTV2Yv35cxgNzS1i9pZKhR3bk95ccx1lDu+tOViJRKn1p8apqQzxTtJrfz11B+bYqjumdzU+/PowzBh+hSx+L1KPSlxarsibEHz9cxcNvl7JhRzUj+3Zm2gXDOXlgjspeZD9U+tLi7Kyu46n3V/HoO6Vs3lXDSf268tuJIzixXxeVvchBqPSlxdhWWcsT761k+t8/p2J3LacU5HLtGQMo1DtpReKm0peEV7G7hunvfs5j761kR1UdXxtyBJPPGKg3Vol8CSp9SVibdlbz6Duf89T7K9lVE+Lso7oz+YwBOs9e5DCo9CXhbNhexUNvl/LHD1dRUxfm3OE9mHzGAAq6dQg6mkiLp9KXhFFeUcnv31rBjHmrCYWd847tydWn96dfbvugo4m0GnGVvpmNA34LpAKPuvu0esv7ANOBXGALcIm7l0WXXQbcGh16h7s/0UjZpZX4YvNuHnyrhGfnlwFw4fG9+K9TB5DXtW3AyURan4OWvpmlAvcDY4AyYJ6ZzXT34phhvwaedPcnzOwM4OfApWbWBbgdKAQcmB9dd2tjfyHS8pRu3Mn9c1bw/II1pKYYF43K43un9qdndpugo4m0WvHs6Y8CSty9FMDMZgATgNjSHwpcF30+B3g++nws8Lq7b4mu+zowDnj68KNLS7Vs/Q7ue7OElxaWk5GWwmUn9eV7p/ajW8esoKOJtHrxlH5PYHXMdBlwQr0xnwAXEDkEdD7Qwcy67mfdnl86rbRoi8q3cd+bJbzy2TraZqRy1Sn9uOrkfuS0zww6mkjSiKf0G3qLo9eb/hFwn5ldDrwNrAHq4lwXM5sETALIy8uLI5K0JJ+sruB3by7nb4s30CEzjWvOGMB3RufTuV1G0NFEkk48pV8G9I6Z7gWUxw5w93LgmwBm1h64wN23mVkZcFq9defW/wfc/WHgYYDCwsJ9fihIy3X368u4943ldGqTzvVjCrjsK33p1CY96FgiSSue0p8HDDSzfCJ78BOBi2MHmFkOsMXdw8BNRM7kAZgN3GVmnaPTZ0WXSxIo3biTB+aUcM7RR/KLC4fTPlNnCIsELeVgA9y9DphMpMAXA8+4+yIzm2pm34gOOw1YambLgG7AndF1twA/I/KDYx4wdc8fdaX1m/bKEjLTUrj9G0NV+CIJIq7vRHd/GXi53rzbYp4/Czy7n3Wn8689f0kSH5Ru5rXi9fzorAKO6KCzckQSxUH39EUOVTjs3DGrmB6dsvjuyf2CjiMiMVT60uieX7CGz9Zs54Zxg8hKTw06jojEUOlLo6qsCfGr2UsZ3qsTE47RWzJEEo1KXxrVo++UsnZbFbeMH6KbkYskIJW+NJoNO6p48K0VjB3WjRP6dQ06jog0QKUvjeae15dRUxdmytlDgo4iIvuh0pdGsWTddv48bzX/cVJf8nPaBR1HRPZDpS+Hzd25c9ZiOmSlc+2ZA4KOIyIHoNKXwzZ32UbeWb6Ja88cSHZbXURNJJGp9OWw1IXC3DVrMX27tuXSE/sEHUdEDkKlL4flz0WrWb5hJ1POHkxGml5OIolO36Xype2oquXu15Yxqm8Xxg7rHnQcEYmDLn0oX9qDc1eweVcNj10xBDO9EUukJdCevnwpZVt38+i7n3PesT0Y3is76DgiEieVvnwpv5q9FANuGDc46CgicghU+nLIFqyu4IUF5Xz35Hx6ZrcJOo6IHAKVvhySyBuxislpn8F/naY3Yom0NCp9OSSvfraOeSu3cv2YQboFokgLpNKXuFXXhZj26hIKurXnW4W9go4jIl9CXKVvZuPMbKmZlZjZlAaW55nZHDP7h5ktNLPx0fl9zazSzBZEP37f2F+ANJ+n3l/Fqs27ueWcoaSlan9BpCU66O/nZpYK3A+MAcqAeWY2092LY4bdCjzj7g+a2VAiN1HvG122wt2PbdzY0ty27qrh3jeWc0pBLqcW5AYdR0S+pHh210YBJe5e6u41wAxgQr0xDnSMPu8ElDdeREkE9765nJ3VddwyXtfKF2nJ4in9nsDqmOmy6LxYPwUuMbMyInv518Qsy48e9nnLzE4+nLASjNKNO3nq/VV8e2Qeg7p3CDqOiByGeEq/offXe73pi4DH3b0XMB54ysxSgLVAnruPAK4H/mRmHeuti5lNMrMiMyvauHHjoX0F0uSmvbKEzLQUrh9TEHQUETlM8ZR+GdA7ZroX+x6+uRJ4BsDd3weygBx3r3b3zdH584EVwD7N4e4Pu3uhuxfm5up4cSL5oHQzrxWv5/unDyC3Q2bQcUTkMMVT+vOAgWaWb2YZwERgZr0xXwBnApjZECKlv9HMcqN/CMbM+gEDgdLGCi9NKxx27phVTI9OWVz51fyg44hIIzjo2TvuXmdmk4HZQCow3d0XmdlUoMjdZwI/BB4xs+uIHPq53N3dzE4BpppZHRAC/tPdtzTZVyON6vkFa/hszXbu+fYxZKWnBh1HRBqBudc/PB+swsJCLyoqCjpG0qusCXH6r+dyRMdMnv/+aFJSdOlkkURmZvPdvfBg4/QOG2nQo++Usm57FbeeM1SFL9KKqPRlHxt2VPHgWysYN6w7o/K7BB1HRBqRSl/2cfdry6gNhZlytq6VL9LaqPRlL0vWbeeZotVcemJf+ua0CzqOiDQylb78U+Ra+YvpkJXOtWfqWvkirZFKX/5p7rKNvLN8E9eeOZDsthlBxxGRJqDSFwDqQmHumrWYvl3bcumJfYKOIyJNRKUvAPy5aDXLN+xkytlDyEjTy0KktdJ3t7Cjqpa7X1vGqL5dGDusW9BxRKQJ6SanwoNzV7B5Vw2PXTEEM70RS6Q1055+kivbuptH3/2c80f0ZHiv7KDjiEgTU+knuV/NXooBN4wdFHQUEWkGKv0ktmB1BS8sKOeqk/vRI7tN0HFEpBmo9JOUu3PHS8XktM/kP0/rH3QcEWkmKv0k9epn6yhatZXrxxTQPlN/zxdJFir9JFRdF2Laq0sY1K0D3yrsFXQcEWlGKv0k9NT7q1i1eTc3nzOEtFS9BESSib7jk8zWXTXc+8ZyTi3I5dQC3YReJNmo9JPMb99Yzs7qOm45Z0jQUUQkACr9JFK6cSf/+8Eqvj0yj4JuHYKOIyIBiKv0zWycmS01sxIzm9LA8jwzm2Nm/zCzhWY2PmbZTdH1lprZ2MYML4dm2itLyExL4foxBUFHEZGAHLT0zSwVuB84GxgKXGRmQ+sNuxV4xt1HABOBB6LrDo1ODwPGAQ9EP580sw9KN/Na8Xq+f/oAcjtkBh1HRAISz57+KKDE3UvdvQaYAUyoN8aBjtHnnYDy6PMJwAx3r3b3z4GS6OeTZhQOO3fMKqZHpyyu/Gp+0HFEJEDxlH5PYHXMdFl0XqyfApeYWRnwMnDNIawrTez5BWv4bM12bhw3mKx0/aIlksziKf2GrrXr9aYvAh53917AeOApM0uJc13MbJKZFZlZ0caNG+OIJPGqrAnxy1eXMrxXJ75xTI+g44hIwOIp/TKgd8x0L/51+GaPK4FnANz9fSALyIlzXdz9YXcvdPfC3FydO96YHn2nlHXbq7j1nKGkpOha+SLJLp7SnwcMNLN8M8sg8ofZmfXGfAGcCWBmQ4iU/sbouIlmlmlm+cBA4KPGCi8HtmFHFQ++tYJxw7ozKr9L0HFEJAEc9Epb7l5nZpOB2UAqMN3dF5nZVKDI3WcCPwQeMbPriBy+udzdHVhkZs8AxUAdcLW7h5rqi5G93f3aMmpDYaacPTjoKCKSIOK6vKK7v0zkD7Sx826LeV4MjN7PuncCdx5GRvkSlqzbzjNFq7lidD59c9oFHUdEEoTekdsKuTtTXyymQ1Y615wxIOg4IpJAVPqt0GvF63lvxWauH1NAdtuMoOOISAJR6bcy1XUh7py1mIJu7fn3E/KCjiMiCUal38pMf3clX2zZzW3nDtO18kVkH2qFVmTD9irue3M5XxvSja8OzAk6jogkIJV+K/Kr2UupCYW5VdfKF5H9UOm3EgvLKvjL/DK+o1M0ReQAVPqtgLvzPy8Wk9M+g8k6RVNEDkCl3wrM/KSc+au2csPYQXTISg86jogkMJV+C1dZE2LaK0s4qmdHLjy+98FXEJGkptJv4X7/1grWbqvitnOHkaqraIrIQaj0W7A1FZU89PYKzh1+pK6iKSJxUem3YNNeWYI73DRep2iKSHxU+i3UvJVbePGTcr53an96ZrcJOo6ItBAq/RYoHHb+58VFHNkpi/88tV/QcUSkBVHpt0DPzi/jszXbmXL2YNpmxHVLBBERQKXf4uyoquWXs5dyfJ/OutG5iBwylX4Lc9+cEjbtrOa2c4diplM0ReTQqPRbkJWbdvHYuyu58PheHNM7O+g4ItICxVX6ZjbOzJaaWYmZTWlg+T1mtiD6sczMKmKWhWKWzWzM8MnmzpcXk55q3Dh2UNBRRKSFOuhfAc0sFbgfGAOUAfPMbGb0ZugAuPt1MeOvAUbEfIpKdz+28SInp3eXb+L14vXcOG4QR3TMCjqOiLRQ8ezpjwJK3L3U3WuAGcCEA4y/CHi6McJJRF0ozNSXFtG7Sxu+Mzo/6Dgi0oLFU/o9gdUx02XRefswsz5APvBmzOwsMysysw/M7LwvnTSJ/emjL1i2fie3jB9KVnpq0HFEpAWL5yTvhk4R8f2MnQg86+6hmHl57l5uZv2AN83sU3dfsdc/YDYJmASQl6ebeceq2F3D3a8v46R+XRk7rFvQcUSkhYtnT78MiL1mby+gfD9jJ1Lv0I67l0cfS4G57H28f8+Yh9290N0Lc3Nz44iUPP7f35azvbKW276uUzRF5PDFU/rzgIFmlm9mGUSKfZ+zcMxsENAZeD9mXmczy4w+zwFGA8X115WGLV+/g6c+WMXFJ+Qx5MiOQccRkVbgoId33L3OzCYDs4FUYLq7LzKzqUCRu+/5AXARMMPdYw/9DAEeMrMwkR8w02LP+pH9c3emvlRMu4xUrh+jUzRFpHHEdeEWd38ZeLnevNvqTf+0gfXeA44+jHxJ643FG3hn+SZuO3coXdplBB1HRFoJvSM3AdXUhbljVjH9c9tx6Ul9go4jIq2ISj8BPf7e56zcvJufnDuU9FT9F4lI41GjJJiNO6r53RslnD4ol9MGHRF0HBFpZVT6CeY3ry2lsjbErecODTqKiLRCKv0E8tmabfy5aDWXfaUv/XPbBx1HRFohlX6CcHemvlhM57YZXHvmwKDjiEgrpdJPEC9/uo6PVm7hh2cV0KlNetBxRKSVUukngKraEHe9vJjB3TswcaSuPSQiTUd31U4AD79dypqKSp6+6kRSU3R9HRFpOtrTD9jabZU8OHcFZx/VnZP6dw06joi0cir9gP3ilSWE3Ll5/JCgo4hIElDpB2j+qq08v6Ccq07Op3eXtkHHEZEkoNIPSDjsTH1xEUd0yOT7pw0IOo6IJAmVfkD+7x9r+KRsGz8eN5h2mfp7uog0D5V+AHZW1/GLV5dwTO9szh/R4O2GRUSahEo/AA/MKWHDjmpu//pQUnSKpog0I5V+M/ti824effdzzh/Rk+PyOgcdR0SSjEq/md318mJSzfjxuMFBRxGRJKTSb0bvrdjEq4vW8f3T+tO9U1bQcUQkCan0m0koHLmKZs/sNlx1Sr+g44hIkoqr9M1snJktNbMSM5vSwPJ7zGxB9GOZmVXELLvMzJZHPy5rzPAtyYx5X7Bk3Q5uHj+ErPTUoOOISJI66AniZpYK3A+MAcqAeWY2092L94xx9+tixl8DjIg+7wLcDhQCDsyPrru1Ub+KBLetspbfvLaMUfldGH9096DjiEgSi2dPfxRQ4u6l7l4DzAAmHGD8RcDT0edjgdfdfUu06F8Hxh1O4Jbo3jeWs3V3DbedOxQznaIpIsGJp/R7Aqtjpsui8/ZhZn2AfODNQ1nXzCaZWZGZFW3cuDGe3C1GyYadPPHeSiaO7M1RPTsFHUdEklw8pd/QrqnvZ+xE4Fl3Dx3Kuu7+sLsXunthbm5uHJFajjtmFdMmPZUfnjUo6CgiInGVfhnQO2a6F1C+n7ET+dehnUNdt9WZs2QDc5du5NozB5LTPjPoOCIicZX+PGCgmeWbWQaRYp9Zf5CZDQI6A+/HzJ4NnGVmnc2sM3BWdF6rVxsK87NZxeTntOOyr/QNOo6ICBDH2TvuXmdmk4mUdSow3d0XmdlUoMjd9/wAuAiY4e4es+4WM/sZkR8cAFPdfUvjfgmJ6cn3V1G6cRd/uKyQjDS9HUJEEoPFdHRCKCws9KKioqBjHJbNO6s57ddzGZHXmSeuGKkzdkSkyZnZfHcvPNg47YI2gbtfX8bumhA/OWeICl9EEopKv5G9V7KJpz/6gktP7MPAbh2CjiMisheVfiN6e9lGrnh8Hv1z23Pd1wqCjiMisg+VfiOZs2QD332yiPycdsyYdCKd2qYHHUlEZB+6OWsjeG3ROq7+08cUdOvA/155Ap3bZQQdSUSkQSr9w/TKp2u55ul/MKxHR578zgnawxeRhKbDO4dh5iflTH76Hwzv1YmnvqvCF5HEpz39L+m5j8v40V8+obBPF6ZfMZL2mdqUIpL41FRfwjPzVvPj5xZyYn5X/nB5IW0ztBlFpGXQ4Z1D9McPV3HjXxfy1QE5TL98pApfRFoUNdYhePzvn/PTF4s5fVAuD15yvG57KCItjko/To++U8odsxYzZmg37rt4BJlpKnwRaXlU+nF4YG4Jv3x1KeOP7s5vJ44gPVVHxUSkZVLpH8Rv/7ace/62jAnH9uA3/3YMaSp8EWnBVPr74e7c/foyfvdmCd88rie/uvAYUlN0xUwRadlU+g1wd6a9uoSH3ipl4sje3HX+0aSo8EWkFVDp1+Pu/OylxUz/++dccmIeU79xlApfRFoNlX6McNi5feYinvpgFVeM7stt5w7VTVBEpFVR6UeFw84tz3/K0x+tZtIp/bjp7MEqfBFpdeI6FcXMxpnZUjMrMbMp+xnzLTMrNrNFZvanmPkhM1sQ/ZjZ0LpBC4WdG/+6kKc/Ws3Vp/dX4YtIq3XQPX0zSwXuB8YAZcA8M5vp7sUxYwYCNwGj3X2rmR0R8ykq3f3YRs7daOpCYX74l094YUE5P/jaQP77zIEqfBFpteLZ0x8FlLh7qbvXADOACfXGXAXc7+5bAdx9Q+PGbBq1oTD//ecFvLCgnBvGDuIHXytQ4YtIqxZP6fcEVsdMl0XnxSoACszs72b2gZmNi1mWZWZF0fnnHWbeRlNTF2bynz5m1sK13Dx+MFefPiDoSCIiTS6eP+Q2tOvrDXyegcBpQC/gHTM7yt0rgDx3LzezfsCbZvapu6/Y6x8wmwRMAsjLyzvEL+HQVdeF+P7/fswbSzZw+9eHcsXo/Cb/N0VEEkE8e/plQO+Y6V5AeQNjXnD3Wnf/HFhK5IcA7l4efSwF5gIj6v8D7v6wuxe6e2Fubu4hfxGHoqo2xKQn5/PGkg387LyjVPgiklTiKf15wEAzyzezDGAiUP8snOeB0wHMLIfI4Z5SM+tsZpkx80cDxQSksibElU/M4+3lG/nFBUdz6Yl9gooiIhKIgx7ecfc6M5sMzAZSgenuvsjMpgJF7j4zuuwsMysGQsAN7r7ZzL4CPGRmYSI/YKbFnvXTnHZV1/Gdx+cxb+UWfn3hMVxwfK8gYoiIBMrc6x+eD1ZhYaEXFRU16ufcUVXLFY/N4x+rK7j7W8cw4dj6f4cWEWnZzGy+uxcebFyrf0futspaLpv+EZ+t2ca9E0dwzvAjg44kIhKYVl36FbtruPQPH7Fk3Xbu//fjGDuse9CRREQC1WpLf8uuGv790Q9ZsWEnD116PGcM7hZ0JBGRwLXK0t+4o5pLHv2QlZt38chlhZxa0LSngYqItBStrvQ3bK/iokc+oLyiiscuH8lXBuQEHUlEJGG0qtJfu62Six/5kA3bq3j8ipGc0K9r0JFERBJKqyn9tdsq+fZDH7B1Vw1PXjmK4/t0CTqSiEjCaTWl3zErnYFHtOeaMwdybO/soOOIiCSkVlP67TLT+MPlI4OOISKS0OK6c5aIiLQOKn0RkSSi0hcRSSIqfRGRJKLSFxFJIip9EZEkotIXEUkiKn0RkSSScHfOMrONwKqgczSRHGBT0CFaAG2ng9M2ik8ybac+7n7QSwonXOm3ZmZWFM/tzJKdttPBaRvFR9tpXzq8IyKSRFT6IiJJRKXfvB4OOkALoe10cNpG8dF2qkfH9EVEkoj29EVEkoiTDWE+AAAC0UlEQVRKvwmZ2Uoz+9TMFphZUXReFzN73cyWRx87B52zuZnZdDPbYGafxcxrcLtYxL1mVmJmC83suOCSN5/9bKOfmtma6OtpgZmNj1l2U3QbLTWzscGkbl5m1tvM5pjZYjNbZGb/HZ2v19IBqPSb3unufmzMaWNTgDfcfSDwRnQ62TwOjKs3b3/b5WxgYPRjEvBgM2UM2uPsu40A7om+no5195cBzGwoMBEYFl3nATNLbbakwakDfujuQ4ATgauj20KvpQNQ6Te/CcAT0edPAOcFmCUQ7v42sKXe7P1tlwnAkx7xAZBtZkc2T9Lg7Gcb7c8EYIa7V7v750AJMKrJwiUId1/r7h9Hn+8AFgM90WvpgFT6TcuB18xsvplNis7r5u5rIfKiBY4ILF1i2d926QmsjhlXFp2XrCZHD01Mjzk0mPTbyMz6AiOAD9Fr6YBU+k1rtLsfR+TXyqvN7JSgA7VA1sC8ZD3l7EGgP3AssBb4TXR+Um8jM2sP/BX4gbtvP9DQBuYlzXbaQ6XfhNy9PPq4Afg/Ir9yr9/zK2X0cUNwCRPK/rZLGdA7ZlwvoLyZsyUEd1/v7iF3DwOP8K9DOEm7jcwsnUjh/9Hdn4vO1mvpAFT6TcTM2plZhz3PgbOAz4CZwGXRYZcBLwSTMOHsb7vMBP4jeubFicC2Pb+6J5t6x5/PJ/J6gsg2mmhmmWaWT+QPlR81d77mZmYG/AFY7O53xyzSa+kA9OasJmJm/Yjs3QOkAX9y9zvNrCvwDJAHfAH8m7vH+we7VsHMngZOI3IFxPXA7cDzNLBdot/Y9xE5K2U3cIW7FwWRuzntZxudRuTQjgMrge/tKS0zuwX4DpEzWn7g7q80e+hmZmZfBd4BPgXC0dk3Ezmur9fSfqj0RUSSiA7viIgkEZW+iEgSUemLiCQRlb6ISBJR6YuIJBGVvohIElHpi4gkEZW+iEgS+f/Pdh2aK5FItAAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"learn_resnet.recorder.plot_metrics()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}