Datasets
Models
Downloads: 1
[404218]:
/
Code /
Efficiency Metrics for Parallel QML Algorithms /
10Q2L 4RYe5RYt Epochs /
_150 Epoch 100.0% 3353.32s kkawchak.ipynb
1839 lines (1838 with data), 123.0 kB
{
"cells": [
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"id": "gDf49Zs-YG8S",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 0
},
"outputId": "411ca767-a288-4f85-e82a-955bbe65a48a"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Time in seconds since beginning of run: 1704989686.7182207\n",
"Thu Jan 11 16:14:46 2024\n"
]
}
],
"source": [
"# This cell is added by sphinx-gallery\n",
"# It can be customized to whatever you like\n",
"%matplotlib inline\n",
"# !pip install pennylane\n",
"import time\n",
"seconds = time.time()\n",
"print(\"Time in seconds since beginning of run:\", seconds)\n",
"local_time = time.ctime(seconds)\n",
"print(local_time)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "m4zD_EUsYG8Y"
},
"source": [
"Turning quantum nodes into Torch Layers\n",
"=======================================\n",
"\n",
"::: {.meta}\n",
":property=\\\"og:description\\\": Learn how to create hybrid ML models in\n",
"PennyLane using Torch :property=\\\"og:image\\\":\n",
"<https://pennylane.ai/qml/_images/PyTorch_icon.png>\n",
":::\n",
"\n",
"::: {.related}\n",
"tutorial\\_qnn\\_module\\_tf Turning quantum nodes into Keras Layers\n",
":::\n",
"\n",
"*Author: Tom Bromley --- Posted: 02 November 2020. Last updated: 28\n",
"January 2021.*\n",
"\n",
"Creating neural networks in [PyTorch](https://pytorch.org/) is easy\n",
"using the [nn module](https://pytorch.org/docs/stable/nn.html). Models\n",
"are constructed from elementary *layers* and can be trained using the\n",
"PyTorch API. For example, the following code defines a two-layer network\n",
"that could be used for binary classification:\n"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"id": "ELkMIhhMYG8c"
},
"outputs": [],
"source": [
"from pennylane import broadcast\n",
"import torch\n",
"from torch.utils.data import DataLoader, TensorDataset\n",
"\n",
"layer_1 = torch.nn.Linear(2, 2)\n",
"layer_2 = torch.nn.Linear(2, 2)\n",
"softmax = torch.nn.Softmax(dim=1)\n",
"\n",
"layers = [layer_1, layer_2, softmax]\n",
"model = torch.nn.Sequential(*layers)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Ym9INdgjYG8d"
},
"source": [
"**What if we want to add a quantum layer to our model?** This is\n",
"possible in PennyLane:\n",
"`QNodes <../glossary/hybrid_computation>`{.interpreted-text role=\"doc\"}\n",
"can be converted into `torch.nn` layers and combined with the wide range\n",
"of built-in classical [layers](https://pytorch.org/docs/stable/nn.html)\n",
"to create truly hybrid models. This tutorial will guide you through a\n",
"simple example to show you how it\\'s done!\n",
"\n",
"::: {.note}\n",
"::: {.title}\n",
"Note\n",
":::\n",
"\n",
"A similar demo explaining how to\n",
"`turn quantum nodes into Keras layers <tutorial_qnn_module_tf>`{.interpreted-text\n",
"role=\"doc\"} is also available.\n",
":::\n",
"\n",
"Fixing the dataset and problem\n",
"==============================\n",
"\n",
"Let us begin by choosing a simple dataset and problem to allow us to\n",
"focus on how the hybrid model is constructed. Our objective is to\n",
"classify points generated from scikit-learn\\'s binary-class\n",
"[make\\_moons()](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_moons.html)\n",
"dataset:\n"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 406
},
"id": "DJnE2JiyYG8d",
"outputId": "fab14522-8206-4e40-f9f6-0926bee25979"
},
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAGFCAYAAABg2vAPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACL2ElEQVR4nO3dd1wUd/rA8c/MLh0ERUFF7L33Ho3RJMYUTe/V9ORyueTufne5kstdLnepl957NZpqeqKx9957RUFAeofdmd8fXzpbYVkWeN6vVyLszs58F5aZZ77leTTTNE2EEEII0WrpTd0AIYQQQjQtCQaEEEKIVk6CASGEEKKVk2BACCGEaOUkGBBCCCFaOQkGhBBCiFZOggEhhBCilZNgQAghhGjlJBgQQgghWjkJBoQQQohWToIBIYQQopWTYEAIIYRo5SQYEEIIIVo5CQaEEEKIVk6CASGEEKKVk2BACCGEaOUkGBBCCCFaOQkGhBBCiFZOggEhhBCilZNgQAghhGjlrE3dACEa286TOXyx+SSn80voGB3K5aO60Cc+qqmbJYQQAUMzTdNs6kYI0RjK7AYPzt/Gwm3JWHQNTEADu2Fy3fiu/POiwei61tTNFEKIJifDBKLFeuy7PXyzPRlQAYDdNLEbKvb9cO1xXlxysCmbJ4QQAUOCAdEiZRWU8sHaY7jq93p9+WGKSu3+a5QQQgQoCQZEi7Ti4GlshusRsPwSGxuOZvqpRUIIEbgkGBAtUnGZZ3f8JTajkVsihBCBT4IB0SIN7NTGo+36yaoCIYSQYEC0TIMTohncuY1aReCARdeY0qc9XWPD/dwyIYQIPBIMiBbrf1cOJyLEUicgsGga7SKC+ffFQ5qoZUIIEVgkz4AIeIZhsurQaT7fdILUXJU46LJRXZjYKxZNc50nICmzkFeXHeLzzScoLjOICLFw5eiu3Dm1J3FtQv30DoQQIrBJMCACWnGZnTs/3MTSfelYdA27YVb+O31AHC9fO5IQq8XtfgzDpKjMTliQRRINCSFELTJMIALaP7/ZzfL96QCVCYMq/l2yN43Hvtvj0X50XSMixCqBgBBCOCDBgAhYmQWlzN+YhLN0AYYJn6xPIqewzL8NE0KIFkaCARGw1h7OcJs4qNRusO5Ihp9aJIQQLZNULRQBy10g4O12jck0TTYdy2LTsSwsusbEXu0Z2NmzXAdCCNHUJBgQAWtIQrTbbTQPt2tMR08XcNdHm9iTkkfFlATDhPE92/HC1SPpEBXSpO0TQgh3ZDWBCGjXvbmONYczKicNVleROOidm8c2QcuUzIJSzn12OZkFpXXaqGkQGmRhQMcohie25drxXenVIbKJWiqEEM7JnAER0J68fCjxbUKovQhA16BzdCiPXzq0aRpW7sO1x8jIL3EYrJgmFJXa2Xw8m/fWHGXG08t4b/VR/zdSCCHckJ4BEfCyCkp5f80xPt1wnNP5pXSICuGqMYncMKE70eFBTdq2M59cwtGMQq9e8+HccUzu076RWiSEEN6TYECIBhj+yM9kF3m+tNGiwaTe7Xl/7rhGbJUQQnhHJhCKFquw1MbXW5NZefA0hmEyqltbLh+V6NPehISYMK+CAbsJqw5mYJqm21TKQgjhL9IzIFqk7SeyufHt9WQVlqFrYAKYEBKk8+p1ozizX1yD9p9XXMbjP+5l3vokr5c2asChx2ZJNkQhRMCQCYSixckqKOW6t9aRU37HbphqMp8JlJQZ3Pb+Rg6n59d7/0Wldq56fS2frDvudSCga6q8sgQCQohAIsGAaHHmb0wir9jmMI2xiQoOGjKr/6N1x9idkou9Hn1qhglzJ/eo97GFEKIxSDAgWpxf9qTiavDLbpj8tOtUvff/4dpj5eMOjumaGgqwVLv7t5R/efXYRGYP71zvYwshRGOQCYSixSkuM9xuU2Jzv40zJ7OLXMUCGCYMT4yhX8coft51ijK7waDO0dw8qTvnDuooEweFEAFHggHhM6Zpsvl4Ft9sSyG7sJSusRFcMboLXdqG+7UdwxOj2ZOS6zAREKg79qFdYuq9/6jQIDILSp0+b9E0urQN4/FLhzZ5UqT6KC6zsys5B7sB/TpGER3WtLkchBCNT4IB4RMFJTbu+nATyw+cxqprqEUqGi/8eoDfzejLfdP7OHydaZqU2g2CLbrP7pivHdeND9ced/q83TC5cWK3eu//khEJvLP6qNNgw26aXDSs+Q0F2OwGL/x6kLdXHSGv2AZAsEXn0lEJPDRrAFGhEhQI0VLJ0kLhE3d9uImfdp1yOGkP4IlLh3LFmMTK77MLS3ljxWE+XnecrMIyQoN0Lh7RhTun9qRbbESD2/P68kM89v1edI3KNlV8feOEbvzjokH1Dj6Ss4uY+exyCkpsdSYRWnSNwZ3b8PldE7Fams+UHNM0+e28rXyzLbnOEIhFg4Gdo1lw5wRCgyxN0j4hROOSYEA02OH0fM56epnT5zWgS9swlv1hGrqucTq/hEteXs2JrMIawYNF1wgLsvDpHeMZ1LnhlQiX7kvjjRWHWXMoA9OEYYkx3DK5BxcO7dTgXoi9p3K564NNHMkoxKJpmJgYJkzt24HnrhpOTHhwg9vvT6sPneaaN9Y5fV4DHr5wIDdNkpUQQrREEgyIBntj+WH+88Mep70CFX753RT6xEfxm4838/3OU44rEWrQrX0Eix+Y6rNhA9M0MU18vrbfNE3WHMpg24kcgiwaU/t2oE98lE+P4S+/nbeFb7enOB360IA+8ZH8/Lup/m2YEMIvZM6AaLDiMru6cLuJK4vLDE7nlzgNBECl6z2cXsC6I5mM7xnrk/ZpmkZjTODXNI2JvdszsXfzLzp0PLPQ6e8E1ErKk1lF/muQEMKvms+gpghY/Tu1cXkhAQiyaHRrH87BtHy32+oa7E7O9WUTW4RSm8Gqg6f5adcpDqTm+XTfsRHBdcpE19bchj6EEJ6TngHRYNP6dSC+TQjpeSUOhwosusbFIxJoExpEsNV9/GmW1xAQimmavLv6KM8tPkB2YVVRpBGJMfzn0iH079imwce4eEQXFu1Jc/q8rsFlo7o0+DhCiMAkZ1zRYFaLzgtXjyTIotfIugfqItK1bRh/Om8AAEMSomnn7g5To8GFhFqSF389yCPf7K4RCABsP5HDpa+s5mBa/essVDhnUDxDEqLr/P5ABXOxkSFcP6H+yzGFEIFNggHhE2N7tGPhvZO5YGgnrOUXlOiwIO6Y2osv75lEuwgVAARZdO6e1svpfnQNZg/rTEJMmF/aHejS80p4dvEBh8/ZTZPiMoOnf97X4OMEWXQ+mDuWqX07AGrCYEVc0L9jFAvumED7yJAGH0cIEZhkNYHwuTK7QXGZnYhgq8MZ/KZp8uh3e3hr5REsuoZhmlg0DZthMrVvB169bhRhwbKeHeDNFYd57HvXKzV0Dbb8/RyfZQo8lJ7PqoOnsdlNRnSNYXhijKRQFqKFkzkDwueCLDpBLhLuaJrG3y4YyFVjElmw6QQns4qICQ9izogERndrKxeealJzi1XA5KJEomFCRn6Jz4KBXh0i6dUh0if7EkI0DxIMiCbTJz6Kh2YNaOpmBLT2kSFuV19oGpXDMNXlFpdxMquIiGArie3CJMgSQjglwYAQAWz28AQe/3Gv0+ctmsaZ/TvUWPaXllfMEz/u5autydjKexT6xUfyu7P7MXNwx0ZvsxCi+ZEJhEIEsI7Rodw2pafD53QNrBaNB8/uV/lYel4Jc15axZdbqgIBgP1p+dz54SY+Xue8gJMQovWSYEB4LLuwlK1J2ew7lYfhLvew8Jn/O7c/D5zdl7BaRYJ6tI/gk9vHM7BzVZ6B5xbvJzW3pM7QQsU04YcX7iS70Hn5ZSFE6ySrCYRb6XklPPb9Hr7Zloyt/CKT2DaM+6b34fLRiW5eLXwlv8TGygPp5BXb6NkhgpFda062LC6zM/yfP1NcZjjdhxQcEkI4InMGhEsZ+SVc/PIqUnKKa9xtJmUV8YfPtpOeX8LdZ/ZuwhbWZZomu5JzSc0tJi4qlMEJbRpl8lxecRlfbD7J+qOZaMD4nrFcPCKBiBDP/6yOni7gq60nOZ1fQsc2oVw8sovTHAuRIVZmDu7kdF/peSUuAwFQCYSOZhR63D4hROsgwYBw6eWlh+oEAtU99dM+LhnRhY7RoX5uWRXTNNmalM2321M4kJrHruRcMgqqusJ7dojg7xcM9GlWw7WHM7j1vY0UlNgqiyB9uz2FJ3/ax9s3jWFUt7YuX283TB5euJMP1x7HomtogGGaPP3Lfu4+sxe/P6ef1wFMVKj7P2cTaOPBdkKI1kWGCYRTNrvBsH/+TEGJ3ek2ugYPnN2Xe8/q47d2GYbJ7pRc8ktsdIgM4Z/f7mbZ/nR0DYfJebTy/715w2imD4hv8PFPZhcx/emllNqMOsfTNQix6vz1goHERYUyvmc7okLrrv9//Me9vLL0kNNjPDSrP7dPcZ6p0Zlr31zLmkMZLpMU/fy7KfRtpqWWhRCNQ24RhFM5RWUuAwEAXdM4num/bucvNp/gmV/2c8JBOV1nF0AT0Ez4+9e7mNYvzmFWRG+8v+YoZQ4CgYo2FJUZ/OXLnQCEBuncMqkHD57TrzLvf25xGW+vPOLyGC8tOcSNE7sTYvUuE+P9M/qy5tAaNNT7rk7X4LzBnSQQEELUIasJhFMRIVYsHnRVO8t8dzK7iP/+sJczn1zC+McWc+t7G1m+P536dka9s+oID8zf5jAQcMcsb8/6o5n1OnZ1P+1KxUVCwBqKywxeWXqIP32+vfKx5fvTKbG5HtvPKSpj49Esr9s2pns7Xr9+NG3CVJxv1bXKGgMXDuvM01cM83qfQoiWT3oGhFOhQRbOGRTPz7tTnc4ZsBkms4cn1Hl8/ZFMbnx7PaU2e+WFMz2/hEV7UhnVtS1pecWcyi0mOiyIS0d14ZZJPYhv43zeQVZBKY99v6fB7yklx/tAorbSMte9JbWZwIJNJ7hlcg8GdGpDoZvelgoFJbZ6tA5mDIxn3UMz+GnXKQ6l5RMeYuW8wR3pFhtRr/0JIVo+6RkQLt17Vm8sWtXdZXW6BmcPjGdwQnSNxwtKbNz63gZKqgUCQGVAsel4FklZRZTZTU7nl/Lm8iOc99wKDqU7L8W7sNqyxoaIjWh45b0hXRyX+nXFomt8vukEAL3jPcv736cB3fmhQRZmD0/ggXP6cefUXhIICCFckmBAuDSoczTv3TK2snytpbzbWUN1O79w9Yg6r/lq60lyi20uJ7FVZzdNcorKuPfjzU6HEE5kFVaWRq6v2IhgJvSKbdA+AG6c0N1tvYDaTNMkLa8EgBGJMfSNj3QYYIH6GY/r0Y4e7VvPBXzt4Qzu+WgzU59cwsxnl/PcogOk5RU3dbOEaDVkmEC4NaFXLKv/dBZL96WzLzWPsCALZw+MJ7FduMPtNx7NwqLh8bg6qF6DPSl5bE3KZkTXusvyYsKDvb4A1/bQrAEuqyl6amLv9twxpSevLT/sdAVDbSbQIUoFVJqm8fTlw7nitTWU2o0a78uia0SGWHnskiENbmdzYJom//lhL68vP4xF1yp/FvtT83hz5WE+mDuO4YkxTdtIIVoBCQYE2YWlfLnlJAfT8okIsTJzcEdG1Kphb7XozBgYz4yB7pfmaZX/9+7irWmwzUEwYJomfeOjPO5pqC0mLIi/nD+AS0d1qd8OHPjTef0ZlhjDG8sPszUp2+07NU1Iy6260x3SJZqF907iucUH+GHnKeyGSZBFY/bwzvx2el+ngVZLs3BbMq8vPwxQIygyTDXcdPM761nz5+mEBnm3qkII4R0JBlq5zzed4M9f7KDMbmDRNUzg9eWHmdgrllevH0UbB2vk3RnfK5Yvtpz0vjEmBFlr3rmvP5LJX7/awf5U5/MJKkKWiktJZIiFSb3bM6Z7O7rHRjClbweCrb4dEdM0jVlDOjFrSCfshklydhEznlnmcpXAN9tTuPesPHYl5/DB2mMcSs8nMtjKjRO7cdHQzvTv1KbVXfTecNG7YpiQVVjGwm3JXCFpr4VoVJJ0qBVbvj+dG99e7/Cu1qJpjO/Zjo9uG+/1fotK7Ux6/FeyC0u9upvXgOV/nFZ5V7zpWCZXvb4Wu2E6TSZkAvFtQnj68uGM7t6WwlI7bUKtWH0wHOANwzAZ9PCPFLlIB6xr0KVtOMczC2tcAHVNLeP85LbxdSZjtmTFZXb6/+1Hl9tYdI05wxNkSaQQjUwmELZiz/96AGdpBOymyapDGWxLyvZ6v2HBFt69eQyRIdYak+RcpSywaDBzcMca3eP//Ga300AAINii8+p1I1n9p+lM7tOe0CAL7SKC/R4IABSW2V0GAqCGCioSNFV/TxVd4re+txGb3fU+WiPTy+EmIYT3JBhopbIKStl4NMvlnbtV1/hx16l67X9olxh+/f2ZPHhOPwZ3bkPvDpHMHtaZaf06AFQuzbOUBwgju7XlicuGVr7+cHo+207kuGxfid3AbuD1Mr/GEBZkIcTNUISrS5phwqncYhbtSfNtwwJYaJCFQZ3buAwS7YbJ+B4NXwEihHBN5gy0UoUeJs4prGfiG4D2kSHcM60390yrqmpomiZbkrKZvyGJpMxCYiNDuHhEAlP6dqhxUT+V635Zma75JomQL1h0jYtHJLBg04l6r3qw6hqbj2cxc3BHH7cucN0+pSe/nbfV4XO6Bm3CgrhwWGf/NkqIVkiCgVaqQ2QIkSFW8l1c7O2G2aDEN45omsbIrm0Z6WD5YO32uWOYVcv1AsE903rz/Y4UCkrt9Q4IAqGXw58uGtaZ7UnZvLXqaI2lhbqmhpveuWkMYcGta1KlEE1BhglaqWCrztVjE53WHtCAkCCd2cOb5q6sd1wkAzpFuexCDg9W+Q4CRWK7cBbcOZG+tTIMWnWNGyd0o2u7cFxd6m2GyaRe7Ru3kQFG0zT+esFAPr5tHGcPjKdL2zB6x0Vy71l9+PXBMx3mnBBC+J6sJmjFcovLuOyV1RxKK8Be7WNg0TQMTJ6/akSTdtGuOnia699ah2k6Hm9/+MKB3Dyph9/b5Y5pmuw4mcPelDxCgnSm9OlA24hgPl53nIe+3OHwNRZdo3eHCH68f0qN/A5CCOEPEgy0crnFZby85BAfrztGbrEaMpjUO5bfnNWH8T2bfuLW0n1pPPTlDpKzq+YQtAm18oeZ/bl+fLcmbJn3TNPkX9/u5u1qXeKaplYZdGkbxrzbx9OlbetINuQre0/lMn/DCU5mF9I2PJjZwxMY37OdBFRCeEmCAQFAmd0gq6CUsGALUfVINNSYDMNk3ZFMTmYX0TY8iMl92hNibb7jyJuPZ/HxumPsT80nKtTKhUM7M3t4goyNe8EwTP757W7eXa0CK8Mw0csDrDP6tOe160cRHixTooTwlAQDQohm57Vlh/jPD3sdPqdrcMHQzjzvoIiWEMIxCQZEwDFNk20ncvhsUxKpuSW0jwzhslEJjOzaVrp/m5hhmKw8eJqNRzNB05jQM9bv3fKlNoNxjy0iq7DM6TaaBiv+OE2GXYTwkAQDIqDY7Aa//2w7X205WTmuXvHvuYPief7qEc16iMAZwzBJKc+t0KlNKElZhaw9nIFpwujubekd59slnvVxMC2f297bwJEMVU7aRC0/7RcfyZs3jvFbcaXNx7O45OXVLrfRgH/NGcx1zWxeiRBNRQbVREB56uf9fF1e5KhizXnFv7/sTuXRb/fwrzmDm6x9vmY3TN5ZdYS3Vh4hJUcFAyFWvU7Bo4m9Ynn2yuHEtQn1W9vyS2zYDZM2oVayCsu48vU1ZJffjduq5VE4mF7AFa+t4effTfHLfJNSF8WgKmkebieEACQYEAEkv8TGu6uPOE3ba5jwyYbjPHB2X9pGBPu1bY3BNE0eXLCNr7ecrPGeHVU+XHckk8tfXcO3901u9Avu9ztSeGXpIXaczAGgW7tw+sRHkplf6vB3YzdMTuUU88Xmk9w4sXujtg2gT1xkjQRFjpgmZBSUNHpbhGgpJOmQCBjrj2RQ7KbYj81usurQaT+1qHEt2ZfGV7UCAWfshsnxzEIWbDzhdlvTNNl5Mocl+9LYlZyDNyOBzy06wN0fbWZXck7lY8czC1m0J81tO7/aWo+y1fUQGxnCBUM7OU2YVeGlJYfYWo9CW0K0RtIzIPymuMzOjztPcTAtn7BgCzMHd6RXh6psfaU2zy5aLaX794O1x9ze4da2YFMSt0x2nmhp2f50/vXtbg6m5Vc+1jc+kocvHMSk3q6zG+5KzuF/i/YDNasqetI6EzieUcicl1aRX2yjb8dIrhvXjQm9YhtlcuHfLxjI8v3pLicRWnSNt1Yc5oVrRvr8+EK0NBIMCL9YtDuVB+ZvJbfYhlXXMEyTJ3/ax3mDO/LMFcMJCy6vYIf7i8/ghGh/NLnR7U/N9yoQMIHTeaVOn/91byq3vrexzs/vQGo+17+1jndvHsuUvh2cvv6jdce9Dk6qyygoJaNAte9IRgHf7zjFteO68uicwT4PCGIjQ+jXMYq1hzOdbmM3TJbsS/fpcYVoqWSYQDS6jUczuf2DjeSVZzi0GWblnedPu05x37wtgMrtX7t6YXUWXWN0t7b09XHxpKYSFeJdLK5rkNA2zOFzhmHy1692qtTNta7lZvl/f/96p8shg93JufUOBGqr2M9H647z4brjPtlnbboHAYav3o8QLZ0EA8IrhmGy4kA6b644zAdrj5Gc7b6E8HOLDzi94zdMtUqgYoz6v5cOIS4qBEut87xF14gJD+KZK4Y3+D0EiguHdcabIoWGCdeM7erwufVHM0nOLnbaq2KacDSjkM3Hs53uPzzY4rKQUn1owBvLD2M0wkV5RNcYlz8/iwbDE2N8flwhWiIJBoTHthzPYsqTS7j+rfU89v0e/v7VTiY9/isPzt9KcZnd4Wtyi8tYeeA0dhfXAouu8d32FAA6RYfx7W8mc8fUXrQLVysGosOCmDu5Bz/cdwZdY1tOEpmrx3YlJjzY7UQ4qLqwzR7huHBUSo77oMzdducN7ui6DbrGiMSYyt8LQFiQxeUF2URNQEzNK3a+UT1dM66by+EHuwk3T+ru8+MK0RLJnIEAV1xm57vtKfy6N41im52Bndpw1diuJMQ47i5uLAfT8rjmjXWU2NRFv/JGz4Qvt5wkv8TGa9ePrvO6ghKb2zkAOlQOIYAaD/7jzP78cWb/ypzzLVG7iGDm3T6eW97dwImsIqzl79NW6y46yKJxycgu/O2CgU4TLrWLCPHomLEutrt4ZBde+PUgGQWldbrXdU21439XDiehbRjHMgrRNPho7THeX3MMw82KhcZIbZYQE8bTlw/jgflb0bSquQ4WTQUCcyf3CKgS10IEMgkGAtjR0wVc8+ZakrOL0TV1AV6yN42Xlhzk3xcP4WonXcaN4cVfD1JqN3DU22uY8NOuVLafyGZol5gaz7WLCCY82EJhqeOeAwCbadK9fYTD51pqIFChb3wUy/4wjaX70lh3RE2GG9+zHSO6tmXnyRxME4Z2iSYm3HVehYm9YmkXHkxmofMJhvFRIYzt0c7p85EhVj65fTw3vLWek9k1g5PIECuv3zC68vfUO06tAhnZtS1vrzrqsm2dokPp2EjJkuaMSKBnhwjeWnmEX/emYTdMhifGcMukHkwfECfpq4XwkAQDAarUZnDdW+tIzVWJUyouwhX/PvTFDrrFhjOxl+vlYr5QZjf4dnuKy8lYVl3jqy3JdYKBEKuFK0Yn8sHaY05fb9U1LhmR4MsmNysWXWP6gHimD6h5F3tGH+cz/2sLsuj8eVZ//vDZdqfbPHT+AKeTMyv06hDJ0j+cyS+7U1l58DR2u8nIbjFcOKyzwyqAqz3I+TB3co9GDeqGdonhuaukKJEQDSHBQID6adcpTmQ5H9/VNXht2WG/BAOFpfY6Xde1mUC2k7vS+6b34de9aZzMLqoREGia6j5++MJBLSKjYFO7fHQipgmPfreb3GrDLjHhQfzt/IHMHu5ZwBVk0Zk1pBOzhnRyuV16XgmfukmCFGTRuHac1AcQItBJMBCgluxLc7nm227C8gPp2OwGVkvjzgONDLESGWIlv8TmcrsuTpa9tYsI5ou7J/LkT/v4cvNJSu0qaVC/+Cjun9GXmW4mrgnPXTEmkdkjOrN0XzppeSXER4VwZr84gq2+/4ws25/udulemd1k+4lsxvWM9fnxhRC+I8FAgCq1GW7TyJomXPn6Gq4c05WLhnUmNKhxqvlZdI2rxiTyzqqj2J20yTBMLh+d6HQf7SNDePzSofztgoGczCoiPNhCl7ZhMqbbCEKsFs4d1PgBlrMVJHW2ayEZI4VoyWRpYYAa2sWzLHubjmXzx8+2c+krq8kpcp6ataHuOrMXHaNDnY4533tWb49K2EaGWOnXMYrEduESCDRzAzq5T/6kaSodshAisEkwEKAuG5XoVff/3pQ8HvpyR6O1JzYyhC/vnsgwB0HKmO5tudkP1epEYBnZta2qIOgkprPoGmf1j6NTtH+XwQohvCfBQIBqFxHMs1cO93h7u2nyw44UTuX4PrlLhe93pDjMYLf5WDaXvrqGHBdFY0TLo2kaz141nNBgS50eI4uu0SEyhH/NHtxErRNCeEOCgQY6crqAb7cn89OuUz7vpp81pBNBzm67HDBM2HjMeeGWhkjLK+Zf3+1x+JzdNDmeUcjLSw82yrFF4BrUOZrvfnMGl45MILi8JysixMJNE7uz8DeT6Ozn5FhCiPqRCYT1dDK7iD8u2MaqQxmVjwVbdK4b35U/nTfAZ7O3gyw6ZXbPJmo1ps82nXA5odFumnyy/jh/OLdfo69uEIGle/sInrhsGP+5ZCiFpTYigq0NyiuQkV/CnpQ8LLrG8MQYwoIbZ2KsEKKKBAP1cDq/hEtfXk16fkmNx0vtBu+sPkpKTjEvXzvSJxPkzuofxw87T3lUfU3XYFS3tg0+piNHTxega5rLtLO5xTZyi220k5wBrZJF14gKDar363MKy3j4m118sy258vMeEWzhpknduX9GX4IkyBSi0chfVz28vfII6XnFDi/Qpgk/7DzF5uNZPjnWbWf0dLvEEMCiaZw3uFOjTdby5CSvaarynRDeKiixccVra2oEAgAFpXZeXnKI++dt9ejvQAhRPxIM1MOnG5LcVuH7bJPrzGyeGpYYw/+uHI5F11xWh+vfKYrHLhnik2M6cv7QTi6zEFp0jRn94xst14Fo2T5ed5z9aXmOA2zgux0plbUbhBC+J8MEXjJNk8wC58VgAOyGSaoPZ/XPHp7AuB6xzNtwnC3Hs8nILyG/xEap3aBzdBhXjElstKRDh9PzeW7RAdYdyUADhxUItfL/7j2rt8+PL+pKzS3m880nSMosIiY8iAuHdmZg5zZN3awG+Xj9cZeVDS26xqcbkhgvmQyFaBQSDHhJ0zTaRQST4SIgsOga8dG+rdLWMTqU+2f09ek+XSm1GTz0xXY+23zS6TYVpWLbhAXx3FXDGZYY47f2tVavLTvEEz/uw8RE1zRM4JWlhzh/SCeevmJYs+uZMU2TwlI7KTnO63CACrBPZBX6qVVCtD4SDNTDlWMSeW3ZIadDBXbD5LJRXfzbKB/7x8JdLgMBUL0E/5o9iCvGJBJibV4XoTryUmH967DtYyjKgjZdYPQtMOomCHafWdEfPtt0gv/8sLfy++qTOX/YmUJYkM5TVwxvgpZ5r9Rm8N7qo7y7+igns10HAqAC7PaRIX5omRCtkwQD9XDL5B58vvkEp/NL64xxahqcO6gjI7s2zqx+fziVU8y8Dcc92jarsKz5BwIZh+Dtc6EwE8zyZZwZB+Gnh2DbJ3DTtxDqWXroxmIYJs8u2u/8eRM+33KS353Tj4Tytf17T+XyzqqjLN6Tis1uMqJrDDdN6sHUvp6XRm4MpTaDW97dwKpDp10ODVRnN0wuGdm8A2whAplMIKyH9pEhfH7XRMb1aFfj8WCLzk0TuvP8VSOadd79X3af8ugkbZiwxUerJpqMacL8G2sGAuoJ9V/qLvjpr03Vukr70/JclrSu8MuuUwD8sCOF859fyWebVNCaXVTG8gOnufHt9Tz+4143e2lc7685yqqDngcClvIls2f1j2vchgnRiknPQD11aRvOx7eN53B6PjuTcwm26EzoGUt0eP3XWTtimibp+SUYBnSICnFaKMiX8kvs6C7KJ1fQoPknGDqxEVJd1HQw7bB9HpzzTwhrut6ewlL3iad0TaOwzE5qbjH3zduCYZg1JnxW/D5fWXqIUV3bMmNgfCO11jnTNHlv9VGHE1Ed0YBzBnXkicuG+uWzL0RrJcFAA/XsEEnPDr6vymaaJp9vPsmryw5xMC0fgPioEG6e3INbJ/do1Itwzw4RHiU5MoEpfdo3Wjv84uQm0HQwXZTZtZdC6m7oPsl/7aqle2wEFjcBmt0w6RsXxSfrj2OvFQhUZ9Hg7VVHmiQYKLEZJLnp4dCAIQnRXDOuK5N6t/eoGqYQomGa+W1dy/XET/v4/YJtHCoPBABS80p4/Ie93PXRZo8u1vV1Vv84YiPdZxGMCQ9izoiERmuHX+gWPOqv1ps2bm4XEcyswR2d3h3rGsRFhXBmvw5sOpaFq4+H3YRNx5pmeMeqa7gbQdN1jX4do7hqbFcJBITwE+kZCEDbT2TzytJDQN11/Sbwy+5Uvt56stEmVAVZdJ65Yji3vLuhTldzhahQKx/cMq5B6WcDQs9pOM6eUE1oNHQa5pfmuPLXCway8VgWaXklNYLBioRUz141HKtFx+LBfBV/d7knZRby+eYTnMoppmu7cJIyC50GLHbD5JxBHZ3u62BaPh+sOcqK8nkHk3rHcsOE7vSNj2qk1gvR8kkwEIA+XnfcZZewrsH7a4416uzqqX07MP+OCTy7aD8rDpyufDwuKoSrx3bllkk9fD4/okm07w19Z8KBX2pNIKygwfi7Ici3eSPqI75NKAvvncwLvx5gwcYTFJXZ0TWYPiCO+87qw+AEteJhcp/2LNuf7nyYQNc4w0/DO6Zp8t8f9/L6ssPomuoVMEzTaSBg0TV6tI9wOlnw660neeDTbaBVzYE4llHAh2uPExcVQtfYcGYPT+CSEQlEhMjpTQhPaaYk/K637MJSdqfkYtE0hnbxXXW1i19axZakbJfbRIZY2fnIuT45njtZBaXkFJXRISqkZZ5gi7Lhw0vK5w9YVFCgW8GwwdArYc4rajghgJTaDLILS4kMtRIeXPN3klNYxqTHf6Ww1ObwoqsBC+6cwOju7eo+6WOvLTtUIzeCIxZNJfOyGSa94yJ5/5axDksfH0rP55xnlmP34JTVpW0Yn94xoXKZpRDCtRZ4Zm98ucVl/Oub3Xy55WRlvv6KGu6+qK4WGWpF01wPZfuzIFDbiGDatuRKhGExcMvPsP8H2D4fCtKgbQ8YeQN0nYDbQe4mEGzViWvjuLciOjyI924Zww1vr6ew1F75ObLoqurko3MG+yUQKLHZeWnJQZfbWDSNcwd3pE2olbMHxnNmvzinQxgfrDmG05zYtaTkFHP7+xv59jeTm/UyXyH8RYIBLxWV2rnq9bXsS8mrcYdSUKKqqx09XciL1zQsz8D5QzrV6JqvzaJpXDSsc733LxywWGHAheq/FmBUt3Ys/8M05m88wa97Uym1GYzq1o5rx3elVyOsfnFk09EscottLrexmyYzB3dkRGIMm45l8e32ZEZ3b+fwjn7lwdMeT5y1Gya7knPZdCzLL4GPEM2dBANemrfhOHuScx3enFRUV7v2UFcm9q7/mOxFwzvzwq8HOZVbt0yyrkFIkM6NE7vXe/+idYiNDOGuM3tx15m9muT4nuRGAHh16SH2pFT9TWkanDuwI49fOrTmvBQvBzQtusbqQxkSDAjhAVla6KWP1rlO02vRNT7dmNSgY4QHW5l3+3i6x6plVVZdw1reddo2PJgPbx0nS65EwOsT71kPxN5TNYNr01QrZq55cy3FZVUBxaTesV6vgpAZUUJ4RnoGvJSSXeTyBsVumBzPbHh1tcR24fzyu6ksP5DOygOnsRkqt/zMwR2bfy0A0Sp0i41gQs9Y1h/NdNi9XzH876jn326qbv5vtiVz+ehEAK6f0J0P1h7z+Ph2w2RM9+ZbI0QIf5JgwEsx4cEUlDrPoKZr0KEB1dXScotZsOkEB9PyCQu2cN7gjjw0awC6pGIVzdB/LhnCxS+vIrfYVic3gmmaYDrv/dc0WLDpRGUw0DsukmeuGM4D87eiaa6zMVYsUZzQK9aXb0eIFkuCAS9dProLzy8+4HSdtGHCxfXMyvfxuuP87eudmKaJBqBpfLzuOEO7RPPOTWOIlRKuopnp3j6Cb+87gxcWH+DLLScpsRlYdY0LhnZi9aEM0vJKnL7WNCE1t7jGY3NGJDCocxveX3OM5QfSSc0tprjMqLHIQNegbXgQr18/SlYSCOEhyTPgpYz8Es57bgUZBXXLF1s0jcEJbfjsrolulxfml9iYt/448zYkkZpbTHiwhdRcxydGi64xtEs0X9w1UU5uotkqtRnkFJURFWolNMjCpa+sZvPxLKfj+roG43rE8snt453u02Y3+H7nKT5ae4wjpwuIDlMpsq8e25V2LXk5rBA+JsFAPRzPKOQ3n2xm24mcyjsSDTh7YDxPXj6M6DDXmfmyCkq5/LU1HErPd9lNWtuCOycwRmZGixZi/oYk/vj5dpfbPHvl8OZf/0KIZqDVDRMkZxfx1daTpOeVEBcVysUjEugY7V2q2a6x4Xx972R2nMhhS1IWFl1jcu/2dIuN8Oj1f/lqB0dOF3g109mqa/y865QEA6LFmD2iMx+uO8bOkzl1ht10DYYnxjBrSKemaZwQrUyr6RkwTZPHf9zH68tVAaDquf/vnNqLP5zbzy9d8Kdyipn438Uuq8o5YtU1rh7blX/NGdw4DROiCeQVl/H3r3excFty5d+jVde4eEQC/7hoUMtMfy1EAGo1f2kvLz3Eq8sOVX5v2M0az7UJC+LOqY2fnGX7iWyvAwEAm2HSv5NUZRMtS1RoEP+7cjgPzRrAluNZaJrGyK4xMllWCD9rFcFAYanNbY70l349yE0TuxMaVL81/PklNn7ZfYrTeaV0jA7l7IHxDvdVn9KxGhAWbGH2cBk7FS1Th6gQl2WLhRCNq1UEA6sPZrhNjZpXYmPN4Qym9XNcOtUZ0zR5a+URnv55f2VJWcNUVQX/dsEArhzTtcb2o7u1I9iiU2o3PNp/xXrs/105nEjpMhVCiMBWmAnrXoXN70F+OkTEwogbYPxdEOGf0uH10SquLgWlroulVG5X4tl21b27+iiPfren8vuKIYD8Ehv/9/kOQqyWGrOho8ODuHJMIh+tO+a0vGywVafEZqBrcFb/OO4+sxcjukomNSEqnMop5tMNSexJySU0SOecQR05e2B8gyuGCkFxDuz7AYqyIKYr9DkHLK5XiFXKTYa3zobcFFUKHSA/DVb+D7Z+DHN/hpjExmt7A7SKYKBne89ypHtbza24zM4zP+93uc1/f9zLhcM61xge+Mv5AziWUcByB5UJO0WH8sXdE4gMDSbEqsvJTYhaPll/nL9+uRMTE9NUKw++2ppMj/YRfHjrOIcVD4VwyzRh+VOw4imwFYOmg2lAeHu48FnPKpou/A3kVQsEKvdth4JU+OouuOnbRml+Q7WKK83ghDYM6BSFs+F6XYMhCdEM6NTGq/0u259OnpvehFM5xWw+nlXjsdAgC7dP7VlZfKi6lJxirn9rA3bDlEBAiFpWHjjNn7/Ygd00McpzdFTMBT6eWcgNb63j6OkCFm5L5tvtyXUyGDpjmiabjmXxt692cs/Hm3ns+z0cTMtrvDciGkdhJqx5CRbeBz8+BMfXeV6tavmTsORRFQiACgQACjPg0+vhwCLXr886CgcXg+FkSNqww9EVcPqAZ+3xs1bRM6BpGk9eNowrXltDic2okyM91Krz+KVDvd5vdmGpR9tlFdTcrtRm8NtPtmI4+JCawOH0Ap76aZ8sIxSilpeXHqycl1Ob3TA5lF7AmU8trXxM1+DCYZ15dM5gokIdd/UWl9m59+PNLNqThlXXMEwTTdN4fflhbp7Unb+dP1BqgzQHWz6Cb+8Howy08snba1+CbpPgqo8hLMb5a4uyVDDgkKl6CRY9DL2nq6IZjqRsw6MUcslboH0f99v5Wau59RycEM3X90zinIHxWMp/mRZN49xB8Xx972QGdvauVwCgS1vPygi/tfIIaw9nVH6/aE8qGQWlTpcY2k2TBZuS6jWHQYiWqrjMzupDGV4tzTVM+GZbMje+vZ4yJ5N2//bVTn7dmwaoJbyGSeUNwzurjvLGisMNbrtoZAcXwdf3gL1U9QQYNvUfwPG18Ol1rnsI9n6nXuuMaUDqTjjtYlhY93BegafzD/ysVfQMVOgTH8Ur140ir7iMrIIy2kYEOb1b8MSEnrF0jgklJbvYZTy48VgWV72+lkcuGsSNE7uzJyUXq65hc3FWKy4zOJFVRL+OkltACMDl34srhgmbj2fz485TXDisc43nUnOL+XzzCZcBxqvLDnHzpB4EW1vNvVPzs+yJ8jF+B130Znn3/MnN0GWU49cXZqjeBEevr72dM90mgiUE7M6Lb6EHQY+pro/RRFrlpzsqNIiuseENCgQAdF3j3xcPQdOc9xxB1V3GPxbuYn9qHqFBFjxJ/BgiJx8hKkUEW+o9OVDX4LVlh3h31RG+2nKS3OIyAJbtS3fb05BVWMa2E9n1Oq7wg8JMSFrn+kKuW2HPQufPRye6DwQAors4fy4sBkbfooISRzQdRt4A4YGZUl6uNg00rV8cH8wdx0APJh/qusaHa48xfUAcdhcnIA3o0T6CbrGeDUMI0RpomsZNE7tTn9F7w4Sdybk88u1u7v90K2MeXcTziw9QXGb3aH8lZZ7lBRFNoKzQg400KCty/nS/WRDi4hyuWaD7FLXU0JWz/wn9L1Bf69aq1wL0ORdm/seDtjYNCQZ8YFLv9nx33xluqxXaDTVjuX/HNkzr1wGLk7OQCdw3vbeUKxailpsmdWdK3w5oUK+goKJDrsRm8Mwv+9lxMsftlC9dg77x3i07Fn4UEef6Qg5q/kCHfs6fDwqF858u/6bWJ0uzgDUEzv23+7ZYg+GK9+HmH2DYVdBrOgy7Gm78Fq7+RO0nQEkw4ENhQe5/nBXjjs9fPYKxPWIBVZjFoqmTjqbB/83sz8UjXHRHCdFKBVl03rxxNP+4aBDd26sqoVZd86hnzpEvt5ygR/sIp4G5Rdc4Z1BH4tp4V9lU+JE1GEbdWHUHXoemLvZDLne9n6FXwJUfQrueNR9PHAu3/ASdPFxxpmlq/sDsl+D6L2DOS9DjDNdjyQGg1VQt9Ie/frWDT9Yn1Vi6WJ2uwYPn9OOeab0BtbZ5/ZFMvt2eQl5xGd1iI7hiTKIkTRHCQza7gUXXKC4zuPjlVRxIzcfuxSlNA35zVm/eXnWUojJ7zWXHmkbH6FC+vHuiBAOBrjgX3p4J6Xtrjv1rFsCEy96GQRd7ti/ThFM7oChTDQvUDg5aKAkGfOhgWh4zn12B3TDrdD3qGoQFWVj6h2l0iArcriIhmqvswlL+8uVOftiZ4vHyQ4uu8cDZfblwaGdeXX6ILzafoLjMoE2olWvGdeP2KT1pFxHcuA0XvlGcq9L+bnwbirPVY72mwxkPQvdJTdq05kCCAQ/YDZNdyTkUlNjp0T6CjtHO7xJ+3nWKez/egs0wKk9IGhARYuWdm8cwpntgziQVoqU4lVPM1qQssovK+NPnO9xu/+yVwyvrh5imSYnNIMSqy5yd5spuU0mEgsIgROZ6eEqCAdQJoNRuEGypewKYvyGJ/y3aT0qOSlGpaXBWvzj+cdEgEts5nu2fllfM/A1JbDiahUXXmNS7PZeN7EJ0eGAmmxCipbruzXWsOZThdOggIsTCxr+cTVhw/UqXC9FStOpg4GR2Ea8tO8Rnm05QWGqnTZiVq8d25bYzetI+MoTXlx/ise/31nmdRdeICQti4W8m1xjf35Wcw5srjvDTrlOU2Az6xUdx08TuXDqqS41CRUII/9h7KpdLXl5NSZnd4XLeJy4byhWjA7OKnBD+1GqDgQOpeVz26hryS2x1Jg3FtQnhzRtHc9GLq5xOBrToGpeMSODJy4cB8MvuVO76cJMqnFL+mooc6jMHdeSla0dKQCBEE9idnMvDC3ey4WhVwbCu7cL548x+XDC0s4tXCtF6tNpg4PznV7D3VJ7Di71F1+jZPoJD6fkuJyIFWTS2PXwOZTaTcY8tosRmOFyzrAGPzB7EDRO6+6r5QggvHT1dQFJWITFhwQxOaCNzAoSoplXVJqiw40QOu5JznT5vN0wOpOVj1R1XR6tQZjfJyC/ll92pTgOBCm+vPML147vJCUiIJtK9fURlbgIhMOxw4BfY/ikUpEF0VxhxncoR0ArP060yGNiVnOPRdu6WJ2lAm7AgdpzMQddwmmLYBI5mFFJUZic8uFX+yIUQInCU5MFHV8Dx1VUFinQLbPsYBl0Kl7wOltZ1rm6VGQhDPMgUCK6DAYumMa1/HNFhQVgrUge6YdVb5Y9bCCECy8L7VHEjqEpSZJT/u+sLWBq4NQQaS6u8Op3Rp4PbyXxRIVYuGZng8Bqva6DrcP+MPgBM6x/ndKIhgEWDib1ipQSqEEI0tezjsOtLF1UKTVj3KpR6UgCp5Whd/SDl2keGcMXoLny6Icnp3f+tZ/Tk7mm9iAqx8uG649gNs3J1QHybUJ65YjhDu8QAcPbAeBLbhpGcU+wwKLCbcMfUXo34joRoOdJyi5m/MYldybkEW3VmDIjn3EEdJZgWvnF4KbgrT1WaDyc3qZoCrUSrXU1QYrNz3ydb+GlXKhZdwzBNdE3DbphcMzaRR+cMQS/vPUjPK2HxnlQKSu30jotkcu/2dXoWjp4u4Jo315KcXVwZNFg0DQOTRy6SlQRCeOKLzSf442fbMUwT00T9TZomiW3D+OjW8XSVst6ioTa+Dd/+zv12138Jvc5q/PYEiFYbDIDKPLg1KZsvNp8ko6CEjm3CuHx0FwbUswJacZmd77an8MvuVIptdgZ0asM1Y7s6zVQohKiy/kgmV76+BkdnJIuu0Tk6lMUPnik9BKJhkrfC61Ndb6Nb4IF9ENnBL00KBK06GBDCJww7nNgARdnQrofruunCqbnvbmDp/nSX829evGaEJAoSDff6NEjZ5njegGaBwZfApW/6v11NqFXOGRDCLcOAnCR1soju6nyZ0daPYfEjkHeq6rGEUXDB/6DTMP+0tQUwDJOl+9Jdlh+26Bq/7kmTYEA03GVvqZLHBadrlTzWIbY3nPdE07WtibT6YCAjv4StSdloGoxIbEtbKVfaupkmbHwLVj2nZh0DhLeHcXfA5N+BpVqxqfVvwPe/r7uP5C3qRDP3F+g42D/tbubspukyEAAwyisKeiO/xEZOURltw4Mkx4eo0q4n3LkS1r8OWz6EokyI6gyjb4bRt0BIVFO30O+a1TBBbnEZn286wXfbU8grsdEvPorrxndjbA/vywLnFZfxj4W7+HprMrbybkmrrnHpyC78/cKBRITIiaNV+v4P6gSBRo0Zx5oGvc+Bqz5WvQTFufBUH7AVO96PZoGeZ8L1X/ih0S3DWU8t5cjpAqfzvHUNHjynH/dM6+12X/tO5fG/Rfv5edcpDFOlDr9waGfun9GXjIIS3lhxhMV7UrHZTQZ0iuLmST24eERC5aRhIVqbZhMMHE7P56rX15KeVwKo07RFV7P/b5zQjX9cNMjjVL8lNjtXvraW7Sey6ywt1DUY2bUtn9w+niCLTFRqVY6vg7fPcb3Nxa/DsCth8wew8F43O9TggT3QppPPmtiSfbDmKH//epfTYMCqa6z583Q6RIW43M/WpGyuen0NZXazZhEyXSPEqlNUakcvP3dAVUGxOcM788wVwyUgEK1Ss7ja2Q2TW97dQEZBKSZV92sVf8zvrTnGvA1JHu9v4dZktibVDQRAnRQ2Hsvi+x0pDW+4aF42vQO6ix4hTYcNb6ivc5NdbwuACXnyOfLU1WO7cvbAeDRUv0wFi66ha/D0FcPcBgKmafLg/K2U2ow6ExHthklhqb1GZVGoyjT61dZkPt98wjdvRohmplkEA8v2p3E0o9DpLGMNeH3ZITzt5Ji3Icll9mBdg0+9CC5EC5G+Dwyb8+dNA04fUF9HtK9KX+pKROtZmtRQVovOK9eN4l9zBtMrLhJQgcD0AXEsuHMCs4cnuN3HpmNZHEovcFtXxBFdg3dXH/X+hUK0AM1iYHz1wQysulY5tl+bCRzJKCQ9r4S4NqFu95eSXeRwLXMFw4ST2UX1bK1otkLbUGeuQG0h6iLFwDnww/+BUeZ4O02HxLEQk+jjRrZsFl3juvHduG58t8qsn95U+jyYll/vYxsm7E7JxTRNqS4qWp1m0TPgaZTv6XZxUaG4+lPXNIhz0x0pWqBBl+AyENAsMOQK9XVELExxsJIAVCCgaTD9H75uYati0TWvL8rhDZz4a5X5AqKVahbBwKhubZ32ClToFB3q8QX8stFdXGamNk24fLRnd3SmaVJYanOZKEU0E0Muh5hu6qJfm2ZRvQJjb6t6bOr/wfS/Q1CtDJNtOsO1n0O3CY3bXlHH1L4dCKlnhkKLriqRSq+AaI2axTDBOYPiiYsK4XR+icO7fw2YO7mHx7OALx3ZhfdWH+Xw6YI6F3GLptG3YyQXDXOd2CSnsIzXVxzi43XHySosq1y6dNeZvegT3/rWqLYIweFw07fw8VWQtqtqgqBhg6h4uHqeutBX0DQ440EYewcc/AWKc6Btd+g+RZW1FH4XHRbELZN78MrSQ0630TQcDhMapsntZ/RsxNYJEbiazdLCnSdzuOaNteSX2CoDAoumKgJeMLQTz101wm1Z4uoyC0r5/YJtLNmbVtlLoKEqED5x2VBiwp0nH8osKOXSV1ZzLKPmRCWLrhFk0fjo1nGM6uZ97gMRIEwTjixX1c1MOySOgz7nOs9CKAKK3TD5x8JdfLj2GLqmoWnqQg9w7biu/LTrFGl5pZVBga6Bhsbjlw3lslFdmrj1QjSNZhMMAKTmFvPh2mMs3JpMQamNvvFRXD++G+cO6ljvtcHHMwrZcDQTTYMx3dt5VFTogU+38uWWkw6HGnRNlThe+X9neRWcCCF861hGAV9tSVZFyKJDuWREFzpGh1JUaueb7cks2ZtGic1gcEI0V49NpFN0WFM3WYgm06yCgUCQllvMuMcWu6uGzds3jeas/vGV3x89XcA7q47w3Y4UissM+nWM4oYJ3bhwaGdJctIcmCbknlTLCdskSC+BEKJFkTOal15eetBtIGDRNfak5FUGA2sPZ3Dj2+uxGVUZ0bYcz2LTsSwW7U7lWS+HOIQfmSZsfk/VKsg8rB6LiINxt8Ok+2vWKhBCiGZKZjl56Yedp9xuYxhm5YzmwlIbt7+/kTK74TDr2bfbU/ho3bFGaavwgZ/+At/8FjKPVD1WkAZL/g2fXgd2F0mKhBCimZBgwAt2wyQ1t8TtdiYwY4DqFfhmWzK5xTaXORDeXHHE4+yJwo9ObIS1L5V/U+v3Y5qw/0fY+ZnfmyWEEL4mwYAXdA2P1jB3jgmle/sIALYcz3Y5BGACxzMLyS2SO8yAs/Ft97UK1r/hv/YIh07lFLP3VC7ZhaU+3W92YSnbT2RzMC1fgnXR4smcAS9omsb5Qzrx1daTLu/0/zJrQOXXuq65zHZYwWKROQMBJ32v+1oFGQf81x5Rw7rDGTz18z42HM0CVLB+zqCO/Glm/8pgvD7Scot57Ps9fLs9pTLZWY/2Efzu7L5u848I0VxJz4CX7pjaC6uuOyx0pGswoFMUMwdXlayd0qe9y+yJugZDEqKJbGAaVdEIQipqFbgQHOmXpoialuxN4+o31rLpWFblY4YJv+xO5aKXVnI4vX41CtLzSpj90iq+qRYIgFoNdN8nW3h31REXrxai+ZJgwEv9Okbx7s1jiA5Ts8iDdK0yn/nwxBg+mDuuxrDAjAHxdGkb5nSowDBhdLe2/LAjhZxCJ0VvhP+U5MHuhbBtHiSMwn2tgsv81jShlNkNfv/ZNkyzbj0Su2FSUGLnkW9212vfzy3eT1peSZ3MpBXfPfrdHjLy3c8bEqK5kTwD9VRis/PDjlPsSs4h2KpzVv94RnaNcZjX/HB6Ple/sZbU3JLKmni6VvdEFmzRuW58V/503gCC65lfXdSTYcDS/8DqF8BWrWKlHqSGA8xa5Yo1i0pffPdaiJasdf70y+5Ubnt/o8ttNGD1n8/yKpFQcZmd4f/8meIyw+k2ugYPzRrArZK2WLQw0jddTyFWC3NGJDBnhPsa6z07RPLrg2fy5ZaT/LjzFDlFpRxIy6fUZtQICErtBu+sPkpaXgkvXD1CCqb4049/gvWv1X3csFN5X1i9VkFEB7j6EwkEmsDR0wUOg+nqTFR2UW+CgcyCUpeBAICuaRzPLPR4n0I0FxIM+ElEiLWyTvuj3+5md0qew5OZaarcA7ee0ZPhiTF+b2erlHkY1r/u5EkD0KH7ZDVsYNggcSz0myUJh5pIVKjVo3LlUaHe/X4iQ62VPXfOmEAbL/crRHMgfdFNYP7GJJclj626xuebTvixRa3c9gVqmaBTBhxfA1P/COf+GwbOlkCgCZ09ML5yno4rh9LzvNpvm9AgpvbrgMVFj5zdMLlQVhQ0T6YJR1fCV3fDexfBF7fDoSVqiFBIMOBvNrtBbrHrnAJ2wyQtr9hPLRLkp+JweUh1hg2Ksv3SHOFabGQIN0/q7na7//t8BwUl3uXv+O30Pmia44+DrsH5QzrRr6OUKG92bKUw/wZ493zY/ikcWaYShn0wBz66HMqK3O6ipZNgwM+sFp02oa5HZ3RdI75NqJ9aJIjq5LjAfXV6EIS19U97hFv3z+iLu9QchaV2vtmW7NV+R3Rty1s3jaFteQlzq66VlziGOcMTePqKYfVssWhSix+Bvd+qrytyhxjlk4IP/wo//F/TtCuAyJyBetidnMsvu1MpKrPTv2MUMwd3JDTI4vHrrxrblbdWHMHu5AJkN0ypq+5Pw65UtQac0S0w+FK1ekAEhNP5JdjdxG9BFo2Dad7nG5jatwPrHprOot2pHEzLJzzEyjkD4z0qby4CUHEObHhTrQpyxDRg60dw1t8gsoN/2xZAJBjwQl5xGb/5ZAtL96VjKc8saDNMohdaefaqEUzrF+fRfm49owdfbTlJRkFpnbkDGjB7eGeGdonxefuFEzFdYdJ9qjJhbZpFJRY6U+4cAkl4sPtTl2FCeD2TeQVZdM4b0sn9hiLwHV8HNjfDroZNDR204rwhMkzgIdM0ueODTazYnw6ou/eKDGW5RTZue28j25KyPdpXXFQoX9w9kfE92tV4PDRI5/apPXnqcumK9LsZj8D0h8uzDlbTZTTM/QXaybryQNIhKoQRiTG4mkdoN0zOG9zR58cuKrWz82QOe1JyKbPL5LOAZ/ewZoW9dSd9k6RDHtp4NJPLXl3j9HmLrnFW/zjeuGG0V/s9crqAPSm5BFt0xvVs5/VyKOFjZUVwbBWUFkD7fhDXv+bzeacg9ySEx0Lb7k3SRKEs3ZfGze9scLgUUNdgWr843rppjM+OV1xm55lf9vPRumMUlKjx5tiIYG6f0pPbzuiJ7sEKB9EEspPg2SG4XjQK3LMBOvT1S5MCkQQDHnrkm118sOaYyzoDmgZ7/jnTq/kDoplI2wM//xUOLqbypNJ5JJz9CPSY0qRNa80+33SCh77cQanNwKJrmKgegWn9OvDiNSOJ8FHNj1KbwfVvrWPD0UyHOQ4uH9WFJy4bKonCAtXHV8KBX+pmEgU1JyhxPNz8vf/bBWryctpuyE9TScza92mSZsicAQ/lFdvcxZWYprp7kGCghUndBW+dDWXF1Li7SNkK78+Bqz6GfjObqHGt26WjujBjYDxfbz3J4fQCIkIsnDe4E4MTon16nM83n2DdkUynzy/YdILLRnVhXM9Ynx5X+MiFz8Pb50D28ZoTCTULRMTDxQ6yj/rDgUXqJiN9T9VjnUfCzP9C13F+bUqTBQMns4v4YtMJUnKLaR8RzJwRCfTsELgV4Hq0j3Bb0zw6LEi6+Vui7/+oAoHadxWmAWjwzX3QezdYJLZuCtFhQdwwoXujHuOjtcfQNOcrUC26xrwNSRIMBKqoeLh9Kax/Aza9p3KLRLSHEdfDuDvU1/6293uYd03dx1O2qnwIN30LXcf7rTl+HyYwTZOnft7Hy0sPoaFyfVd07V09NpF/zR6M1RJ48xrTcouZ8J9fnS4H1IC7zuzFH2f2d/i8aKYyj8Dzw91vN+RyOLFBLWNq2wPGzIUhV4A1uNGbKHwvLbeYX/emUVRmp198FLd/sJH8EgddzNUM6xLN1/dO9lMLRbNm2OF/AyEvFYdzGTQd4gfDnSv81iS/38q8tfIILy05BKgfgVHt4jpvfRJtQoP486wB/m6WW3FtQvnbBQP4xze7Hd4hmMB3O1KIiwrhmnHdpOpgS5F11LPtdnxG5R91cQ58fQ9s+wSu/QyCPC+WI5pWic3Ow1/vYv7GJAyTyr91ZyXIK+gaxIRL4Cc8dHipmozsjGnAqe1waid0HOyXJvn1ilVis/PirwedPm8C76w6Sk5RYC7xuGlSD165diT94h2nIz2eUcgj3+xm7nsbKLXJkqNmwTDUxJ2CDMd9wGExHu6o2msrxiSPrYYljzW0hcKPHpy/rTIQgKqPhOGmMpJhqvwgQngkJ8m32/mAX4OBTceyyHZzoS+1GywrX8sfiM4b0olP7xhPsIOhDLP8v5UHT/POqiN+b5vwgmGHNS/Bc0PgqT7wZE94ZSJsm1czKOg4DGK61e8YpgEb35a8583EruQcvt2e4riaqIvXWXSN3nGRzJIkRcJT4R7OLfF0Ox/wazBQVOp6zK1CoZfFRfztNx9vodRFshHThHdWH3U74VA0EcOAz26Gn/4COdWqQ6btgS/vUHnMK+g6zHi4/scqzYeDi+r/euE3C7cmux0OqGDRtcraCCMSY/j4tnGyikh4rveMugnOaotOhATv8tY0hF+Dgd5xnq0W6OOkGz4QfLc9heUHTrvd7lROccAOd7R6u76A3V9T936v/PuV/4OTm6seHnwpXPQCBEWo73Urasqoh2vKv7wTTh9oWJtFo8ssKPXoN/rytSM4q38cbSPUHIFtJ7L561c72XQsq3EbKAKX3QZ7voEv7oBPr4NfH1XLGJ0JCoOz/up6n2f/U92M+Ilfg4FusRFM6BnrNPrWNegTF8nIrjH+bJZXXlt+yONtgwJwVUTAKsyEZU/Cs0Ph3x3hueHqolyc4/tjrX9DrS92Rreq7v3qRt4AD+6BPudSNSDkYc9PWRF8//t6Nlb4S0LbMLfFK8ODLXy1JZlfdqeSka/S3JbZTRbvSePyV1ez0MsqiaIFyDsFr05WQcCOBbDnW1jxDDw3FNa+6vx1Y29X+QSCy28ytPLrRWgMzHkVBl/S6E2vzu9LC4+cLuDil1eRV2yrUaTHomuEWHXm3T4+YIv0FJfZ6f+3Hz3admyPdsy/Y0Ijt6iFyDkJb5+r0vzWSAiiq2V6t/zk22pi/+0Gxdmut+k8Em5fUvOxbx8oDxLq+Sfz222SwjiAJWUWMuWJJU5/uxZdY2yPdqw9lOF0myCLxuo/TadDVEhjNVMEEtOE16ZA2q6qksi1XfUJ9J/lfB+lBbDvByhIhzYJ0PdcsPr/8+P3W9ce7SP45t7JXDoyoXISnkXXmDW4IwvvnRSwgQC4L3lf3V1n9mq8hrQ0X90Fucl1S4yahlra9+1vfXu8IHelaDUIqTVUlXEINr5FvQOBin2IgJXYLpx7z+rt8DmLrtEhMoSUbNeTQe2GyYJN/psBLprYkWVqCaCzQEDTYeUzrvcRHKGqJY6/CwZe1CSBADRRBsLEduE8cdkw/jl7MDlFZbQJDSIsOPAn34QFW+gXH8n+1HyXl4TZwzp7XM641Tt9QP1BOWPaVaaunBMqb7cvDL4E1r7iOE+5OigMnF3zoe2fqqEFp6/xgLsJQ6LJPXB2X9pHhvDirwdJzy8B1PDl9AFx/PX8AUx5YqnL1xsmbDmW3fgNFYFh/09qWNFwMundNFQysqIsCGvr37Z5qUnzp4YGWZrdDNy5Z/Tkj59td/p8ZIiF/1w6xI8tauaqT9RzyoTkrb4LBsberrr7bcV1eyP08lzlQ6+s+Xh+WnkGmnoeM6oTJIys54uFv2iaxo0Tu3PtuK7sOJlDUZmd3h0iiWsTimGY6BoOlx5Wt/ZwBgUlNp8VSRIBzFaCRxOJbR6WUW5CMsPNS5eP6sLVYxOBmlnJLJpGWJCFd24eS3iwnAQ8pnsYDOoN/Jkmb1Gz+p/uD+/MhO6TqoYCdGvV/mO6qZzgIbVWvrTp7N04UW3T/uL5exVNzmrRGdG1LRN7tSeuTSgAuq5xZr843K0+zCuxMX+jDBW0Ch2HgOFm1VhEXNPUPvCSlDCuB9M0WbIvjfdWH2N3ci6hQTqzhnbi+vHd6NLW3Xi0qCEvFf43wPmYG4AlGH5/wItsgLVsfAe+/Z26GFd051V07Y24Xo3R6VboOQ36nO34op11DJ4bhlddA5quhhZmPAwTf1O/touAsv5IJle8tsblNhrQNz6Kn34npa1bvJJ8eLovlBbitMbAmQ/B1D/4vWnekmBANL2v7irP/OcgkZOmw5hbYdaT9dv3qR3w6hk4v4hrcNcqiB/kfl+/PAyrnvXgoBp0mwAD56gCRuHtPG6uCHwT/7OY5Jxil9u0DQ9iy9/P8VOLRJPa/zPMu1p9XTl3oLz7qPskuO6LJpsU6A0ZJhBNb9ZT0P0M9XXF+v+Kf/ucA+c8Wv99r3vddfe8blF5Bzwx4x8w4xEIiXa+jWaB9n3hmvmqNKoEAi1O/05tXA4VaEBcVKjf2iOaWN9z4LZfVfBvKS9W1bYbnPvvZhMIgPQMiEBhGCpt77aPIS8F2nSBEddCjzMbloXrueGQ5aZORGwv+I0nExnLlRXD0RWw5SM48DOUFajHdQsMuhTOe1yCgBbsx50p3Pmh88+LBvz9woHcPKmH/xolAoNpqiFPS/ObN9b8WixaJl1XEXZfH3etap4kmPUy2AgKVXML+pytxgqTN4O9TNUf92VyJBGQZgyIZ2z3tmw8llVnZYFF1+geG84VoxObpnGiaWlaswwEQIYJREvXe4br1MOaBXpPr//+g8Oh+2ToNU0CgVbCatF55+axXDKyS40VRZoGZ/WPY/4dE2RZoWh2ZJhAtGzp++GV8U5WK2iqR+Lu9dDecea5JpF9HIpzISYRQl3MTxANklVQyoJNSWw6loVF15jcuwOzh3f26kKenlfChqOZGKbJ8MQYWU3U3JQWqAJD2cfV0N6A2a02qJdgQLR8O7+AL24try1UHhRoFnUrd+lbMGhOU7auyv6fYcmjkLJNfa8HqYqJMx5WeQ6EzyzZm8ZdH26ixG6AqT4Khgkx4UG8f8vYgE6LLnxky0fwwx9UQKBb1Q2DblHLgM/6u+u5SrZSOPQr5J+CyHjoNR2swf5reyOQYEC0DpmHYcNbcGS5+r7nVBg9F9oFyCSvrZ/AV3eqpZTVl1jqFpW05LYl0KZT07WvBTmYls95zy3HZjfrLDjVNYgMtbLs99MqSxSLFmj31zD/BufPT/mD8xLD2xfAj/8HhRlVj4W1hXMfg+HX+LadfiTBgBBNKfs4LH8KNr/nfBvdotIjz3nFf+1qwf761Q4+WZ9Uo2pqdZoGD503gNum9PRzy4RfmCa8MBIyj+A0/4glGH6/v249gR2fwedzne97zqsw/GqfNdWfZAKhEE0lbY9KiLTlA9fbGXZ1EirJ80+7WrifdqU6DQRAXSt+3n3Kjy0SfpW6U/UUusomai9VBdKqM+zws5Peggq//E2tLGqGJBgQoimYJnw2V13gHWVerM1eCrkpjd+uVqDM7v7nXWLz4HcimqeibPfbaDoU19ru2GqVA8WVgvSqochmRoIBIZrCiY2Qtsu7ksgVhZVEgwxNiK6xJLA2i64xPDHGfw0S/mMYar6AO6YBbbvXfKwg3bNjFJz2ulmBQBbDupCRX8Jnm06wOyWXEKvOjAHxnNU/DqtF50RWIe+uOso325MpLLXTq0Mk14/vxpwRCS5PNEIAcGo7KledB1N2NB0SRssEQh+5aVJ3lh9wfsK2GybXjuvmxxYJv1n8D9jgLv24BuGxKhV6dW0SPDtGM135I8GAE19vPcnvF2zDZphoqDrn8zeeoGeHCP583gDun7eFYptROfa4/UQ2Dy7I5vudKbx63SiCLNLpIlywhuBxBUTThGkPNWpzWpNp/eK4cUI33ltzDL18SSGoHgG7YfLX8wfQr6P0wrQ4eamw+kX322kaXPQ8WIJqPp44Ftr2gKyjOP7b1SA6AbpN8kFj/U+uWA5sPJrJ/Z9upcxuqlTTJpUX/WMZhdz5wUYKS+01JiFVfPnrnjTeWHG4KZotmpNe09UdvztBEXD5OyrDofAJTdP4x0WDeOHqEQzrEqMyyGoaE3vF8v4tY7n1DFlF0CLt/gqPAvApf4B+s+o+rmlw/lPlKc5r9/6Wfz/r6YbVUmlCsrTQgbnvbWDpvnSXM45d6RAVwto/T5fhAuHaV3fDtk+cTyAcOFstVQqWrHaNyTBMNE0FCaIFW/IYrHgGDA9m+w++DC5+zXGdgYOL4cc/w+l9VY/F9oGZ/1H1SpopGSaoxW6YLNmbVqcAiTfS80o4lVtMQkyY7xomWp7zn1aJS/b/WDMDmmGD0bfCrCeb7V1Gc6J7ELSbpsmWpGwWbEziZFYR7SNDmDMigcm923v0euEjZcWQvgcyDsORpXB4OWCq+iBjb4fOw52/NjrRs0AAYOfnqgzx9L/Xfa73dLhnHaRshbzyDISdR3hYFC1wSc9ALcVldvr/7ccG72fdQ9OJbyM1zYUbpglJ62DbPCg8rU5YI66D+EHe76usGI6thJJ8aN+nfvsQddgNkz9+to3PN5+snFdQ8e/EXrG8eeNowoPlvqpR2ctg2ROw7jUoyan7vG5RKwXOfxrGOEkKVJwLT/UFW5FnxwyOgN8fUP+2AhIMODDliSUkZRZ6Or2rBg3o3j6CXx+cKt2Owj9ME1a/oDIZVj9Rdh4BF70IHQc3XdtagP/9sp/nFx9weD7QNbhwWGeeu2qE39vVahgGzL+uPAmQu7OyBrcvUZ99Rza+A9/e7/mxb/wGekzxfPtmTPogHbhpYne32zjrGTSBu6b2kkBA+M+Sx1Tms9p3TCnb4e1zIH2f49cJt4rL7Ly18ojTS5BhwsJtySRne3i3Kby3/0fY+x0eTf7TddV74Mzom1VxsrB2nh27mWYTrA8JBhy4fkI3zujTntpzRi26hgb8ZdYAOpfPB6i45ldMFrx9Sk8uH93Fr+0VTSh5qyqAtOk9yDrm/+PnpsCKpxw/Z9rV0MGv//Zvm1qQrUnZ5JfYXG5jmrDigIcJaYT3Nr2rqox6wrDDkWWutxlyGVz3pft96RboOMSz47YAMtDlQJBF562bxvD+mmO8u+oISVlFaBpM6dOeO6b2YnzPWK4d35WFW5P5dkcK+cU2+sZHcu24bgyTzGWtQ+ZhlU44eTNVyYM0tQLgohcgtI1/2rFjgevnTTvs/RaKcyA02j9takE8SV2sAaV2GW1tNJkHvcvUWWfZnwMJwyFhlJoEaDjYt2aBgRdDZJwXx23eJBhwIsiiM3dyD+ZO7kFRqR2rRauRSCg82MpVY7ty1diuTdhK0STy0+HtmdXSjppV/+75BvJT4abv1J1FY8s7pU5cruobmHbVVgkGvNa/Y5saiYkcMVEpjkUjCYsF3BQWqqBboaeHOTkufbPq77h6sKHpENtbreZpRVpkMJBZUMr8jUks3pNKic1gRGIM143vRp/4+mUVCwv2w0ldNB/rX6t7Aqlg2uH4GjjwC/Sb2fhtiYxzf9ek6Sq9qvBah6gQzh/Sie93nnKYd8SiawzoGCU9go1p2JVwYr1n2xp2GHeHZ9u26wl3rlRzDLZ+CIWZKpXwqJth9C3+690LEC1uNcHWpGxueGsdeSU2Kt6ZRdcwDJO/XziQmyf1aNoGiubv6QGQl+z8ec0CAy+Cy99t/LbknIRnBzvvGdAs0O88uOqjxm9LC5VZUMqlr6zmWEZBjR4Ci6YREx7Egjsn0LNDZNM1sKUryYdXJ0P2ceeBr2YBTLV6ZsS1fm1eS9GiJhDmFZdx49vrya8WCIBaJ2wCj3yzm9UHm2dFKRFAijJdP2/aIT/NP22JToCJ9zl+TrOoGgjT/uKftrQgxWV20vKKKS6z0y4imK/vncSD5/Sjc3QoFk2jXUQwt07pwfe/PUMCgcYWEqmG3RJGqu81ncpLlzUE2vZUqwTuWiOBQAO0qGGCr7acJLeozOnIkkXXeGPFYSb2bu/XdokWpk1nNYHQGd0KMX6sejfjH6q88cpnoLSg6vEO/WH2ixA/0H9taeZWHjjNi0sOsOFoJnYDgiwaFw3rzG/O6sM903pzz7TeTd3E1ik6AW5dBCc3wdGVaglHt0nQZXTzzvxnt8GJDVCSC+16Qfum+3y1qGGCOz7YyM+7Ul1OMwmyaBz4t4MiFEJ4atVzsOgfrift3fwDdJvotyYBKhA4vLQ8A2Fv6DyyeZ8o/ejI6QLu+2QLO07WzW6naxARYuWzOydKNcOWylaq8hlkHoKQNjDgwsZfSbD5fbXsN/9U1WNdx8P5zzRJ9tAWFQzc+t4GFu1x3T1r0TQOPnaeJAUS9VeSB2+eDaf3OxjD1GDwJSqxiXzGmoWT2UVc8PwKsgqdJ5ixaBqDE9rw9b2T/dgy4Rf7foCv71F1QipW5ug6jLkdznnUcbGihlrzMvz057qPaxYICoPbfoUO/Xx/XBda1JyBkd3aOs0MCCrCH9EtRgIB0TAhUXDz9yp5iV7tRBEcCWc8CBe/LoFAM/LykoPkFLnONGc3TbadyGFPSq6fWiX84sgKmHeNWkkA5cG9qVYlrHvV8QW7oYqyYNHDjp8z7VBWBIse8f1x3WhRwcCVoxMJsuhOU04YJsyV1QTCF8LbwSWvw4P74Pqv4MZvVVGT6X9rnDsJ0ShsdoPPN5/wuErp/tS8xm2Q8K9f/1X+haMPgAnr31Ardnxp5+eu0xybdtj3PRRk+Pa4brSoYCA2MoSXrhmJxaJVpgeGqlTBcyf3YObgjk3VPNESRbSHXtOgxxkQHN7UrRFeKiixU1zmPstghbAgyTnSYuQmq4qhrub+aBrs8iB1sbfH1d3dMJg15xL4QYu7hZkxMJ4ff3sG76w6ys+7UymzGwzrEsNNk7pzZt8OMkQghKgUEWIhxKpTYnMfEIQFW5gkK5FajqIs99toFvdLib0V3t6z9Mp+ThTW4oIBgN5xUfz74iH8++LWU2RCCOE9q0VnzogEPtt0wmGGwepuO6MnESEt8pTZOkV1Kp8w6OLCbNhqLhNO3wcnNqpU493PUEsevTX4Evj5r86f13S17yj/9mLLJ1sI0ardO603P+xMoaDYjt3J4qobJ3Tj/ul9/Nwy0ajC26lMobsXOg8IrKEw6GI1b+DLO+DoiqrnNB0GXQIXPqsmFXsqqiNMug9W/q/uc5quApTpf/fqrfhCi1paKIQQ9XEgNY8HF2xj+4mqPAMWDYZ3bcvjlw6hd5zkF2iRMo/AG9OgOLdWQFBeifSiF1TOgVfPUCnIa1c41CyQOFZNIPZm4rBhwPInVUBgK6p6PKYrzH5ZzUHyMwkGhBAtmmmaHs8V2p2cy/7UPEKDdCb2bk+b0KBGbp1ocpmH4ceHVNKhilUFsb3hrL/BoDmw/ClY8m/XEw2v/AgGXOD9sYtz4eAv6t92PdXwgN408/olGBBCtDhJmYW8seIwX24+SV6JjU7RoVw3vhs3TuxOpIz7C0fyTkHWMVWtsEP/qlwhzw2HrCPOX9dCioE1+2DANE2W7U/nux0paMDMwR05s28cuqvsQ0KIFmvnyRyuen0tRWX2GpMCdQ36xEUx/84JRIfJHb/w0GNdoNRNfomE0XDbYv+0p5E06xB5/6k8rnx9TY00ovM3niAmPIh5t42nf6fWVY9aBLDs46qSYWQ8xCQ2dWtaLMMwueejzRSV1p0MaJhwMD2f/3y/h/9eOrSJWiianaiOkJGP48REqJ6BFvA33WyTDqXnlXD+C47ziWcXljH7pVWk55U0QcuEqOb4OnjrHHh2CLw5HZ4dDO/MghObmrplLdLqQxkcyyx0uirAbph8sfkkucWu0w8LUWnUTa6fN+0w4jq/NKUxNdtg4G9f76DM7nyEo8Rm8NTP+/zYIiFqOboS3j1flSit7vgaeGcmHF/bNO1qwXaczMHiZrJgqd3gUFq+n1okmiVbCWybB+9eCBvfUcWDHCW613ToOxN6nuX3Jvpasw0GluxNd7vNd9tT/NASIRwwTVh4n1qKVHsWsmmoZCbf/k5tJ3wmyKJhuixirgRbm+2pTzS2oizVm/flHXBsJWQeBFsxdYYJrKEw9g644v0mWwHgS832HXiSPrS4zIOUj6LlCYQLbNJ6VRsdJ59T04C03ZC8xa/Naumm9Y9zW3QoLiqEfvGSN0A4sfA+OLVDfV0RyFf8q+nQazpc+zn8fj+c91+whjRNO32s2U4gDLJoLocJAKJCvXt7pmmy8VgWH609xt5TeUSFWrlgaGcuGZlAlKw3DmxJ62HV83DgZzDKIH4wjL8Lhl7VNFF71lHPt0sY6f3+DQNK8yEoXKokVtOrQyQzBsSxZG+603kDd07thdXSbO+DRGPKToI93+B0sqBpwJHlqkx5aLRfm9bYmu1fxDkD491uc8nILh7vzzRN/rFwF5e/uoZvtqew91QeG49m8Y+Fuzj7meUcPV3QkOaKxrTtU9Wtt/8HsJeoP9jUnfDVXaqrz/C8Kp3PhMV4uF1b7/ZbmAm//B2e6A7/TYR/d4Qvboe0vd62sMV65srhjOwWA1RVLK3495ZJPbh5UvcmapkIeMdW4TQQqGCUwUtjYc1LdTMSNmPNNs9ASnYRU59cQqmT3oE2oVY2/GUGIR6WHP143XEe+nKHw+csukbXduEsfmCq5C8INLnJaqa+YXO+zZxXYPg1/msTQFkxPN0XinOcbxPeHh7cCxYPe53y0+GtGerupXrqVN0CejDcuFClRhUYhsmKg6f5eutJcgrLSGwXzlVjE+nfUZYbCxe2fgJf3enhxpqqTXDpmy1izkCzDQYA9qfmcu0b60jPL63xeM8OESy4YwKxkZ6N5ZimyZlPLeV4RqHLmPCdm8cwrV9cA1osfG7pf2HZ485ThWq6GjK4c4Xj550xDHWXkJcCER1UmlBvu+PXvQY//NH58xc8C6Nv9nx/X9wOOz5zXFRFs0CbTvDbHS3ixNQS/LI7lbdXHmHz8Sx0TWNy7/bcekYPxvX0b2la4YXMw/D8CO9ec8UHquBRM9esBxv7xrdh/V9msOFoJkv2pmOxaMwZlkDv+Eiv9pOWV8KxjEKX21h1jdUHT0swEGhStrrOGV4xZGCaVelF3dn7PfzwB8g5UfVYZDyc828YernnbRt7u5qFvOTfaqmSblU9GNYwmPGwd4FAYSbs/Nx5dTXTrtp76FfoM8Pz/YpG8fiPe3ll6SF0jcoJjb/uS+OXPan8a85grh/fzfUORNNo1xN6n63+jlyVNq6gWWDDGxIMBAJN0xjbI5axPeofbRsedo64m6UsmoAlWN39uwoI9CDPA4H9P8E8B0MK+anwxa3qOMOu9GxfmgaTfquSluxeCPmnIKqzOnF4U/IUIOOg66EQUCem1J0SDLiQU1TGu6uO8vH6Y6TllRATFsRlo7owd3JPOkaH+uQYy/en88rSQ0DNc0ZFauS/f7WTCT3bSSXEQDXnFXh3Fpw+gNv5A6a9xczXkf5EIC4qlI5tXA8p2AyT0d28nOwlGl+fc9wEAlboe65n+zJN+OH/Kr5xvM1PD4HdTfa61N2w+kVY+SwcWQEhbWDk9TDlDzDiWu8DAVBrmt0xjfLkKMKR0/klzHlxJc8t3k9qbgmmCVmFZby96iiznlvBQR8lInpn9ZHKCYuO6LrGh2uP++RYohFEdoDblsCsJ9VqHXeCIxq/TX4gwQBqguAtk3s4yi8FqAIn8W1CmOHBCgbhZ4MvhahO6q7YEcMOE3/j2b5ObiqvTubibqDwNBxa4uS5THh/DrwyAX75Gyz+J7x3gZp5nLbHszY4Ez8I2nR2s5GpJhc2xeqJZuAfC3dxPKuoTg+f3TDJKS7jt/N8k/Nh6/HsGgWSarMbJpuOZfnkWKKW0kIoyGj4LP+QSBh7G0x/GIeZBytoFhhyWcOOFSAkGCh3y6QenDekI6Au/hV0DSJDrLx14xiCZG1y4AkKgxu+VpP8QA0ZVPyrW+Hi1zyfYZ+f6uF2p+o+Zi+DD+aoNcig7tIrxhwzDsE756mVD/WlW+CM37vfbs2L8M1v63+cFio9r4Tvd6Q4vUjbDZNdyblsS8pu8LE8OU+ESAZE30paDx9dDo91hid7whM9YdE/XK/mASjOVXOEdn0Jpw/WfX741Wq+kKObDc2igobRc33yFppas58z4CtWi86LV4/khyGn+HDtMfal5hEZYuWiYZ25bnw3n40nikbQoR/8divs/AIO/KQm63UeCSNvUDPsPRXZ0bPtohzsc+93kLLN8famXZ101r0GZz/ieXtqG32LCliWPe5iIxO2vK/mKXQZVf9jtTB7T+W6nfOjAdtP5jAsMaZBx5o+IJ75G5OcBh66BmcNkInIPrP3e/i0olBQ+c+8OFslIdv3I8z9qW6CILsNfv2n+pu0FVc93n0yzH4J2nZX34dGw03fwUeXqV5DvfySadjUDcg1n3p3jglgzXppoRA+ZZrwwii1vMjZUEFEB3hgT93cAPOug33fu56BHNkRfu+D4llPD4A8F70MuhVGXA8XPtvwY7UQaw9ncNXr7gtDPXHpUK4Y07BytAdS8zjvuRXYjbpVEnQNwoOtLPvDmR4vfRYulBbCU32gtACHf7OaBcbdATP/U/VYfhosuNlxgiHNAuGxailyVLWbA8MOB35RPX+mAV3HQf8LPM8R0gxIX5UQFTQNzqu463YyTnjufxyfAAoz3C9Fctdl6amCNNfPGzbIlglq1Q1PjCEqxHVHqK7BGX3bN/hYfeKjeOnakVgtWuWQo1b+X0SIlfduGSOBgK/s+kKl5XaaPtgOm99TScCKc+GLO+DpfqoAkaPXmHb1t7z6hZqP6xboNxNmPqbqEQy6uEUFAiDBgBA19TkbrpkPMV1rPh7VCS5723megdieVV2IDmnQtquL573gLid6xd2NqBQaZGHuGT2cPq9rcNGwznSK9s1qjHMHdWTVn87i/hl9mdKnPWf268BfLxjIyv87i1Hd2vnkGAI1MVd3c1EuLYD9P8Lr02D7fNerj6AqgGhlneYyZ0CI2vqeA723won1kHsSIuKg20R1d+DMyBthy4eu9zvqFt+0b9jVsPYV1wmIhl7hm2O1IL85qw8ns4pYsOkEFl3DbpiV/07oFctjlwzx6fHiokK5b3ofn+5T1BIUhttcAAALbvRuvyV5au5RUOuZKyZzBoTwBdNUpU+3vF/3Oc0CnYeriUi+yAOQmwyvTFTdnrUDAs2iVk/c9L2kJXZiW1I28zcmcTK7iPaRIVwyIoEJvWLRPE1MJQJH8lZ4fWrj7Hv4dXDhc62mKqgEA0L4imHA6ufVeGPhafWYNUwlHJr+sFqG5Cvp+2DBTZC2uzwDY/mf8YALYPbLECoFeUQr8f6c8ol9vq4gWJ5BtCErgJoRCQaE8DV7GaTuUjOQO/StX8ZBT5imWl+dvFlNZuo1Hdo5HxcXokUqzoF518LRFdWW/tnxaPjAHWso/P5AqwiuJRgQQgjRvFUExru/UuP9ZcWwc4Fv9n3F+zBwtm/2FcBkUFEIIUTzpmlq7f/M/0DPM+HkRt/tu6zY/TYtQOuYGSFEc2UrURUPd34GRVkQ20dlF0wc09QtEyLwLHtClQx3VU/AW/EDfbevACbDBEIEqvw0eO8iSN9TVaZZt6jx0NFzYdZTsmJAiAppe+Dl8b7bn2aBTsPgdieFyVoYOZMIEag+vR5O71dfVyRKqajGtvEtWPdq07RLiEC06V03ib+8oFvUxN85r/hmf82ABANCBKKTmyFprevlUqufa3ipViFairTdKhV3Q1lDVI6BO5ZBXP+G76+ZkDkDQgSiQ7+qbkpXwUDeKTh9oFWdsIRwKjiyajjNW5oOCaNVFcKQNq0m0VB10jMgRCAy7GqGtNvtfHAnJERLMHC2B4GAk78p04AzHoTwdq0yEADpGRDCf9L3w9YPITtJFRIaegV0GeP4ot9llPsLfUgUxPZqnLYK0dwMnANL/6P+vuqk6dZVQaOI9qreSEWvm2YBDDjnMVWVsBWT1QRCNDbThJ//CmteVBOTTFOdnAwb9J0Jl79bt2aBYcALIxyf2EC9fsI9cM6jfnkLQjQL2cfho8shfW/5ZEINjDIIjYErP4TEcbBnIez9FsoKIW6gKjImmTslGBCi0a1+QQUDjmg6DL0SLnawMuDUTnj3fJVRrTIgKO9F6Doerv/SN4WPhGhJDAMOL4GDi1Rq8IRRMGiO/K24IcGAEI3JXgZP94PCDOfbaDrcvxOiE+o+l5sM616D7Z+qKoXtusOYW2H4tWrWsxBC+IAEA0I0pqT18NbZ7re74FkYfXOjN0cIIRyR1QRCNCabh3nNt3ykhgWEEKIJSM+AEI0pPw2e7u++1rpWHpfPeQWGXdX47RLCl46sUBkxj61Wn+VeZ8H4uyBhZFO3THhIggEhGtv8m9QMZncBAagT6d3roEPfRm+WED6x4hlY/EhV3Qwo/9qA2S/BiGubtn3CIzJMIERjO+9xNTlQs3iwsabqDgjRHBxbrQIBqJka27ADJiy8V2XJ9FZBBmx8G5Y/CdvmQWmBT5ornJNgQIjGFhUPty2FSffhtrSqaYcjy/3RKiEabu2rqhfAKU1d1D1lmrDkMbUC59sHYOl/4cs74Kk+sPn9BjdXOCfBgBD+EBELM/4BbTq731aTP0vRTBxf47pYlmmHo6s839+yJ2DZ4ypREGZVFs7SAlj4G9j5eYOaK5yTs44Q/tRruusyq5oFek3zX3tEy2crhZTtkLINyjxc3eIp3YNLSEXPQX4aZB6GsiLH2xXnwMpnXO9r0SNqLoLwOalNIIQ/jb9T1SdwSFO9AqPn+rVJooWy22DF07DuFSjKUo+FRMPYW2Hqn8Aa3PBj9D4btn3ivI6GpkO7XvDmDDixQT0WFKZKBJ/5Z9VjVmHfj+6X4mYfg1PboPOIhrdd1CCrCYTwt23z4Ku7VIGiii5WzaJOnJe/CwMuaNLmNZhhh4OL4cT6qp6OxHGeVWEUvmEYsOAmtYqFWqd4TYeeZ6lyvQ2t0HdqJ7w2xclKGU31ghk29buvXlFQs0DbbnDrYlUpEFSmzR/+r257axt0Mcx4RL2+tqxjagLuwV/VfrqfoTJ2tu9dzzfYekgwIERTOH0ANryp1mdrmlqXPWYutO3u/b4KM2HrR5C0Tp1ke06FIVdASKTr15mm+s+Trl5PndoJ865Rd3AVwyGGDToNh6s+dpxyWfje/p/g4ytcb3PpWzDksoYfa8dnapKfaVYFBRVVAjHBXur4dZpFXahnPeF5myv2relw6ZsqMKiw9zuYf6MKOsxqQTYmzHkVhl1Z33fYKkgwIERzdmARzL9Oda+aoFYrGBDWFq77XBVpqe3YGlj9PBz4Wd3FdxwC4+6EYVc3LDDITYGXx9cqrFROs6hA565VUjDGHz6+Sv1+neW20CzQbQLc9J1vjpd1VK0aqJ50yBqixvhd3ekHhcMfj0BQqBrW+N9ANbfAXe8AqLkId66GuP7q+C+MLh+ucPBaTYc7VkDHwfV6e62BTCAUork6fRDmXa0mhZkm6iRY3hVbnAsfXKzWa1e35SN45zx1F1Zx4kzdCV/fDV/e3rDJWetfdxwIgHos8xDs/KL++xeeyzjgOsmVaYeMQ747XtvucPY/4dZFMPdnOPNP6qLuarIsqDLC+afU1xYrXPi86inzaEWNButfU19ufLt8GMJJEKHpahhCOCXBgBDN1frXqpK71Gba1YV5ywdVj+WcUMuzMGteKCrGcncsUNUR62v7fNcXIE2HnZ/Vf//Cc2HtcJvTIqxt47YhOBKP7vCDo6q+7jcTrv9KDSu5Y9jgwC/q64O/uv7sGTY4tNj9PlsxCQaEaK72fOPm7s9Q46gV3CVt0XSVX76+SnJdP28aavmYaHzuxsc1vfFrYAy8yPkqAygfqphYc0UBqDkvty+BWA8m/VV+/j3p0ZIRcVckGBCiubKXud/GVlL1dfJW98HDqR31b09sb9fdu7oVYvvUf//Cc0OvgpiujlNgaxaIjIcR1/v2mJlH1DDQ7oVqeKrjEOh3npM03OWrC6b+n/P99ZzmJieHDlGdVYDZ/QzX6b51i9pGOCXBgBDNVcJI9yfALtUmEFqDcfsnbwmqf3vGzK25fKw2wwajb67//oXnQiLh5u+h0zD1vWap+qzEDVDPVSzpa6i8U/DhpfD8cPjsZph/vUon/M39cNFL0Pfcqjbo5Z+voDC1GqDnmc73O+ZWN9kNDTi5UaUqtoS4+ezZYewdXr6x1kVWEwjRXB1YBB9d6nqbO1dVzaDe8iF8fY/zbXUrDLgILn+nfu2xl8FHl8ORZY5PzKNugQuekXwD/mSacHJTeb0LE7pOhK7jffc7KMqG16dCdpKDFSTl+QyuXQDpe1SPQWk+tO8Dgy+FkCiHu6xh07sqqNAtroccAIZeqea9aHrVthV5Ds57EsbdXo832HpIMCBEc2Wa8OOfVYY5Ta+6AGsWdWI+51GY+Juq7UsL4YWRapa3o+ECTYe5i2r2JnjLVqIqza1/A4qz1WNtOsPE38K4OyQQAMhNho3vwP4fwV4CXcbB2Nug09Cmbpn3lj8FS/7t+q78us+h94z6H+PkJljzUvlKFBeXq9BouO5LFUAcWkxl0qGxdzTsM91KSDAgRHNmmrDrS1j7suoyRYceU2DivY5PwOn74P3ZkJdSHkCYVUu5Ln7NN0loQOXDzzqi7szadndT2a4VObICPr5cBU0VF9CKu9dzH4MJLnpuGkNZkeo1KM6F2F4qza83AduzQ1WCKWd0CwycA5d5UbnQkcPL4P2L3G939adqRYLwmtQmEKI50zQYfIn6zzDKL+wuTuYd+sF9W1QAceBn1bXfeTiMuEGVWvYVa7A6lqhSmAmfXFkzEICqLu2fHlKT7npMafy2mCaseVFVCay+CiRuEMx+wXGyKkcK0lw/b9gh92T921mhoraCr7YTdUgwIERL4Wn2wKAwGH6N+k/4z5YPVZIdZ52xukV1h/sjGFj2OCz9T93H0/fAO7NU8qCOQ9zvJyLOdc+AZoE2PkhB7agOgcPtujf8WK2UrCYQQgh/OLLceSAA6i768LLGb0d+Oix/wvFzpqF6ixb/y7N9jbzB9XJS0+6boLPTcIgb6OJYOrTrqSZHinqRngEhmqvDy1QK4BMbwBIM/WbB2Nsbr0Lb6QOw83PVFRvTTc3erp0wRrjgyfQsP0zh2vm566DEtKshpILTENHe9b7G3KqyXLpaTdDzrIa3WdPggmfhvQvUsEqNCojlhYsufE4mqDaA9AwI0Rwt/peaULX/B8hPhZwkVQXxlQlVKVp9xVYKX94JL45WY8wb3oKf/wJP94W1r/j2WC1Z1wmu76I1C3Sb1PjtyD/lOj8FACYUpLvfV1gM3PKTKkxUPf2xHgQjb4KrPvJdVcyu4+DmH9XPsbouY1TBJX8Mr7RgsppAiOZm3w/wibNUspqqFve7Xe7v6jy18L7yVMZOThWXvAFDPSg929rlp8Gzg1Vw5exnee3n0KcBy/A8sfZV+OlPrnsH0OAPB737DGUegeQtanVEt0mN22uUfVwlO4qMcz1PwFaietCKMlVGxq4TpPfACQkGhGhu3j1flSF2Wp5Wh+l/h8m/a9hxTFPVKvjxT663a9tDrVAI9JNs6i5Vue7gIvWz6zZJlW5OHOu/Nuz/CT69Ts0PqPj9VSwtPPMhONNFel5fyU+HZ/o5z+6nWaDP2XBNA4pWBYINb8Hif1bluwAVOFzwLPSa1kSNClwSDAjR3PyrA9hLXW/T51y4dn7DjvPzX2H1C55te/2X5V3FAWr7AlWi2VF2On+v7884pIZ09n2vJut1Gauy43Wb6L82LHtCJQuqTbOo+Se3LqrKXNkcrX0VfnQQWGm6Clpv+Aa6+2FIphmRYECI5ubRuJoFiOrQVOKVq+fV/xjHVsM753nxAg1GXg+zni6vgRBAMg7Bi2NcF2m65Wc1Jt1amKZKVLXs8ZqVJOMHw0UvqLoXzVVJvqpXUFbo+HlNVzUbbl/q12YFOllNIERz030KHFyM87KtJnRrYIW2DW9V3Tl7xFTr6O1lcHEDyiA3ho1ust/pVjUc0pqCAU1TvSGj58LRFSrxULte6iIZ6MM97uz9znkgAGolQvIWOH2w8VbeNEOymkCI5mbivbit3149q1x9nNruRSBQzjRg2yfqJBtIjq5y3Stg2ODoSv+1J5AEhar5AYMvVZkom3sgAGp1jdvVEuXbiUoSDAjR3CSMdl9qeO3LKu98fQVH1O91mgV2fVH/4zYGT5a2uVryJ5qXqE6ug7/K7To2fluaEfkLEKK5ObJMdce7UpLbsLvdgbPrd4HU9MDLD99zmus7Rd0KvQN48qPwTv9ZroNZTVe5CWJ7+a9NzYAEA0I0N6UFHm6XX/9jjLhBlYT1pLu1OsPmfN13WZGaV/DuhfDyBJh3nUqQZLgZ8mio0TerCz5OusANu1piKFqG4Ag420k6ZU1Xn+lzHKykaOUkGBAiUKXvg/VvwLrXIWVb1ePt+3r2+vYNqBoYEQs3fgMRHdT3urX8guqGJQiGXF738fw0eG0KfH0PHFsJabvV0rqPLoMFN7nv6WiI6C5w5YdqyVz14Ea3qIvD7JfUxDnRcoyZq36vEXE1H4/tAzd83bomi3pIlhYKEWgKTsPnt8LhJVTdzZpqPfplb6uL22tTVBIdR2OjmkUtDbt1UcPaUVoIO7+Agz9Dbgq07armK6z4HxSernVsTbVx1lMw9ra6+3pnFiStdZLoRoMpf4Cz/uJ520xT9UK4mztRXXaSWllwaLHqjeg+SeXWb9/H8334mmGoORbrX1e/z6AwGDhH9VTITPeGs9vg2KqqDISdR7aMSZKNQIIBIQJJWTG8MQ1O76t74dQtanLUnatUOtZ3Zqrtq1+UNQsEh6t18/EDPT+uYVdd9qk7wBqqvl/+pBpq0K3lx9DVRWrcHbDoYdi9sOrYsX1g2kMw+JK6+z61A16d7Pr4IdHw+/1qdrsrmYdh1XOwfb5aPhbRAUbfAuPvVnnymxPDDp/fBrs+Vz0UFcV3dIv6mV+zAHpObdo2ilZD8gwIEUh2fq660B0x7JB7UtUJmHQf3LZU1aTf87V6TrfCoIvhzD97Nznq+DpYcCPkpZTnFrBTI3d+5RJDu1qloOlw+btQkAFZRyEkUg1dOLvjOrKi5sXOkZIcdWfcZZTzbZK3qlTMtqKqQKkgXQUtOz9XAZCn+fBtJWroIjgCwtt59hpf2/h21cqL6j8bw656Pj69Fh7Yq36+gejICvV5OLICMKH7ZBWUSQDTLEnPgBCB5L0L1SoAVxfODv3hnnVV35fkqxn84e28XxKYvg9emwr2EtfHrE63qotUZAfPtl/9IvzyN/f7n7sIEsc4fs4w4IWRqkfE2dDI0CvcJzwqylKpeDe/VzURs9skmPp/DbuI2W1q6MQa6lkPhWmq95N5BJdliy98DkbdVP92NZY1L8FPD6mfe2WNBYsKZGY8ApPvb9LmCe/JBEIhAklBuvuLZmFGze9DIiEmsX65AVY9C0ap54EAqBP+3m883z5xnPv9B4W7HtY4uhyyjjhfP27aYcdnUJjpfB9FWfDm2apYUfUVGcfXwvuz1eu9VVoAvz4KT/WGp/vB493g7ZnlGSJdKMlVQx6uAgHdCic2eNYOuw3yUqG4gcmmPJG8VQUCUPP3UdFbs+hhSPKw3SJgSDAgRCBp28P1cj5NVxOhfMEwYMfnzqvXOaPrNfPZu9NlNHQc6nw1gqbDqJtdBzOndrjPe2CUwen9zp9f+l91Aa4dUFR8v/BeKMlzfYzqSgvUsMWKp2vmVkhaDx9eqpZROuPpkk13KzhK8mDRI/BkL3i6L/w3US3dPLLcs/3Xx/o3XLdLt6oJkaJZkWBAiEAy8kbX2dNMQ02Y8wV7ifrPW4bddQ352jQNrnhfTfarfkGv+LrrBJj+N9f7sISornV3rCGOHy8rVnMtnP5sTbXNjgXuj1Fh1fNqyWftXg+zfM7FN/erlSGOhERCwijXAY5hc10JsiQP3j5P9e5UL9N7bBW8d5Gq1NgYkta6TlVt2CBpnfPnRUCSYECIQNLnHOh7nuOLhKZD4ngYcoX6Pm2vKjO84Cb47vfqjtSbKUDWUAhv72UDNQhrC/1mefeydj3grtVw1t/UZMPw9mqZ1+yX4fqv1JI6V/rMcH+MyHiIH+L4ufxU18VrQN3RprvoWajOMFQZYlfDH6Ydtn7k/PlJ9zt/vWaB6MSaP+f0/bDrK9j/s1r2ueIZSNvlPBhZeA8UZXv2fryhe7Cc05OcFCKgyG9MiECi6+ouesmj6mJTMbZtDYUR18PZj6gT7fd/hPWvlS/7M1SgsOEN6He+ykXgbokeqDv2MXNh+VOe5XLXdBVsXPic8ztwV8LbwRkPqP+81a6nSpG8Z6HzC+jk34HFySnNo/kUhufzLkpy1IRBVzRdTdB0ZuBFcNZf1ZyDygqR5fkaIuPg+i9VHoXTB9UQxvE1Va8NjlTbuwpGbKVqCea42z17T57qe64ajnH1mfE0MZYIGLKaQIhAVVoAKdvVCb/jYJUeGGDl/2DRPxy/RtNh+DUq+5oninPUpLqMg+4Dgk4jYMbfXXddN6aSfLXc7vDSqiWQukVdFCfcC+c86jqhzFvnqAl5ri6gd6yATkPdt6WsCP7dCbcTAEfdDOc/5Xpfqbtg4zuqUmRQuAp6hlyuhhKyj6sEU8W5ngVsNY4fBCOvhwv+593r3Mk+Di+OVsszXZn2F5j6R98eWzQaCQaEaE5sJWrWuqtiQJoFHtjteVW2wkw1CW37J1Un+OhEmPJ7NWSRe1INDbTr0fD256fD1o9h//dqzDtuIIyeC90mePZ601RLL3csKM8q1031mMT1d//ag4vVxD5HF3DNooKc67xYUfDhZXDoV9cX6Ru/gR5TPN9nbd/8Vk1E9LacNKhgZNydcK6DPPwnN8G+H1XOhvghKgDxpDepwsHF8PGVatKmK54GV6LJSTAgRHNybDW8c5777Wa/BCOu827fxTlqtr01VNU18KT0r6cOL1NDH0nraz5esTZ97O1w3hONnyp226fwzX0q6NGtQHla495nw+XvQEiU5/s6tgbenVU+T6PWaVSzQMIIlTuhvu/JXgb/6QK24vq9HlQipup5+AszYf4NcHSF+tmjqwt6aIwaXuo93fN9vzkDTmzEae+IblEFry58tv7tF34jcwaEaE48ujBo9buAhEZD5xHev86dLR+qAkWOVCxrXP86xA9q/AQ7w66EfjPVWPrpA6orfsBF0Hm49/vqNkFdQL+8szy4KF8uaNjUSoGr5zUsuCnJq38goFlUEJA4tuoxw4CPr4CTm8u/twPlP//iXHWnf9tiz4s2nd6Py2ESw66WhIpmQYIBIZqTuIHuU/tiOp9V7295qaqr2y1NLdUbeWPj9w6ERjsuplQfgy5Wwwvb50PaHrUqov8F0HW84/dR0RHryXsMiVKVFu2lzrfRdEBTn4fqwUjXcXDVxzWPc2SZiyRGBqDDymdVD4knrGGAq3wTmqqTIZoFCQaEaE6iOqoVA/u+d56Wt33fmneETWnLBx5mNzQh8xDknYI2nRq9WT7lSXCx/2dY/YLKAYAJXcaoSY8DLnQeGFiC1DLS7Z84TwxlGqo65YlNkL5XrYYYcJH6/dfe7+6vqq1acMCwqdUaFRMz3Rk4u3x5pYs5EwMucr8fERAkGBCiuZn1JKRsUWWFq5+IdQsERcClbwZOmVZnRZec8SYtsiP5aWp2viVYddV7Mymusax4Ghb/s2Ye/xMbYf71KtfA2Y84f+2U36vUzyX5Di66Goy8QQUWXZzUdKiuJN/9z9ewqSEPT+7ox9+pajzYzbr71SwQ0R6GXul+PyIgSNIhIZqbNp3g9mUw8Tdq4heo7umRN8Idy9UyxEBhDXOfRrhCmwRVork+8tNhwc3wdH/4YI6a2PdUX1j2pBorbyonN6tAAGpezCu+XvWsmlzpTLsecMtPdWfkW0NVXgVvlg3G9kblMXAhMt59AqjKtvWEaz9TOQ+gasii4uue01RwJpoFWU0gRHNmmmqSmTU0cHoDqtv7Pcy72oMNNTjnXyrAqc401SoHS5DzhEBFWfDGWZB1zPnd80XP16f1DffVPbB9nvOued0KfWfCVS4yFVZI2a6GAoLCoMdUCG3jXVuyk+C5Ic6zVGo6nPkQTP2Dd/stLVB5Lza8WXNlhWZRX89+GYZ78hkQTUl6BoRozjRNXRwCMRAAla2uQ3/3hXkGzobxd1d9b7fB2lfguaGqEuBjnVXSoL3f133t2lch66iTsWtTdWUnb23Am2iAkxvd5/E/ucmzfXUaqso0D7jQ+0AAVGXLsx8t/6bW50WzQMchMP4u7/dblAWb3qu7xNK0q+GDr+9WgYwIaBIMCCEaj25RaXWdJUDq0B8ufw8ue6fabHg7LLgBfvyzuputcGKD6mVY/ULNfWx6x/VYuG51XUGwMXmSttmb1M4leSpb4Q9/UsMPyVu8a8/Ee+Hyd9WqlAohUTDhbrjpO7XU0lsb3ykPeFz0OKx7zfv9Cr+SCYRCiMa1/g2VxbAi7z5UfT3oEhg0p+b22z6Bvd/V3U/FBf/nv6nMiO17q7vR/FTXxzdskHOiQW+h3vqdr9bauypI1P8Cz/a160v46m5VcEkPAkw1ObHXWSqg8rS3YNDFMHAO5CarIaY2CQ2baOkuC6Nhg0OL6r9/4RfSMyCEaDxHVsDKZ8q/qX7nWP710scgqdba9/Wvu550qOmqNwDU8EhFzQZndCtExHrTat8ZdZOqN+Dw/WiqsNKYW93v5+hKNUGyrEh9b5RVDT8cXqayCnpD0yA6AWJ7NXzFhScrQGRqWsCTYEAI0XjWv+F6voBuUdUWq0vb4740cOrOqu+HX+v6GIat6Za4RcXDdV+UpznWqs2411SQcPWnntV8WPZ4+WsdXFRNOxxe4vncA1/rNsnN79gK3Sf7rz2iXmSYQAhRf4YBhxZD0jp1Qeh5Zs3se8mb3XQh2+texKyhrrPuoasLaYUJ96ihBUeV/TQdepwJ3c/w/D156tQOVaI4KFwVI3I23t51HNy/E9a8pGoCWIKhzzlqhn1YW/fHKc6BI8tdb6NbYffXKreCv42ZC+tedf68YYOxd/ivPaJeJBgQQtRP6i6Yd42aya+Xn0qW/Vfltr/qY4ju4uEEulrd1ANnq4u701n4hppRXyG6i1qL/9kt5T0G5fMRNF2VAr7gWd+utkjdrWotJG+ueiwoHCbeB1P/r26Bp5Ob1GTIpHVVjxVmQPxAFTy5U1roQaM0lVSoKcT2gjmvwFd3qp95xe+tItvhuY/VLJYkApLkGRBCeC/vFLw83vHduG6B6K5w12oVHKx+0XnvgKbDlD/CtD9XPZa2B16bopYX4iCzXZvOcO+GuslxTFNl9kvZqoKQXtPVuLgvZRyC189Ua+sdvadxd8J5j1d9f2IjvDNLjfHXGPrQVcxy9Ty1/NIVexk80UOtJHBKg5n/VVkBm8qpHWrVwKHF6r12mwzj7gic1NjCJQkGhBDeW/wvWPk/10MAF72g7nxfHAv2Egcpa3V1Qb93U916BAcXqUlxpQVVvQ6GTWW9u+5z9W9T+OIO2LHAxfvW4L7NVe17bSqc2u5kDoSmllz+bpf7WgA//xXWvOz8uJYQeHAvhLfz9J0IUYNMIBRCeG/HfNeBABrs/BxiusK1C8rH+Csm0JVPoguJguu+dFyYqPcMeHAfnP8MDLsKRlyvhh7u2dB0gUBZkXpPrt63psO2T9XXqbtVL4XTyZAm5KXA4aXuj33G71U64doT9SomJF7wPwkERIPInAEhhPeKXXVZA5hQlK2+7HEGPLAbts0rr9qnqdnlw64qn2XvREiUmpzGXN+0uaGKc1R3vyuaBvmn1NfZxz3bryfbhcXA3J9h2RMqo2Jp+fyAhNFqnkKfGZ4dSwgnJBgQQngvtlf5SgEnd726Bdr3qfo+NFqNH49rxrPKQ6NVsh9XAYFpQmR5tkVP79Q93S4sBmY+BjMeVnM2giNUZUAhfECGCYQQ3hsz13UuAMMOo272X3v8ISgMhlzmenzfNGBYeU6DhNFqpYMrwZHQ+2zv2mENgbbdJBAQPiXBgBDCe0OuUGlwnWUKHHkjdJvo3zb5w9Q/qgu4syQ74+6smtOg6zDjEdf7m/YQBIe73kYIP5DVBEKI+rGVwoqn1HKy4mz1WFQnmPRblWSm9np7fyvJVxMd9/+kkhh1Gg6jboS23Ru237S9qhJf9WRJwREw6X410a/2+97yIfz4J7U0ULOoCYjWMLWccuJ9gVtxUrQqEgwIIRrGVgpZR9SFrl0P98vk/CF1N7w/GwrSqEpCZFH/XvgcjPQyl7/DY+yC9L0QFKEmSQZHON+2rEgVX8pNhogO0P/8+pUhFqKRSDAghGhZSgvh+eFQcNrJMkBNlevtPsnfLRMiYMmcASFEy7LrC1XW2FXWw9Uv+LdNQgQ4CQaEEC3LwUWuSyCbdjj4i5TVFaIaCQaEEC2Lvcz9hd60SzAgRDUSDAghWpbOI1zP0Nd06Di06Vc7CBFA5K9BCNGyjLyhfOWAk4DANGD83X5tkhCBTlYTCCFanl1fwmdzVQ+BYVOPaboKBIZfA7Nfdtx7YJpw6FfY8oGqGRARp2oo9L8ALJK9XbRcEgwIIVqm5C2w5iXY90N50qGhMPZOlVLYUSBgK4XPblL5ACqSA1X8mzAarv9C1ScQogWSYEAIIQB++iuseRFwcErULNB/Flz5od+bJYQ/yJwBIYTITYG1L+EwEADVO7DnW8g65tdmCeEvMggmhGgd8tPUXIC0PaoCYf8LofcMtargczdVGAEw4cgyaOuDVMZCBBgJBoQQLd/mD+Db+8sv+JqaM7D5fYgfArNfgGOrPNuP4SSroRDNnMwZEEK0bId+hQ8udvycboGozpCT5Nm+7lwFHQf7rm1CBAiZMyCEaNmWP12ed8ABw+55INC+nwQCosWSYEAI0XKV5MOxlc6LFoHzQKG2i1/zTZuECEASDAghWi5bifttNA0iO7oICjQYcBEkjPBp04QIJBIMCCFarrC2KougK4YdxsyF6IRa1Q419V/cALjo+cZspRBNToIBIUTLpesw9nYXJY01sIaobe5YDtP/Du16QUgb6NAfznscbl2kggohWjBZTSCEaNnKiuGDOZC0rmYugYphgcvehkFzmqJlQgQMCQaEEC1fWTGsewXWvw65yaqnoO9MmPw7SBzb1K0ToslJMCCEaD1ME8qKwBIsVQiFqEaCASGEEKKVkwmEQgghRCsnwYAQQgjRykkwIIQQQrRyEgwIIYQQrZwEA0IIIUQrJ8GAEEII0cpJMCCEEEK0chIMCCGEEK2cBANCCCFEKyfBgBBCCNHKSTAghBBCtHISDAghhBCtnAQDQgghRCsnwYAQQgjRykkwIIQQQrRyEgwIIYQQrZwEA0IIIUQrJ8GAEEII0cpJMCCEEEK0chIMCCGEEK3c/wO0oBvlRnJgeQAAAABJRU5ErkJggg==\n"
},
"metadata": {}
}
],
"source": [
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"from sklearn.datasets import make_moons\n",
"\n",
"# Set random seeds\n",
"torch.manual_seed(42)\n",
"np.random.seed(42)\n",
"\n",
"X, y = make_moons(n_samples=200, noise=0.1)\n",
"y_ = torch.unsqueeze(torch.tensor(y), 1) # used for one-hot encoded labels\n",
"y_hot = torch.scatter(torch.zeros((200, 2)), 1, y_, 1)\n",
"\n",
"c = [\"#1f77b4\" if y_ == 0 else \"#ff7f0e\" for y_ in y] # colours for each class\n",
"plt.axis(\"off\")\n",
"plt.scatter(X[:, 0], X[:, 1], c=c)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "QwENIn2CYG8e"
},
"source": [
"Defining a QNode\n",
"================\n",
"\n",
"Our next step is to define the QNode that we want to interface with\n",
"`torch.nn`. Any combination of device, operations and measurements that\n",
"is valid in PennyLane can be used to compose the QNode. However, the\n",
"QNode arguments must satisfy additional `conditions\n",
"<code/api/pennylane.qnn.TorchLayer>`{.interpreted-text role=\"doc\"}\n",
"including having an argument called `inputs`. All other arguments must\n",
"be arrays or tensors and are treated as trainable weights in the model.\n",
"We fix a two-qubit QNode using the\n",
"`default.qubit <code/api/pennylane.devices.default_qubit.DefaultQubit>`{.interpreted-text\n",
"role=\"doc\"} simulator and operations from the\n",
"`templates <introduction/templates>`{.interpreted-text role=\"doc\"}\n",
"module.\n"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"id": "TO-sU02MYG8f"
},
"outputs": [],
"source": [
"import pennylane as qml\n",
"\n",
"n_qubits = 10\n",
"dev = qml.device(\"default.qubit\", wires=n_qubits)\n",
"\n",
"@qml.qnode(dev)\n",
"def qnode(inputs, weights):\n",
" qml.AngleEmbedding(inputs, wires=range(n_qubits), rotation='Y')\n",
" qml.AngleEmbedding(inputs, wires=range(n_qubits), rotation='Y')\n",
" qml.AngleEmbedding(inputs, wires=range(n_qubits), rotation='Y')\n",
" qml.AngleEmbedding(inputs, wires=range(n_qubits), rotation='Y')\n",
" qml.RandomLayers(weights, wires=range(n_qubits), ratio_imprim=0, rotations=[qml.RY], seed=42)\n",
" return [qml.expval(qml.PauliZ(wires=i)) for i in range(n_qubits)]\n",
"weights = np.array([1,2,3,4,5,6,7,8,9,10]);"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vwo-ucc4YG8g"
},
"source": [
"Interfacing with Torch\n",
"======================\n",
"\n",
"With the QNode defined, we are ready to interface with `torch.nn`. This\n",
"is achieved using the `~pennylane.qnn.TorchLayer`{.interpreted-text\n",
"role=\"class\"} class of the `~pennylane.qnn`{.interpreted-text\n",
"role=\"mod\"} module, which converts the QNode to the elementary building\n",
"block of `torch.nn`: a *layer*. We shall see in the following how the\n",
"resultant layer can be combined with other well-known neural network\n",
"layers to form a hybrid model.\n",
"\n",
"We must first define the `weight_shapes` dictionary. Recall that all of\n",
"the arguments of the QNode (except the one named `inputs`) are treated\n",
"as trainable weights. For the QNode to be successfully converted to a\n",
"layer in `torch.nn`, we need to provide the details of the shape of each\n",
"trainable weight for them to be initialized. The `weight_shapes`\n",
"dictionary maps from the argument names of the QNode to corresponding\n",
"shapes:\n"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"id": "O-RFeXqmYG8h"
},
"outputs": [],
"source": [
"n_layers = 5\n",
"weight_shapes = {\"weights\": (n_layers, n_qubits)}"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1uMLnLnGYG8h"
},
"source": [
"In our example, the `weights` argument of the QNode is trainable and has\n",
"shape given by `(n_layers, n_qubits)`, which is passed to\n",
"`~pennylane.templates.layers.BasicEntanglerLayers`{.interpreted-text\n",
"role=\"func\"}.\n",
"\n",
"Now that `weight_shapes` is defined, it is easy to then convert the\n",
"QNode:\n"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"id": "m5fcIRAWYG8i"
},
"outputs": [],
"source": [
"qlayer = qml.qnn.TorchLayer(qnode, weight_shapes)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "p9-wiN-0YG8i"
},
"source": [
"With this done, the QNode can now be treated just like any other\n",
"`torch.nn` layer and we can proceed using the familiar Torch workflow.\n",
"\n",
"Creating a hybrid model\n",
"=======================\n",
"\n",
"Let\\'s create a basic three-layered hybrid model consisting of:\n",
"\n",
"1. a 2-neuron fully connected classical layer\n",
"2. our 2-qubit QNode converted into a layer\n",
"3. another 2-neuron fully connected classical layer\n",
"4. a softmax activation to convert to a probability vector\n",
"\n",
"A diagram of the model can be seen in the figure below.\n",
"\n",
"{.align-center\n",
"width=\"100.0%\"}\n",
"\n",
"We can construct the model using the\n",
"[Sequential](https://pytorch.org/docs/stable/generated/torch.nn.Sequential.html)\n",
"API:\n"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"id": "Pgc-TkPpYG8j"
},
"outputs": [],
"source": [
"clayer_1 = torch.nn.Linear(2, 10)\n",
"clayer_2 = torch.nn.Linear(10, 2)\n",
"softmax = torch.nn.Softmax(dim=1)\n",
"layers = [clayer_1, qlayer, clayer_2, softmax]\n",
"model = torch.nn.Sequential(*layers)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "suZkU2tEYG8j"
},
"source": [
"Training the model\n",
"==================\n",
"\n",
"We can now train our hybrid model on the classification dataset using\n",
"the usual Torch approach. We\\'ll use the standard\n",
"[SGD](https://pytorch.org/docs/stable/optim.html#torch.optim.SGD)\n",
"optimizer and the mean absolute error loss function:\n"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"id": "57MwgNnEYG8j"
},
"outputs": [],
"source": [
"opt = torch.optim.SGD(model.parameters(), lr=0.2)\n",
"loss_func = torch.nn.L1Loss()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6Y_yohUlYG8j"
},
"source": [
"Note that there are more advanced combinations of optimizer and loss\n",
"function, but here we are focusing on the basics.\n",
"\n",
"The model is now ready to be trained!\n"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"id": "NtA9KgZ-YG8k",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 0
},
"outputId": "e3926978-1c1d-4f28-dae4-d82a2953c7b2"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Train - Epoch 1: Loss: 0.1927\n",
"Train Accuracy: 94.00%\n",
"Test - Epoch 1: Loss: 0.0990\n",
"Test Accuracy: 94.00%\n",
"Train - Epoch 2: Loss: 0.0650\n",
"Train Accuracy: 98.00%\n",
"Test - Epoch 2: Loss: 0.0470\n",
"Test Accuracy: 98.00%\n",
"Train - Epoch 3: Loss: 0.0380\n",
"Train Accuracy: 98.00%\n",
"Test - Epoch 3: Loss: 0.0369\n",
"Test Accuracy: 98.00%\n",
"Train - Epoch 4: Loss: 0.0313\n",
"Train Accuracy: 98.00%\n",
"Test - Epoch 4: Loss: 0.0289\n",
"Test Accuracy: 98.00%\n",
"Train - Epoch 5: Loss: 0.0273\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 5: Loss: 0.0226\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 6: Loss: 0.0249\n",
"Train Accuracy: 98.50%\n",
"Test - Epoch 6: Loss: 0.0213\n",
"Test Accuracy: 98.50%\n",
"Train - Epoch 7: Loss: 0.0182\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 7: Loss: 0.0226\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 8: Loss: 0.0185\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 8: Loss: 0.0224\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 9: Loss: 0.0203\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 9: Loss: 0.0144\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 10: Loss: 0.0169\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 10: Loss: 0.0157\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 11: Loss: 0.0153\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 11: Loss: 0.0155\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 12: Loss: 0.0193\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 12: Loss: 0.0121\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 13: Loss: 0.0145\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 13: Loss: 0.0177\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 14: Loss: 0.0154\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 14: Loss: 0.0125\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 15: Loss: 0.0169\n",
"Train Accuracy: 98.50%\n",
"Test - Epoch 15: Loss: 0.0284\n",
"Test Accuracy: 98.50%\n",
"Train - Epoch 16: Loss: 0.0131\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 16: Loss: 0.0104\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 17: Loss: 0.0133\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 17: Loss: 0.0104\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 18: Loss: 0.0110\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 18: Loss: 0.0156\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 19: Loss: 0.0107\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 19: Loss: 0.0109\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 20: Loss: 0.0150\n",
"Train Accuracy: 99.00%\n",
"Test - Epoch 20: Loss: 0.0117\n",
"Test Accuracy: 99.00%\n",
"Train - Epoch 21: Loss: 0.0170\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 21: Loss: 0.0097\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 22: Loss: 0.0124\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 22: Loss: 0.0089\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 23: Loss: 0.0155\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 23: Loss: 0.0083\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 24: Loss: 0.0097\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 24: Loss: 0.0102\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 25: Loss: 0.0078\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 25: Loss: 0.0083\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 26: Loss: 0.0090\n",
"Train Accuracy: 99.00%\n",
"Test - Epoch 26: Loss: 0.0180\n",
"Test Accuracy: 99.00%\n",
"Train - Epoch 27: Loss: 0.0117\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 27: Loss: 0.0077\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 28: Loss: 0.0093\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 28: Loss: 0.0077\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 29: Loss: 0.0079\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 29: Loss: 0.0075\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 30: Loss: 0.0100\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 30: Loss: 0.0087\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 31: Loss: 0.0079\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 31: Loss: 0.0103\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 32: Loss: 0.0082\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 32: Loss: 0.0070\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 33: Loss: 0.0089\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 33: Loss: 0.0085\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 34: Loss: 0.0082\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 34: Loss: 0.0081\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 35: Loss: 0.0064\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 35: Loss: 0.0101\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 36: Loss: 0.0097\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 36: Loss: 0.0081\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 37: Loss: 0.0090\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 37: Loss: 0.0109\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 38: Loss: 0.0153\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 38: Loss: 0.0056\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 39: Loss: 0.0067\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 39: Loss: 0.0062\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 40: Loss: 0.0086\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 40: Loss: 0.0086\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 41: Loss: 0.0075\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 41: Loss: 0.0055\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 42: Loss: 0.0060\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 42: Loss: 0.0083\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 43: Loss: 0.0090\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 43: Loss: 0.0053\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 44: Loss: 0.0070\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 44: Loss: 0.0052\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 45: Loss: 0.0091\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 45: Loss: 0.0115\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 46: Loss: 0.0067\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 46: Loss: 0.0057\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 47: Loss: 0.0084\n",
"Train Accuracy: 98.50%\n",
"Test - Epoch 47: Loss: 0.0157\n",
"Test Accuracy: 98.50%\n",
"Train - Epoch 48: Loss: 0.0085\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 48: Loss: 0.0055\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 49: Loss: 0.0071\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 49: Loss: 0.0053\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 50: Loss: 0.0065\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 50: Loss: 0.0061\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 51: Loss: 0.0059\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 51: Loss: 0.0043\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 52: Loss: 0.0095\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 52: Loss: 0.0048\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 53: Loss: 0.0067\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 53: Loss: 0.0058\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 54: Loss: 0.0151\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 54: Loss: 0.0061\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 55: Loss: 0.0065\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 55: Loss: 0.0053\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 56: Loss: 0.0066\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 56: Loss: 0.0046\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 57: Loss: 0.0068\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 57: Loss: 0.0048\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 58: Loss: 0.0067\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 58: Loss: 0.0058\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 59: Loss: 0.0076\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 59: Loss: 0.0049\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 60: Loss: 0.0068\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 60: Loss: 0.0044\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 61: Loss: 0.0065\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 61: Loss: 0.0042\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 62: Loss: 0.0069\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 62: Loss: 0.0064\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 63: Loss: 0.0083\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 63: Loss: 0.0056\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 64: Loss: 0.0078\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 64: Loss: 0.0046\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 65: Loss: 0.0066\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 65: Loss: 0.0039\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 66: Loss: 0.0059\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 66: Loss: 0.0039\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 67: Loss: 0.0070\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 67: Loss: 0.0049\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 68: Loss: 0.0055\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 68: Loss: 0.0040\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 69: Loss: 0.0057\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 69: Loss: 0.0036\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 70: Loss: 0.0052\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 70: Loss: 0.0076\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 71: Loss: 0.0098\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 71: Loss: 0.0054\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 72: Loss: 0.0071\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 72: Loss: 0.0041\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 73: Loss: 0.0055\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 73: Loss: 0.0042\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 74: Loss: 0.0044\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 74: Loss: 0.0138\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 75: Loss: 0.0083\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 75: Loss: 0.0059\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 76: Loss: 0.0057\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 76: Loss: 0.0036\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 77: Loss: 0.0056\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 77: Loss: 0.0031\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 78: Loss: 0.0077\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 78: Loss: 0.0034\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 79: Loss: 0.0038\n",
"Train Accuracy: 98.50%\n",
"Test - Epoch 79: Loss: 0.0311\n",
"Test Accuracy: 98.50%\n",
"Train - Epoch 80: Loss: 0.0038\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 80: Loss: 0.0047\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 81: Loss: 0.0061\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 81: Loss: 0.0038\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 82: Loss: 0.0059\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 82: Loss: 0.0032\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 83: Loss: 0.0134\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 83: Loss: 0.0058\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 84: Loss: 0.0065\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 84: Loss: 0.0058\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 85: Loss: 0.0057\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 85: Loss: 0.0051\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 86: Loss: 0.0056\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 86: Loss: 0.0044\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 87: Loss: 0.0056\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 87: Loss: 0.0038\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 88: Loss: 0.0056\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 88: Loss: 0.0060\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 89: Loss: 0.0143\n",
"Train Accuracy: 99.00%\n",
"Test - Epoch 89: Loss: 0.0135\n",
"Test Accuracy: 99.00%\n",
"Train - Epoch 90: Loss: 0.0095\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 90: Loss: 0.0036\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 91: Loss: 0.0054\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 91: Loss: 0.0034\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 92: Loss: 0.0047\n",
"Train Accuracy: 96.00%\n",
"Test - Epoch 92: Loss: 0.0392\n",
"Test Accuracy: 96.00%\n",
"Train - Epoch 93: Loss: 0.0098\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 93: Loss: 0.0079\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 94: Loss: 0.0057\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 94: Loss: 0.0049\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 95: Loss: 0.0046\n",
"Train Accuracy: 99.00%\n",
"Test - Epoch 95: Loss: 0.0219\n",
"Test Accuracy: 99.00%\n",
"Train - Epoch 96: Loss: 0.0119\n",
"Train Accuracy: 98.00%\n",
"Test - Epoch 96: Loss: 0.0271\n",
"Test Accuracy: 98.00%\n",
"Train - Epoch 97: Loss: 0.0077\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 97: Loss: 0.0032\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 98: Loss: 0.0068\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 98: Loss: 0.0048\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 99: Loss: 0.0054\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 99: Loss: 0.0029\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 100: Loss: 0.0075\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 100: Loss: 0.0051\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 101: Loss: 0.0076\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 101: Loss: 0.0033\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 102: Loss: 0.0049\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 102: Loss: 0.0046\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 103: Loss: 0.0048\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 103: Loss: 0.0037\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 104: Loss: 0.0050\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 104: Loss: 0.0052\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 105: Loss: 0.0053\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 105: Loss: 0.0045\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 106: Loss: 0.0061\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 106: Loss: 0.0047\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 107: Loss: 0.0073\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 107: Loss: 0.0031\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 108: Loss: 0.0049\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 108: Loss: 0.0032\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 109: Loss: 0.0048\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 109: Loss: 0.0066\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 110: Loss: 0.0200\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 110: Loss: 0.0074\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 111: Loss: 0.0072\n",
"Train Accuracy: 99.00%\n",
"Test - Epoch 111: Loss: 0.0098\n",
"Test Accuracy: 99.00%\n",
"Train - Epoch 112: Loss: 0.0051\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 112: Loss: 0.0071\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 113: Loss: 0.0127\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 113: Loss: 0.0027\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 114: Loss: 0.0042\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 114: Loss: 0.0027\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 115: Loss: 0.0050\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 115: Loss: 0.0022\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 116: Loss: 0.0028\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 116: Loss: 0.0040\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 117: Loss: 0.0091\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 117: Loss: 0.0055\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 118: Loss: 0.0056\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 118: Loss: 0.0047\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 119: Loss: 0.0054\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 119: Loss: 0.0028\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 120: Loss: 0.0063\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 120: Loss: 0.0026\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 121: Loss: 0.0108\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 121: Loss: 0.0047\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 122: Loss: 0.0057\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 122: Loss: 0.0084\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 123: Loss: 0.0095\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 123: Loss: 0.0045\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 124: Loss: 0.0076\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 124: Loss: 0.0026\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 125: Loss: 0.0076\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 125: Loss: 0.0053\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 126: Loss: 0.0086\n",
"Train Accuracy: 99.00%\n",
"Test - Epoch 126: Loss: 0.0128\n",
"Test Accuracy: 99.00%\n",
"Train - Epoch 127: Loss: 0.0111\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 127: Loss: 0.0064\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 128: Loss: 0.0089\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 128: Loss: 0.0051\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 129: Loss: 0.0059\n",
"Train Accuracy: 98.50%\n",
"Test - Epoch 129: Loss: 0.0146\n",
"Test Accuracy: 98.50%\n",
"Train - Epoch 130: Loss: 0.0059\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 130: Loss: 0.0039\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 131: Loss: 0.0049\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 131: Loss: 0.0026\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 132: Loss: 0.0046\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 132: Loss: 0.0041\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 133: Loss: 0.0063\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 133: Loss: 0.0077\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 134: Loss: 0.0059\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 134: Loss: 0.0028\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 135: Loss: 0.0039\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 135: Loss: 0.0144\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 136: Loss: 0.0048\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 136: Loss: 0.0135\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 137: Loss: 0.0062\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 137: Loss: 0.0050\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 138: Loss: 0.0055\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 138: Loss: 0.0028\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 139: Loss: 0.0050\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 139: Loss: 0.0042\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 140: Loss: 0.0067\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 140: Loss: 0.0035\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 141: Loss: 0.0044\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 141: Loss: 0.0020\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 142: Loss: 0.0049\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 142: Loss: 0.0060\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 143: Loss: 0.0057\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 143: Loss: 0.0043\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 144: Loss: 0.0051\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 144: Loss: 0.0040\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 145: Loss: 0.0112\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 145: Loss: 0.0071\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 146: Loss: 0.0075\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 146: Loss: 0.0036\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 147: Loss: 0.0045\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 147: Loss: 0.0080\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 148: Loss: 0.0125\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 148: Loss: 0.0048\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 149: Loss: 0.0053\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 149: Loss: 0.0033\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 150: Loss: 0.0054\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 150: Loss: 0.0024\n",
"Test Accuracy: 100.00%\n"
]
}
],
"source": [
"X = torch.tensor(X, requires_grad=True).float()\n",
"y_hot = y_hot.float()\n",
"\n",
"batch_size = 5\n",
"batches = 200 // batch_size\n",
"\n",
"train_dataset = TensorDataset(X, y_hot)\n",
"train_loader = DataLoader(train_dataset, batch_size=5, shuffle=True, drop_last=True)\n",
"\n",
"epochs = 150\n",
"\n",
"for epoch in range(epochs):\n",
" train_running_loss = 0\n",
"\n",
" for xs, ys in train_loader:\n",
" opt.zero_grad()\n",
"\n",
" loss_evaluated = loss_func(model(xs), ys)\n",
" loss_evaluated.backward()\n",
"\n",
" opt.step()\n",
"\n",
" train_running_loss += loss_evaluated.item()\n",
"\n",
" train_avg_loss = train_running_loss / batches\n",
" print(f\"Train - Epoch {epoch + 1}: Loss: {train_avg_loss:.4f}\")\n",
"\n",
" # Calculate train accuracy\n",
" y_pred_train = model(X)\n",
" predictions_train = torch.argmax(y_pred_train, axis=1).detach().numpy()\n",
"\n",
" correct_train = [1 if p == p_true else 0 for p, p_true in zip(predictions_train, y)]\n",
" train_accuracy = sum(correct_train) / len(correct_train)\n",
" print(f\"Train Accuracy: {train_accuracy * 100:.2f}%\")\n",
"\n",
" # Calculate test loss and accuracy\n",
" test_loss = loss_func(model(X), y_hot).item()\n",
" print(f\"Test - Epoch {epoch + 1}: Loss: {test_loss:.4f}\")\n",
"\n",
" y_pred_test = model(X)\n",
" predictions_test = torch.argmax(y_pred_test, axis=1).detach().numpy()\n",
"\n",
" correct_test = [1 if p == p_true else 0 for p, p_true in zip(predictions_test, y)]\n",
" test_accuracy = sum(correct_test) / len(correct_test)\n",
" print(f\"Test Accuracy: {test_accuracy * 100:.2f}%\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "orKWhPb5YG8k"
},
"source": [
"How did we do? The model looks to have successfully trained and the\n",
"accuracy is reasonably high. In practice, we would aim to push the\n",
"accuracy higher by thinking carefully about the model design and the\n",
"choice of hyperparameters such as the learning rate.\n",
"\n",
"Creating non-sequential models\n",
"==============================\n",
"\n",
"The model we created above was composed of a sequence of classical and\n",
"quantum layers. This type of model is very common and is suitable in a\n",
"lot of situations. However, in some cases we may want a greater degree\n",
"of control over how the model is constructed, for example when we have\n",
"multiple inputs and outputs or when we want to distribute the output of\n",
"one layer into multiple subsequent layers.\n",
"\n",
"Suppose we want to make a hybrid model consisting of:\n",
"\n",
"1. a 4-neuron fully connected classical layer\n",
"2. a 2-qubit quantum layer connected to the first two neurons of the\n",
" previous classical layer\n",
"3. a 2-qubit quantum layer connected to the second two neurons of the\n",
" previous classical layer\n",
"4. a 2-neuron fully connected classical layer which takes a\n",
" 4-dimensional input from the combination of the previous quantum\n",
" layers\n",
"5. a softmax activation to convert to a probability vector\n",
"\n",
"A diagram of the model can be seen in the figure below.\n",
"\n",
"{.align-center\n",
"width=\"100.0%\"}\n",
"\n",
"This model can also be constructed by creating a new class that inherits\n",
"from the `torch.nn`\n",
"[Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) and\n",
"overriding the `forward()` method:\n"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"id": "3nRj296bYG8k"
},
"outputs": [],
"source": [
"class HybridModel(torch.nn.Module):\n",
" def __init__(self):\n",
" super().__init__()\n",
" self.clayer_1 = torch.nn.Linear(2, 20)\n",
" self.qlayer_1 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
" self.qlayer_2 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
" self.clayer_2 = torch.nn.Linear(20, 2)\n",
" self.softmax = torch.nn.Softmax(dim=1)\n",
"\n",
" def forward(self, x):\n",
" x = self.clayer_1(x)\n",
" x_1, x_2 = torch.split(x, 10, dim=1)\n",
" x_1 = self.qlayer_1(x_1)\n",
" x_2 = self.qlayer_2(x_2)\n",
" x = torch.cat([x_1, x_2], axis=1)\n",
" x = self.clayer_2(x)\n",
" return self.softmax(x)\n",
"\n",
"model = HybridModel()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XU2-uOcYYG8l"
},
"source": [
"As a final step, let\\'s train the model to check if it\\'s working:\n"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 0
},
"id": "0TOzLpJBYG8l",
"outputId": "2b4ea03a-8507-4f12-e92c-84430533596d"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Train - Epoch 1: Loss: 0.1588\n",
"Train Accuracy: 98.00%\n",
"Test - Epoch 1: Loss: 0.0555\n",
"Test Accuracy: 98.00%\n",
"Train - Epoch 2: Loss: 0.0420\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 2: Loss: 0.0258\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 3: Loss: 0.0231\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 3: Loss: 0.0190\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 4: Loss: 0.0173\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 4: Loss: 0.0147\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 5: Loss: 0.0153\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 5: Loss: 0.0122\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 6: Loss: 0.0132\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 6: Loss: 0.0123\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 7: Loss: 0.0122\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 7: Loss: 0.0098\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 8: Loss: 0.0103\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 8: Loss: 0.0095\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 9: Loss: 0.0097\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 9: Loss: 0.0085\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 10: Loss: 0.0090\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 10: Loss: 0.0083\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 11: Loss: 0.0087\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 11: Loss: 0.0077\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 12: Loss: 0.0085\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 12: Loss: 0.0075\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 13: Loss: 0.0092\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 13: Loss: 0.0071\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 14: Loss: 0.0082\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 14: Loss: 0.0070\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 15: Loss: 0.0078\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 15: Loss: 0.0067\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 16: Loss: 0.0077\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 16: Loss: 0.0061\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 17: Loss: 0.0070\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 17: Loss: 0.0060\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 18: Loss: 0.0084\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 18: Loss: 0.0060\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 19: Loss: 0.0068\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 19: Loss: 0.0054\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 20: Loss: 0.0060\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 20: Loss: 0.0053\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 21: Loss: 0.0066\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 21: Loss: 0.0053\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 22: Loss: 0.0067\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 22: Loss: 0.0051\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 23: Loss: 0.0062\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 23: Loss: 0.0049\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 24: Loss: 0.0060\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 24: Loss: 0.0048\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 25: Loss: 0.0064\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 25: Loss: 0.0053\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 26: Loss: 0.0059\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 26: Loss: 0.0043\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 27: Loss: 0.0051\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 27: Loss: 0.0043\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 28: Loss: 0.0059\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 28: Loss: 0.0041\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 29: Loss: 0.0054\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 29: Loss: 0.0038\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 30: Loss: 0.0049\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 30: Loss: 0.0091\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 31: Loss: 0.0040\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 31: Loss: 0.0044\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 32: Loss: 0.0057\n",
"Train Accuracy: 99.50%\n",
"Test - Epoch 32: Loss: 0.0043\n",
"Test Accuracy: 99.50%\n",
"Train - Epoch 33: Loss: 0.0059\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 33: Loss: 0.0035\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 34: Loss: 0.0051\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 34: Loss: 0.0036\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 35: Loss: 0.0055\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 35: Loss: 0.0041\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 36: Loss: 0.0053\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 36: Loss: 0.0036\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 37: Loss: 0.0053\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 37: Loss: 0.0039\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 38: Loss: 0.0043\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 38: Loss: 0.0033\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 39: Loss: 0.0051\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 39: Loss: 0.0029\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 40: Loss: 0.0050\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 40: Loss: 0.0028\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 41: Loss: 0.0040\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 41: Loss: 0.0056\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 42: Loss: 0.0060\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 42: Loss: 0.0026\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 43: Loss: 0.0039\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 43: Loss: 0.0028\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 44: Loss: 0.0046\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 44: Loss: 0.0026\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 45: Loss: 0.0046\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 45: Loss: 0.0031\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 46: Loss: 0.0028\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 46: Loss: 0.0027\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 47: Loss: 0.0035\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 47: Loss: 0.0025\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 48: Loss: 0.0041\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 48: Loss: 0.0029\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 49: Loss: 0.0036\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 49: Loss: 0.0037\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 50: Loss: 0.0038\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 50: Loss: 0.0038\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 51: Loss: 0.0035\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 51: Loss: 0.0021\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 52: Loss: 0.0028\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 52: Loss: 0.0088\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 53: Loss: 0.0042\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 53: Loss: 0.0022\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 54: Loss: 0.0020\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 54: Loss: 0.0029\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 55: Loss: 0.0036\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 55: Loss: 0.0024\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 56: Loss: 0.0030\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 56: Loss: 0.0019\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 57: Loss: 0.0032\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 57: Loss: 0.0019\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 58: Loss: 0.0038\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 58: Loss: 0.0021\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 59: Loss: 0.0034\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 59: Loss: 0.0033\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 60: Loss: 0.0042\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 60: Loss: 0.0039\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 61: Loss: 0.0041\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 61: Loss: 0.0017\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 62: Loss: 0.0029\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 62: Loss: 0.0017\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 63: Loss: 0.0030\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 63: Loss: 0.0016\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 64: Loss: 0.0029\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 64: Loss: 0.0017\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 65: Loss: 0.0025\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 65: Loss: 0.0021\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 66: Loss: 0.0029\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 66: Loss: 0.0017\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 67: Loss: 0.0034\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 67: Loss: 0.0015\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 68: Loss: 0.0027\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 68: Loss: 0.0015\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 69: Loss: 0.0028\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 69: Loss: 0.0024\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 70: Loss: 0.0026\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 70: Loss: 0.0016\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 71: Loss: 0.0023\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 71: Loss: 0.0015\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 72: Loss: 0.0023\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 72: Loss: 0.0014\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 73: Loss: 0.0018\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 73: Loss: 0.0021\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 74: Loss: 0.0030\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 74: Loss: 0.0013\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 75: Loss: 0.0024\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 75: Loss: 0.0016\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 76: Loss: 0.0019\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 76: Loss: 0.0019\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 77: Loss: 0.0012\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 77: Loss: 0.0015\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 78: Loss: 0.0030\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 78: Loss: 0.0020\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 79: Loss: 0.0027\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 79: Loss: 0.0069\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 80: Loss: 0.0034\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 80: Loss: 0.0014\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 81: Loss: 0.0014\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 81: Loss: 0.0013\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 82: Loss: 0.0019\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 82: Loss: 0.0013\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 83: Loss: 0.0022\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 83: Loss: 0.0012\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 84: Loss: 0.0015\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 84: Loss: 0.0014\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 85: Loss: 0.0021\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 85: Loss: 0.0012\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 86: Loss: 0.0019\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 86: Loss: 0.0012\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 87: Loss: 0.0017\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 87: Loss: 0.0012\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 88: Loss: 0.0017\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 88: Loss: 0.0014\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 89: Loss: 0.0015\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 89: Loss: 0.0019\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 90: Loss: 0.0013\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 90: Loss: 0.0010\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 91: Loss: 0.0013\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 91: Loss: 0.0012\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 92: Loss: 0.0018\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 92: Loss: 0.0010\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 93: Loss: 0.0014\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 93: Loss: 0.0010\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 94: Loss: 0.0015\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 94: Loss: 0.0010\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 95: Loss: 0.0013\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 95: Loss: 0.0010\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 96: Loss: 0.0015\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 96: Loss: 0.0011\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 97: Loss: 0.0018\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 97: Loss: 0.0012\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 98: Loss: 0.0013\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 98: Loss: 0.0010\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 99: Loss: 0.0014\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 99: Loss: 0.0010\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 100: Loss: 0.0013\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 100: Loss: 0.0009\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 101: Loss: 0.0011\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 101: Loss: 0.0009\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 102: Loss: 0.0015\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 102: Loss: 0.0009\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 103: Loss: 0.0013\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 103: Loss: 0.0009\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 104: Loss: 0.0012\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 104: Loss: 0.0009\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 105: Loss: 0.0013\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 105: Loss: 0.0008\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 106: Loss: 0.0011\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 106: Loss: 0.0010\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 107: Loss: 0.0013\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 107: Loss: 0.0013\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 108: Loss: 0.0012\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 108: Loss: 0.0009\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 109: Loss: 0.0011\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 109: Loss: 0.0009\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 110: Loss: 0.0009\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 110: Loss: 0.0008\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 111: Loss: 0.0011\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 111: Loss: 0.0009\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 112: Loss: 0.0012\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 112: Loss: 0.0010\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 113: Loss: 0.0011\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 113: Loss: 0.0008\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 114: Loss: 0.0010\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 114: Loss: 0.0010\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 115: Loss: 0.0008\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 115: Loss: 0.0008\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 116: Loss: 0.0012\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 116: Loss: 0.0008\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 117: Loss: 0.0011\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 117: Loss: 0.0008\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 118: Loss: 0.0011\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 118: Loss: 0.0009\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 119: Loss: 0.0010\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 119: Loss: 0.0007\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 120: Loss: 0.0009\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 120: Loss: 0.0010\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 121: Loss: 0.0010\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 121: Loss: 0.0007\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 122: Loss: 0.0009\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 122: Loss: 0.0007\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 123: Loss: 0.0010\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 123: Loss: 0.0008\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 124: Loss: 0.0010\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 124: Loss: 0.0008\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 125: Loss: 0.0009\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 125: Loss: 0.0007\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 126: Loss: 0.0009\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 126: Loss: 0.0008\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 127: Loss: 0.0012\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 127: Loss: 0.0007\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 128: Loss: 0.0008\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 128: Loss: 0.0009\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 129: Loss: 0.0010\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 129: Loss: 0.0007\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 130: Loss: 0.0009\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 130: Loss: 0.0008\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 131: Loss: 0.0009\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 131: Loss: 0.0007\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 132: Loss: 0.0009\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 132: Loss: 0.0006\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 133: Loss: 0.0009\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 133: Loss: 0.0007\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 134: Loss: 0.0009\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 134: Loss: 0.0007\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 135: Loss: 0.0007\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 135: Loss: 0.0007\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 136: Loss: 0.0009\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 136: Loss: 0.0006\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 137: Loss: 0.0008\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 137: Loss: 0.0007\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 138: Loss: 0.0009\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 138: Loss: 0.0006\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 139: Loss: 0.0008\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 139: Loss: 0.0006\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 140: Loss: 0.0009\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 140: Loss: 0.0006\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 141: Loss: 0.0007\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 141: Loss: 0.0008\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 142: Loss: 0.0008\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 142: Loss: 0.0006\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 143: Loss: 0.0008\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 143: Loss: 0.0006\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 144: Loss: 0.0009\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 144: Loss: 0.0006\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 145: Loss: 0.0008\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 145: Loss: 0.0006\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 146: Loss: 0.0007\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 146: Loss: 0.0006\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 147: Loss: 0.0007\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 147: Loss: 0.0006\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 148: Loss: 0.0007\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 148: Loss: 0.0006\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 149: Loss: 0.0007\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 149: Loss: 0.0006\n",
"Test Accuracy: 100.00%\n",
"Train - Epoch 150: Loss: 0.0007\n",
"Train Accuracy: 100.00%\n",
"Test - Epoch 150: Loss: 0.0006\n",
"Test Accuracy: 100.00%\n"
]
}
],
"source": [
"opt = torch.optim.SGD(model.parameters(), lr=0.2)\n",
"epochs = 150\n",
"\n",
"for epoch in range(epochs):\n",
" train_running_loss = 0\n",
"\n",
" for xs, ys in train_loader:\n",
" opt.zero_grad()\n",
"\n",
" loss_evaluated = loss_func(model(xs), ys)\n",
" loss_evaluated.backward()\n",
"\n",
" opt.step()\n",
"\n",
" train_running_loss += loss_evaluated.item()\n",
"\n",
" train_avg_loss = train_running_loss / batches\n",
" print(f\"Train - Epoch {epoch + 1}: Loss: {train_avg_loss:.4f}\")\n",
"\n",
" # Calculate train accuracy\n",
" y_pred_train = model(X)\n",
" predictions_train = torch.argmax(y_pred_train, axis=1).detach().numpy()\n",
"\n",
" correct_train = [1 if p == p_true else 0 for p, p_true in zip(predictions_train, y)]\n",
" train_accuracy = sum(correct_train) / len(correct_train)\n",
" print(f\"Train Accuracy: {train_accuracy * 100:.2f}%\")\n",
"\n",
" # Calculate test loss and accuracy\n",
" test_loss = loss_func(model(X), y_hot).item()\n",
" print(f\"Test - Epoch {epoch + 1}: Loss: {test_loss:.4f}\")\n",
"\n",
" y_pred_test = model(X)\n",
" predictions_test = torch.argmax(y_pred_test, axis=1).detach().numpy()\n",
"\n",
" correct_test = [1 if p == p_true else 0 for p, p_true in zip(predictions_test, y)]\n",
" test_accuracy = sum(correct_test) / len(correct_test)\n",
" print(f\"Test Accuracy: {test_accuracy * 100:.2f}%\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5hYXv8CkYG8l"
},
"source": [
"Great! We\\'ve mastered the basics of constructing hybrid\n",
"classical-quantum models using PennyLane and Torch. Can you think of any\n",
"interesting hybrid models to construct? How do they perform on realistic\n",
"datasets?\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9QsGJNbCYG8m"
},
"source": [
"About the author\n",
"================\n"
]
},
{
"cell_type": "code",
"source": [
"seconds = time.time()\n",
"print(\"Time in seconds since end of run:\", seconds)\n",
"local_time = time.ctime(seconds)\n",
"print(local_time)"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 0
},
"id": "MNv-uf3nGrec",
"outputId": "9a702a46-dc99-41ae-9cae-bc137afc7417"
},
"execution_count": 25,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Time in seconds since end of run: 1704993040.0414505\n",
"Thu Jan 11 17:10:40 2024\n"
]
}
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.18"
},
"colab": {
"provenance": [],
"machine_shape": "hm"
}
},
"nbformat": 4,
"nbformat_minor": 0
}