686 lines (686 with data), 73.4 kB

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": 667,
      "metadata": {
        "id": "Mqkq8qOoIjmA",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "outputId": "69bc3bd0-ca45-4248-f7f2-80e42eaa4c41"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Time in seconds since beginning of run: 1701839379.7491345\n",
            "Wed Dec  6 05:09:39 2023\n"
          ]
        }
      ],
      "source": [
        "# This cell is added by sphinx-gallery\n",
        "# It can be customized to whatever you like\n",
        "%matplotlib inline\n",
        "# !pip install pennylane\n",
        "import time\n",
        "seconds = time.time()\n",
        "print(\"Time in seconds since beginning of run:\", seconds)\n",
        "local_time = time.ctime(seconds)\n",
        "print(local_time)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OagJVyhUIjmB"
      },
      "source": [
        "Turning quantum nodes into Torch Layers\n",
        "=======================================\n",
        "\n",
        "::: {.meta}\n",
        ":property=\\\"og:description\\\": Learn how to create hybrid ML models in\n",
        "PennyLane using Torch :property=\\\"og:image\\\":\n",
        "<https://pennylane.ai/qml/_images/PyTorch_icon.png>\n",
        ":::\n",
        "\n",
        "::: {.related}\n",
        "tutorial\\_qnn\\_module\\_tf Turning quantum nodes into Keras Layers\n",
        ":::\n",
        "\n",
        "*Author: Tom Bromley --- Posted: 02 November 2020. Last updated: 28\n",
        "January 2021.*\n",
        "\n",
        "Creating neural networks in [PyTorch](https://pytorch.org/) is easy\n",
        "using the [nn module](https://pytorch.org/docs/stable/nn.html). Models\n",
        "are constructed from elementary *layers* and can be trained using the\n",
        "PyTorch API. For example, the following code defines a two-layer network\n",
        "that could be used for binary classification:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 668,
      "metadata": {
        "id": "MOhMWnZeIjmC"
      },
      "outputs": [],
      "source": [
        "import torch\n",
        "\n",
        "layer_1 = torch.nn.Linear(2, 2)\n",
        "layer_2 = torch.nn.Linear(2, 2)\n",
        "softmax = torch.nn.Softmax(dim=1)\n",
        "\n",
        "layers = [layer_1, layer_2, softmax]\n",
        "model = torch.nn.Sequential(*layers)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ak3WCr3IIjmC"
      },
      "source": [
        "**What if we want to add a quantum layer to our model?** This is\n",
        "possible in PennyLane:\n",
        "`QNodes <../glossary/hybrid_computation>`{.interpreted-text role=\"doc\"}\n",
        "can be converted into `torch.nn` layers and combined with the wide range\n",
        "of built-in classical [layers](https://pytorch.org/docs/stable/nn.html)\n",
        "to create truly hybrid models. This tutorial will guide you through a\n",
        "simple example to show you how it\\'s done!\n",
        "\n",
        "::: {.note}\n",
        "::: {.title}\n",
        "Note\n",
        ":::\n",
        "\n",
        "A similar demo explaining how to\n",
        "`turn quantum nodes into Keras layers <tutorial_qnn_module_tf>`{.interpreted-text\n",
        "role=\"doc\"} is also available.\n",
        ":::\n",
        "\n",
        "Fixing the dataset and problem\n",
        "==============================\n",
        "\n",
        "Let us begin by choosing a simple dataset and problem to allow us to\n",
        "focus on how the hybrid model is constructed. Our objective is to\n",
        "classify points generated from scikit-learn\\'s binary-class\n",
        "[make\\_moons()](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_moons.html)\n",
        "dataset:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 669,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 406
        },
        "id": "b-ucArFtIjmC",
        "outputId": "487dfae3-c4ef-4ee3-ce3a-6aeacb5fde8d"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAGFCAYAAABg2vAPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACL2ElEQVR4nO3dd1wUd/rA8c/MLh0ERUFF7L33Ho3RJMYUTe/V9ORyueTufne5kstdLnepl957NZpqeqKx9957RUFAeofdmd8fXzpbYVkWeN6vVyLszs58F5aZZ77leTTTNE2EEEII0WrpTd0AIYQQQjQtCQaEEEKIVk6CASGEEKKVk2BACCGEaOUkGBBCCCFaOQkGhBBCiFZOggEhhBCilZNgQAghhGjlJBgQQgghWjkJBoQQQohWToIBIYQQopWTYEAIIYRo5SQYEEIIIVo5CQaEEEKIVk6CASGEEKKVk2BACCGEaOUkGBBCCCFaOQkGhBBCiFZOggEhhBCilZNgQAghhGjlrE3dACEa286TOXyx+SSn80voGB3K5aO60Cc+qqmbJYQQAUMzTdNs6kYI0RjK7AYPzt/Gwm3JWHQNTEADu2Fy3fiu/POiwei61tTNFEKIJifDBKLFeuy7PXyzPRlQAYDdNLEbKvb9cO1xXlxysCmbJ4QQAUOCAdEiZRWU8sHaY7jq93p9+WGKSu3+a5QQQgQoCQZEi7Ti4GlshusRsPwSGxuOZvqpRUIIEbgkGBAtUnGZZ3f8JTajkVsihBCBT4IB0SIN7NTGo+36yaoCIYSQYEC0TIMTohncuY1aReCARdeY0qc9XWPD/dwyIYQIPBIMiBbrf1cOJyLEUicgsGga7SKC+ffFQ5qoZUIIEVgkz4AIeIZhsurQaT7fdILUXJU46LJRXZjYKxZNc50nICmzkFeXHeLzzScoLjOICLFw5eiu3Dm1J3FtQv30DoQQIrBJMCACWnGZnTs/3MTSfelYdA27YVb+O31AHC9fO5IQq8XtfgzDpKjMTliQRRINCSFELTJMIALaP7/ZzfL96QCVCYMq/l2yN43Hvtvj0X50XSMixCqBgBBCOCDBgAhYmQWlzN+YhLN0AYYJn6xPIqewzL8NE0KIFkaCARGw1h7OcJs4qNRusO5Ihp9aJIQQLZNULRQBy10g4O12jck0TTYdy2LTsSwsusbEXu0Z2NmzXAdCCNHUJBgQAWtIQrTbbTQPt2tMR08XcNdHm9iTkkfFlATDhPE92/HC1SPpEBXSpO0TQgh3ZDWBCGjXvbmONYczKicNVleROOidm8c2QcuUzIJSzn12OZkFpXXaqGkQGmRhQMcohie25drxXenVIbKJWiqEEM7JnAER0J68fCjxbUKovQhA16BzdCiPXzq0aRpW7sO1x8jIL3EYrJgmFJXa2Xw8m/fWHGXG08t4b/VR/zdSCCHckJ4BEfCyCkp5f80xPt1wnNP5pXSICuGqMYncMKE70eFBTdq2M59cwtGMQq9e8+HccUzu076RWiSEEN6TYECIBhj+yM9kF3m+tNGiwaTe7Xl/7rhGbJUQQnhHJhCKFquw1MbXW5NZefA0hmEyqltbLh+V6NPehISYMK+CAbsJqw5mYJqm21TKQgjhL9IzIFqk7SeyufHt9WQVlqFrYAKYEBKk8+p1ozizX1yD9p9XXMbjP+5l3vokr5c2asChx2ZJNkQhRMCQCYSixckqKOW6t9aRU37HbphqMp8JlJQZ3Pb+Rg6n59d7/0Wldq56fS2frDvudSCga6q8sgQCQohAIsGAaHHmb0wir9jmMI2xiQoOGjKr/6N1x9idkou9Hn1qhglzJ/eo97GFEKIxSDAgWpxf9qTiavDLbpj8tOtUvff/4dpj5eMOjumaGgqwVLv7t5R/efXYRGYP71zvYwshRGOQCYSixSkuM9xuU2Jzv40zJ7OLXMUCGCYMT4yhX8coft51ijK7waDO0dw8qTvnDuooEweFEAFHggHhM6Zpsvl4Ft9sSyG7sJSusRFcMboLXdqG+7UdwxOj2ZOS6zAREKg79qFdYuq9/6jQIDILSp0+b9E0urQN4/FLhzZ5UqT6KC6zsys5B7sB/TpGER3WtLkchBCNT4IB4RMFJTbu+nATyw+cxqprqEUqGi/8eoDfzejLfdP7OHydaZqU2g2CLbrP7pivHdeND9ced/q83TC5cWK3eu//khEJvLP6qNNgw26aXDSs+Q0F2OwGL/x6kLdXHSGv2AZAsEXn0lEJPDRrAFGhEhQI0VLJ0kLhE3d9uImfdp1yOGkP4IlLh3LFmMTK77MLS3ljxWE+XnecrMIyQoN0Lh7RhTun9qRbbESD2/P68kM89v1edI3KNlV8feOEbvzjokH1Dj6Ss4uY+exyCkpsdSYRWnSNwZ3b8PldE7Fams+UHNM0+e28rXyzLbnOEIhFg4Gdo1lw5wRCgyxN0j4hROOSYEA02OH0fM56epnT5zWgS9swlv1hGrqucTq/hEteXs2JrMIawYNF1wgLsvDpHeMZ1LnhlQiX7kvjjRWHWXMoA9OEYYkx3DK5BxcO7dTgXoi9p3K564NNHMkoxKJpmJgYJkzt24HnrhpOTHhwg9vvT6sPneaaN9Y5fV4DHr5wIDdNkpUQQrREEgyIBntj+WH+88Mep70CFX753RT6xEfxm4838/3OU44rEWrQrX0Eix+Y6rNhA9M0MU18vrbfNE3WHMpg24kcgiwaU/t2oE98lE+P4S+/nbeFb7enOB360IA+8ZH8/Lup/m2YEMIvZM6AaLDiMru6cLuJK4vLDE7nlzgNBECl6z2cXsC6I5mM7xnrk/ZpmkZjTODXNI2JvdszsXfzLzp0PLPQ6e8E1ErKk1lF/muQEMKvms+gpghY/Tu1cXkhAQiyaHRrH87BtHy32+oa7E7O9WUTW4RSm8Gqg6f5adcpDqTm+XTfsRHBdcpE19bchj6EEJ6TngHRYNP6dSC+TQjpeSUOhwosusbFIxJoExpEsNV9/GmW1xAQimmavLv6KM8tPkB2YVVRpBGJMfzn0iH079imwce4eEQXFu1Jc/q8rsFlo7o0+DhCiMAkZ1zRYFaLzgtXjyTIotfIugfqItK1bRh/Om8AAEMSomnn7g5To8GFhFqSF389yCPf7K4RCABsP5HDpa+s5mBa/essVDhnUDxDEqLr/P5ABXOxkSFcP6H+yzGFEIFNggHhE2N7tGPhvZO5YGgnrOUXlOiwIO6Y2osv75lEuwgVAARZdO6e1svpfnQNZg/rTEJMmF/aHejS80p4dvEBh8/ZTZPiMoOnf97X4OMEWXQ+mDuWqX07AGrCYEVc0L9jFAvumED7yJAGH0cIEZhkNYHwuTK7QXGZnYhgq8MZ/KZp8uh3e3hr5REsuoZhmlg0DZthMrVvB169bhRhwbKeHeDNFYd57HvXKzV0Dbb8/RyfZQo8lJ7PqoOnsdlNRnSNYXhijKRQFqKFkzkDwueCLDpBLhLuaJrG3y4YyFVjElmw6QQns4qICQ9izogERndrKxeealJzi1XA5KJEomFCRn6Jz4KBXh0i6dUh0if7EkI0DxIMiCbTJz6Kh2YNaOpmBLT2kSFuV19oGpXDMNXlFpdxMquIiGArie3CJMgSQjglwYAQAWz28AQe/3Gv0+ctmsaZ/TvUWPaXllfMEz/u5autydjKexT6xUfyu7P7MXNwx0ZvsxCi+ZEJhEIEsI7Rodw2pafD53QNrBaNB8/uV/lYel4Jc15axZdbqgIBgP1p+dz54SY+Xue8gJMQovWSYEB4LLuwlK1J2ew7lYfhLvew8Jn/O7c/D5zdl7BaRYJ6tI/gk9vHM7BzVZ6B5xbvJzW3pM7QQsU04YcX7iS70Hn5ZSFE6ySrCYRb6XklPPb9Hr7Zloyt/CKT2DaM+6b34fLRiW5eLXwlv8TGygPp5BXb6NkhgpFda062LC6zM/yfP1NcZjjdhxQcEkI4InMGhEsZ+SVc/PIqUnKKa9xtJmUV8YfPtpOeX8LdZ/ZuwhbWZZomu5JzSc0tJi4qlMEJbRpl8lxecRlfbD7J+qOZaMD4nrFcPCKBiBDP/6yOni7gq60nOZ1fQsc2oVw8sovTHAuRIVZmDu7kdF/peSUuAwFQCYSOZhR63D4hROsgwYBw6eWlh+oEAtU99dM+LhnRhY7RoX5uWRXTNNmalM2321M4kJrHruRcMgqqusJ7dojg7xcM9GlWw7WHM7j1vY0UlNgqiyB9uz2FJ3/ax9s3jWFUt7YuX283TB5euJMP1x7HomtogGGaPP3Lfu4+sxe/P6ef1wFMVKj7P2cTaOPBdkKI1kWGCYRTNrvBsH/+TEGJ3ek2ugYPnN2Xe8/q47d2GYbJ7pRc8ktsdIgM4Z/f7mbZ/nR0DYfJebTy/715w2imD4hv8PFPZhcx/emllNqMOsfTNQix6vz1goHERYUyvmc7okLrrv9//Me9vLL0kNNjPDSrP7dPcZ6p0Zlr31zLmkMZLpMU/fy7KfRtpqWWhRCNQ24RhFM5RWUuAwEAXdM4num/bucvNp/gmV/2c8JBOV1nF0AT0Ez4+9e7mNYvzmFWRG+8v+YoZQ4CgYo2FJUZ/OXLnQCEBuncMqkHD57TrzLvf25xGW+vPOLyGC8tOcSNE7sTYvUuE+P9M/qy5tAaNNT7rk7X4LzBnSQQEELUIasJhFMRIVYsHnRVO8t8dzK7iP/+sJczn1zC+McWc+t7G1m+P536dka9s+oID8zf5jAQcMcsb8/6o5n1OnZ1P+1KxUVCwBqKywxeWXqIP32+vfKx5fvTKbG5HtvPKSpj49Esr9s2pns7Xr9+NG3CVJxv1bXKGgMXDuvM01cM83qfQoiWT3oGhFOhQRbOGRTPz7tTnc4ZsBkms4cn1Hl8/ZFMbnx7PaU2e+WFMz2/hEV7UhnVtS1pecWcyi0mOiyIS0d14ZZJPYhv43zeQVZBKY99v6fB7yklx/tAorbSMte9JbWZwIJNJ7hlcg8GdGpDoZvelgoFJbZ6tA5mDIxn3UMz+GnXKQ6l5RMeYuW8wR3pFhtRr/0JIVo+6RkQLt17Vm8sWtXdZXW6BmcPjGdwQnSNxwtKbNz63gZKqgUCQGVAsel4FklZRZTZTU7nl/Lm8iOc99wKDqU7L8W7sNqyxoaIjWh45b0hXRyX+nXFomt8vukEAL3jPcv736cB3fmhQRZmD0/ggXP6cefUXhIICCFckmBAuDSoczTv3TK2snytpbzbWUN1O79w9Yg6r/lq60lyi20uJ7FVZzdNcorKuPfjzU6HEE5kFVaWRq6v2IhgJvSKbdA+AG6c0N1tvYDaTNMkLa8EgBGJMfSNj3QYYIH6GY/r0Y4e7VvPBXzt4Qzu+WgzU59cwsxnl/PcogOk5RU3dbOEaDVkmEC4NaFXLKv/dBZL96WzLzWPsCALZw+MJ7FduMPtNx7NwqLh8bg6qF6DPSl5bE3KZkTXusvyYsKDvb4A1/bQrAEuqyl6amLv9twxpSevLT/sdAVDbSbQIUoFVJqm8fTlw7nitTWU2o0a78uia0SGWHnskiENbmdzYJom//lhL68vP4xF1yp/FvtT83hz5WE+mDuO4YkxTdtIIVoBCQYE2YWlfLnlJAfT8okIsTJzcEdG1Kphb7XozBgYz4yB7pfmaZX/9+7irWmwzUEwYJomfeOjPO5pqC0mLIi/nD+AS0d1qd8OHPjTef0ZlhjDG8sPszUp2+07NU1Iy6260x3SJZqF907iucUH+GHnKeyGSZBFY/bwzvx2el+ngVZLs3BbMq8vPwxQIygyTDXcdPM761nz5+mEBnm3qkII4R0JBlq5zzed4M9f7KDMbmDRNUzg9eWHmdgrllevH0UbB2vk3RnfK5Yvtpz0vjEmBFlr3rmvP5LJX7/awf5U5/MJKkKWiktJZIiFSb3bM6Z7O7rHRjClbweCrb4dEdM0jVlDOjFrSCfshklydhEznlnmcpXAN9tTuPesPHYl5/DB2mMcSs8nMtjKjRO7cdHQzvTv1KbVXfTecNG7YpiQVVjGwm3JXCFpr4VoVJJ0qBVbvj+dG99e7/Cu1qJpjO/Zjo9uG+/1fotK7Ux6/FeyC0u9upvXgOV/nFZ5V7zpWCZXvb4Wu2E6TSZkAvFtQnj68uGM7t6WwlI7bUKtWH0wHOANwzAZ9PCPFLlIB6xr0KVtOMczC2tcAHVNLeP85LbxdSZjtmTFZXb6/+1Hl9tYdI05wxNkSaQQjUwmELZiz/96AGdpBOymyapDGWxLyvZ6v2HBFt69eQyRIdYak+RcpSywaDBzcMca3eP//Ga300AAINii8+p1I1n9p+lM7tOe0CAL7SKC/R4IABSW2V0GAqCGCioSNFV/TxVd4re+txGb3fU+WiPTy+EmIYT3JBhopbIKStl4NMvlnbtV1/hx16l67X9olxh+/f2ZPHhOPwZ3bkPvDpHMHtaZaf06AFQuzbOUBwgju7XlicuGVr7+cHo+207kuGxfid3AbuD1Mr/GEBZkIcTNUISrS5phwqncYhbtSfNtwwJYaJCFQZ3buAwS7YbJ+B4NXwEihHBN5gy0UoUeJs4prGfiG4D2kSHcM60390yrqmpomiZbkrKZvyGJpMxCYiNDuHhEAlP6dqhxUT+V635Zma75JomQL1h0jYtHJLBg04l6r3qw6hqbj2cxc3BHH7cucN0+pSe/nbfV4XO6Bm3CgrhwWGf/NkqIVkiCgVaqQ2QIkSFW8l1c7O2G2aDEN45omsbIrm0Z6WD5YO32uWOYVcv1AsE903rz/Y4UCkrt9Q4IAqGXw58uGtaZ7UnZvLXqaI2lhbqmhpveuWkMYcGta1KlEE1BhglaqWCrztVjE53WHtCAkCCd2cOb5q6sd1wkAzpFuexCDg9W+Q4CRWK7cBbcOZG+tTIMWnWNGyd0o2u7cFxd6m2GyaRe7Ru3kQFG0zT+esFAPr5tHGcPjKdL2zB6x0Vy71l9+PXBMx3mnBBC+J6sJmjFcovLuOyV1RxKK8Be7WNg0TQMTJ6/akSTdtGuOnia699ah2k6Hm9/+MKB3Dyph9/b5Y5pmuw4mcPelDxCgnSm9OlA24hgPl53nIe+3OHwNRZdo3eHCH68f0qN/A5CCOEPEgy0crnFZby85BAfrztGbrEaMpjUO5bfnNWH8T2bfuLW0n1pPPTlDpKzq+YQtAm18oeZ/bl+fLcmbJn3TNPkX9/u5u1qXeKaplYZdGkbxrzbx9OlbetINuQre0/lMn/DCU5mF9I2PJjZwxMY37OdBFRCeEmCAQFAmd0gq6CUsGALUfVINNSYDMNk3ZFMTmYX0TY8iMl92hNibb7jyJuPZ/HxumPsT80nKtTKhUM7M3t4goyNe8EwTP757W7eXa0CK8Mw0csDrDP6tOe160cRHixTooTwlAQDQohm57Vlh/jPD3sdPqdrcMHQzjzvoIiWEMIxCQZEwDFNk20ncvhsUxKpuSW0jwzhslEJjOzaVrp/m5hhmKw8eJqNRzNB05jQM9bv3fKlNoNxjy0iq7DM6TaaBiv+OE2GXYTwkAQDIqDY7Aa//2w7X205WTmuXvHvuYPief7qEc16iMAZwzBJKc+t0KlNKElZhaw9nIFpwujubekd59slnvVxMC2f297bwJEMVU7aRC0/7RcfyZs3jvFbcaXNx7O45OXVLrfRgH/NGcx1zWxeiRBNRQbVREB56uf9fF1e5KhizXnFv7/sTuXRb/fwrzmDm6x9vmY3TN5ZdYS3Vh4hJUcFAyFWvU7Bo4m9Ynn2yuHEtQn1W9vyS2zYDZM2oVayCsu48vU1ZJffjduq5VE4mF7AFa+t4effTfHLfJNSF8WgKmkebieEACQYEAEkv8TGu6uPOE3ba5jwyYbjPHB2X9pGBPu1bY3BNE0eXLCNr7ecrPGeHVU+XHckk8tfXcO3901u9Avu9ztSeGXpIXaczAGgW7tw+sRHkplf6vB3YzdMTuUU88Xmk9w4sXujtg2gT1xkjQRFjpgmZBSUNHpbhGgpJOmQCBjrj2RQ7KbYj81usurQaT+1qHEt2ZfGV7UCAWfshsnxzEIWbDzhdlvTNNl5Mocl+9LYlZyDNyOBzy06wN0fbWZXck7lY8czC1m0J81tO7/aWo+y1fUQGxnCBUM7OU2YVeGlJYfYWo9CW0K0RtIzIPymuMzOjztPcTAtn7BgCzMHd6RXh6psfaU2zy5aLaX794O1x9ze4da2YFMSt0x2nmhp2f50/vXtbg6m5Vc+1jc+kocvHMSk3q6zG+5KzuF/i/YDNasqetI6EzieUcicl1aRX2yjb8dIrhvXjQm9YhtlcuHfLxjI8v3pLicRWnSNt1Yc5oVrRvr8+EK0NBIMCL9YtDuVB+ZvJbfYhlXXMEyTJ3/ax3mDO/LMFcMJCy6vYIf7i8/ghGh/NLnR7U/N9yoQMIHTeaVOn/91byq3vrexzs/vQGo+17+1jndvHsuUvh2cvv6jdce9Dk6qyygoJaNAte9IRgHf7zjFteO68uicwT4PCGIjQ+jXMYq1hzOdbmM3TJbsS/fpcYVoqWSYQDS6jUczuf2DjeSVZzi0GWblnedPu05x37wtgMrtX7t6YXUWXWN0t7b09XHxpKYSFeJdLK5rkNA2zOFzhmHy1692qtTNta7lZvl/f/96p8shg93JufUOBGqr2M9H647z4brjPtlnbboHAYav3o8QLZ0EA8IrhmGy4kA6b644zAdrj5Gc7b6E8HOLDzi94zdMtUqgYoz6v5cOIS4qBEut87xF14gJD+KZK4Y3+D0EiguHdcabIoWGCdeM7erwufVHM0nOLnbaq2KacDSjkM3Hs53uPzzY4rKQUn1owBvLD2M0wkV5RNcYlz8/iwbDE2N8flwhWiIJBoTHthzPYsqTS7j+rfU89v0e/v7VTiY9/isPzt9KcZnd4Wtyi8tYeeA0dhfXAouu8d32FAA6RYfx7W8mc8fUXrQLVysGosOCmDu5Bz/cdwZdY1tOEpmrx3YlJjzY7UQ4qLqwzR7huHBUSo77oMzdducN7ui6DbrGiMSYyt8LQFiQxeUF2URNQEzNK3a+UT1dM66by+EHuwk3T+ru8+MK0RLJnIEAV1xm57vtKfy6N41im52Bndpw1diuJMQ47i5uLAfT8rjmjXWU2NRFv/JGz4Qvt5wkv8TGa9ePrvO6ghKb2zkAOlQOIYAaD/7jzP78cWb/ypzzLVG7iGDm3T6eW97dwImsIqzl79NW6y46yKJxycgu/O2CgU4TLrWLCPHomLEutrt4ZBde+PUgGQWldbrXdU21439XDiehbRjHMgrRNPho7THeX3MMw82KhcZIbZYQE8bTlw/jgflb0bSquQ4WTQUCcyf3CKgS10IEMgkGAtjR0wVc8+ZakrOL0TV1AV6yN42Xlhzk3xcP4WonXcaN4cVfD1JqN3DU22uY8NOuVLafyGZol5gaz7WLCCY82EJhqeOeAwCbadK9fYTD51pqIFChb3wUy/4wjaX70lh3RE2GG9+zHSO6tmXnyRxME4Z2iSYm3HVehYm9YmkXHkxmofMJhvFRIYzt0c7p85EhVj65fTw3vLWek9k1g5PIECuv3zC68vfUO06tAhnZtS1vrzrqsm2dokPp2EjJkuaMSKBnhwjeWnmEX/emYTdMhifGcMukHkwfECfpq4XwkAQDAarUZnDdW+tIzVWJUyouwhX/PvTFDrrFhjOxl+vlYr5QZjf4dnuKy8lYVl3jqy3JdYKBEKuFK0Yn8sHaY05fb9U1LhmR4MsmNysWXWP6gHimD6h5F3tGH+cz/2sLsuj8eVZ//vDZdqfbPHT+AKeTMyv06hDJ0j+cyS+7U1l58DR2u8nIbjFcOKyzwyqAqz3I+TB3co9GDeqGdonhuaukKJEQDSHBQID6adcpTmQ5H9/VNXht2WG/BAOFpfY6Xde1mUC2k7vS+6b34de9aZzMLqoREGia6j5++MJBLSKjYFO7fHQipgmPfreb3GrDLjHhQfzt/IHMHu5ZwBVk0Zk1pBOzhnRyuV16XgmfukmCFGTRuHac1AcQItBJMBCgluxLc7nm227C8gPp2OwGVkvjzgONDLESGWIlv8TmcrsuTpa9tYsI5ou7J/LkT/v4cvNJSu0qaVC/+Cjun9GXmW4mrgnPXTEmkdkjOrN0XzppeSXER4VwZr84gq2+/4ws25/udulemd1k+4lsxvWM9fnxhRC+I8FAgCq1GW7TyJomXPn6Gq4c05WLhnUmNKhxqvlZdI2rxiTyzqqj2J20yTBMLh+d6HQf7SNDePzSofztgoGczCoiPNhCl7ZhMqbbCEKsFs4d1PgBlrMVJHW2ayEZI4VoyWRpYYAa2sWzLHubjmXzx8+2c+krq8kpcp6ataHuOrMXHaNDnY4533tWb49K2EaGWOnXMYrEduESCDRzAzq5T/6kaSodshAisEkwEKAuG5XoVff/3pQ8HvpyR6O1JzYyhC/vnsgwB0HKmO5tudkP1epEYBnZta2qIOgkprPoGmf1j6NTtH+XwQohvCfBQIBqFxHMs1cO93h7u2nyw44UTuX4PrlLhe93pDjMYLf5WDaXvrqGHBdFY0TLo2kaz141nNBgS50eI4uu0SEyhH/NHtxErRNCeEOCgQY6crqAb7cn89OuUz7vpp81pBNBzm67HDBM2HjMeeGWhkjLK+Zf3+1x+JzdNDmeUcjLSw82yrFF4BrUOZrvfnMGl45MILi8JysixMJNE7uz8DeT6Ozn5FhCiPqRCYT1dDK7iD8u2MaqQxmVjwVbdK4b35U/nTfAZ7O3gyw6ZXbPJmo1ps82nXA5odFumnyy/jh/OLdfo69uEIGle/sInrhsGP+5ZCiFpTYigq0NyiuQkV/CnpQ8LLrG8MQYwoIbZ2KsEKKKBAP1cDq/hEtfXk16fkmNx0vtBu+sPkpKTjEvXzvSJxPkzuofxw87T3lUfU3XYFS3tg0+piNHTxega5rLtLO5xTZyi220k5wBrZJF14gKDar363MKy3j4m118sy258vMeEWzhpknduX9GX4IkyBSi0chfVz28vfII6XnFDi/Qpgk/7DzF5uNZPjnWbWf0dLvEEMCiaZw3uFOjTdby5CSvaarynRDeKiixccVra2oEAgAFpXZeXnKI++dt9ejvQAhRPxIM1MOnG5LcVuH7bJPrzGyeGpYYw/+uHI5F11xWh+vfKYrHLhnik2M6cv7QTi6zEFp0jRn94xst14Fo2T5ed5z9aXmOA2zgux0plbUbhBC+J8MEXjJNk8wC58VgAOyGSaoPZ/XPHp7AuB6xzNtwnC3Hs8nILyG/xEap3aBzdBhXjElstKRDh9PzeW7RAdYdyUADhxUItfL/7j2rt8+PL+pKzS3m880nSMosIiY8iAuHdmZg5zZN3awG+Xj9cZeVDS26xqcbkhgvmQyFaBQSDHhJ0zTaRQST4SIgsOga8dG+rdLWMTqU+2f09ek+XSm1GTz0xXY+23zS6TYVpWLbhAXx3FXDGZYY47f2tVavLTvEEz/uw8RE1zRM4JWlhzh/SCeevmJYs+uZMU2TwlI7KTnO63CACrBPZBX6qVVCtD4SDNTDlWMSeW3ZIadDBXbD5LJRXfzbKB/7x8JdLgMBUL0E/5o9iCvGJBJibV4XoTryUmH967DtYyjKgjZdYPQtMOomCHafWdEfPtt0gv/8sLfy++qTOX/YmUJYkM5TVwxvgpZ5r9Rm8N7qo7y7+igns10HAqAC7PaRIX5omRCtkwQD9XDL5B58vvkEp/NL64xxahqcO6gjI7s2zqx+fziVU8y8Dcc92jarsKz5BwIZh+Dtc6EwE8zyZZwZB+Gnh2DbJ3DTtxDqWXroxmIYJs8u2u/8eRM+33KS353Tj4Tytf17T+XyzqqjLN6Tis1uMqJrDDdN6sHUvp6XRm4MpTaDW97dwKpDp10ODVRnN0wuGdm8A2whAplMIKyH9pEhfH7XRMb1aFfj8WCLzk0TuvP8VSOadd79X3af8ugkbZiwxUerJpqMacL8G2sGAuoJ9V/qLvjpr03Vukr70/JclrSu8MuuUwD8sCOF859fyWebVNCaXVTG8gOnufHt9Tz+4143e2lc7685yqqDngcClvIls2f1j2vchgnRiknPQD11aRvOx7eN53B6PjuTcwm26EzoGUt0eP3XWTtimibp+SUYBnSICnFaKMiX8kvs6C7KJ1fQoPknGDqxEVJd1HQw7bB9HpzzTwhrut6ewlL3iad0TaOwzE5qbjH3zduCYZg1JnxW/D5fWXqIUV3bMmNgfCO11jnTNHlv9VGHE1Ed0YBzBnXkicuG+uWzL0RrJcFAA/XsEEnPDr6vymaaJp9vPsmryw5xMC0fgPioEG6e3INbJ/do1Itwzw4RHiU5MoEpfdo3Wjv84uQm0HQwXZTZtZdC6m7oPsl/7aqle2wEFjcBmt0w6RsXxSfrj2OvFQhUZ9Hg7VVHmiQYKLEZJLnp4dCAIQnRXDOuK5N6t/eoGqYQomGa+W1dy/XET/v4/YJtHCoPBABS80p4/Ie93PXRZo8u1vV1Vv84YiPdZxGMCQ9izoiERmuHX+gWPOqv1ps2bm4XEcyswR2d3h3rGsRFhXBmvw5sOpaFq4+H3YRNx5pmeMeqa7gbQdN1jX4do7hqbFcJBITwE+kZCEDbT2TzytJDQN11/Sbwy+5Uvt56stEmVAVZdJ65Yji3vLuhTldzhahQKx/cMq5B6WcDQs9pOM6eUE1oNHQa5pfmuPLXCway8VgWaXklNYLBioRUz141HKtFx+LBfBV/d7knZRby+eYTnMoppmu7cJIyC50GLHbD5JxBHZ3u62BaPh+sOcqK8nkHk3rHcsOE7vSNj2qk1gvR8kkwEIA+XnfcZZewrsH7a4416uzqqX07MP+OCTy7aD8rDpyufDwuKoSrx3bllkk9fD4/okm07w19Z8KBX2pNIKygwfi7Ici3eSPqI75NKAvvncwLvx5gwcYTFJXZ0TWYPiCO+87qw+AEteJhcp/2LNuf7nyYQNc4w0/DO6Zp8t8f9/L6ssPomuoVMEzTaSBg0TV6tI9wOlnw660neeDTbaBVzYE4llHAh2uPExcVQtfYcGYPT+CSEQlEhMjpTQhPaaYk/K637MJSdqfkYtE0hnbxXXW1i19axZakbJfbRIZY2fnIuT45njtZBaXkFJXRISqkZZ5gi7Lhw0vK5w9YVFCgW8GwwdArYc4rajghgJTaDLILS4kMtRIeXPN3klNYxqTHf6Ww1ObwoqsBC+6cwOju7eo+6WOvLTtUIzeCIxZNJfOyGSa94yJ5/5axDksfH0rP55xnlmP34JTVpW0Yn94xoXKZpRDCtRZ4Zm98ucVl/Oub3Xy55WRlvv6KGu6+qK4WGWpF01wPZfuzIFDbiGDatuRKhGExcMvPsP8H2D4fCtKgbQ8YeQN0nYDbQe4mEGzViWvjuLciOjyI924Zww1vr6ew1F75ObLoqurko3MG+yUQKLHZeWnJQZfbWDSNcwd3pE2olbMHxnNmvzinQxgfrDmG05zYtaTkFHP7+xv59jeTm/UyXyH8RYIBLxWV2rnq9bXsS8mrcYdSUKKqqx09XciL1zQsz8D5QzrV6JqvzaJpXDSsc733LxywWGHAheq/FmBUt3Ys/8M05m88wa97Uym1GYzq1o5rx3elVyOsfnFk09EscottLrexmyYzB3dkRGIMm45l8e32ZEZ3b+fwjn7lwdMeT5y1Gya7knPZdCzLL4GPEM2dBANemrfhOHuScx3enFRUV7v2UFcm9q7/mOxFwzvzwq8HOZVbt0yyrkFIkM6NE7vXe/+idYiNDOGuM3tx15m9muT4nuRGAHh16SH2pFT9TWkanDuwI49fOrTmvBQvBzQtusbqQxkSDAjhAVla6KWP1rlO02vRNT7dmNSgY4QHW5l3+3i6x6plVVZdw1reddo2PJgPbx0nS65EwOsT71kPxN5TNYNr01QrZq55cy3FZVUBxaTesV6vgpAZUUJ4RnoGvJSSXeTyBsVumBzPbHh1tcR24fzyu6ksP5DOygOnsRkqt/zMwR2bfy0A0Sp0i41gQs9Y1h/NdNi9XzH876jn326qbv5vtiVz+ehEAK6f0J0P1h7z+Ph2w2RM9+ZbI0QIf5JgwEsx4cEUlDrPoKZr0KEB1dXScotZsOkEB9PyCQu2cN7gjjw0awC6pGIVzdB/LhnCxS+vIrfYVic3gmmaYDrv/dc0WLDpRGUw0DsukmeuGM4D87eiaa6zMVYsUZzQK9aXb0eIFkuCAS9dProLzy8+4HSdtGHCxfXMyvfxuuP87eudmKaJBqBpfLzuOEO7RPPOTWOIlRKuopnp3j6Cb+87gxcWH+DLLScpsRlYdY0LhnZi9aEM0vJKnL7WNCE1t7jGY3NGJDCocxveX3OM5QfSSc0tprjMqLHIQNegbXgQr18/SlYSCOEhyTPgpYz8Es57bgUZBXXLF1s0jcEJbfjsrolulxfml9iYt/448zYkkZpbTHiwhdRcxydGi64xtEs0X9w1UU5uotkqtRnkFJURFWolNMjCpa+sZvPxLKfj+roG43rE8snt453u02Y3+H7nKT5ae4wjpwuIDlMpsq8e25V2LXk5rBA+JsFAPRzPKOQ3n2xm24mcyjsSDTh7YDxPXj6M6DDXmfmyCkq5/LU1HErPd9lNWtuCOycwRmZGixZi/oYk/vj5dpfbPHvl8OZf/0KIZqDVDRMkZxfx1daTpOeVEBcVysUjEugY7V2q2a6x4Xx972R2nMhhS1IWFl1jcu/2dIuN8Oj1f/lqB0dOF3g109mqa/y865QEA6LFmD2iMx+uO8bOkzl1ht10DYYnxjBrSKemaZwQrUyr6RkwTZPHf9zH68tVAaDquf/vnNqLP5zbzy9d8Kdyipn438Uuq8o5YtU1rh7blX/NGdw4DROiCeQVl/H3r3excFty5d+jVde4eEQC/7hoUMtMfy1EAGo1f2kvLz3Eq8sOVX5v2M0az7UJC+LOqY2fnGX7iWyvAwEAm2HSv5NUZRMtS1RoEP+7cjgPzRrAluNZaJrGyK4xMllWCD9rFcFAYanNbY70l349yE0TuxMaVL81/PklNn7ZfYrTeaV0jA7l7IHxDvdVn9KxGhAWbGH2cBk7FS1Th6gQl2WLhRCNq1UEA6sPZrhNjZpXYmPN4Qym9XNcOtUZ0zR5a+URnv55f2VJWcNUVQX/dsEArhzTtcb2o7u1I9iiU2o3PNp/xXrs/105nEjpMhVCiMBWmAnrXoXN70F+OkTEwogbYPxdEOGf0uH10SquLgWlroulVG5X4tl21b27+iiPfren8vuKIYD8Ehv/9/kOQqyWGrOho8ODuHJMIh+tO+a0vGywVafEZqBrcFb/OO4+sxcjukomNSEqnMop5tMNSexJySU0SOecQR05e2B8gyuGCkFxDuz7AYqyIKYr9DkHLK5XiFXKTYa3zobcFFUKHSA/DVb+D7Z+DHN/hpjExmt7A7SKYKBne89ypHtbza24zM4zP+93uc1/f9zLhcM61xge+Mv5AziWUcByB5UJO0WH8sXdE4gMDSbEqsvJTYhaPll/nL9+uRMTE9NUKw++2ppMj/YRfHjrOIcVD4VwyzRh+VOw4imwFYOmg2lAeHu48FnPKpou/A3kVQsEKvdth4JU+OouuOnbRml+Q7WKK83ghDYM6BSFs+F6XYMhCdEM6NTGq/0u259OnpvehFM5xWw+nlXjsdAgC7dP7VlZfKi6lJxirn9rA3bDlEBAiFpWHjjNn7/Ygd00McpzdFTMBT6eWcgNb63j6OkCFm5L5tvtyXUyGDpjmiabjmXxt692cs/Hm3ns+z0cTMtrvDciGkdhJqx5CRbeBz8+BMfXeV6tavmTsORRFQiACgQACjPg0+vhwCLXr886CgcXg+FkSNqww9EVcPqAZ+3xs1bRM6BpGk9eNowrXltDic2okyM91Krz+KVDvd5vdmGpR9tlFdTcrtRm8NtPtmI4+JCawOH0Ap76aZ8sIxSilpeXHqycl1Ob3TA5lF7AmU8trXxM1+DCYZ15dM5gokIdd/UWl9m59+PNLNqThlXXMEwTTdN4fflhbp7Unb+dP1BqgzQHWz6Cb+8Howy08snba1+CbpPgqo8hLMb5a4uyVDDgkKl6CRY9DL2nq6IZjqRsw6MUcslboH0f99v5Wau59RycEM3X90zinIHxWMp/mRZN49xB8Xx972QGdvauVwCgS1vPygi/tfIIaw9nVH6/aE8qGQWlTpcY2k2TBZuS6jWHQYiWqrjMzupDGV4tzTVM+GZbMje+vZ4yJ5N2//bVTn7dmwaoJbyGSeUNwzurjvLGisMNbrtoZAcXwdf3gL1U9QQYNvUfwPG18Ol1rnsI9n6nXuuMaUDqTjjtYlhY93BegafzD/ysVfQMVOgTH8Ur140ir7iMrIIy2kYEOb1b8MSEnrF0jgklJbvYZTy48VgWV72+lkcuGsSNE7uzJyUXq65hc3FWKy4zOJFVRL+OkltACMDl34srhgmbj2fz485TXDisc43nUnOL+XzzCZcBxqvLDnHzpB4EW1vNvVPzs+yJ8jF+B130Znn3/MnN0GWU49cXZqjeBEevr72dM90mgiUE7M6Lb6EHQY+pro/RRFrlpzsqNIiuseENCgQAdF3j3xcPQdOc9xxB1V3GPxbuYn9qHqFBFjxJ/BgiJx8hKkUEW+o9OVDX4LVlh3h31RG+2nKS3OIyAJbtS3fb05BVWMa2E9n1Oq7wg8JMSFrn+kKuW2HPQufPRye6DwQAors4fy4sBkbfooISRzQdRt4A4YGZUl6uNg00rV8cH8wdx0APJh/qusaHa48xfUAcdhcnIA3o0T6CbrGeDUMI0RpomsZNE7tTn9F7w4Sdybk88u1u7v90K2MeXcTziw9QXGb3aH8lZZ7lBRFNoKzQg400KCty/nS/WRDi4hyuWaD7FLXU0JWz/wn9L1Bf69aq1wL0ORdm/seDtjYNCQZ8YFLv9nx33xluqxXaDTVjuX/HNkzr1wGLk7OQCdw3vbeUKxailpsmdWdK3w5oUK+goKJDrsRm8Mwv+9lxMsftlC9dg77x3i07Fn4UEef6Qg5q/kCHfs6fDwqF858u/6bWJ0uzgDUEzv23+7ZYg+GK9+HmH2DYVdBrOgy7Gm78Fq7+RO0nQEkw4ENhQe5/nBXjjs9fPYKxPWIBVZjFoqmTjqbB/83sz8UjXHRHCdFKBVl03rxxNP+4aBDd26sqoVZd86hnzpEvt5ygR/sIp4G5Rdc4Z1BH4tp4V9lU+JE1GEbdWHUHXoemLvZDLne9n6FXwJUfQrueNR9PHAu3/ASdPFxxpmlq/sDsl+D6L2DOS9DjDNdjyQGg1VQt9Ie/frWDT9Yn1Vi6WJ2uwYPn9OOeab0BtbZ5/ZFMvt2eQl5xGd1iI7hiTKIkTRHCQza7gUXXKC4zuPjlVRxIzcfuxSlNA35zVm/eXnWUojJ7zWXHmkbH6FC+vHuiBAOBrjgX3p4J6Xtrjv1rFsCEy96GQRd7ti/ThFM7oChTDQvUDg5aKAkGfOhgWh4zn12B3TDrdD3qGoQFWVj6h2l0iArcriIhmqvswlL+8uVOftiZ4vHyQ4uu8cDZfblwaGdeXX6ILzafoLjMoE2olWvGdeP2KT1pFxHcuA0XvlGcq9L+bnwbirPVY72mwxkPQvdJTdq05kCCAQ/YDZNdyTkUlNjp0T6CjtHO7xJ+3nWKez/egs0wKk9IGhARYuWdm8cwpntgziQVoqU4lVPM1qQssovK+NPnO9xu/+yVwyvrh5imSYnNIMSqy5yd5spuU0mEgsIgROZ6eEqCAdQJoNRuEGypewKYvyGJ/y3aT0qOSlGpaXBWvzj+cdEgEts5nu2fllfM/A1JbDiahUXXmNS7PZeN7EJ0eGAmmxCipbruzXWsOZThdOggIsTCxr+cTVhw/UqXC9FStOpg4GR2Ea8tO8Rnm05QWGqnTZiVq8d25bYzetI+MoTXlx/ise/31nmdRdeICQti4W8m1xjf35Wcw5srjvDTrlOU2Az6xUdx08TuXDqqS41CRUII/9h7KpdLXl5NSZnd4XLeJy4byhWjA7OKnBD+1GqDgQOpeVz26hryS2x1Jg3FtQnhzRtHc9GLq5xOBrToGpeMSODJy4cB8MvuVO76cJMqnFL+mooc6jMHdeSla0dKQCBEE9idnMvDC3ey4WhVwbCu7cL548x+XDC0s4tXCtF6tNpg4PznV7D3VJ7Di71F1+jZPoJD6fkuJyIFWTS2PXwOZTaTcY8tosRmOFyzrAGPzB7EDRO6+6r5QggvHT1dQFJWITFhwQxOaCNzAoSoplXVJqiw40QOu5JznT5vN0wOpOVj1R1XR6tQZjfJyC/ll92pTgOBCm+vPML147vJCUiIJtK9fURlbgIhMOxw4BfY/ikUpEF0VxhxncoR0ArP060yGNiVnOPRdu6WJ2lAm7AgdpzMQddwmmLYBI5mFFJUZic8uFX+yIUQInCU5MFHV8Dx1VUFinQLbPsYBl0Kl7wOltZ1rm6VGQhDPMgUCK6DAYumMa1/HNFhQVgrUge6YdVb5Y9bCCECy8L7VHEjqEpSZJT/u+sLWBq4NQQaS6u8Op3Rp4PbyXxRIVYuGZng8Bqva6DrcP+MPgBM6x/ndKIhgEWDib1ipQSqEEI0tezjsOtLF1UKTVj3KpR6UgCp5Whd/SDl2keGcMXoLny6Icnp3f+tZ/Tk7mm9iAqx8uG649gNs3J1QHybUJ65YjhDu8QAcPbAeBLbhpGcU+wwKLCbcMfUXo34joRoOdJyi5m/MYldybkEW3VmDIjn3EEdJZgWvnF4KbgrT1WaDyc3qZoCrUSrXU1QYrNz3ydb+GlXKhZdwzBNdE3DbphcMzaRR+cMQS/vPUjPK2HxnlQKSu30jotkcu/2dXoWjp4u4Jo315KcXVwZNFg0DQOTRy6SlQRCeOKLzSf442fbMUwT00T9TZomiW3D+OjW8XSVst6ioTa+Dd/+zv12138Jvc5q/PYEiFYbDIDKPLg1KZsvNp8ko6CEjm3CuHx0FwbUswJacZmd77an8MvuVIptdgZ0asM1Y7s6zVQohKiy/kgmV76+BkdnJIuu0Tk6lMUPnik9BKJhkrfC61Ndb6Nb4IF9ENnBL00KBK06GBDCJww7nNgARdnQrofruunCqbnvbmDp/nSX829evGaEJAoSDff6NEjZ5njegGaBwZfApW/6v11NqFXOGRDCLcOAnCR1soju6nyZ0daPYfEjkHeq6rGEUXDB/6DTMP+0tQUwDJOl+9Jdlh+26Bq/7kmTYEA03GVvqZLHBadrlTzWIbY3nPdE07WtibT6YCAjv4StSdloGoxIbEtbKVfaupkmbHwLVj2nZh0DhLeHcXfA5N+BpVqxqfVvwPe/r7uP5C3qRDP3F+g42D/tbubspukyEAAwyisKeiO/xEZOURltw4Mkx4eo0q4n3LkS1r8OWz6EokyI6gyjb4bRt0BIVFO30O+a1TBBbnEZn286wXfbU8grsdEvPorrxndjbA/vywLnFZfxj4W7+HprMrbybkmrrnHpyC78/cKBRITIiaNV+v4P6gSBRo0Zx5oGvc+Bqz5WvQTFufBUH7AVO96PZoGeZ8L1X/ih0S3DWU8t5cjpAqfzvHUNHjynH/dM6+12X/tO5fG/Rfv5edcpDFOlDr9waGfun9GXjIIS3lhxhMV7UrHZTQZ0iuLmST24eERC5aRhIVqbZhMMHE7P56rX15KeVwKo07RFV7P/b5zQjX9cNMjjVL8lNjtXvraW7Sey6ywt1DUY2bUtn9w+niCLTFRqVY6vg7fPcb3Nxa/DsCth8wew8F43O9TggT3QppPPmtiSfbDmKH//epfTYMCqa6z583Q6RIW43M/WpGyuen0NZXazZhEyXSPEqlNUakcvP3dAVUGxOcM788wVwyUgEK1Ss7ja2Q2TW97dQEZBKSZV92sVf8zvrTnGvA1JHu9v4dZktibVDQRAnRQ2Hsvi+x0pDW+4aF42vQO6ix4hTYcNb6ivc5NdbwuACXnyOfLU1WO7cvbAeDRUv0wFi66ha/D0FcPcBgKmafLg/K2U2ow6ExHthklhqb1GZVGoyjT61dZkPt98wjdvRohmplkEA8v2p3E0o9DpLGMNeH3ZITzt5Ji3Icll9mBdg0+9CC5EC5G+Dwyb8+dNA04fUF9HtK9KX+pKROtZmtRQVovOK9eN4l9zBtMrLhJQgcD0AXEsuHMCs4cnuN3HpmNZHEovcFtXxBFdg3dXH/X+hUK0AM1iYHz1wQysulY5tl+bCRzJKCQ9r4S4NqFu95eSXeRwLXMFw4ST2UX1bK1otkLbUGeuQG0h6iLFwDnww/+BUeZ4O02HxLEQk+jjRrZsFl3juvHduG58t8qsn95U+jyYll/vYxsm7E7JxTRNqS4qWp1m0TPgaZTv6XZxUaG4+lPXNIhz0x0pWqBBl+AyENAsMOQK9XVELExxsJIAVCCgaTD9H75uYati0TWvL8rhDZz4a5X5AqKVahbBwKhubZ32ClToFB3q8QX8stFdXGamNk24fLRnd3SmaVJYanOZKEU0E0Muh5hu6qJfm2ZRvQJjb6t6bOr/wfS/Q1CtDJNtOsO1n0O3CY3bXlHH1L4dCKlnhkKLriqRSq+AaI2axTDBOYPiiYsK4XR+icO7fw2YO7mHx7OALx3ZhfdWH+Xw6YI6F3GLptG3YyQXDXOd2CSnsIzXVxzi43XHySosq1y6dNeZvegT3/rWqLYIweFw07fw8VWQtqtqgqBhg6h4uHqeutBX0DQ440EYewcc/AWKc6Btd+g+RZW1FH4XHRbELZN78MrSQ0630TQcDhMapsntZ/RsxNYJEbiazdLCnSdzuOaNteSX2CoDAoumKgJeMLQTz101wm1Z4uoyC0r5/YJtLNmbVtlLoKEqED5x2VBiwp0nH8osKOXSV1ZzLKPmRCWLrhFk0fjo1nGM6uZ97gMRIEwTjixX1c1MOySOgz7nOs9CKAKK3TD5x8JdfLj2GLqmoWnqQg9w7biu/LTrFGl5pZVBga6Bhsbjlw3lslFdmrj1QjSNZhMMAKTmFvPh2mMs3JpMQamNvvFRXD++G+cO6ljvtcHHMwrZcDQTTYMx3dt5VFTogU+38uWWkw6HGnRNlThe+X9neRWcCCF861hGAV9tSVZFyKJDuWREFzpGh1JUaueb7cks2ZtGic1gcEI0V49NpFN0WFM3WYgm06yCgUCQllvMuMcWu6uGzds3jeas/vGV3x89XcA7q47w3Y4UissM+nWM4oYJ3bhwaGdJctIcmCbknlTLCdskSC+BEKJFkTOal15eetBtIGDRNfak5FUGA2sPZ3Dj2+uxGVUZ0bYcz2LTsSwW7U7lWS+HOIQfmSZsfk/VKsg8rB6LiINxt8Ok+2vWKhBCiGZKZjl56Yedp9xuYxhm5YzmwlIbt7+/kTK74TDr2bfbU/ho3bFGaavwgZ/+At/8FjKPVD1WkAZL/g2fXgd2F0mKhBCimZBgwAt2wyQ1t8TtdiYwY4DqFfhmWzK5xTaXORDeXHHE4+yJwo9ObIS1L5V/U+v3Y5qw/0fY+ZnfmyWEEL4mwYAXdA2P1jB3jgmle/sIALYcz3Y5BGACxzMLyS2SO8yAs/Ft97UK1r/hv/YIh07lFLP3VC7ZhaU+3W92YSnbT2RzMC1fgnXR4smcAS9omsb5Qzrx1daTLu/0/zJrQOXXuq65zHZYwWKROQMBJ32v+1oFGQf81x5Rw7rDGTz18z42HM0CVLB+zqCO/Glm/8pgvD7Scot57Ps9fLs9pTLZWY/2Efzu7L5u848I0VxJz4CX7pjaC6uuOyx0pGswoFMUMwdXlayd0qe9y+yJugZDEqKJbGAaVdEIQipqFbgQHOmXpoialuxN4+o31rLpWFblY4YJv+xO5aKXVnI4vX41CtLzSpj90iq+qRYIgFoNdN8nW3h31REXrxai+ZJgwEv9Okbx7s1jiA5Ts8iDdK0yn/nwxBg+mDuuxrDAjAHxdGkb5nSowDBhdLe2/LAjhZxCJ0VvhP+U5MHuhbBtHiSMwn2tgsv81jShlNkNfv/ZNkyzbj0Su2FSUGLnkW9212vfzy3eT1peSZ3MpBXfPfrdHjLy3c8bEqK5kTwD9VRis/PDjlPsSs4h2KpzVv94RnaNcZjX/HB6Ple/sZbU3JLKmni6VvdEFmzRuW58V/503gCC65lfXdSTYcDS/8DqF8BWrWKlHqSGA8xa5Yo1i0pffPdaiJasdf70y+5Ubnt/o8ttNGD1n8/yKpFQcZmd4f/8meIyw+k2ugYPzRrArZK2WLQw0jddTyFWC3NGJDBnhPsa6z07RPLrg2fy5ZaT/LjzFDlFpRxIy6fUZtQICErtBu+sPkpaXgkvXD1CCqb4049/gvWv1X3csFN5X1i9VkFEB7j6EwkEmsDR0wUOg+nqTFR2UW+CgcyCUpeBAICuaRzPLPR4n0I0FxIM+ElEiLWyTvuj3+5md0qew5OZaarcA7ee0ZPhiTF+b2erlHkY1r/u5EkD0KH7ZDVsYNggcSz0myUJh5pIVKjVo3LlUaHe/X4iQ62VPXfOmEAbL/crRHMgfdFNYP7GJJclj626xuebTvixRa3c9gVqmaBTBhxfA1P/COf+GwbOlkCgCZ09ML5yno4rh9LzvNpvm9AgpvbrgMVFj5zdMLlQVhQ0T6YJR1fCV3fDexfBF7fDoSVqiFBIMOBvNrtBbrHrnAJ2wyQtr9hPLRLkp+JweUh1hg2Ksv3SHOFabGQIN0/q7na7//t8BwUl3uXv+O30Pmia44+DrsH5QzrRr6OUKG92bKUw/wZ493zY/ikcWaYShn0wBz66HMqK3O6ipZNgwM+sFp02oa5HZ3RdI75NqJ9aJIjq5LjAfXV6EIS19U97hFv3z+iLu9QchaV2vtmW7NV+R3Rty1s3jaFteQlzq66VlziGOcMTePqKYfVssWhSix+Bvd+qrytyhxjlk4IP/wo//F/TtCuAyJyBetidnMsvu1MpKrPTv2MUMwd3JDTI4vHrrxrblbdWHMHu5AJkN0ypq+5Pw65UtQac0S0w+FK1ekAEhNP5JdjdxG9BFo2Dad7nG5jatwPrHprOot2pHEzLJzzEyjkD4z0qby4CUHEObHhTrQpyxDRg60dw1t8gsoN/2xZAJBjwQl5xGb/5ZAtL96VjKc8saDNMohdaefaqEUzrF+fRfm49owdfbTlJRkFpnbkDGjB7eGeGdonxefuFEzFdYdJ9qjJhbZpFJRY6U+4cAkl4sPtTl2FCeD2TeQVZdM4b0sn9hiLwHV8HNjfDroZNDR204rwhMkzgIdM0ueODTazYnw6ou/eKDGW5RTZue28j25KyPdpXXFQoX9w9kfE92tV4PDRI5/apPXnqcumK9LsZj8D0h8uzDlbTZTTM/QXaybryQNIhKoQRiTG4mkdoN0zOG9zR58cuKrWz82QOe1JyKbPL5LOAZ/ewZoW9dSd9k6RDHtp4NJPLXl3j9HmLrnFW/zjeuGG0V/s9crqAPSm5BFt0xvVs5/VyKOFjZUVwbBWUFkD7fhDXv+bzeacg9ySEx0Lb7k3SRKEs3ZfGze9scLgUUNdgWr843rppjM+OV1xm55lf9vPRumMUlKjx5tiIYG6f0pPbzuiJ7sEKB9EEspPg2SG4XjQK3LMBOvT1S5MCkQQDHnrkm118sOaYyzoDmgZ7/jnTq/kDoplI2wM//xUOLqbypNJ5JJz9CPSY0qRNa80+33SCh77cQanNwKJrmKgegWn9OvDiNSOJ8FHNj1KbwfVvrWPD0UyHOQ4uH9WFJy4bKonCAtXHV8KBX+pmEgU1JyhxPNz8vf/bBWryctpuyE9TScza92mSZsicAQ/lFdvcxZWYprp7kGCghUndBW+dDWXF1Li7SNkK78+Bqz6GfjObqHGt26WjujBjYDxfbz3J4fQCIkIsnDe4E4MTon16nM83n2DdkUynzy/YdILLRnVhXM9Ynx5X+MiFz8Pb50D28ZoTCTULRMTDxQ6yj/rDgUXqJiN9T9VjnUfCzP9C13F+bUqTBQMns4v4YtMJUnKLaR8RzJwRCfTsELgV4Hq0j3Bb0zw6LEi6+Vui7/+oAoHadxWmAWjwzX3QezdYJLZuCtFhQdwwoXujHuOjtcfQNOcrUC26xrwNSRIMBKqoeLh9Kax/Aza9p3KLRLSHEdfDuDvU1/6293uYd03dx1O2qnwIN30LXcf7rTl+HyYwTZOnft7Hy0sPoaFyfVd07V09NpF/zR6M1RJ48xrTcouZ8J9fnS4H1IC7zuzFH2f2d/i8aKYyj8Dzw91vN+RyOLFBLWNq2wPGzIUhV4A1uNGbKHwvLbeYX/emUVRmp198FLd/sJH8EgddzNUM6xLN1/dO9lMLRbNm2OF/AyEvFYdzGTQd4gfDnSv81iS/38q8tfIILy05BKgfgVHt4jpvfRJtQoP486wB/m6WW3FtQvnbBQP4xze7Hd4hmMB3O1KIiwrhmnHdpOpgS5F11LPtdnxG5R91cQ58fQ9s+wSu/QyCPC+WI5pWic3Ow1/vYv7GJAyTyr91ZyXIK+gaxIRL4Cc8dHipmozsjGnAqe1waid0HOyXJvn1ilVis/PirwedPm8C76w6Sk5RYC7xuGlSD165diT94h2nIz2eUcgj3+xm7nsbKLXJkqNmwTDUxJ2CDMd9wGExHu6o2msrxiSPrYYljzW0hcKPHpy/rTIQgKqPhOGmMpJhqvwgQngkJ8m32/mAX4OBTceyyHZzoS+1GywrX8sfiM4b0olP7xhPsIOhDLP8v5UHT/POqiN+b5vwgmGHNS/Bc0PgqT7wZE94ZSJsm1czKOg4DGK61e8YpgEb35a8583EruQcvt2e4riaqIvXWXSN3nGRzJIkRcJT4R7OLfF0Ox/wazBQVOp6zK1CoZfFRfztNx9vodRFshHThHdWH3U74VA0EcOAz26Gn/4COdWqQ6btgS/vUHnMK+g6zHi4/scqzYeDi+r/euE3C7cmux0OqGDRtcraCCMSY/j4tnGyikh4rveMugnOaotOhATv8tY0hF+Dgd5xnq0W6OOkGz4QfLc9heUHTrvd7lROccAOd7R6u76A3V9T936v/PuV/4OTm6seHnwpXPQCBEWo73Urasqoh2vKv7wTTh9oWJtFo8ssKPXoN/rytSM4q38cbSPUHIFtJ7L561c72XQsq3EbKAKX3QZ7voEv7oBPr4NfH1XLGJ0JCoOz/up6n2f/U92M+Ilfg4FusRFM6BnrNPrWNegTF8nIrjH+bJZXXlt+yONtgwJwVUTAKsyEZU/Cs0Ph3x3hueHqolyc4/tjrX9DrS92Rreq7v3qRt4AD+6BPudSNSDkYc9PWRF8//t6Nlb4S0LbMLfFK8ODLXy1JZlfdqeSka/S3JbZTRbvSePyV1ez0MsqiaIFyDsFr05WQcCOBbDnW1jxDDw3FNa+6vx1Y29X+QSCy28ytPLrRWgMzHkVBl/S6E2vzu9LC4+cLuDil1eRV2yrUaTHomuEWHXm3T4+YIv0FJfZ6f+3Hz3admyPdsy/Y0Ijt6iFyDkJb5+r0vzWSAiiq2V6t/zk22pi/+0Gxdmut+k8Em5fUvOxbx8oDxLq+Sfz222SwjiAJWUWMuWJJU5/uxZdY2yPdqw9lOF0myCLxuo/TadDVEhjNVMEEtOE16ZA2q6qksi1XfUJ9J/lfB+lBbDvByhIhzYJ0PdcsPr/8+P3W9ce7SP45t7JXDoyoXISnkXXmDW4IwvvnRSwgQC4L3lf3V1n9mq8hrQ0X90Fucl1S4yahlra9+1vfXu8IHelaDUIqTVUlXEINr5FvQOBin2IgJXYLpx7z+rt8DmLrtEhMoSUbNeTQe2GyYJN/psBLprYkWVqCaCzQEDTYeUzrvcRHKGqJY6/CwZe1CSBADRRBsLEduE8cdkw/jl7MDlFZbQJDSIsOPAn34QFW+gXH8n+1HyXl4TZwzp7XM641Tt9QP1BOWPaVaaunBMqb7cvDL4E1r7iOE+5OigMnF3zoe2fqqEFp6/xgLsJQ6LJPXB2X9pHhvDirwdJzy8B1PDl9AFx/PX8AUx5YqnL1xsmbDmW3fgNFYFh/09qWNFwMundNFQysqIsCGvr37Z5qUnzp4YGWZrdDNy5Z/Tkj59td/p8ZIiF/1w6xI8tauaqT9RzyoTkrb4LBsberrr7bcV1eyP08lzlQ6+s+Xh+WnkGmnoeM6oTJIys54uFv2iaxo0Tu3PtuK7sOJlDUZmd3h0iiWsTimGY6BoOlx5Wt/ZwBgUlNp8VSRIBzFaCRxOJbR6WUW5CMsPNS5eP6sLVYxOBmlnJLJpGWJCFd24eS3iwnAQ8pnsYDOoN/Jkmb1Gz+p/uD+/MhO6TqoYCdGvV/mO6qZzgIbVWvrTp7N04UW3T/uL5exVNzmrRGdG1LRN7tSeuTSgAuq5xZr843K0+zCuxMX+jDBW0Ch2HgOFm1VhEXNPUPvCSlDCuB9M0WbIvjfdWH2N3ci6hQTqzhnbi+vHd6NLW3Xi0qCEvFf43wPmYG4AlGH5/wItsgLVsfAe+/Z26GFd051V07Y24Xo3R6VboOQ36nO34op11DJ4bhlddA5quhhZmPAwTf1O/touAsv5IJle8tsblNhrQNz6Kn34npa1bvJJ8eLovlBbitMbAmQ/B1D/4vWnekmBANL2v7irP/OcgkZOmw5hbYdaT9dv3qR3w6hk4v4hrcNcqiB/kfl+/PAyrnvXgoBp0mwAD56gCRuHtPG6uCHwT/7OY5Jxil9u0DQ9iy9/P8VOLRJPa/zPMu1p9XTl3oLz7qPskuO6LJpsU6A0ZJhBNb9ZT0P0M9XXF+v+Kf/ucA+c8Wv99r3vddfe8blF5Bzwx4x8w4xEIiXa+jWaB9n3hmvmqNKoEAi1O/05tXA4VaEBcVKjf2iOaWN9z4LZfVfBvKS9W1bYbnPvvZhMIgPQMiEBhGCpt77aPIS8F2nSBEddCjzMbloXrueGQ5aZORGwv+I0nExnLlRXD0RWw5SM48DOUFajHdQsMuhTOe1yCgBbsx50p3Pmh88+LBvz9woHcPKmH/xolAoNpqiFPS/ObN9b8WixaJl1XEXZfH3etap4kmPUy2AgKVXML+pytxgqTN4O9TNUf92VyJBGQZgyIZ2z3tmw8llVnZYFF1+geG84VoxObpnGiaWlaswwEQIYJREvXe4br1MOaBXpPr//+g8Oh+2ToNU0CgVbCatF55+axXDKyS40VRZoGZ/WPY/4dE2RZoWh2ZJhAtGzp++GV8U5WK2iqR+Lu9dDecea5JpF9HIpzISYRQl3MTxANklVQyoJNSWw6loVF15jcuwOzh3f26kKenlfChqOZGKbJ8MQYWU3U3JQWqAJD2cfV0N6A2a02qJdgQLR8O7+AL24try1UHhRoFnUrd+lbMGhOU7auyv6fYcmjkLJNfa8HqYqJMx5WeQ6EzyzZm8ZdH26ixG6AqT4Khgkx4UG8f8vYgE6LLnxky0fwwx9UQKBb1Q2DblHLgM/6u+u5SrZSOPQr5J+CyHjoNR2swf5reyOQYEC0DpmHYcNbcGS5+r7nVBg9F9oFyCSvrZ/AV3eqpZTVl1jqFpW05LYl0KZT07WvBTmYls95zy3HZjfrLDjVNYgMtbLs99MqSxSLFmj31zD/BufPT/mD8xLD2xfAj/8HhRlVj4W1hXMfg+HX+LadfiTBgBBNKfs4LH8KNr/nfBvdotIjz3nFf+1qwf761Q4+WZ9Uo2pqdZoGD503gNum9PRzy4RfmCa8MBIyj+A0/4glGH6/v249gR2fwedzne97zqsw/GqfNdWfZAKhEE0lbY9KiLTlA9fbGXZ1EirJ80+7WrifdqU6DQRAXSt+3n3Kjy0SfpW6U/UUusomai9VBdKqM+zws5Peggq//E2tLGqGJBgQoimYJnw2V13gHWVerM1eCrkpjd+uVqDM7v7nXWLz4HcimqeibPfbaDoU19ru2GqVA8WVgvSqochmRoIBIZrCiY2Qtsu7ksgVhZVEgwxNiK6xJLA2i64xPDHGfw0S/mMYar6AO6YBbbvXfKwg3bNjFJz2ulmBQBbDupCRX8Jnm06wOyWXEKvOjAHxnNU/DqtF50RWIe+uOso325MpLLXTq0Mk14/vxpwRCS5PNEIAcGo7KledB1N2NB0SRssEQh+5aVJ3lh9wfsK2GybXjuvmxxYJv1n8D9jgLv24BuGxKhV6dW0SPDtGM135I8GAE19vPcnvF2zDZphoqDrn8zeeoGeHCP583gDun7eFYptROfa4/UQ2Dy7I5vudKbx63SiCLNLpIlywhuBxBUTThGkPNWpzWpNp/eK4cUI33ltzDL18SSGoHgG7YfLX8wfQr6P0wrQ4eamw+kX322kaXPQ8WIJqPp44Ftr2gKyjOP7b1SA6AbpN8kFj/U+uWA5sPJrJ/Z9upcxuqlTTJpUX/WMZhdz5wUYKS+01JiFVfPnrnjTeWHG4KZotmpNe09UdvztBEXD5OyrDofAJTdP4x0WDeOHqEQzrEqMyyGoaE3vF8v4tY7n1DFlF0CLt/gqPAvApf4B+s+o+rmlw/lPlKc5r9/6Wfz/r6YbVUmlCsrTQgbnvbWDpvnSXM45d6RAVwto/T5fhAuHaV3fDtk+cTyAcOFstVQqWrHaNyTBMNE0FCaIFW/IYrHgGDA9m+w++DC5+zXGdgYOL4cc/w+l9VY/F9oGZ/1H1SpopGSaoxW6YLNmbVqcAiTfS80o4lVtMQkyY7xomWp7zn1aJS/b/WDMDmmGD0bfCrCeb7V1Gc6J7ELSbpsmWpGwWbEziZFYR7SNDmDMigcm923v0euEjZcWQvgcyDsORpXB4OWCq+iBjb4fOw52/NjrRs0AAYOfnqgzx9L/Xfa73dLhnHaRshbzyDISdR3hYFC1wSc9ALcVldvr/7ccG72fdQ9OJbyM1zYUbpglJ62DbPCg8rU5YI66D+EHe76usGI6thJJ8aN+nfvsQddgNkz9+to3PN5+snFdQ8e/EXrG8eeNowoPlvqpR2ctg2ROw7jUoyan7vG5RKwXOfxrGOEkKVJwLT/UFW5FnxwyOgN8fUP+2AhIMODDliSUkZRZ6Or2rBg3o3j6CXx+cKt2Owj9ME1a/oDIZVj9Rdh4BF70IHQc3XdtagP/9sp/nFx9weD7QNbhwWGeeu2qE39vVahgGzL+uPAmQu7OyBrcvUZ99Rza+A9/e7/mxb/wGekzxfPtmTPogHbhpYne32zjrGTSBu6b2kkBA+M+Sx1Tms9p3TCnb4e1zIH2f49cJt4rL7Ly18ojTS5BhwsJtySRne3i3Kby3/0fY+x0eTf7TddV74Mzom1VxsrB2nh27mWYTrA8JBhy4fkI3zujTntpzRi26hgb8ZdYAOpfPB6i45ldMFrx9Sk8uH93Fr+0VTSh5qyqAtOk9yDrm/+PnpsCKpxw/Z9rV0MGv//Zvm1qQrUnZ5JfYXG5jmrDigIcJaYT3Nr2rqox6wrDDkWWutxlyGVz3pft96RboOMSz47YAMtDlQJBF562bxvD+mmO8u+oISVlFaBpM6dOeO6b2YnzPWK4d35WFW5P5dkcK+cU2+sZHcu24bgyTzGWtQ+ZhlU44eTNVyYM0tQLgohcgtI1/2rFjgevnTTvs/RaKcyA02j9takE8SV2sAaV2GW1tNJkHvcvUWWfZnwMJwyFhlJoEaDjYt2aBgRdDZJwXx23eJBhwIsiiM3dyD+ZO7kFRqR2rRauRSCg82MpVY7ty1diuTdhK0STy0+HtmdXSjppV/+75BvJT4abv1J1FY8s7pU5cruobmHbVVgkGvNa/Y5saiYkcMVEpjkUjCYsF3BQWqqBboaeHOTkufbPq77h6sKHpENtbreZpRVpkMJBZUMr8jUks3pNKic1gRGIM143vRp/4+mUVCwv2w0ldNB/rX6t7Aqlg2uH4GjjwC/Sb2fhtiYxzf9ek6Sq9qvBah6gQzh/Sie93nnKYd8SiawzoGCU9go1p2JVwYr1n2xp2GHeHZ9u26wl3rlRzDLZ+CIWZKpXwqJth9C3+690LEC1uNcHWpGxueGsdeSU2Kt6ZRdcwDJO/XziQmyf1aNoGiubv6QGQl+z8ec0CAy+Cy99t/LbknIRnBzvvGdAs0O88uOqjxm9LC5VZUMqlr6zmWEZBjR4Ci6YREx7Egjsn0LNDZNM1sKUryYdXJ0P2ceeBr2YBTLV6ZsS1fm1eS9GiJhDmFZdx49vrya8WCIBaJ2wCj3yzm9UHm2dFKRFAijJdP2/aIT/NP22JToCJ9zl+TrOoGgjT/uKftrQgxWV20vKKKS6z0y4imK/vncSD5/Sjc3QoFk2jXUQwt07pwfe/PUMCgcYWEqmG3RJGqu81ncpLlzUE2vZUqwTuWiOBQAO0qGGCr7acJLeozOnIkkXXeGPFYSb2bu/XdokWpk1nNYHQGd0KMX6sejfjH6q88cpnoLSg6vEO/WH2ixA/0H9taeZWHjjNi0sOsOFoJnYDgiwaFw3rzG/O6sM903pzz7TeTd3E1ik6AW5dBCc3wdGVaglHt0nQZXTzzvxnt8GJDVCSC+16Qfum+3y1qGGCOz7YyM+7Ul1OMwmyaBz4t4MiFEJ4atVzsOgfrift3fwDdJvotyYBKhA4vLQ8A2Fv6DyyeZ8o/ejI6QLu+2QLO07WzW6naxARYuWzOydKNcOWylaq8hlkHoKQNjDgwsZfSbD5fbXsN/9U1WNdx8P5zzRJ9tAWFQzc+t4GFu1x3T1r0TQOPnaeJAUS9VeSB2+eDaf3OxjD1GDwJSqxiXzGmoWT2UVc8PwKsgqdJ5ixaBqDE9rw9b2T/dgy4Rf7foCv71F1QipW5ug6jLkdznnUcbGihlrzMvz057qPaxYICoPbfoUO/Xx/XBda1JyBkd3aOs0MCCrCH9EtRgIB0TAhUXDz9yp5iV7tRBEcCWc8CBe/LoFAM/LykoPkFLnONGc3TbadyGFPSq6fWiX84sgKmHeNWkkA5cG9qVYlrHvV8QW7oYqyYNHDjp8z7VBWBIse8f1x3WhRwcCVoxMJsuhOU04YJsyV1QTCF8LbwSWvw4P74Pqv4MZvVVGT6X9rnDsJ0ShsdoPPN5/wuErp/tS8xm2Q8K9f/1X+haMPgAnr31Ardnxp5+eu0xybdtj3PRRk+Pa4brSoYCA2MoSXrhmJxaJVpgeGqlTBcyf3YObgjk3VPNESRbSHXtOgxxkQHN7UrRFeKiixU1zmPstghbAgyTnSYuQmq4qhrub+aBrs8iB1sbfH1d3dMJg15xL4QYu7hZkxMJ4ff3sG76w6ys+7UymzGwzrEsNNk7pzZt8OMkQghKgUEWIhxKpTYnMfEIQFW5gkK5FajqIs99toFvdLib0V3t6z9Mp+ThTW4oIBgN5xUfz74iH8++LWU2RCCOE9q0VnzogEPtt0wmGGwepuO6MnESEt8pTZOkV1Kp8w6OLCbNhqLhNO3wcnNqpU493PUEsevTX4Evj5r86f13S17yj/9mLLJ1sI0ardO603P+xMoaDYjt3J4qobJ3Tj/ul9/Nwy0ajC26lMobsXOg8IrKEw6GI1b+DLO+DoiqrnNB0GXQIXPqsmFXsqqiNMug9W/q/uc5quApTpf/fqrfhCi1paKIQQ9XEgNY8HF2xj+4mqPAMWDYZ3bcvjlw6hd5zkF2iRMo/AG9OgOLdWQFBeifSiF1TOgVfPUCnIa1c41CyQOFZNIPZm4rBhwPInVUBgK6p6PKYrzH5ZzUHyMwkGhBAtmmmaHs8V2p2cy/7UPEKDdCb2bk+b0KBGbp1ocpmH4ceHVNKhilUFsb3hrL/BoDmw/ClY8m/XEw2v/AgGXOD9sYtz4eAv6t92PdXwgN408/olGBBCtDhJmYW8seIwX24+SV6JjU7RoVw3vhs3TuxOpIz7C0fyTkHWMVWtsEP/qlwhzw2HrCPOX9dCioE1+2DANE2W7U/nux0paMDMwR05s28cuqvsQ0KIFmvnyRyuen0tRWX2GpMCdQ36xEUx/84JRIfJHb/w0GNdoNRNfomE0XDbYv+0p5E06xB5/6k8rnx9TY00ovM3niAmPIh5t42nf6fWVY9aBLDs46qSYWQ8xCQ2dWtaLMMwueejzRSV1p0MaJhwMD2f/3y/h/9eOrSJWiianaiOkJGP48REqJ6BFvA33WyTDqXnlXD+C47ziWcXljH7pVWk55U0QcuEqOb4OnjrHHh2CLw5HZ4dDO/MghObmrplLdLqQxkcyyx0uirAbph8sfkkucWu0w8LUWnUTa6fN+0w4jq/NKUxNdtg4G9f76DM7nyEo8Rm8NTP+/zYIiFqOboS3j1flSit7vgaeGcmHF/bNO1qwXaczMHiZrJgqd3gUFq+n1okmiVbCWybB+9eCBvfUcWDHCW613ToOxN6nuX3Jvpasw0GluxNd7vNd9tT/NASIRwwTVh4n1qKVHsWsmmoZCbf/k5tJ3wmyKJhuixirgRbm+2pTzS2oizVm/flHXBsJWQeBFsxdYYJrKEw9g644v0mWwHgS832HXiSPrS4zIOUj6LlCYQLbNJ6VRsdJ59T04C03ZC8xa/Naumm9Y9zW3QoLiqEfvGSN0A4sfA+OLVDfV0RyFf8q+nQazpc+zn8fj+c91+whjRNO32s2U4gDLJoLocJAKJCvXt7pmmy8VgWH609xt5TeUSFWrlgaGcuGZlAlKw3DmxJ62HV83DgZzDKIH4wjL8Lhl7VNFF71lHPt0sY6f3+DQNK8yEoXKokVtOrQyQzBsSxZG+603kDd07thdXSbO+DRGPKToI93+B0sqBpwJHlqkx5aLRfm9bYmu1fxDkD491uc8nILh7vzzRN/rFwF5e/uoZvtqew91QeG49m8Y+Fuzj7meUcPV3QkOaKxrTtU9Wtt/8HsJeoP9jUnfDVXaqrz/C8Kp3PhMV4uF1b7/ZbmAm//B2e6A7/TYR/d4Qvboe0vd62sMV65srhjOwWA1RVLK3495ZJPbh5UvcmapkIeMdW4TQQqGCUwUtjYc1LdTMSNmPNNs9ASnYRU59cQqmT3oE2oVY2/GUGIR6WHP143XEe+nKHw+csukbXduEsfmCq5C8INLnJaqa+YXO+zZxXYPg1/msTQFkxPN0XinOcbxPeHh7cCxYPe53y0+GtGerupXrqVN0CejDcuFClRhUYhsmKg6f5eutJcgrLSGwXzlVjE+nfUZYbCxe2fgJf3enhxpqqTXDpmy1izkCzDQYA9qfmcu0b60jPL63xeM8OESy4YwKxkZ6N5ZimyZlPLeV4RqHLmPCdm8cwrV9cA1osfG7pf2HZ485ThWq6GjK4c4Xj550xDHWXkJcCER1UmlBvu+PXvQY//NH58xc8C6Nv9nx/X9wOOz5zXFRFs0CbTvDbHS3ixNQS/LI7lbdXHmHz8Sx0TWNy7/bcekYPxvX0b2la4YXMw/D8CO9ec8UHquBRM9esBxv7xrdh/V9msOFoJkv2pmOxaMwZlkDv+Eiv9pOWV8KxjEKX21h1jdUHT0swEGhStrrOGV4xZGCaVelF3dn7PfzwB8g5UfVYZDyc828YernnbRt7u5qFvOTfaqmSblU9GNYwmPGwd4FAYSbs/Nx5dTXTrtp76FfoM8Pz/YpG8fiPe3ll6SF0jcoJjb/uS+OXPan8a85grh/fzfUORNNo1xN6n63+jlyVNq6gWWDDGxIMBAJN0xjbI5axPeofbRsedo64m6UsmoAlWN39uwoI9CDPA4H9P8E8B0MK+anwxa3qOMOu9GxfmgaTfquSluxeCPmnIKqzOnF4U/IUIOOg66EQUCem1J0SDLiQU1TGu6uO8vH6Y6TllRATFsRlo7owd3JPOkaH+uQYy/en88rSQ0DNc0ZFauS/f7WTCT3bSSXEQDXnFXh3Fpw+gNv5A6a9xczXkf5EIC4qlI5tXA8p2AyT0d28nOwlGl+fc9wEAlboe65n+zJN+OH/Kr5xvM1PD4HdTfa61N2w+kVY+SwcWQEhbWDk9TDlDzDiWu8DAVBrmt0xjfLkKMKR0/klzHlxJc8t3k9qbgmmCVmFZby96iiznlvBQR8lInpn9ZHKCYuO6LrGh2uP++RYohFEdoDblsCsJ9VqHXeCIxq/TX4gwQBqguAtk3s4yi8FqAIn8W1CmOHBCgbhZ4MvhahO6q7YEcMOE3/j2b5ObiqvTubibqDwNBxa4uS5THh/DrwyAX75Gyz+J7x3gZp5nLbHszY4Ez8I2nR2s5GpJhc2xeqJZuAfC3dxPKuoTg+f3TDJKS7jt/N8k/Nh6/HsGgWSarMbJpuOZfnkWKKW0kIoyGj4LP+QSBh7G0x/GIeZBytoFhhyWcOOFSAkGCh3y6QenDekI6Au/hV0DSJDrLx14xiCZG1y4AkKgxu+VpP8QA0ZVPyrW+Hi1zyfYZ+f6uF2p+o+Zi+DD+aoNcig7tIrxhwzDsE756mVD/WlW+CM37vfbs2L8M1v63+cFio9r4Tvd6Q4vUjbDZNdyblsS8pu8LE8OU+ESAZE30paDx9dDo91hid7whM9YdE/XK/mASjOVXOEdn0Jpw/WfX741Wq+kKObDc2igobRc33yFppas58z4CtWi86LV4/khyGn+HDtMfal5hEZYuWiYZ25bnw3n40nikbQoR/8divs/AIO/KQm63UeCSNvUDPsPRXZ0bPtohzsc+93kLLN8famXZ101r0GZz/ieXtqG32LCliWPe5iIxO2vK/mKXQZVf9jtTB7T+W6nfOjAdtP5jAsMaZBx5o+IJ75G5OcBh66BmcNkInIPrP3e/i0olBQ+c+8OFslIdv3I8z9qW6CILsNfv2n+pu0FVc93n0yzH4J2nZX34dGw03fwUeXqV5DvfySadjUDcg1n3p3jglgzXppoRA+ZZrwwii1vMjZUEFEB3hgT93cAPOug33fu56BHNkRfu+D4llPD4A8F70MuhVGXA8XPtvwY7UQaw9ncNXr7gtDPXHpUK4Y07BytAdS8zjvuRXYjbpVEnQNwoOtLPvDmR4vfRYulBbCU32gtACHf7OaBcbdATP/U/VYfhosuNlxgiHNAuGxailyVLWbA8MOB35RPX+mAV3HQf8LPM8R0gxIX5UQFTQNzqu463YyTnjufxyfAAoz3C9Fctdl6amCNNfPGzbIlglq1Q1PjCEqxHVHqK7BGX3bN/hYfeKjeOnakVgtWuWQo1b+X0SIlfduGSOBgK/s+kKl5XaaPtgOm99TScCKc+GLO+DpfqoAkaPXmHb1t7z6hZqP6xboNxNmPqbqEQy6uEUFAiDBgBA19TkbrpkPMV1rPh7VCS5723megdieVV2IDmnQtquL573gLid6xd2NqBQaZGHuGT2cPq9rcNGwznSK9s1qjHMHdWTVn87i/hl9mdKnPWf268BfLxjIyv87i1Hd2vnkGAI1MVd3c1EuLYD9P8Lr02D7fNerj6AqgGhlneYyZ0CI2vqeA723won1kHsSIuKg20R1d+DMyBthy4eu9zvqFt+0b9jVsPYV1wmIhl7hm2O1IL85qw8ns4pYsOkEFl3DbpiV/07oFctjlwzx6fHiokK5b3ofn+5T1BIUhttcAAALbvRuvyV5au5RUOuZKyZzBoTwBdNUpU+3vF/3Oc0CnYeriUi+yAOQmwyvTFTdnrUDAs2iVk/c9L2kJXZiW1I28zcmcTK7iPaRIVwyIoEJvWLRPE1MJQJH8lZ4fWrj7Hv4dXDhc62mKqgEA0L4imHA6ufVeGPhafWYNUwlHJr+sFqG5Cvp+2DBTZC2uzwDY/mf8YALYPbLECoFeUQr8f6c8ol9vq4gWJ5BtCErgJoRCQaE8DV7GaTuUjOQO/StX8ZBT5imWl+dvFlNZuo1Hdo5HxcXokUqzoF518LRFdWW/tnxaPjAHWso/P5AqwiuJRgQQgjRvFUExru/UuP9ZcWwc4Fv9n3F+zBwtm/2FcBkUFEIIUTzpmlq7f/M/0DPM+HkRt/tu6zY/TYtQOuYGSFEc2UrURUPd34GRVkQ20dlF0wc09QtEyLwLHtClQx3VU/AW/EDfbevACbDBEIEqvw0eO8iSN9TVaZZt6jx0NFzYdZTsmJAiAppe+Dl8b7bn2aBTsPgdieFyVoYOZMIEag+vR5O71dfVyRKqajGtvEtWPdq07RLiEC06V03ib+8oFvUxN85r/hmf82ABANCBKKTmyFprevlUqufa3ipViFairTdKhV3Q1lDVI6BO5ZBXP+G76+ZkDkDQgSiQ7+qbkpXwUDeKTh9oFWdsIRwKjiyajjNW5oOCaNVFcKQNq0m0VB10jMgRCAy7GqGtNvtfHAnJERLMHC2B4GAk78p04AzHoTwdq0yEADpGRDCf9L3w9YPITtJFRIaegV0GeP4ot9llPsLfUgUxPZqnLYK0dwMnANL/6P+vuqk6dZVQaOI9qreSEWvm2YBDDjnMVWVsBWT1QRCNDbThJ//CmteVBOTTFOdnAwb9J0Jl79bt2aBYcALIxyf2EC9fsI9cM6jfnkLQjQL2cfho8shfW/5ZEINjDIIjYErP4TEcbBnIez9FsoKIW6gKjImmTslGBCi0a1+QQUDjmg6DL0SLnawMuDUTnj3fJVRrTIgKO9F6Doerv/SN4WPhGhJDAMOL4GDi1Rq8IRRMGiO/K24IcGAEI3JXgZP94PCDOfbaDrcvxOiE+o+l5sM616D7Z+qKoXtusOYW2H4tWrWsxBC+IAEA0I0pqT18NbZ7re74FkYfXOjN0cIIRyR1QRCNCabh3nNt3ykhgWEEKIJSM+AEI0pPw2e7u++1rpWHpfPeQWGXdX47RLCl46sUBkxj61Wn+VeZ8H4uyBhZFO3THhIggEhGtv8m9QMZncBAagT6d3roEPfRm+WED6x4hlY/EhV3Qwo/9qA2S/BiGubtn3CIzJMIERjO+9xNTlQs3iwsabqDgjRHBxbrQIBqJka27ADJiy8V2XJ9FZBBmx8G5Y/CdvmQWmBT5ornJNgQIjGFhUPty2FSffhtrSqaYcjy/3RKiEabu2rqhfAKU1d1D1lmrDkMbUC59sHYOl/4cs74Kk+sPn9BjdXOCfBgBD+EBELM/4BbTq731aTP0vRTBxf47pYlmmHo6s839+yJ2DZ4ypREGZVFs7SAlj4G9j5eYOaK5yTs44Q/tRruusyq5oFek3zX3tEy2crhZTtkLINyjxc3eIp3YNLSEXPQX4aZB6GsiLH2xXnwMpnXO9r0SNqLoLwOalNIIQ/jb9T1SdwSFO9AqPn+rVJooWy22DF07DuFSjKUo+FRMPYW2Hqn8Aa3PBj9D4btn3ivI6GpkO7XvDmDDixQT0WFKZKBJ/5Z9VjVmHfj+6X4mYfg1PboPOIhrdd1CCrCYTwt23z4Ku7VIGiii5WzaJOnJe/CwMuaNLmNZhhh4OL4cT6qp6OxHGeVWEUvmEYsOAmtYqFWqd4TYeeZ6lyvQ2t0HdqJ7w2xclKGU31ghk29buvXlFQs0DbbnDrYlUpEFSmzR/+r257axt0Mcx4RL2+tqxjagLuwV/VfrqfoTJ2tu9dzzfYekgwIERTOH0ANryp1mdrmlqXPWYutO3u/b4KM2HrR5C0Tp1ke06FIVdASKTr15mm+s+Trl5PndoJ865Rd3AVwyGGDToNh6s+dpxyWfje/p/g4ytcb3PpWzDksoYfa8dnapKfaVYFBRVVAjHBXur4dZpFXahnPeF5myv2relw6ZsqMKiw9zuYf6MKOsxqQTYmzHkVhl1Z33fYKkgwIERzdmARzL9Oda+aoFYrGBDWFq77XBVpqe3YGlj9PBz4Wd3FdxwC4+6EYVc3LDDITYGXx9cqrFROs6hA565VUjDGHz6+Sv1+neW20CzQbQLc9J1vjpd1VK0aqJ50yBqixvhd3ekHhcMfj0BQqBrW+N9ANbfAXe8AqLkId66GuP7q+C+MLh+ucPBaTYc7VkDHwfV6e62BTCAUork6fRDmXa0mhZkm6iRY3hVbnAsfXKzWa1e35SN45zx1F1Zx4kzdCV/fDV/e3rDJWetfdxwIgHos8xDs/KL++xeeyzjgOsmVaYeMQ747XtvucPY/4dZFMPdnOPNP6qLuarIsqDLC+afU1xYrXPi86inzaEWNButfU19ufLt8GMJJEKHpahhCOCXBgBDN1frXqpK71Gba1YV5ywdVj+WcUMuzMGteKCrGcncsUNUR62v7fNcXIE2HnZ/Vf//Cc2HtcJvTIqxt47YhOBKP7vCDo6q+7jcTrv9KDSu5Y9jgwC/q64O/uv7sGTY4tNj9PlsxCQaEaK72fOPm7s9Q46gV3CVt0XSVX76+SnJdP28aavmYaHzuxsc1vfFrYAy8yPkqAygfqphYc0UBqDkvty+BWA8m/VV+/j3p0ZIRcVckGBCiubKXud/GVlL1dfJW98HDqR31b09sb9fdu7oVYvvUf//Cc0OvgpiujlNgaxaIjIcR1/v2mJlH1DDQ7oVqeKrjEOh3npM03OWrC6b+n/P99ZzmJieHDlGdVYDZ/QzX6b51i9pGOCXBgBDNVcJI9yfALtUmEFqDcfsnbwmqf3vGzK25fKw2wwajb67//oXnQiLh5u+h0zD1vWap+qzEDVDPVSzpa6i8U/DhpfD8cPjsZph/vUon/M39cNFL0Pfcqjbo5Z+voDC1GqDnmc73O+ZWN9kNDTi5UaUqtoS4+ezZYewdXr6x1kVWEwjRXB1YBB9d6nqbO1dVzaDe8iF8fY/zbXUrDLgILn+nfu2xl8FHl8ORZY5PzKNugQuekXwD/mSacHJTeb0LE7pOhK7jffc7KMqG16dCdpKDFSTl+QyuXQDpe1SPQWk+tO8Dgy+FkCiHu6xh07sqqNAtroccAIZeqea9aHrVthV5Ds57EsbdXo832HpIMCBEc2Wa8OOfVYY5Ta+6AGsWdWI+51GY+Juq7UsL4YWRapa3o+ECTYe5i2r2JnjLVqIqza1/A4qz1WNtOsPE38K4OyQQAMhNho3vwP4fwV4CXcbB2Nug09Cmbpn3lj8FS/7t+q78us+h94z6H+PkJljzUvlKFBeXq9BouO5LFUAcWkxl0qGxdzTsM91KSDAgRHNmmrDrS1j7suoyRYceU2DivY5PwOn74P3ZkJdSHkCYVUu5Ln7NN0loQOXDzzqi7szadndT2a4VObICPr5cBU0VF9CKu9dzH4MJLnpuGkNZkeo1KM6F2F4qza83AduzQ1WCKWd0CwycA5d5UbnQkcPL4P2L3G939adqRYLwmtQmEKI50zQYfIn6zzDKL+wuTuYd+sF9W1QAceBn1bXfeTiMuEGVWvYVa7A6lqhSmAmfXFkzEICqLu2fHlKT7npMafy2mCaseVFVCay+CiRuEMx+wXGyKkcK0lw/b9gh92T921mhoraCr7YTdUgwIERL4Wn2wKAwGH6N+k/4z5YPVZIdZ52xukV1h/sjGFj2OCz9T93H0/fAO7NU8qCOQ9zvJyLOdc+AZoE2PkhB7agOgcPtujf8WK2UrCYQQgh/OLLceSAA6i768LLGb0d+Oix/wvFzpqF6ixb/y7N9jbzB9XJS0+6boLPTcIgb6OJYOrTrqSZHinqRngEhmqvDy1QK4BMbwBIM/WbB2Nsbr0Lb6QOw83PVFRvTTc3erp0wRrjgyfQsP0zh2vm566DEtKshpILTENHe9b7G3KqyXLpaTdDzrIa3WdPggmfhvQvUsEqNCojlhYsufE4mqDaA9AwI0Rwt/peaULX/B8hPhZwkVQXxlQlVKVp9xVYKX94JL45WY8wb3oKf/wJP94W1r/j2WC1Z1wmu76I1C3Sb1PjtyD/lOj8FACYUpLvfV1gM3PKTKkxUPf2xHgQjb4KrPvJdVcyu4+DmH9XPsbouY1TBJX8Mr7RgsppAiOZm3w/wibNUspqqFve7Xe7v6jy18L7yVMZOThWXvAFDPSg929rlp8Gzg1Vw5exnee3n0KcBy/A8sfZV+OlPrnsH0OAPB737DGUegeQtanVEt0mN22uUfVwlO4qMcz1PwFaietCKMlVGxq4TpPfACQkGhGhu3j1flSF2Wp5Wh+l/h8m/a9hxTFPVKvjxT663a9tDrVAI9JNs6i5Vue7gIvWz6zZJlW5OHOu/Nuz/CT69Ts0PqPj9VSwtPPMhONNFel5fyU+HZ/o5z+6nWaDP2XBNA4pWBYINb8Hif1bluwAVOFzwLPSa1kSNClwSDAjR3PyrA9hLXW/T51y4dn7DjvPzX2H1C55te/2X5V3FAWr7AlWi2VF2On+v7884pIZ09n2vJut1Gauy43Wb6L82LHtCJQuqTbOo+Se3LqrKXNkcrX0VfnQQWGm6Clpv+Aa6+2FIphmRYECI5ubRuJoFiOrQVOKVq+fV/xjHVsM753nxAg1GXg+zni6vgRBAMg7Bi2NcF2m65Wc1Jt1amKZKVLXs8ZqVJOMHw0UvqLoXzVVJvqpXUFbo+HlNVzUbbl/q12YFOllNIERz030KHFyM87KtJnRrYIW2DW9V3Tl7xFTr6O1lcHEDyiA3ho1ust/pVjUc0pqCAU1TvSGj58LRFSrxULte6iIZ6MM97uz9znkgAGolQvIWOH2w8VbeNEOymkCI5mbivbit3149q1x9nNruRSBQzjRg2yfqJBtIjq5y3Stg2ODoSv+1J5AEhar5AYMvVZkom3sgAGp1jdvVEuXbiUoSDAjR3CSMdl9qeO3LKu98fQVH1O91mgV2fVH/4zYGT5a2uVryJ5qXqE6ug7/K7To2fluaEfkLEKK5ObJMdce7UpLbsLvdgbPrd4HU9MDLD99zmus7Rd0KvQN48qPwTv9ZroNZTVe5CWJ7+a9NzYAEA0I0N6UFHm6XX/9jjLhBlYT1pLu1OsPmfN13WZGaV/DuhfDyBJh3nUqQZLgZ8mio0TerCz5OusANu1piKFqG4Ag420k6ZU1Xn+lzHKykaOUkGBAiUKXvg/VvwLrXIWVb1ePt+3r2+vYNqBoYEQs3fgMRHdT3urX8guqGJQiGXF738fw0eG0KfH0PHFsJabvV0rqPLoMFN7nv6WiI6C5w5YdqyVz14Ea3qIvD7JfUxDnRcoyZq36vEXE1H4/tAzd83bomi3pIlhYKEWgKTsPnt8LhJVTdzZpqPfplb6uL22tTVBIdR2OjmkUtDbt1UcPaUVoIO7+Agz9Dbgq07armK6z4HxSernVsTbVx1lMw9ra6+3pnFiStdZLoRoMpf4Cz/uJ520xT9UK4mztRXXaSWllwaLHqjeg+SeXWb9/H8334mmGoORbrX1e/z6AwGDhH9VTITPeGs9vg2KqqDISdR7aMSZKNQIIBIQJJWTG8MQ1O76t74dQtanLUnatUOtZ3Zqrtq1+UNQsEh6t18/EDPT+uYVdd9qk7wBqqvl/+pBpq0K3lx9DVRWrcHbDoYdi9sOrYsX1g2kMw+JK6+z61A16d7Pr4IdHw+/1qdrsrmYdh1XOwfb5aPhbRAUbfAuPvVnnymxPDDp/fBrs+Vz0UFcV3dIv6mV+zAHpObdo2ilZD8gwIEUh2fq660B0x7JB7UtUJmHQf3LZU1aTf87V6TrfCoIvhzD97Nznq+DpYcCPkpZTnFrBTI3d+5RJDu1qloOlw+btQkAFZRyEkUg1dOLvjOrKi5sXOkZIcdWfcZZTzbZK3qlTMtqKqQKkgXQUtOz9XAZCn+fBtJWroIjgCwtt59hpf2/h21cqL6j8bw656Pj69Fh7Yq36+gejICvV5OLICMKH7ZBWUSQDTLEnPgBCB5L0L1SoAVxfODv3hnnVV35fkqxn84e28XxKYvg9emwr2EtfHrE63qotUZAfPtl/9IvzyN/f7n7sIEsc4fs4w4IWRqkfE2dDI0CvcJzwqylKpeDe/VzURs9skmPp/DbuI2W1q6MQa6lkPhWmq95N5BJdliy98DkbdVP92NZY1L8FPD6mfe2WNBYsKZGY8ApPvb9LmCe/JBEIhAklBuvuLZmFGze9DIiEmsX65AVY9C0ap54EAqBP+3m883z5xnPv9B4W7HtY4uhyyjjhfP27aYcdnUJjpfB9FWfDm2apYUfUVGcfXwvuz1eu9VVoAvz4KT/WGp/vB493g7ZnlGSJdKMlVQx6uAgHdCic2eNYOuw3yUqG4gcmmPJG8VQUCUPP3UdFbs+hhSPKw3SJgSDAgRCBp28P1cj5NVxOhfMEwYMfnzqvXOaPrNfPZu9NlNHQc6nw1gqbDqJtdBzOndrjPe2CUwen9zp9f+l91Aa4dUFR8v/BeKMlzfYzqSgvUsMWKp2vmVkhaDx9eqpZROuPpkk13KzhK8mDRI/BkL3i6L/w3US3dPLLcs/3Xx/o3XLdLt6oJkaJZkWBAiEAy8kbX2dNMQ02Y8wV7ifrPW4bddQ352jQNrnhfTfarfkGv+LrrBJj+N9f7sISornV3rCGOHy8rVnMtnP5sTbXNjgXuj1Fh1fNqyWftXg+zfM7FN/erlSGOhERCwijXAY5hc10JsiQP3j5P9e5UL9N7bBW8d5Gq1NgYkta6TlVt2CBpnfPnRUCSYECIQNLnHOh7nuOLhKZD4ngYcoX6Pm2vKjO84Cb47vfqjtSbKUDWUAhv72UDNQhrC/1mefeydj3grtVw1t/UZMPw9mqZ1+yX4fqv1JI6V/rMcH+MyHiIH+L4ufxU18VrQN3RprvoWajOMFQZYlfDH6Ydtn7k/PlJ9zt/vWaB6MSaP+f0/bDrK9j/s1r2ueIZSNvlPBhZeA8UZXv2fryhe7Cc05OcFCKgyG9MiECi6+ouesmj6mJTMbZtDYUR18PZj6gT7fd/hPWvlS/7M1SgsOEN6He+ykXgbokeqDv2MXNh+VOe5XLXdBVsXPic8ztwV8LbwRkPqP+81a6nSpG8Z6HzC+jk34HFySnNo/kUhufzLkpy1IRBVzRdTdB0ZuBFcNZf1ZyDygqR5fkaIuPg+i9VHoXTB9UQxvE1Va8NjlTbuwpGbKVqCea42z17T57qe64ajnH1mfE0MZYIGLKaQIhAVVoAKdvVCb/jYJUeGGDl/2DRPxy/RtNh+DUq+5oninPUpLqMg+4Dgk4jYMbfXXddN6aSfLXc7vDSqiWQukVdFCfcC+c86jqhzFvnqAl5ri6gd6yATkPdt6WsCP7dCbcTAEfdDOc/5Xpfqbtg4zuqUmRQuAp6hlyuhhKyj6sEU8W5ngVsNY4fBCOvhwv+593r3Mk+Di+OVsszXZn2F5j6R98eWzQaCQaEaE5sJWrWuqtiQJoFHtjteVW2wkw1CW37J1Un+OhEmPJ7NWSRe1INDbTr0fD256fD1o9h//dqzDtuIIyeC90mePZ601RLL3csKM8q1031mMT1d//ag4vVxD5HF3DNooKc67xYUfDhZXDoV9cX6Ru/gR5TPN9nbd/8Vk1E9LacNKhgZNydcK6DPPwnN8G+H1XOhvghKgDxpDepwsHF8PGVatKmK54GV6LJSTAgRHNybDW8c5777Wa/BCOu827fxTlqtr01VNU18KT0r6cOL1NDH0nraz5esTZ97O1w3hONnyp226fwzX0q6NGtQHla495nw+XvQEiU5/s6tgbenVU+T6PWaVSzQMIIlTuhvu/JXgb/6QK24vq9HlQipup5+AszYf4NcHSF+tmjqwt6aIwaXuo93fN9vzkDTmzEae+IblEFry58tv7tF34jcwaEaE48ujBo9buAhEZD5xHev86dLR+qAkWOVCxrXP86xA9q/AQ7w66EfjPVWPrpA6orfsBF0Hm49/vqNkFdQL+8szy4KF8uaNjUSoGr5zUsuCnJq38goFlUEJA4tuoxw4CPr4CTm8u/twPlP//iXHWnf9tiz4s2nd6Py2ESw66WhIpmQYIBIZqTuIHuU/tiOp9V7295qaqr2y1NLdUbeWPj9w6ERjsuplQfgy5Wwwvb50PaHrUqov8F0HW84/dR0RHryXsMiVKVFu2lzrfRdEBTn4fqwUjXcXDVxzWPc2SZiyRGBqDDymdVD4knrGGAq3wTmqqTIZoFCQaEaE6iOqoVA/u+d56Wt33fmneETWnLBx5mNzQh8xDknYI2nRq9WT7lSXCx/2dY/YLKAYAJXcaoSY8DLnQeGFiC1DLS7Z84TwxlGqo65YlNkL5XrYYYcJH6/dfe7+6vqq1acMCwqdUaFRMz3Rk4u3x5pYs5EwMucr8fERAkGBCiuZn1JKRsUWWFq5+IdQsERcClbwZOmVZnRZec8SYtsiP5aWp2viVYddV7Mymusax4Ghb/s2Ye/xMbYf71KtfA2Y84f+2U36vUzyX5Di66Goy8QQUWXZzUdKiuJN/9z9ewqSEPT+7ox9+pajzYzbr71SwQ0R6GXul+PyIgSNIhIZqbNp3g9mUw8Tdq4heo7umRN8Idy9UyxEBhDXOfRrhCmwRVork+8tNhwc3wdH/4YI6a2PdUX1j2pBorbyonN6tAAGpezCu+XvWsmlzpTLsecMtPdWfkW0NVXgVvlg3G9kblMXAhMt59AqjKtvWEaz9TOQ+gasii4uue01RwJpoFWU0gRHNmmmqSmTU0cHoDqtv7Pcy72oMNNTjnXyrAqc401SoHS5DzhEBFWfDGWZB1zPnd80XP16f1DffVPbB9nvOued0KfWfCVS4yFVZI2a6GAoLCoMdUCG3jXVuyk+C5Ic6zVGo6nPkQTP2Dd/stLVB5Lza8WXNlhWZRX89+GYZ78hkQTUl6BoRozjRNXRwCMRAAla2uQ3/3hXkGzobxd1d9b7fB2lfguaGqEuBjnVXSoL3f133t2lch66iTsWtTdWUnb23Am2iAkxvd5/E/ucmzfXUaqso0D7jQ+0AAVGXLsx8t/6bW50WzQMchMP4u7/dblAWb3qu7xNK0q+GDr+9WgYwIaBIMCCEaj25RaXWdJUDq0B8ufw8ue6fabHg7LLgBfvyzuputcGKD6mVY/ULNfWx6x/VYuG51XUGwMXmSttmb1M4leSpb4Q9/UsMPyVu8a8/Ee+Hyd9WqlAohUTDhbrjpO7XU0lsb3ykPeFz0OKx7zfv9Cr+SCYRCiMa1/g2VxbAi7z5UfT3oEhg0p+b22z6Bvd/V3U/FBf/nv6nMiO17q7vR/FTXxzdskHOiQW+h3vqdr9bauypI1P8Cz/a160v46m5VcEkPAkw1ObHXWSqg8rS3YNDFMHAO5CarIaY2CQ2baOkuC6Nhg0OL6r9/4RfSMyCEaDxHVsDKZ8q/qX7nWP710scgqdba9/Wvu550qOmqNwDU8EhFzQZndCtExHrTat8ZdZOqN+Dw/WiqsNKYW93v5+hKNUGyrEh9b5RVDT8cXqayCnpD0yA6AWJ7NXzFhScrQGRqWsCTYEAI0XjWv+F6voBuUdUWq0vb4740cOrOqu+HX+v6GIat6Za4RcXDdV+UpznWqs2411SQcPWnntV8WPZ4+WsdXFRNOxxe4vncA1/rNsnN79gK3Sf7rz2iXmSYQAhRf4YBhxZD0jp1Qeh5Zs3se8mb3XQh2+texKyhrrPuoasLaYUJ96ihBUeV/TQdepwJ3c/w/D156tQOVaI4KFwVI3I23t51HNy/E9a8pGoCWIKhzzlqhn1YW/fHKc6BI8tdb6NbYffXKreCv42ZC+tedf68YYOxd/ivPaJeJBgQQtRP6i6Yd42aya+Xn0qW/Vfltr/qY4ju4uEEulrd1ANnq4u701n4hppRXyG6i1qL/9kt5T0G5fMRNF2VAr7gWd+utkjdrWotJG+ueiwoHCbeB1P/r26Bp5Ob1GTIpHVVjxVmQPxAFTy5U1roQaM0lVSoKcT2gjmvwFd3qp95xe+tItvhuY/VLJYkApLkGRBCeC/vFLw83vHduG6B6K5w12oVHKx+0XnvgKbDlD/CtD9XPZa2B16bopYX4iCzXZvOcO+GuslxTFNl9kvZqoKQXtPVuLgvZRyC189Ua+sdvadxd8J5j1d9f2IjvDNLjfHXGPrQVcxy9Ty1/NIVexk80UOtJHBKg5n/VVkBm8qpHWrVwKHF6r12mwzj7gic1NjCJQkGhBDeW/wvWPk/10MAF72g7nxfHAv2Egcpa3V1Qb93U916BAcXqUlxpQVVvQ6GTWW9u+5z9W9T+OIO2LHAxfvW4L7NVe17bSqc2u5kDoSmllz+bpf7WgA//xXWvOz8uJYQeHAvhLfz9J0IUYNMIBRCeG/HfNeBABrs/BxiusK1C8rH+Csm0JVPoguJguu+dFyYqPcMeHAfnP8MDLsKRlyvhh7u2dB0gUBZkXpPrt63psO2T9XXqbtVL4XTyZAm5KXA4aXuj33G71U64doT9SomJF7wPwkERIPInAEhhPeKXXVZA5hQlK2+7HEGPLAbts0rr9qnqdnlw64qn2XvREiUmpzGXN+0uaGKc1R3vyuaBvmn1NfZxz3bryfbhcXA3J9h2RMqo2Jp+fyAhNFqnkKfGZ4dSwgnJBgQQngvtlf5SgEnd726Bdr3qfo+NFqNH49rxrPKQ6NVsh9XAYFpQmR5tkVP79Q93S4sBmY+BjMeVnM2giNUZUAhfECGCYQQ3hsz13UuAMMOo272X3v8ISgMhlzmenzfNGBYeU6DhNFqpYMrwZHQ+2zv2mENgbbdJBAQPiXBgBDCe0OuUGlwnWUKHHkjdJvo3zb5w9Q/qgu4syQ74+6smtOg6zDjEdf7m/YQBIe73kYIP5DVBEKI+rGVwoqn1HKy4mz1WFQnmPRblWSm9np7fyvJVxMd9/+kkhh1Gg6jboS23Ru237S9qhJf9WRJwREw6X410a/2+97yIfz4J7U0ULOoCYjWMLWccuJ9gVtxUrQqEgwIIRrGVgpZR9SFrl0P98vk/CF1N7w/GwrSqEpCZFH/XvgcjPQyl7/DY+yC9L0QFKEmSQZHON+2rEgVX8pNhogO0P/8+pUhFqKRSDAghGhZSgvh+eFQcNrJMkBNlevtPsnfLRMiYMmcASFEy7LrC1XW2FXWw9Uv+LdNQgQ4CQaEEC3LwUWuSyCbdjj4i5TVFaIaCQaEEC2Lvcz9hd60SzAgRDUSDAghWpbOI1zP0Nd06Di06Vc7CBFA5K9BCNGyjLyhfOWAk4DANGD83X5tkhCBTlYTCCFanl1fwmdzVQ+BYVOPaboKBIZfA7Nfdtx7YJpw6FfY8oGqGRARp2oo9L8ALJK9XbRcEgwIIVqm5C2w5iXY90N50qGhMPZOlVLYUSBgK4XPblL5ACqSA1X8mzAarv9C1ScQogWSYEAIIQB++iuseRFwcErULNB/Flz5od+bJYQ/yJwBIYTITYG1L+EwEADVO7DnW8g65tdmCeEvMggmhGgd8tPUXIC0PaoCYf8LofcMtargczdVGAEw4cgyaOuDVMZCBBgJBoQQLd/mD+Db+8sv+JqaM7D5fYgfArNfgGOrPNuP4SSroRDNnMwZEEK0bId+hQ8udvycboGozpCT5Nm+7lwFHQf7rm1CBAiZMyCEaNmWP12ed8ABw+55INC+nwQCosWSYEAI0XKV5MOxlc6LFoHzQKG2i1/zTZuECEASDAghWi5bifttNA0iO7oICjQYcBEkjPBp04QIJBIMCCFarrC2KougK4YdxsyF6IRa1Q419V/cALjo+cZspRBNToIBIUTLpesw9nYXJY01sIaobe5YDtP/Du16QUgb6NAfznscbl2kggohWjBZTSCEaNnKiuGDOZC0rmYugYphgcvehkFzmqJlQgQMCQaEEC1fWTGsewXWvw65yaqnoO9MmPw7SBzb1K0ToslJMCCEaD1ME8qKwBIsVQiFqEaCASGEEKKVkwmEQgghRCsnwYAQQgjRykkwIIQQQrRyEgwIIYQQrZwEA0IIIUQrJ8GAEEII0cpJMCCEEEK0chIMCCGEEK2cBANCCCFEKyfBgBBCCNHKSTAghBBCtHISDAghhBCtnAQDQgghRCsnwYAQQgjRykkwIIQQQrRyEgwIIYQQrZwEA0IIIUQrJ8GAEEII0cpJMCCEEEK0chIMCCGEEK3c/wO0oBvlRnJgeQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "from sklearn.datasets import make_moons\n",
        "\n",
        "# Set random seeds\n",
        "torch.manual_seed(42)\n",
        "np.random.seed(42)\n",
        "\n",
        "X, y = make_moons(n_samples=200, noise=0.1)\n",
        "y_ = torch.unsqueeze(torch.tensor(y), 1)  # used for one-hot encoded labels\n",
        "y_hot = torch.scatter(torch.zeros((200, 2)), 1, y_, 1)\n",
        "\n",
        "c = [\"#1f77b4\" if y_ == 0 else \"#ff7f0e\" for y_ in y]  # colours for each class\n",
        "plt.axis(\"off\")\n",
        "plt.scatter(X[:, 0], X[:, 1], c=c)\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ApsohoY-IjmC"
      },
      "source": [
        "Defining a QNode\n",
        "================\n",
        "\n",
        "Our next step is to define the QNode that we want to interface with\n",
        "`torch.nn`. Any combination of device, operations and measurements that\n",
        "is valid in PennyLane can be used to compose the QNode. However, the\n",
        "QNode arguments must satisfy additional `conditions\n",
        "<code/api/pennylane.qnn.TorchLayer>`{.interpreted-text role=\"doc\"}\n",
        "including having an argument called `inputs`. All other arguments must\n",
        "be arrays or tensors and are treated as trainable weights in the model.\n",
        "We fix a two-qubit QNode using the\n",
        "`default.qubit <code/api/pennylane.devices.default_qubit.DefaultQubit>`{.interpreted-text\n",
        "role=\"doc\"} simulator and operations from the\n",
        "`templates <introduction/templates>`{.interpreted-text role=\"doc\"}\n",
        "module.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 670,
      "metadata": {
        "id": "ljTpsaAzIjmD"
      },
      "outputs": [],
      "source": [
        "import pennylane as qml\n",
        "\n",
        "n_qubits = 2\n",
        "dev = qml.device(\"default.qubit\", wires=n_qubits)\n",
        "\n",
        "@qml.qnode(dev)\n",
        "def qnode(inputs, weights):\n",
        "    qml.AngleEmbedding(inputs, wires=range(n_qubits), rotation='Y')\n",
        "    qml.AngleEmbedding(inputs, wires=range(n_qubits), rotation='Y')\n",
        "    qml.AngleEmbedding(inputs, wires=range(n_qubits), rotation='Y')\n",
        "    qml.AngleEmbedding(inputs, wires=range(n_qubits), rotation='Y')\n",
        "    return [qml.expval(qml.PauliZ(wires=i)) for i in range(n_qubits)]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pHCzxvEdIjmD"
      },
      "source": [
        "Interfacing with Torch\n",
        "======================\n",
        "\n",
        "With the QNode defined, we are ready to interface with `torch.nn`. This\n",
        "is achieved using the `~pennylane.qnn.TorchLayer`{.interpreted-text\n",
        "role=\"class\"} class of the `~pennylane.qnn`{.interpreted-text\n",
        "role=\"mod\"} module, which converts the QNode to the elementary building\n",
        "block of `torch.nn`: a *layer*. We shall see in the following how the\n",
        "resultant layer can be combined with other well-known neural network\n",
        "layers to form a hybrid model.\n",
        "\n",
        "We must first define the `weight_shapes` dictionary. Recall that all of\n",
        "the arguments of the QNode (except the one named `inputs`) are treated\n",
        "as trainable weights. For the QNode to be successfully converted to a\n",
        "layer in `torch.nn`, we need to provide the details of the shape of each\n",
        "trainable weight for them to be initialized. The `weight_shapes`\n",
        "dictionary maps from the argument names of the QNode to corresponding\n",
        "shapes:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 671,
      "metadata": {
        "id": "BuCmykAyIjmD"
      },
      "outputs": [],
      "source": [
        "n_layers = 6\n",
        "weight_shapes = {\"weights\": (n_layers, n_qubits)}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X26c79bNIjmD"
      },
      "source": [
        "In our example, the `weights` argument of the QNode is trainable and has\n",
        "shape given by `(n_layers, n_qubits)`, which is passed to\n",
        "`~pennylane.templates.layers.BasicEntanglerLayers`{.interpreted-text\n",
        "role=\"func\"}.\n",
        "\n",
        "Now that `weight_shapes` is defined, it is easy to then convert the\n",
        "QNode:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 672,
      "metadata": {
        "id": "kbwSMCeQIjmD"
      },
      "outputs": [],
      "source": [
        "qlayer = qml.qnn.TorchLayer(qnode, weight_shapes)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4qcVUw0BIjmD"
      },
      "source": [
        "With this done, the QNode can now be treated just like any other\n",
        "`torch.nn` layer and we can proceed using the familiar Torch workflow.\n",
        "\n",
        "Creating a hybrid model\n",
        "=======================\n",
        "\n",
        "Let\\'s create a basic three-layered hybrid model consisting of:\n",
        "\n",
        "1.  a 2-neuron fully connected classical layer\n",
        "2.  our 2-qubit QNode converted into a layer\n",
        "3.  another 2-neuron fully connected classical layer\n",
        "4.  a softmax activation to convert to a probability vector\n",
        "\n",
        "A diagram of the model can be seen in the figure below.\n",
        "\n",
        "![](/demonstrations/qnn_module/qnn_torch.png){.align-center\n",
        "width=\"100.0%\"}\n",
        "\n",
        "We can construct the model using the\n",
        "[Sequential](https://pytorch.org/docs/stable/generated/torch.nn.Sequential.html)\n",
        "API:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 673,
      "metadata": {
        "id": "BOtxpC8rIjmD"
      },
      "outputs": [],
      "source": [
        "clayer_1 = torch.nn.Linear(2, 2)\n",
        "clayer_2 = torch.nn.Linear(2, 2)\n",
        "softmax = torch.nn.Softmax(dim=1)\n",
        "layers = [clayer_1, qlayer, clayer_2, softmax]\n",
        "model = torch.nn.Sequential(*layers)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8Yqu7PtgIjmD"
      },
      "source": [
        "Training the model\n",
        "==================\n",
        "\n",
        "We can now train our hybrid model on the classification dataset using\n",
        "the usual Torch approach. We\\'ll use the standard\n",
        "[SGD](https://pytorch.org/docs/stable/optim.html#torch.optim.SGD)\n",
        "optimizer and the mean absolute error loss function:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 674,
      "metadata": {
        "id": "KW3xjlDhIjmE"
      },
      "outputs": [],
      "source": [
        "opt = torch.optim.SGD(model.parameters(), lr=0.2)\n",
        "loss = torch.nn.L1Loss()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0ufrRD6KIjmE"
      },
      "source": [
        "Note that there are more advanced combinations of optimizer and loss\n",
        "function, but here we are focusing on the basics.\n",
        "\n",
        "The model is now ready to be trained!\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 675,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "hB32s2yPIjmE",
        "outputId": "7ea74ef2-338b-443f-c6fc-effe28344bdb"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Average loss over epoch 1: 0.3808\n",
            "Average loss over epoch 2: 0.2490\n",
            "Average loss over epoch 3: 0.3245\n",
            "Average loss over epoch 4: 0.3950\n",
            "Average loss over epoch 5: 0.4331\n",
            "Average loss over epoch 6: 0.2657\n",
            "Accuracy: 85.5%\n"
          ]
        }
      ],
      "source": [
        "X = torch.tensor(X, requires_grad=True).float()\n",
        "y_hot = y_hot.float()\n",
        "\n",
        "batch_size = 5\n",
        "batches = 200 // batch_size\n",
        "\n",
        "data_loader = torch.utils.data.DataLoader(\n",
        "    list(zip(X, y_hot)), batch_size=5, shuffle=True, drop_last=True\n",
        ")\n",
        "\n",
        "epochs = 6\n",
        "\n",
        "for epoch in range(epochs):\n",
        "\n",
        "    running_loss = 0\n",
        "\n",
        "    for xs, ys in data_loader:\n",
        "        opt.zero_grad()\n",
        "\n",
        "        loss_evaluated = loss(model(xs), ys)\n",
        "        loss_evaluated.backward()\n",
        "\n",
        "        opt.step()\n",
        "\n",
        "        running_loss += loss_evaluated\n",
        "\n",
        "    avg_loss = running_loss / batches\n",
        "    print(\"Average loss over epoch {}: {:.4f}\".format(epoch + 1, avg_loss))\n",
        "\n",
        "y_pred = model(X)\n",
        "predictions = torch.argmax(y_pred, axis=1).detach().numpy()\n",
        "\n",
        "correct = [1 if p == p_true else 0 for p, p_true in zip(predictions, y)]\n",
        "accuracy = sum(correct) / len(correct)\n",
        "print(f\"Accuracy: {accuracy * 100}%\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-SB6eGhSIjmE"
      },
      "source": [
        "How did we do? The model looks to have successfully trained and the\n",
        "accuracy is reasonably high. In practice, we would aim to push the\n",
        "accuracy higher by thinking carefully about the model design and the\n",
        "choice of hyperparameters such as the learning rate.\n",
        "\n",
        "Creating non-sequential models\n",
        "==============================\n",
        "\n",
        "The model we created above was composed of a sequence of classical and\n",
        "quantum layers. This type of model is very common and is suitable in a\n",
        "lot of situations. However, in some cases we may want a greater degree\n",
        "of control over how the model is constructed, for example when we have\n",
        "multiple inputs and outputs or when we want to distribute the output of\n",
        "one layer into multiple subsequent layers.\n",
        "\n",
        "Suppose we want to make a hybrid model consisting of:\n",
        "\n",
        "1.  a 4-neuron fully connected classical layer\n",
        "2.  a 2-qubit quantum layer connected to the first two neurons of the\n",
        "    previous classical layer\n",
        "3.  a 2-qubit quantum layer connected to the second two neurons of the\n",
        "    previous classical layer\n",
        "4.  a 2-neuron fully connected classical layer which takes a\n",
        "    4-dimensional input from the combination of the previous quantum\n",
        "    layers\n",
        "5.  a softmax activation to convert to a probability vector\n",
        "\n",
        "A diagram of the model can be seen in the figure below.\n",
        "\n",
        "![](/demonstrations/qnn_module/qnn2_torch.png){.align-center\n",
        "width=\"100.0%\"}\n",
        "\n",
        "This model can also be constructed by creating a new class that inherits\n",
        "from the `torch.nn`\n",
        "[Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) and\n",
        "overriding the `forward()` method:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 676,
      "metadata": {
        "id": "0VfTBf67IjmE"
      },
      "outputs": [],
      "source": [
        "class HybridModel(torch.nn.Module):\n",
        "    def __init__(self):\n",
        "        super().__init__()\n",
        "        self.clayer_1 = torch.nn.Linear(2, 64)\n",
        "        self.qlayer_1 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_2 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_3 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_4 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_5 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_6 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_7 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_8 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_9 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_10 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_11 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_12 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_13 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_14 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_15 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_16 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_17 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_18 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_19 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_20 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_21 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_22 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_23 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_24 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_25 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_26 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_27 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_28 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_29 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_30 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_31 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.qlayer_32 = qml.qnn.TorchLayer(qnode, weight_shapes)\n",
        "        self.clayer_2 = torch.nn.Linear(64, 2)\n",
        "        self.softmax = torch.nn.Softmax(dim=1)\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = self.clayer_1(x)\n",
        "        x_1, x_2, x_3, x_4, x_5, x_6, x_7, x_8, x_9, x_10, x_11, x_12, x_13, x_14, x_15, x_16, x_17, x_18, x_19, x_20, x_21, x_22, x_23, x_24, x_25, x_26, x_27, x_28, x_29, x_30, x_31, x_32 = torch.split(x, 2, dim=1)\n",
        "        x_1 = self.qlayer_1(x_1)\n",
        "        x_2 = self.qlayer_2(x_2)\n",
        "        x_3 = self.qlayer_3(x_3)\n",
        "        x_4 = self.qlayer_4(x_4)\n",
        "        x_5 = self.qlayer_5(x_5)\n",
        "        x_6 = self.qlayer_6(x_6)\n",
        "        x_7 = self.qlayer_7(x_7)\n",
        "        x_8 = self.qlayer_8(x_8)\n",
        "        x_9 = self.qlayer_9(x_9)\n",
        "        x_10 = self.qlayer_10(x_10)\n",
        "        x_11 = self.qlayer_11(x_11)\n",
        "        x_12 = self.qlayer_12(x_12)\n",
        "        x_13 = self.qlayer_13(x_13)\n",
        "        x_14 = self.qlayer_14(x_14)\n",
        "        x_15 = self.qlayer_15(x_15)\n",
        "        x_16 = self.qlayer_16(x_16)\n",
        "        x_17 = self.qlayer_17(x_17)\n",
        "        x_18 = self.qlayer_18(x_18)\n",
        "        x_19 = self.qlayer_19(x_19)\n",
        "        x_20 = self.qlayer_20(x_20)\n",
        "        x_21 = self.qlayer_21(x_21)\n",
        "        x_22 = self.qlayer_22(x_22)\n",
        "        x_23 = self.qlayer_23(x_23)\n",
        "        x_24 = self.qlayer_24(x_24)\n",
        "        x_25 = self.qlayer_25(x_25)\n",
        "        x_26 = self.qlayer_26(x_26)\n",
        "        x_27 = self.qlayer_27(x_27)\n",
        "        x_28 = self.qlayer_28(x_28)\n",
        "        x_29 = self.qlayer_29(x_29)\n",
        "        x_30 = self.qlayer_30(x_30)\n",
        "        x_31 = self.qlayer_31(x_31)\n",
        "        x_32 = self.qlayer_32(x_32)\n",
        "        x = torch.cat([x_1, x_2, x_3, x_4, x_5, x_6, x_7, x_8, x_9, x_10, x_11, x_12, x_13, x_14, x_15, x_16, x_17, x_18, x_19, x_20, x_21, x_22, x_23, x_24, x_25, x_26, x_27, x_28, x_29, x_30, x_31, x_32], axis=1)\n",
        "        x = self.clayer_2(x)\n",
        "        return self.softmax(x)\n",
        "\n",
        "model = HybridModel()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tMt6fZUgIjmE"
      },
      "source": [
        "As a final step, let\\'s train the model to check if it\\'s working:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 677,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "gurIJukfIjmE",
        "outputId": "8d6a686a-9863-4870-b4a3-2e3cc97b66ed"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Average loss over epoch 1: 0.1261\n",
            "Average loss over epoch 2: 0.0252\n",
            "Average loss over epoch 3: 0.0141\n",
            "Average loss over epoch 4: 0.0114\n",
            "Average loss over epoch 5: 0.0090\n",
            "Average loss over epoch 6: 0.0084\n",
            "Accuracy: 99.5%\n"
          ]
        }
      ],
      "source": [
        "opt = torch.optim.SGD(model.parameters(), lr=0.2)\n",
        "epochs = 6\n",
        "\n",
        "for epoch in range(epochs):\n",
        "\n",
        "    running_loss = 0\n",
        "\n",
        "    for xs, ys in data_loader:\n",
        "        opt.zero_grad()\n",
        "\n",
        "        loss_evaluated = loss(model(xs), ys)\n",
        "        loss_evaluated.backward()\n",
        "\n",
        "        opt.step()\n",
        "\n",
        "        running_loss += loss_evaluated\n",
        "\n",
        "    avg_loss = running_loss / batches\n",
        "    print(\"Average loss over epoch {}: {:.4f}\".format(epoch + 1, avg_loss))\n",
        "\n",
        "y_pred = model(X)\n",
        "predictions = torch.argmax(y_pred, axis=1).detach().numpy()\n",
        "\n",
        "correct = [1 if p == p_true else 0 for p, p_true in zip(predictions, y)]\n",
        "accuracy = sum(correct) / len(correct)\n",
        "print(f\"Accuracy: {accuracy * 100}%\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZeRyQGCmIjmE"
      },
      "source": [
        "Great! We\\'ve mastered the basics of constructing hybrid\n",
        "classical-quantum models using PennyLane and Torch. Can you think of any\n",
        "interesting hybrid models to construct? How do they perform on realistic\n",
        "datasets?\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QDn0A3AuIjmE"
      },
      "source": [
        "About the author\n",
        "================\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "seconds = time.time()\n",
        "print(\"Time in seconds since end of run:\", seconds)\n",
        "local_time = time.ctime(seconds)\n",
        "print(local_time)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "5d2KCuaYAemE",
        "outputId": "a26a02d1-70ce-4d2c-c750-01a6fa4116c2"
      },
      "execution_count": 678,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Time in seconds since end of run: 1701839520.1162884\n",
            "Wed Dec  6 05:12:00 2023\n"
          ]
        }
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.17"
    },
    "colab": {
      "provenance": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}