--- a
+++ b/nbs/Appendix_pre_trained_models.ipynb
@@ -0,0 +1,112 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "caa16d01",
+   "metadata": {},
+   "source": [
+    "# Using pre-trained models on text data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "d9f0014c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%matplotlib inline\n",
+    "\n",
+    "import pandas as pd\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ec0f3fba",
+   "metadata": {},
+   "source": [
+    "Repositories like Hugging Face provide a huge number of pre-trained models that can tranform text data into highly representative features.\n",
+    "\n",
+    "In this example we use ClinicalBERT, a language model initialized from the more general language model BERT and then further trained on a large multicenter dataset with a large corpus of 1.2B words of diverse diseases. This approach was inspired by [this blogpost](https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/BERT/Fine_tuning_BERT_(and_friends)_for_multi_label_text_classification.ipynb#scrollTo=4wxY3x-ZZz8h)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "01dbc978",
+   "metadata": {},
+   "source": [
+    "First install the libraries for HuggingFace: Transformers and Datasets"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6a160ce7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install -q transformers datasets"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6297d597",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from transformers import AutoTokenizer, AutoModel\n",
+    "tokenizer = AutoTokenizer.from_pretrained(\"medicalai/ClinicalBERT\")\n",
+    "model = AutoModel.from_pretrained(\"medicalai/ClinicalBERT\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1835f4f3",
+   "metadata": {},
+   "source": [
+    "Load in prior dataset"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "0a790b4c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# load data as a pandas dataframe\n",
+    "df = pd.read_csv('../data/overview-of-recordings.csv')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f6245713",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.7"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}