Downloads: 1
Switch to side-by-side view
--- a +++ b/1. Applying AI to 2D Medical Imaging Data/2. Apply Machine Learning Exercise/solution.ipynb @@ -0,0 +1,1314 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "import numpy as np" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Worklist prioritization: Emergency Setting" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "## First, read in the file of the current worklist with the probabilities that your two algorithms have\n", + "## generated for the two types of findings you're most concerned with:\n", + "\n", + "worklist = pd.read_csv('probabilities.csv')" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th></th>\n", + " <th>Image_Type</th>\n", + " <th>Brain_bleed_probability</th>\n", + " <th>Aortic_dissection_probability</th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>0</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.05</td>\n", + " </tr>\n", + " <tr>\n", + " <th>1</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.17</td>\n", + " </tr>\n", + " <tr>\n", + " <th>2</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.00</td>\n", + " </tr>\n", + " <tr>\n", + " <th>3</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.04</td>\n", + " </tr>\n", + " <tr>\n", + " <th>4</th>\n", + " <td>wrist_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.00</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + " Image_Type Brain_bleed_probability Aortic_dissection_probability\n", + "0 chest_xray 0.0 0.05\n", + "1 chest_xray 0.0 0.17\n", + "2 chest_xray 0.0 0.00\n", + "3 chest_xray 0.0 0.04\n", + "4 wrist_xray 0.0 0.00" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "worklist.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here, I'm just creating a new column to address the first question in the exercise, showing that every image taking 6 minutes to read will be read in the order that they are presented in this list." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "worklist['time_to_read'] = np.arange(6, 6*(len(worklist)+1),6)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th></th>\n", + " <th>Image_Type</th>\n", + " <th>Brain_bleed_probability</th>\n", + " <th>Aortic_dissection_probability</th>\n", + " <th>time_to_read</th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>0</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.05</td>\n", + " <td>6</td>\n", + " </tr>\n", + " <tr>\n", + " <th>1</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.17</td>\n", + " <td>12</td>\n", + " </tr>\n", + " <tr>\n", + " <th>2</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.00</td>\n", + " <td>18</td>\n", + " </tr>\n", + " <tr>\n", + " <th>3</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.04</td>\n", + " <td>24</td>\n", + " </tr>\n", + " <tr>\n", + " <th>4</th>\n", + " <td>wrist_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.00</td>\n", + " <td>30</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + " Image_Type Brain_bleed_probability Aortic_dissection_probability \\\n", + "0 chest_xray 0.0 0.05 \n", + "1 chest_xray 0.0 0.17 \n", + "2 chest_xray 0.0 0.00 \n", + "3 chest_xray 0.0 0.04 \n", + "4 wrist_xray 0.0 0.00 \n", + "\n", + " time_to_read \n", + "0 6 \n", + "1 12 \n", + "2 18 \n", + "3 24 \n", + "4 30 " + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "worklist.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, for each image, I want to see if brain bleed or aortic dissection are likely. I'll create a new column showing the max probability between the two of them" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "worklist['max_prob'] = worklist[[\"Brain_bleed_probability\", \"Aortic_dissection_probability\"]].max(axis=1)" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th></th>\n", + " <th>Image_Type</th>\n", + " <th>Brain_bleed_probability</th>\n", + " <th>Aortic_dissection_probability</th>\n", + " <th>time_to_read</th>\n", + " <th>max_prob</th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>0</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.05</td>\n", + " <td>6</td>\n", + " <td>0.05</td>\n", + " </tr>\n", + " <tr>\n", + " <th>1</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.17</td>\n", + " <td>12</td>\n", + " <td>0.17</td>\n", + " </tr>\n", + " <tr>\n", + " <th>2</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.00</td>\n", + " <td>18</td>\n", + " <td>0.00</td>\n", + " </tr>\n", + " <tr>\n", + " <th>3</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.04</td>\n", + " <td>24</td>\n", + " <td>0.04</td>\n", + " </tr>\n", + " <tr>\n", + " <th>4</th>\n", + " <td>wrist_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.00</td>\n", + " <td>30</td>\n", + " <td>0.00</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + " Image_Type Brain_bleed_probability Aortic_dissection_probability \\\n", + "0 chest_xray 0.0 0.05 \n", + "1 chest_xray 0.0 0.17 \n", + "2 chest_xray 0.0 0.00 \n", + "3 chest_xray 0.0 0.04 \n", + "4 wrist_xray 0.0 0.00 \n", + "\n", + " time_to_read max_prob \n", + "0 6 0.05 \n", + "1 12 0.17 \n", + "2 18 0.00 \n", + "3 24 0.04 \n", + "4 30 0.00 " + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "worklist.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Great, now I want to re-order my worklist based on probabilities of critical findings:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "worklist_prioritized = worklist.sort_values(by=['max_prob'],ascending=False)" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th></th>\n", + " <th>Image_Type</th>\n", + " <th>Brain_bleed_probability</th>\n", + " <th>Aortic_dissection_probability</th>\n", + " <th>time_to_read</th>\n", + " <th>max_prob</th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>25</th>\n", + " <td>head_ct</td>\n", + " <td>0.99</td>\n", + " <td>0.00</td>\n", + " <td>156</td>\n", + " <td>0.99</td>\n", + " </tr>\n", + " <tr>\n", + " <th>15</th>\n", + " <td>chest_xray</td>\n", + " <td>0.00</td>\n", + " <td>0.95</td>\n", + " <td>96</td>\n", + " <td>0.95</td>\n", + " </tr>\n", + " <tr>\n", + " <th>10</th>\n", + " <td>chest_xray</td>\n", + " <td>0.00</td>\n", + " <td>0.94</td>\n", + " <td>66</td>\n", + " <td>0.94</td>\n", + " </tr>\n", + " <tr>\n", + " <th>75</th>\n", + " <td>chest_xray</td>\n", + " <td>0.00</td>\n", + " <td>0.93</td>\n", + " <td>456</td>\n", + " <td>0.93</td>\n", + " </tr>\n", + " <tr>\n", + " <th>47</th>\n", + " <td>chest_xray</td>\n", + " <td>0.00</td>\n", + " <td>0.93</td>\n", + " <td>288</td>\n", + " <td>0.93</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + " Image_Type Brain_bleed_probability Aortic_dissection_probability \\\n", + "25 head_ct 0.99 0.00 \n", + "15 chest_xray 0.00 0.95 \n", + "10 chest_xray 0.00 0.94 \n", + "75 chest_xray 0.00 0.93 \n", + "47 chest_xray 0.00 0.93 \n", + "\n", + " time_to_read max_prob \n", + "25 156 0.99 \n", + "15 96 0.95 \n", + "10 66 0.94 \n", + "75 456 0.93 \n", + "47 288 0.93 " + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "worklist_prioritized.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "worklist_prioritized['time_to_read_prioritized'] = np.arange(6, 6*(len(worklist)+1),6)" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "worklist_prioritized['time_delta'] = worklist_prioritized['time_to_read'] - worklist_prioritized['time_to_read_prioritized']" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th></th>\n", + " <th>Image_Type</th>\n", + " <th>Brain_bleed_probability</th>\n", + " <th>Aortic_dissection_probability</th>\n", + " <th>time_to_read</th>\n", + " <th>max_prob</th>\n", + " <th>time_to_read_prioritized</th>\n", + " <th>time_delta</th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>25</th>\n", + " <td>head_ct</td>\n", + " <td>0.99</td>\n", + " <td>0.00</td>\n", + " <td>156</td>\n", + " <td>0.99</td>\n", + " <td>6</td>\n", + " <td>150</td>\n", + " </tr>\n", + " <tr>\n", + " <th>15</th>\n", + " <td>chest_xray</td>\n", + " <td>0.00</td>\n", + " <td>0.95</td>\n", + " <td>96</td>\n", + " <td>0.95</td>\n", + " <td>12</td>\n", + " <td>84</td>\n", + " </tr>\n", + " <tr>\n", + " <th>10</th>\n", + " <td>chest_xray</td>\n", + " <td>0.00</td>\n", + " <td>0.94</td>\n", + " <td>66</td>\n", + " <td>0.94</td>\n", + " <td>18</td>\n", + " <td>48</td>\n", + " </tr>\n", + " <tr>\n", + " <th>75</th>\n", + " <td>chest_xray</td>\n", + " <td>0.00</td>\n", + " <td>0.93</td>\n", + " <td>456</td>\n", + " <td>0.93</td>\n", + " <td>24</td>\n", + " <td>432</td>\n", + " </tr>\n", + " <tr>\n", + " <th>47</th>\n", + " <td>chest_xray</td>\n", + " <td>0.00</td>\n", + " <td>0.93</td>\n", + " <td>288</td>\n", + " <td>0.93</td>\n", + " <td>30</td>\n", + " <td>258</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + " Image_Type Brain_bleed_probability Aortic_dissection_probability \\\n", + "25 head_ct 0.99 0.00 \n", + "15 chest_xray 0.00 0.95 \n", + "10 chest_xray 0.00 0.94 \n", + "75 chest_xray 0.00 0.93 \n", + "47 chest_xray 0.00 0.93 \n", + "\n", + " time_to_read max_prob time_to_read_prioritized time_delta \n", + "25 156 0.99 6 150 \n", + "15 96 0.95 12 84 \n", + "10 66 0.94 18 48 \n", + "75 456 0.93 24 432 \n", + "47 288 0.93 30 258 " + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "worklist_prioritized.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, I want to find places where my algorithm saved at least 30 minutes for brain bleeds:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th></th>\n", + " <th>Image_Type</th>\n", + " <th>Brain_bleed_probability</th>\n", + " <th>Aortic_dissection_probability</th>\n", + " <th>time_to_read</th>\n", + " <th>max_prob</th>\n", + " <th>time_to_read_prioritized</th>\n", + " <th>time_delta</th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>25</th>\n", + " <td>head_ct</td>\n", + " <td>0.99</td>\n", + " <td>0.0</td>\n", + " <td>156</td>\n", + " <td>0.99</td>\n", + " <td>6</td>\n", + " <td>150</td>\n", + " </tr>\n", + " <tr>\n", + " <th>84</th>\n", + " <td>head_ct</td>\n", + " <td>0.91</td>\n", + " <td>0.0</td>\n", + " <td>510</td>\n", + " <td>0.91</td>\n", + " <td>36</td>\n", + " <td>474</td>\n", + " </tr>\n", + " <tr>\n", + " <th>95</th>\n", + " <td>head_ct</td>\n", + " <td>0.90</td>\n", + " <td>0.0</td>\n", + " <td>576</td>\n", + " <td>0.90</td>\n", + " <td>42</td>\n", + " <td>534</td>\n", + " </tr>\n", + " <tr>\n", + " <th>42</th>\n", + " <td>head_ct</td>\n", + " <td>0.89</td>\n", + " <td>0.0</td>\n", + " <td>258</td>\n", + " <td>0.89</td>\n", + " <td>48</td>\n", + " <td>210</td>\n", + " </tr>\n", + " <tr>\n", + " <th>59</th>\n", + " <td>head_ct</td>\n", + " <td>0.89</td>\n", + " <td>0.0</td>\n", + " <td>360</td>\n", + " <td>0.89</td>\n", + " <td>54</td>\n", + " <td>306</td>\n", + " </tr>\n", + " <tr>\n", + " <th>89</th>\n", + " <td>head_ct</td>\n", + " <td>0.78</td>\n", + " <td>0.0</td>\n", + " <td>540</td>\n", + " <td>0.78</td>\n", + " <td>96</td>\n", + " <td>444</td>\n", + " </tr>\n", + " <tr>\n", + " <th>39</th>\n", + " <td>head_ct</td>\n", + " <td>0.77</td>\n", + " <td>0.0</td>\n", + " <td>240</td>\n", + " <td>0.77</td>\n", + " <td>102</td>\n", + " <td>138</td>\n", + " </tr>\n", + " <tr>\n", + " <th>45</th>\n", + " <td>head_ct</td>\n", + " <td>0.75</td>\n", + " <td>0.0</td>\n", + " <td>276</td>\n", + " <td>0.75</td>\n", + " <td>108</td>\n", + " <td>168</td>\n", + " </tr>\n", + " <tr>\n", + " <th>76</th>\n", + " <td>head_ct</td>\n", + " <td>0.69</td>\n", + " <td>0.0</td>\n", + " <td>462</td>\n", + " <td>0.69</td>\n", + " <td>144</td>\n", + " <td>318</td>\n", + " </tr>\n", + " <tr>\n", + " <th>96</th>\n", + " <td>head_ct</td>\n", + " <td>0.45</td>\n", + " <td>0.0</td>\n", + " <td>582</td>\n", + " <td>0.45</td>\n", + " <td>198</td>\n", + " <td>384</td>\n", + " </tr>\n", + " <tr>\n", + " <th>78</th>\n", + " <td>head_ct</td>\n", + " <td>0.44</td>\n", + " <td>0.0</td>\n", + " <td>474</td>\n", + " <td>0.44</td>\n", + " <td>204</td>\n", + " <td>270</td>\n", + " </tr>\n", + " <tr>\n", + " <th>56</th>\n", + " <td>head_ct</td>\n", + " <td>0.22</td>\n", + " <td>0.0</td>\n", + " <td>342</td>\n", + " <td>0.22</td>\n", + " <td>258</td>\n", + " <td>84</td>\n", + " </tr>\n", + " <tr>\n", + " <th>68</th>\n", + " <td>head_ct</td>\n", + " <td>0.07</td>\n", + " <td>0.0</td>\n", + " <td>414</td>\n", + " <td>0.07</td>\n", + " <td>324</td>\n", + " <td>90</td>\n", + " </tr>\n", + " <tr>\n", + " <th>82</th>\n", + " <td>head_ct</td>\n", + " <td>0.05</td>\n", + " <td>0.0</td>\n", + " <td>498</td>\n", + " <td>0.05</td>\n", + " <td>336</td>\n", + " <td>162</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + " Image_Type Brain_bleed_probability Aortic_dissection_probability \\\n", + "25 head_ct 0.99 0.0 \n", + "84 head_ct 0.91 0.0 \n", + "95 head_ct 0.90 0.0 \n", + "42 head_ct 0.89 0.0 \n", + "59 head_ct 0.89 0.0 \n", + "89 head_ct 0.78 0.0 \n", + "39 head_ct 0.77 0.0 \n", + "45 head_ct 0.75 0.0 \n", + "76 head_ct 0.69 0.0 \n", + "96 head_ct 0.45 0.0 \n", + "78 head_ct 0.44 0.0 \n", + "56 head_ct 0.22 0.0 \n", + "68 head_ct 0.07 0.0 \n", + "82 head_ct 0.05 0.0 \n", + "\n", + " time_to_read max_prob time_to_read_prioritized time_delta \n", + "25 156 0.99 6 150 \n", + "84 510 0.91 36 474 \n", + "95 576 0.90 42 534 \n", + "42 258 0.89 48 210 \n", + "59 360 0.89 54 306 \n", + "89 540 0.78 96 444 \n", + "39 240 0.77 102 138 \n", + "45 276 0.75 108 168 \n", + "76 462 0.69 144 318 \n", + "96 582 0.45 198 384 \n", + "78 474 0.44 204 270 \n", + "56 342 0.22 258 84 \n", + "68 414 0.07 324 90 \n", + "82 498 0.05 336 162 " + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "worklist_prioritized[((worklist_prioritized.time_delta>30)&(worklist_prioritized.Image_Type=='head_ct'))]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Looks like there are 14 head CTs that were read more than 30 minutes faster than their original order. All but the last three had a probability of brain bleed < 0.4." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Do the same analysis for saving at least 15 minutes with aortic dissections: " + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "<div>\n", + "<style scoped>\n", + " .dataframe tbody tr th:only-of-type {\n", + " vertical-align: middle;\n", + " }\n", + "\n", + " .dataframe tbody tr th {\n", + " vertical-align: top;\n", + " }\n", + "\n", + " .dataframe thead th {\n", + " text-align: right;\n", + " }\n", + "</style>\n", + "<table border=\"1\" class=\"dataframe\">\n", + " <thead>\n", + " <tr style=\"text-align: right;\">\n", + " <th></th>\n", + " <th>Image_Type</th>\n", + " <th>Brain_bleed_probability</th>\n", + " <th>Aortic_dissection_probability</th>\n", + " <th>time_to_read</th>\n", + " <th>max_prob</th>\n", + " <th>time_to_read_prioritized</th>\n", + " <th>time_delta</th>\n", + " </tr>\n", + " </thead>\n", + " <tbody>\n", + " <tr>\n", + " <th>15</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.95</td>\n", + " <td>96</td>\n", + " <td>0.95</td>\n", + " <td>12</td>\n", + " <td>84</td>\n", + " </tr>\n", + " <tr>\n", + " <th>10</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.94</td>\n", + " <td>66</td>\n", + " <td>0.94</td>\n", + " <td>18</td>\n", + " <td>48</td>\n", + " </tr>\n", + " <tr>\n", + " <th>75</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.93</td>\n", + " <td>456</td>\n", + " <td>0.93</td>\n", + " <td>24</td>\n", + " <td>432</td>\n", + " </tr>\n", + " <tr>\n", + " <th>47</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.93</td>\n", + " <td>288</td>\n", + " <td>0.93</td>\n", + " <td>30</td>\n", + " <td>258</td>\n", + " </tr>\n", + " <tr>\n", + " <th>48</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.84</td>\n", + " <td>294</td>\n", + " <td>0.84</td>\n", + " <td>60</td>\n", + " <td>234</td>\n", + " </tr>\n", + " <tr>\n", + " <th>38</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.83</td>\n", + " <td>234</td>\n", + " <td>0.83</td>\n", + " <td>66</td>\n", + " <td>168</td>\n", + " </tr>\n", + " <tr>\n", + " <th>62</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.82</td>\n", + " <td>378</td>\n", + " <td>0.82</td>\n", + " <td>72</td>\n", + " <td>306</td>\n", + " </tr>\n", + " <tr>\n", + " <th>87</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.82</td>\n", + " <td>528</td>\n", + " <td>0.82</td>\n", + " <td>78</td>\n", + " <td>450</td>\n", + " </tr>\n", + " <tr>\n", + " <th>44</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.81</td>\n", + " <td>270</td>\n", + " <td>0.81</td>\n", + " <td>84</td>\n", + " <td>186</td>\n", + " </tr>\n", + " <tr>\n", + " <th>85</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.79</td>\n", + " <td>516</td>\n", + " <td>0.79</td>\n", + " <td>90</td>\n", + " <td>426</td>\n", + " </tr>\n", + " <tr>\n", + " <th>81</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.73</td>\n", + " <td>492</td>\n", + " <td>0.73</td>\n", + " <td>114</td>\n", + " <td>378</td>\n", + " </tr>\n", + " <tr>\n", + " <th>53</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.72</td>\n", + " <td>324</td>\n", + " <td>0.72</td>\n", + " <td>120</td>\n", + " <td>204</td>\n", + " </tr>\n", + " <tr>\n", + " <th>27</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.71</td>\n", + " <td>168</td>\n", + " <td>0.71</td>\n", + " <td>126</td>\n", + " <td>42</td>\n", + " </tr>\n", + " <tr>\n", + " <th>51</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.71</td>\n", + " <td>312</td>\n", + " <td>0.71</td>\n", + " <td>132</td>\n", + " <td>180</td>\n", + " </tr>\n", + " <tr>\n", + " <th>66</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.70</td>\n", + " <td>402</td>\n", + " <td>0.70</td>\n", + " <td>138</td>\n", + " <td>264</td>\n", + " </tr>\n", + " <tr>\n", + " <th>46</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.60</td>\n", + " <td>282</td>\n", + " <td>0.60</td>\n", + " <td>168</td>\n", + " <td>114</td>\n", + " </tr>\n", + " <tr>\n", + " <th>69</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.60</td>\n", + " <td>420</td>\n", + " <td>0.60</td>\n", + " <td>174</td>\n", + " <td>246</td>\n", + " </tr>\n", + " <tr>\n", + " <th>93</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.49</td>\n", + " <td>564</td>\n", + " <td>0.49</td>\n", + " <td>186</td>\n", + " <td>378</td>\n", + " </tr>\n", + " <tr>\n", + " <th>70</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.48</td>\n", + " <td>426</td>\n", + " <td>0.48</td>\n", + " <td>192</td>\n", + " <td>234</td>\n", + " </tr>\n", + " <tr>\n", + " <th>72</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.38</td>\n", + " <td>438</td>\n", + " <td>0.38</td>\n", + " <td>210</td>\n", + " <td>228</td>\n", + " </tr>\n", + " <tr>\n", + " <th>92</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.26</td>\n", + " <td>558</td>\n", + " <td>0.26</td>\n", + " <td>234</td>\n", + " <td>324</td>\n", + " </tr>\n", + " <tr>\n", + " <th>61</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.26</td>\n", + " <td>372</td>\n", + " <td>0.26</td>\n", + " <td>240</td>\n", + " <td>132</td>\n", + " </tr>\n", + " <tr>\n", + " <th>74</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.15</td>\n", + " <td>450</td>\n", + " <td>0.15</td>\n", + " <td>288</td>\n", + " <td>162</td>\n", + " </tr>\n", + " <tr>\n", + " <th>90</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.15</td>\n", + " <td>546</td>\n", + " <td>0.15</td>\n", + " <td>294</td>\n", + " <td>252</td>\n", + " </tr>\n", + " <tr>\n", + " <th>79</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.11</td>\n", + " <td>480</td>\n", + " <td>0.11</td>\n", + " <td>306</td>\n", + " <td>174</td>\n", + " </tr>\n", + " <tr>\n", + " <th>83</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.11</td>\n", + " <td>504</td>\n", + " <td>0.11</td>\n", + " <td>312</td>\n", + " <td>192</td>\n", + " </tr>\n", + " <tr>\n", + " <th>94</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.06</td>\n", + " <td>570</td>\n", + " <td>0.06</td>\n", + " <td>330</td>\n", + " <td>240</td>\n", + " </tr>\n", + " <tr>\n", + " <th>77</th>\n", + " <td>chest_xray</td>\n", + " <td>0.0</td>\n", + " <td>0.04</td>\n", + " <td>468</td>\n", + " <td>0.04</td>\n", + " <td>348</td>\n", + " <td>120</td>\n", + " </tr>\n", + " </tbody>\n", + "</table>\n", + "</div>" + ], + "text/plain": [ + " Image_Type Brain_bleed_probability Aortic_dissection_probability \\\n", + "15 chest_xray 0.0 0.95 \n", + "10 chest_xray 0.0 0.94 \n", + "75 chest_xray 0.0 0.93 \n", + "47 chest_xray 0.0 0.93 \n", + "48 chest_xray 0.0 0.84 \n", + "38 chest_xray 0.0 0.83 \n", + "62 chest_xray 0.0 0.82 \n", + "87 chest_xray 0.0 0.82 \n", + "44 chest_xray 0.0 0.81 \n", + "85 chest_xray 0.0 0.79 \n", + "81 chest_xray 0.0 0.73 \n", + "53 chest_xray 0.0 0.72 \n", + "27 chest_xray 0.0 0.71 \n", + "51 chest_xray 0.0 0.71 \n", + "66 chest_xray 0.0 0.70 \n", + "46 chest_xray 0.0 0.60 \n", + "69 chest_xray 0.0 0.60 \n", + "93 chest_xray 0.0 0.49 \n", + "70 chest_xray 0.0 0.48 \n", + "72 chest_xray 0.0 0.38 \n", + "92 chest_xray 0.0 0.26 \n", + "61 chest_xray 0.0 0.26 \n", + "74 chest_xray 0.0 0.15 \n", + "90 chest_xray 0.0 0.15 \n", + "79 chest_xray 0.0 0.11 \n", + "83 chest_xray 0.0 0.11 \n", + "94 chest_xray 0.0 0.06 \n", + "77 chest_xray 0.0 0.04 \n", + "\n", + " time_to_read max_prob time_to_read_prioritized time_delta \n", + "15 96 0.95 12 84 \n", + "10 66 0.94 18 48 \n", + "75 456 0.93 24 432 \n", + "47 288 0.93 30 258 \n", + "48 294 0.84 60 234 \n", + "38 234 0.83 66 168 \n", + "62 378 0.82 72 306 \n", + "87 528 0.82 78 450 \n", + "44 270 0.81 84 186 \n", + "85 516 0.79 90 426 \n", + "81 492 0.73 114 378 \n", + "53 324 0.72 120 204 \n", + "27 168 0.71 126 42 \n", + "51 312 0.71 132 180 \n", + "66 402 0.70 138 264 \n", + "46 282 0.60 168 114 \n", + "69 420 0.60 174 246 \n", + "93 564 0.49 186 378 \n", + "70 426 0.48 192 234 \n", + "72 438 0.38 210 228 \n", + "92 558 0.26 234 324 \n", + "61 372 0.26 240 132 \n", + "74 450 0.15 288 162 \n", + "90 546 0.15 294 252 \n", + "79 480 0.11 306 174 \n", + "83 504 0.11 312 192 \n", + "94 570 0.06 330 240 \n", + "77 468 0.04 348 120 " + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "worklist_prioritized[((worklist_prioritized.time_delta>=15)&(worklist_prioritized.Image_Type=='chest_xray'))]" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "28" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "len(worklist_prioritized[((worklist_prioritized.time_delta>=15)&(worklist_prioritized.Image_Type=='chest_xray'))])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Looks like there are 28 chest x-rays that were read more than 15 minutes faster than their original order. All but the last nine had a probability of aortic dissection < 0.4." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, I'll take a look at anywhere that my algorithm made brain bleeds or aortic dissections with a probability of 0.5 or higher be read _slower._" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "worklist_prioritized[((worklist_prioritized.time_delta<0)&(worklist_prioritized.max_prob>=0.5))]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Looks like there were two cases where my algorithm caused an image to be read slower than the priority order it came in. Given that I had images with probabilities <0.5 that were read faster, it is definitely possible to improve my algorithm by adding some more heuristics. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}