<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Installation — DeepProg documentation</title>
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<!--[if lt IE 9]>
<script src="_static/js/html5shiv.min.js"></script>
<![endif]-->
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script src="_static/jquery.js"></script>
<script src="_static/underscore.js"></script>
<script src="_static/doctools.js"></script>
<script src="_static/language_data.js"></script>
<script type="text/javascript" src="_static/js/theme.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Tutorial: Simple DeepProg model" href="usage.html" />
<link rel="prev" title="Welcome to DeepProg’s documentation!" href="index.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="index.html" class="icon icon-home"> DeepProg
</a>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
<ul class="current">
<li class="toctree-l1 current"><a class="current reference internal" href="#">Installation</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#requirements">Requirements</a></li>
<li class="toctree-l2"><a class="reference internal" href="#tested-python-package-versions">Tested python package versions</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#installation-local">installation (local)</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#deep-learning-packages-installation">Deep-Learning packages installation</a></li>
<li class="toctree-l2"><a class="reference internal" href="#support-for-cntk-tensorflow">Support for CNTK / tensorflow</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#r-installation-depreciated">R installation (Depreciated)</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#distributed-computation">Distributed computation</a></li>
<li class="toctree-l2"><a class="reference internal" href="#visualisation-module-experimental">Visualisation module (Experimental)</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#id1">Distributed computation</a></li>
<li class="toctree-l3"><a class="reference internal" href="#id2">Visualisation module (Experimental)</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#usage">Usage</a></li>
<li class="toctree-l2"><a class="reference internal" href="#example-scripts">Example scripts</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="usage.html">Tutorial: Simple DeepProg model</a></li>
<li class="toctree-l1"><a class="reference internal" href="usage_ensemble.html">Tutorial: Ensemble of DeepProg model</a></li>
<li class="toctree-l1"><a class="reference internal" href="usage_advanced.html">Tutorial: Advanced usage of DeepProg model</a></li>
<li class="toctree-l1"><a class="reference internal" href="case_study.html">Case study: Analyzing TCGA HCC dataset</a></li>
<li class="toctree-l1"><a class="reference internal" href="usage_tuning.html">Tutorial: Tuning DeepProg</a></li>
<li class="toctree-l1"><a class="reference internal" href="LICENSE.html">License</a></li>
<li class="toctree-l1"><a class="reference internal" href="api/simdeep.html">simdeep package</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="index.html">DeepProg</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li><a href="index.html" class="icon icon-home"></a> »</li>
<li>Installation</li>
<li class="wy-breadcrumbs-aside">
<a href="_sources/installation.md.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<div class="section" id="installation">
<h1>Installation<a class="headerlink" href="#installation" title="Permalink to this headline">¶</a></h1>
<p>Here we describe how to install the DeepProg package. We assume that the installation will be done locally, using the <code class="docutils literal notranslate"><span class="pre">--user</span></code> flag from pip. Alternatively, the package can be installed using a virtual environment or globally with sudo. Both python2.7 or python3.6 (or higher) can be used. We only tested the installation on a linux environment but it should also work on a OSX environment.</p>
<div class="section" id="requirements">
<h2>Requirements<a class="headerlink" href="#requirements" title="Permalink to this headline">¶</a></h2>
<ul class="simple">
<li><p>Python 2 or 3 (Python3 is recommended)</p></li>
<li><p>Either theano, tensorflow or CNTK (theano is recommended)</p></li>
<li><p><a class="reference external" href="http://deeplearning.net/software/theano/install.html">theano</a> (the used version for the manuscript was 0.8.2)</p></li>
<li><p><a class="reference external" href="https://www.tensorflow.org/">tensorflow</a> as a more robust alternative to theano</p></li>
<li><p><a class="reference external" href="https://github.com/microsoft/CNTK">cntk</a> CNTK is anoter DL library that can present some advantages compared to tensorflow or theano. See <a class="reference external" href="https://docs.microsoft.com/en-us/cognitive-toolkit/">https://docs.microsoft.com/en-us/cognitive-toolkit/</a></p></li>
<li><p>scikit-learn (>=0.18)</p></li>
<li><p>numpy, scipy</p></li>
<li><p>lifelines</p></li>
<li><p>(if using python3) scikit-survival</p></li>
<li><p>(For distributed computing) ray (ray >= 0.8.4) framework</p></li>
<li><p>(For hyperparameter tuning) scikit-optimize</p></li>
</ul>
</div>
<div class="section" id="tested-python-package-versions">
<h2>Tested python package versions<a class="headerlink" href="#tested-python-package-versions" title="Permalink to this headline">¶</a></h2>
<ul class="simple">
<li><p>tensorflow == 2.4.1</p></li>
<li><p>keras == 2.4.3</p></li>
<li><p>ray == 0.8.4</p></li>
<li><p>scikit-learn == 0.23.2</p></li>
<li><p>scikit-survival == 0.14.0</p></li>
<li><p>lifelines == 0.25.5</p></li>
<li><p>scikit-optimize == 0.8.1</p></li>
<li><p>mpld3 == 0.5.1</p></li>
</ul>
<p>Since ray and tensorflow are rapidly evolving libraries, newest versions might unfortunatly break DeepProg’s API. To avoid any dependencies issues, we recommand working inside a Python 3 <a class="reference external" href="https://docs.python.org/3/tutorial/venv.html">virtual environement</a> (<code class="docutils literal notranslate"><span class="pre">virtualenv</span></code>) and install the tested packages.</p>
<div class="section" id="installation-local">
<h3>installation (local)<a class="headerlink" href="#installation-local" title="Permalink to this headline">¶</a></h3>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="c1"># The downloading can take few minutes due to the size of th git project</span>
git clone https://github.com/lanagarmire/DeepProg.git
<span class="nb">cd</span> SimDeep
<span class="c1"># The installation should only take a short amount of time</span>
pip3 install -e . -r requirements.txt --user
<span class="c1"># To intall the distributed frameworks</span>
pip3 install -r requirements_distributed.txt --user
<span class="c1"># Installing scikit-survival (python3 only)</span>
pip3 install -r requirements_pip3.txt --user
<span class="c1"># DeepProg is working also with python2/pip2 however there is no support for scikit-survival in python2</span>
pip2 install -r requirements.txt --user
pip2 install -r requirements_distributed.txt --user
<span class="c1"># to install the tested python library versions</span>
pip install -r requirements_tested.txt
</pre></div>
</div>
</div>
</div>
<div class="section" id="deep-learning-packages-installation">
<h2>Deep-Learning packages installation<a class="headerlink" href="#deep-learning-packages-installation" title="Permalink to this headline">¶</a></h2>
<p>The required python packages can be installed using pip:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>pip install theano --user <span class="c1"># Original backend used OR</span>
pip install tensorflow --user <span class="c1"># Alternative backend for keras</span>
pip install keras --user
</pre></div>
</div>
</div>
<div class="section" id="support-for-cntk-tensorflow">
<h2>Support for CNTK / tensorflow<a class="headerlink" href="#support-for-cntk-tensorflow" title="Permalink to this headline">¶</a></h2>
<p>We originally used Keras with theano as backend plateform. However, <a class="reference external" href="https://www.tensorflow.org/">Tensorflow</a> or <a class="reference external" href="https://docs.microsoft.com/en-us/cognitive-toolkit/">CNTK</a> are more recent DL framework that can be faster or more stable than theano. Because keras supports these 3 backends, it is possible to use them as alternative to theano. To install CNTK, please refer to the official <a class="reference external" href="https://docs.microsoft.com/en-us/cognitive-toolkit/setup-cntk-on-your-machine">guidelines</a> . To change backend, please configure the <code class="docutils literal notranslate"><span class="pre">$HOME/.keras/keras.json</span></code> file. (See official instruction <a class="reference external" href="https://keras.io/backend/">here</a>).</p>
<p>The default configuration file: <code class="docutils literal notranslate"> <span class="pre">~/.keras/keras.json</span></code> looks like this:</p>
<div class="highlight-json notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
<span class="nt">"image_data_format"</span><span class="p">:</span> <span class="s2">"channels_last"</span><span class="p">,</span>
<span class="nt">"epsilon"</span><span class="p">:</span> <span class="mf">1e-07</span><span class="p">,</span>
<span class="nt">"floatx"</span><span class="p">:</span> <span class="s2">"float32"</span><span class="p">,</span>
<span class="nt">"backend"</span><span class="p">:</span> <span class="s2">"tensorflow"</span>
<span class="p">}</span>
</pre></div>
</div>
<div class="section" id="r-installation-depreciated">
<h3>R installation (Depreciated)<a class="headerlink" href="#r-installation-depreciated" title="Permalink to this headline">¶</a></h3>
<p>In his first implementation, DeepProg used the R survival toolkits to fit the survival functions. Thse functions have been replaced with the python toolkits lifelines and scikit-survival for more convenience and avoid any compatibility issue.</p>
<ul class="simple">
<li><p>R</p></li>
<li><p>the R “survival” package installed.</p></li>
<li><p>rpy2 2.8.6 (for python2 rpy2 can be install with: pip install rpy2==2.8.6, for python3 pip3 install rpy2==2.8.6). It seems that newer version of rpy2 might not work due to a bug (not tested)</p></li>
</ul>
<div class="highlight-R notranslate"><div class="highlight"><pre><span></span><span class="nf">install.package</span><span class="p">(</span><span class="s">"survival"</span><span class="p">)</span>
<span class="nf">install.package</span><span class="p">(</span><span class="s">"glmnet"</span><span class="p">)</span>
<span class="nf">source</span><span class="p">(</span><span class="s">"https://bioconductor.org/biocLite.R"</span><span class="p">)</span>
<span class="nf">biocLite</span><span class="p">(</span><span class="s">"survcomp"</span><span class="p">)</span>
</pre></div>
</div>
</div>
</div>
<div class="section" id="distributed-computation">
<h2>Distributed computation<a class="headerlink" href="#distributed-computation" title="Permalink to this headline">¶</a></h2>
<p>It is possible to use the python ray framework <a class="reference external" href="https://github.com/ray-project/ray">https://github.com/ray-project/ray</a> to control the parallel computation of the multiple models. To use this framework, it is required to install it: <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">ray</span> <span class="pre">--user</span></code>.
Alternatively, it is also possible to create the model one by one without the need of the ray framework</p>
</div>
<div class="section" id="visualisation-module-experimental">
<h2>Visualisation module (Experimental)<a class="headerlink" href="#visualisation-module-experimental" title="Permalink to this headline">¶</a></h2>
<p>To visualise test sets projected into the multi-omic survival space, it is required to install <code class="docutils literal notranslate"><span class="pre">mpld3</span></code> module.
Note that the pip version of mpld3 installed with pip on my computer presented a <a class="reference external" href="https://github.com/mpld3/mpld3/issues/434">bug</a>: <code class="docutils literal notranslate"><span class="pre">TypeError:</span> <span class="pre">array([1.])</span> <span class="pre">is</span> <span class="pre">not</span> <span class="pre">JSON</span> <span class="pre">serializable</span> </code>. However, the <a class="reference external" href="https://github.com/mpld3/mpld3">newest</a> version of the mpld3 available from the github solved this issue. Rather than executing <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">mpld3</span> <span class="pre">--user</span></code> It is therefore recommended to install the newest version to avoid this issue directly from the github repository:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>git clone https://github.com/mpld3/mpld3
<span class="nb">cd</span> mpld3
pip install -e . --user
</pre></div>
</div>
<div class="section" id="id1">
<h3>Distributed computation<a class="headerlink" href="#id1" title="Permalink to this headline">¶</a></h3>
<ul class="simple">
<li><p>It is possible to use the python ray framework <a class="reference external" href="https://github.com/ray-project/ray">https://github.com/ray-project/ray</a> to control the parallel computation of the multiple models. To use this framework, it is required to install it: <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">ray</span></code></p></li>
<li><p>Alternatively, it is also possible to create the model one by one without the need of the ray framework</p></li>
</ul>
</div>
<div class="section" id="id2">
<h3>Visualisation module (Experimental)<a class="headerlink" href="#id2" title="Permalink to this headline">¶</a></h3>
<ul class="simple">
<li><p>To visualise test sets projected into the multi-omic survival space, it is required to install <code class="docutils literal notranslate"><span class="pre">mpld3</span></code> module: <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">mpld3</span></code></p></li>
<li><p>Note that the pip version of mpld3 installed on my computer presented a <a class="reference external" href="https://github.com/mpld3/mpld3/issues/434">bug</a>: <code class="docutils literal notranslate"><span class="pre">TypeError:</span> <span class="pre">array([1.])</span> <span class="pre">is</span> <span class="pre">not</span> <span class="pre">JSON</span> <span class="pre">serializable</span> </code>. However, the <a class="reference external" href="https://github.com/mpld3/mpld3">newest</a> version of the mpld3 available from the github solved this issue. It is therefore recommended to install the newest version to avoid this issue.</p></li>
</ul>
</div>
</div>
<div class="section" id="usage">
<h2>Usage<a class="headerlink" href="#usage" title="Permalink to this headline">¶</a></h2>
<ul class="simple">
<li><p>test if simdeep is functional (all the software are correctly installed):</p></li>
</ul>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span> python3 test/test_simdeep.py -v <span class="c1">#</span>
</pre></div>
</div>
<ul class="simple">
<li><p>All the default parameters are defined in the config file: <code class="docutils literal notranslate"><span class="pre">./simdeep/config.py</span></code> but can be passed dynamically. Three types of parameters must be defined:</p>
<ul>
<li><p>The training dataset (omics + survival input files)</p>
<ul>
<li><p>In addition, the parameters of the test set, i.e. the omic dataset and the survival file</p></li>
</ul>
</li>
<li><p>The parameters of the autoencoder (the default parameters works but it might be fine-tuned.</p></li>
<li><p>The parameters of the classification procedures (default are still good)</p></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="example-scripts">
<h2>Example scripts<a class="headerlink" href="#example-scripts" title="Permalink to this headline">¶</a></h2>
<p>Example scripts are availables in ./examples/ which will assist you to build a model from scratch with test and real data:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>examples
├── example_hyperparameters_tuning.py
├── example_hyperparameters_tuning_with_test_dataset.py
├── example_with_dummy_data_distributed.py
├── example_with_dummy_data.py
└── load_3_omics_model.py
</pre></div>
</div>
</div>
</div>
</div>
</div>
<footer>
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="usage.html" class="btn btn-neutral float-right" title="Tutorial: Simple DeepProg model" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
<a href="index.html" class="btn btn-neutral float-left" title="Welcome to DeepProg’s documentation!" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
</div>
<hr/>
<div role="contentinfo">
<p>
© Copyright 2019, Olivier Poirion.
</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script type="text/javascript">
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>
Datasets
Models
