Datasets

From a project

If you are working in the context of a :ref:`Project <project_setup>`, a dataset can be quickly created using :meth:`Project.dataset`. A dataset can be loaded from a given Project with the following parameters:

• tile_px is the tile size, in pixels
• tile_um is the tile size, in microns (int) or magnification ('40x')
• sources is an optional list of dataset sources to use

import slideflow as sf

P = sf.load_project('/project/path')
dataset = P.dataset(tile_px=299, tile_um='10x', sources=['Source1'])

If sources is not provided, all available sources will be used.

Alternatively, you can accomplish the same by creating a :class:`Dataset` object directly, passing in the project :ref:`dataset configuration file <dataset_sources>` to the config argument, and a path to the annotations file to annotations:

dataset = sf.Dataset(
    config='config.json',
    sources=['Source1'],
    annotations='annotations.csv',
    tile_px=299,
    tile_um='10x'
)

Manually from paths

You can also create a dataset by manually supplying paths to slides, destination for image tiles, and clinical annotations. A single dataset source will be created from the provided arguments, which include:

• tile_px is the tile size, in pixels
• tile_um is the size in microns or magnification
• slides is the directory containing whole-slide images
• roi is the directory containing Regions of Interest *.csv files
• tfrecords is the path to where image tiles should be stored in TFRecords
• tiles is the path to where image tiles should be stored as *.jpg images
• annotations is either an annotations file (CSV) or Pandas DataFrame.

For example, to create a dataset from a set of slides, with a configured TFRecord directory and annotations provided via Pandas DataFrame:

import pandas as pd

# Create some clinical annotations
df = pd.DataFrame(...)

# Create a dataset
dataset = sf.Dataset(
    slides='/slides',
    tfrecords='/tfrecords',
    annotations=df,
    tile_px=299,
    tile_um='10x'
)

When creating a :class:`Dataset` manually from paths, tfrecords should be organized into subdirectories named according to tile size. Using the above example, the tfrecords directory should look like:

/tfrecords
└── 299px_10x
    ├── slide1.tfrecords
    ├── slide2.tfrecords
    ├── slide3.tfrecords
    └── ...

Creating a split

Datasets can be split into training and validation or test datasets with :meth:`Dataset.split`. The result of this function is two datasets - the first training, the second validation - each a separate instance of :class:`Dataset`.

Slideflow provides several options for preparing a validation plan, including:

• strategy: 'bootstrap', 'k-fold', 'k-fold-manual', 'k-fold-preserved-site', 'fixed', and 'none'
• fraction: (float between 0-1) [not used for k-fold validation]
• k_fold: int

The default validation strategy is three-fold cross-validation (strategy='k-fold' and k=3).

# Split a dataset into training and validation
# using 5-fold cross-validation, with this being
# the first cross-fold.
train_dataset, test_dataset = dataset.split(
    model_type='classification', # Categorical labels
    labels='subtype',            # Label to balance between datasets
    k_fold=5,                    # Total number of crossfolds
    k_fold_iter=1,               # Cross-fold iteration
    splits='splits.json'         # Where to save/load crossfold splits
)

You can also use :meth:`Dataset.kfold_split` to iterate through cross-fold splits:

# Split a dataset into training and validation
# using 5-fold cross-validation
for train, test in dataset.kfold_split(k=5, labels='subtype'):
    ...

Validation strategies

The strategy option determines how the validation data is selected.

If fixed, a certain percentage of your training data is set aside for testing (determined by fraction).

If bootstrap, validation data will be selected at random (percentage determined by fraction), and all training iterations will be repeated a number of times equal to k_fold. When used during training, the reported model training metrics will be an average of all bootstrap iterations.

If k-fold, training data will be automatically separated into k number of groups (where k is equal to k_fold), and all training iterations will be repeated k number of times using k-fold cross validation. The saved and reported model training metrics will be an average of all k-fold iterations.

Datasets can be separated into manually-curated k-folds using the k-fold-manual strategy. Assign each slide to a k-fold cohort in the annotations file, and designate the appropriate column header with k_fold_header

The k-fold-preserved-site strategy is a cross-validation strategy that ensures site is preserved across the training/validation sets, in order to reduce bias from batch effect as described by Howard, et al. This strategy is recommended when using data from The Cancer Genome Atlas (TCGA).

If none, no validation testing will be performed.

Re-using splits

For all validation strategies, training/validation splits can be logged to a JSON file automatically if a splits configuration file is provided to the argument splits. When provided, :meth:`Dataset.split` will prioritize using previously-generated training/validation splits rather than generating a new split. This aids with experiment reproducibility and hyperparameter tuning. If training/validation splits are being prepared by a :ref:`Project-level function <project>`, splits will be automatically logged to a splits.json file in the project root directory.