FullStudy_ExtractPyRadiomics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Extract PyRadiomics Features Full Study\n",
    "## Extraction on full feature selection\n",
    "This notebook extracts pyradiomics features on the mpReview segmentations in the designated folder. Results will be saved in the \"EvalData\" folder located next to this notebook file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### User Input\n",
    "Specify the folder with the data (MPReview-style directory/file structure).\n",
    "Specify settings file to configure the feature extractor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "MPREVIEW_BASE_FOLDER = \"D:/Dropbox (Partners HealthCare)/Repeat_studies-Cases\"\n",
    "SETTINGS_BASE_FILE_NAMES = [\"FullStudySettings_2D\",\n",
    "                            \"FullStudySettings_3D\",\n",
    "                            \"FullStudySettings_noNormalization_2D\",\n",
    "                            \"FullStudySettings_noNormalization_3D\"]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preparation\n",
    "First do some basic setup, import all libs we need, parse the mpReview directory for all available cases with segmentations and select which cases we want to process. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from mpReviewUtils import MpReviewParser\n",
    "import radiomics\n",
    "import pandas as pd\n",
    "\n",
    "# tell pandas the display width (more ore less), since it cannot autodetect in a browser, \n",
    "# lower this if it is too high for your display (and you encounter ugly wrapping effects)\n",
    "pd.set_option('display.width', 120)\n",
    "\n",
    "mpReviewParser = MpReviewParser(MPREVIEW_BASE_FOLDER)\n",
    "dfSegs = pd.DataFrame(mpReviewParser.getSegmentationRecords())\n",
    "dfSelectedSegs = dfSegs[(dfSegs.measurements.notnull())]\n",
    "\n",
    "assert dfSelectedSegs.shape[0] == 326\n",
    "#print(\"Selected\", dfSelectedSegs.shape[0])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Extract the features\n",
    "NOTE: This only works with 64bit Python. With 32bit we get memory allocation porblems\n",
    "\n",
    "Extract features for each extractor setting in the file list. For each setting, features are extracted for several bin widths and the filenames are tagged accordingly. Changes to other parameters of the extraction (e.g. normalization, 2D/3D texture features) should be done in the settings files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for settingsBaseFileName in SETTINGS_BASE_FILE_NAMES:\n",
    "  settingsFile = os.path.join(os.getcwd(), \"PyRadiomicsSettings\", settingsBaseFileName + \".yaml\")\n",
    "  extractor = radiomics.featureextractor.RadiomicsFeaturesExtractor(settingsFile)\n",
    "  \n",
    "  #print(extractor._enabledFeatures)\n",
    "  #print(list(dfSelectedSegs))\n",
    "  #extractor.settings\n",
    "\n",
    "  for binWidth in [5, 10, 15, 20, 40]:\n",
    "    extractor.settings[\"binWidth\"] = binWidth\n",
    "    features = []\n",
    "    print(\"Extracting features for (image/mask): \")\n",
    "    for idx, row in dfSelectedSegs.iterrows():\n",
    "      print(idx, row.labelFileName)\n",
    "      featureVector = extractor.execute(row.origFileName, row.labelFileName, label = row.labelValue)\n",
    "      featureVector[\"study\"] = row.study\n",
    "      featureVector[\"series\"] = row.series\n",
    "      featureVector[\"canonicalType\"] = row.canonicalType\n",
    "      featureVector[\"segmentedStructure\"] = row.segmentedStructure\n",
    "      features.append(featureVector)\n",
    "    print(\"------------ DONE ------------\")\n",
    "\n",
    "\n",
    "    # postprocess and check features\n",
    "    dfFeatures = pd.DataFrame(features)\n",
    "    print(dfFeatures.shape)\n",
    "\n",
    "    dfNanSel = dfFeatures[(dfFeatures.isnull().any(axis = 1))]\n",
    "    print(dfNanSel.shape)\n",
    "    print(dfNanSel.loc[:, dfNanSel.isnull().any()])\n",
    "\n",
    "    # save to disk\n",
    "    outputBaseDir = os.path.join(os.getcwd(), \"EvalData\")\n",
    "    if not os.path.exists(outputBaseDir):\n",
    "      os.mkdir(outputBaseDir)\n",
    "    outFileName = os.path.join(outputBaseDir, \"pyRadiomicsFeatures_\" + settingsBaseFileName + \"_bin\" + str(binWidth) + \".csv\")\n",
    "    dfFeatures.to_csv(outFileName, sep = \";\", index = False)\n",
    "    print(\"Features saved to\", outFileName)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}