Kristine_Wang_LS_DS_223_assignment.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.3"
    },
    "colab": {
      "name": "Kristine_Wang_LS_DS_223_assignment.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/KristineYW/DS-Unit-2-Kaggle-Challenge/blob/master/Kristine_Wang_LS_DS_223_assignment.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z5JpnVwknpls",
        "colab_type": "text"
      },
      "source": [
        "Lambda School Data Science\n",
        "\n",
        "*Unit 2, Sprint 2, Module 3*\n",
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_wmJMsDynplt",
        "colab_type": "text"
      },
      "source": [
        "# Cross-Validation\n",
        "\n",
        "\n",
        "## Assignment\n",
        "- [ ] [Review requirements for your portfolio project](https://lambdaschool.github.io/ds/unit2), then submit your dataset.\n",
        "- [ ] Continue to participate in our Kaggle challenge. \n",
        "- [ ] Use scikit-learn for hyperparameter optimization with RandomizedSearchCV.\n",
        "- [ ] Submit your predictions to our Kaggle competition. (Go to our Kaggle InClass competition webpage. Use the blue **Submit Predictions** button to upload your CSV file. Or you can use the Kaggle API to submit your predictions.)\n",
        "- [ ] Commit your notebook to your fork of the GitHub repo.\n",
        "\n",
        "\n",
        "**You can't just copy** from the lesson notebook to this assignment.\n",
        "\n",
        "- Because the lesson was **regression**, but the assignment is **classification.**\n",
        "- Because the lesson used [TargetEncoder](https://contrib.scikit-learn.org/categorical-encoding/targetencoder.html), which doesn't work as-is for _multi-class_ classification.\n",
        "\n",
        "So you will have to adapt the example, which is good real-world practice.\n",
        "\n",
        "1. Use a model for classification, such as [RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html)\n",
        "2. Use hyperparameters that match the classifier, such as `randomforestclassifier__ ...`\n",
        "3. Use a metric for classification, such as [`scoring='accuracy'`](https://scikit-learn.org/stable/modules/model_evaluation.html#common-cases-predefined-values)\n",
        "4. If you’re doing a multi-class classification problem — such as whether a waterpump is functional, functional needs repair, or nonfunctional — then use a categorical encoding that works for multi-class classification, such as [OrdinalEncoder](https://contrib.scikit-learn.org/categorical-encoding/ordinal.html) (not [TargetEncoder](https://contrib.scikit-learn.org/categorical-encoding/targetencoder.html))\n",
        "\n",
        "\n",
        "\n",
        "## Stretch Goals\n",
        "\n",
        "### Reading\n",
        "- Jake VanderPlas, [Python Data Science Handbook, Chapter 5.3](https://jakevdp.github.io/PythonDataScienceHandbook/05.03-hyperparameters-and-model-validation.html), Hyperparameters and Model Validation\n",
        "- Jake VanderPlas, [Statistics for Hackers](https://speakerdeck.com/jakevdp/statistics-for-hackers?slide=107)\n",
        "- Ron Zacharski, [A Programmer's Guide to Data Mining, Chapter 5](http://guidetodatamining.com/chapter5/), 10-fold cross validation\n",
        "- Sebastian Raschka, [A Basic Pipeline and Grid Search Setup](https://github.com/rasbt/python-machine-learning-book/blob/master/code/bonus/svm_iris_pipeline_and_gridsearch.ipynb)\n",
        "- Peter Worcester, [A Comparison of Grid Search and Randomized Search Using Scikit Learn](https://blog.usejournal.com/a-comparison-of-grid-search-and-randomized-search-using-scikit-learn-29823179bc85)\n",
        "\n",
        "### Doing\n",
        "- Add your own stretch goals!\n",
        "- Try other [categorical encodings](https://contrib.scikit-learn.org/categorical-encoding/). See the previous assignment notebook for details.\n",
        "- In additon to `RandomizedSearchCV`, scikit-learn has [`GridSearchCV`](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html). Another library called scikit-optimize has [`BayesSearchCV`](https://scikit-optimize.github.io/notebooks/sklearn-gridsearchcv-replacement.html). Experiment with these alternatives.\n",
        "- _[Introduction to Machine Learning with Python](http://shop.oreilly.com/product/0636920030515.do)_ discusses options for \"Grid-Searching Which Model To Use\" in Chapter 6:\n",
        "\n",
        "> You can even go further in combining GridSearchCV and Pipeline: it is also possible to search over the actual steps being performed in the pipeline (say whether to use StandardScaler or MinMaxScaler). This leads to an even bigger search space and should be considered carefully. Trying all possible solutions is usually not a viable machine learning strategy. However, here is an example comparing a RandomForestClassifier and an SVC ...\n",
        "\n",
        "The example is shown in [the accompanying notebook](https://github.com/amueller/introduction_to_ml_with_python/blob/master/06-algorithm-chains-and-pipelines.ipynb), code cells 35-37. Could you apply this concept to your own pipelines?\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FpAU8X9Wnplu",
        "colab_type": "text"
      },
      "source": [
        "### BONUS: Stacking!\n",
        "\n",
        "Here's some code you can use to \"stack\" multiple submissions, which is another form of ensembling:\n",
        "\n",
        "```python\n",
        "import pandas as pd\n",
        "\n",
        "# Filenames of your submissions you want to ensemble\n",
        "files = ['submission-01.csv', 'submission-02.csv', 'submission-03.csv']\n",
        "\n",
        "target = 'status_group'\n",
        "submissions = (pd.read_csv(file)[[target]] for file in files)\n",
        "ensemble = pd.concat(submissions, axis='columns')\n",
        "majority_vote = ensemble.mode(axis='columns')[0]\n",
        "\n",
        "sample_submission = pd.read_csv('sample_submission.csv')\n",
        "submission = sample_submission.copy()\n",
        "submission[target] = majority_vote\n",
        "submission.to_csv('my-ultimate-ensemble-submission.csv', index=False)\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hMyMA042nplu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%capture\n",
        "import sys\n",
        "\n",
        "# If you're on Colab:\n",
        "if 'google.colab' in sys.modules:\n",
        "    DATA_PATH = 'https://raw.githubusercontent.com/LambdaSchool/DS-Unit-2-Kaggle-Challenge/master/data/'\n",
        "    !pip install category_encoders==2.*\n",
        "\n",
        "# If you're working locally:\n",
        "else:\n",
        "    DATA_PATH = '../data/'"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lKuygOV-nplx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "\n",
        "# Merge train_features.csv & train_labels.csv\n",
        "train = pd.merge(pd.read_csv(DATA_PATH+'waterpumps/train_features.csv'), \n",
        "                 pd.read_csv(DATA_PATH+'waterpumps/train_labels.csv'))\n",
        "\n",
        "# Read test_features.csv & sample_submission.csv\n",
        "test = pd.read_csv(DATA_PATH+'waterpumps/test_features.csv')\n",
        "sample_submission = pd.read_csv(DATA_PATH+'waterpumps/sample_submission.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "v5wXFPRkAwfV",
        "colab_type": "code",
        "outputId": "bcc22502-d381-4c35-8657-bc05e1120728",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "test.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(14358, 40)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "w7ipuo5Vnpl0",
        "colab_type": "code",
        "outputId": "c52f0bc9-40c5-4146-e923-21e63adf6085",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "import numpy as np\n",
        "import category_encoders as ce\n",
        "from sklearn.preprocessing import RobustScaler\n",
        "from sklearn.impute import SimpleImputer\n",
        "\n",
        "def wrangle(df):\n",
        "\n",
        "  # Prevent error message\n",
        "  X = df.copy()\n",
        "  target='status_group'\n",
        "\n",
        "  # Treat near-zero values like zero \n",
        "  X['latitude'] = X['latitude'].replace(-2e-08,0)\n",
        "\n",
        "  # Replace zero values with nulls so we can impute later\n",
        "  cols_with_zeros = ['longitude', 'latitude', 'construction_year', \n",
        "                       'gps_height', 'population']\n",
        "  for col in cols_with_zeros:\n",
        "      X[col] = X[col].replace(0, np.nan)\n",
        "  \n",
        "  # Drop super high unique value percentage values \n",
        "  X = X.drop(columns=['recorded_by','id'])\n",
        "\n",
        "  # Convert date_recorded to datetime format so that we can calculate time between \n",
        "  # construction year and date recorded \n",
        "  X['date_recorded'] = pd.to_datetime(X['date_recorded'], infer_datetime_format=True)\n",
        "  \n",
        "  # Extract components from date_recorded, then drop the original column\n",
        "  X['year_recorded'] = X['date_recorded'].dt.year\n",
        "\n",
        "  # Engineer feature: how many years from construction_year to date_recorded\n",
        "  X['years'] = X['year_recorded'] - X['construction_year']\n",
        "  X['years_MISSING'] = X['years'].isnull()\n",
        "  \n",
        "  # Find amount of water per person\n",
        "  X['water/person'] = X['amount_tsh'].replace({0:1}) / X['population'].replace({0:1})\n",
        "  \n",
        "  # Find average population served per year \n",
        "  X['pop/year'] = X['population'].replace({0:1}) / X['years'].replace({0:1})\n",
        "\n",
        "  X = X.drop(columns=['date_recorded'])\n",
        "\n",
        "  # Return wrangled dataframe\n",
        "  return X"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.\n",
            "  import pandas.util.testing as tm\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3KDUL9Y6r9bI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Apply feature engineering to train data \n",
        "train = wrangle(train)\n",
        "test = wrangle(test)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Sdmu-ReVPWEj",
        "colab_type": "code",
        "outputId": "66cdb406-db6d-4bb8-d03b-ffaf09db1463",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "train.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(59400, 43)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fyFnv1lLPUnV",
        "colab_type": "code",
        "outputId": "2f693d5a-ab0e-49b9-9a21-bbee113431c0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "test.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(14358, 42)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vyTkv8--La4U",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def encoder(df):\n",
        "  target = 'status_group'\n",
        "  if target in df.columns:\n",
        "    df_features = df.drop(columns=[target])\n",
        "  else:\n",
        "    df_features = df\n",
        "  numeric_features = df_features.select_dtypes(include='number')\n",
        "  categorical = df_features.select_dtypes(exclude='number').nunique()\n",
        "  cat_features = categorical[categorical <=150].index.tolist()\n",
        "  high_cardinality_features = pd.DataFrame(categorical[categorical > 150])\n",
        "  df_high = df[high_cardinality_features.T.columns]\n",
        "  one_hot_encode = ce.OneHotEncoder(use_cat_names=True)\n",
        "  one_hot_encode.fit(df[cat_features])\n",
        "  low_cardinality_features = one_hot_encode.transform(df[cat_features])\n",
        "  df1 = numeric_features.join(low_cardinality_features, how='outer')\n",
        "  df2 = df1.join(df_high, how='outer')\n",
        "\n",
        "\n",
        "  return df2"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gSjPpRQEOPQc",
        "colab_type": "code",
        "outputId": "be77168e-905c-40bb-8609-c1ee003c3970",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 372
        }
      },
      "source": [
        "X_train = encoder(train)\n",
        "\n",
        "X_train.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>amount_tsh</th>\n",
              "      <th>gps_height</th>\n",
              "      <th>longitude</th>\n",
              "      <th>latitude</th>\n",
              "      <th>num_private</th>\n",
              "      <th>region_code</th>\n",
              "      <th>district_code</th>\n",
              "      <th>population</th>\n",
              "      <th>construction_year</th>\n",
              "      <th>year_recorded</th>\n",
              "      <th>years</th>\n",
              "      <th>water/person</th>\n",
              "      <th>pop/year</th>\n",
              "      <th>basin_Lake Nyasa</th>\n",
              "      <th>basin_Lake Victoria</th>\n",
              "      <th>basin_Pangani</th>\n",
              "      <th>basin_Ruvuma / Southern Coast</th>\n",
              "      <th>basin_Internal</th>\n",
              "      <th>basin_Lake Tanganyika</th>\n",
              "      <th>basin_Wami / Ruvu</th>\n",
              "      <th>basin_Rufiji</th>\n",
              "      <th>basin_Lake Rukwa</th>\n",
              "      <th>region_Iringa</th>\n",
              "      <th>region_Mara</th>\n",
              "      <th>region_Manyara</th>\n",
              "      <th>region_Mtwara</th>\n",
              "      <th>region_Kagera</th>\n",
              "      <th>region_Tanga</th>\n",
              "      <th>region_Shinyanga</th>\n",
              "      <th>region_Tabora</th>\n",
              "      <th>region_Pwani</th>\n",
              "      <th>region_Ruvuma</th>\n",
              "      <th>region_Kilimanjaro</th>\n",
              "      <th>region_Rukwa</th>\n",
              "      <th>region_Mwanza</th>\n",
              "      <th>region_Kigoma</th>\n",
              "      <th>region_Lindi</th>\n",
              "      <th>region_Dodoma</th>\n",
              "      <th>region_Arusha</th>\n",
              "      <th>region_Mbeya</th>\n",
              "      <th>...</th>\n",
              "      <th>source_spring</th>\n",
              "      <th>source_rainwater harvesting</th>\n",
              "      <th>source_dam</th>\n",
              "      <th>source_machine dbh</th>\n",
              "      <th>source_other</th>\n",
              "      <th>source_shallow well</th>\n",
              "      <th>source_river</th>\n",
              "      <th>source_hand dtw</th>\n",
              "      <th>source_lake</th>\n",
              "      <th>source_unknown</th>\n",
              "      <th>source_type_spring</th>\n",
              "      <th>source_type_rainwater harvesting</th>\n",
              "      <th>source_type_dam</th>\n",
              "      <th>source_type_borehole</th>\n",
              "      <th>source_type_other</th>\n",
              "      <th>source_type_shallow well</th>\n",
              "      <th>source_type_river/lake</th>\n",
              "      <th>source_class_groundwater</th>\n",
              "      <th>source_class_surface</th>\n",
              "      <th>source_class_unknown</th>\n",
              "      <th>waterpoint_type_communal standpipe</th>\n",
              "      <th>waterpoint_type_communal standpipe multiple</th>\n",
              "      <th>waterpoint_type_hand pump</th>\n",
              "      <th>waterpoint_type_other</th>\n",
              "      <th>waterpoint_type_improved spring</th>\n",
              "      <th>waterpoint_type_cattle trough</th>\n",
              "      <th>waterpoint_type_dam</th>\n",
              "      <th>waterpoint_type_group_communal standpipe</th>\n",
              "      <th>waterpoint_type_group_hand pump</th>\n",
              "      <th>waterpoint_type_group_other</th>\n",
              "      <th>waterpoint_type_group_improved spring</th>\n",
              "      <th>waterpoint_type_group_cattle trough</th>\n",
              "      <th>waterpoint_type_group_dam</th>\n",
              "      <th>years_MISSING</th>\n",
              "      <th>funder</th>\n",
              "      <th>installer</th>\n",
              "      <th>wpt_name</th>\n",
              "      <th>subvillage</th>\n",
              "      <th>ward</th>\n",
              "      <th>scheme_name</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>6000.0</td>\n",
              "      <td>1390.0</td>\n",
              "      <td>34.938093</td>\n",
              "      <td>-9.856322</td>\n",
              "      <td>0</td>\n",
              "      <td>11</td>\n",
              "      <td>5</td>\n",
              "      <td>109.0</td>\n",
              "      <td>1999.0</td>\n",
              "      <td>2011</td>\n",
              "      <td>12.0</td>\n",
              "      <td>55.045872</td>\n",
              "      <td>9.083333</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>False</td>\n",
              "      <td>Roman</td>\n",
              "      <td>Roman</td>\n",
              "      <td>none</td>\n",
              "      <td>Mnyusi B</td>\n",
              "      <td>Mundindi</td>\n",
              "      <td>Roman</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.0</td>\n",
              "      <td>1399.0</td>\n",
              "      <td>34.698766</td>\n",
              "      <td>-2.147466</td>\n",
              "      <td>0</td>\n",
              "      <td>20</td>\n",
              "      <td>2</td>\n",
              "      <td>280.0</td>\n",
              "      <td>2010.0</td>\n",
              "      <td>2013</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.003571</td>\n",
              "      <td>93.333333</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>False</td>\n",
              "      <td>Grumeti</td>\n",
              "      <td>GRUMETI</td>\n",
              "      <td>Zahanati</td>\n",
              "      <td>Nyamara</td>\n",
              "      <td>Natta</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>25.0</td>\n",
              "      <td>686.0</td>\n",
              "      <td>37.460664</td>\n",
              "      <td>-3.821329</td>\n",
              "      <td>0</td>\n",
              "      <td>21</td>\n",
              "      <td>4</td>\n",
              "      <td>250.0</td>\n",
              "      <td>2009.0</td>\n",
              "      <td>2013</td>\n",
              "      <td>4.0</td>\n",
              "      <td>0.100000</td>\n",
              "      <td>62.500000</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>False</td>\n",
              "      <td>Lottery Club</td>\n",
              "      <td>World vision</td>\n",
              "      <td>Kwa Mahundi</td>\n",
              "      <td>Majengo</td>\n",
              "      <td>Ngorika</td>\n",
              "      <td>Nyumba ya mungu pipe scheme</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.0</td>\n",
              "      <td>263.0</td>\n",
              "      <td>38.486161</td>\n",
              "      <td>-11.155298</td>\n",
              "      <td>0</td>\n",
              "      <td>90</td>\n",
              "      <td>63</td>\n",
              "      <td>58.0</td>\n",
              "      <td>1986.0</td>\n",
              "      <td>2013</td>\n",
              "      <td>27.0</td>\n",
              "      <td>0.017241</td>\n",
              "      <td>2.148148</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>False</td>\n",
              "      <td>Unicef</td>\n",
              "      <td>UNICEF</td>\n",
              "      <td>Zahanati Ya Nanyumbu</td>\n",
              "      <td>Mahakamani</td>\n",
              "      <td>Nanyumbu</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>31.130847</td>\n",
              "      <td>-1.825359</td>\n",
              "      <td>0</td>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2011</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>True</td>\n",
              "      <td>Action In A</td>\n",
              "      <td>Artisan</td>\n",
              "      <td>Shuleni</td>\n",
              "      <td>Kyanyamisa</td>\n",
              "      <td>Nyakasimbi</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 320 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "   amount_tsh  gps_height  ...        ward                  scheme_name\n",
              "0      6000.0      1390.0  ...    Mundindi                        Roman\n",
              "1         0.0      1399.0  ...       Natta                          NaN\n",
              "2        25.0       686.0  ...     Ngorika  Nyumba ya mungu pipe scheme\n",
              "3         0.0       263.0  ...    Nanyumbu                          NaN\n",
              "4         0.0         NaN  ...  Nyakasimbi                          NaN\n",
              "\n",
              "[5 rows x 320 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_h3AOX_xPIPm",
        "colab_type": "code",
        "outputId": "9ca9a22d-e733-477b-fe5e-2fe28b8dcd76",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "X_train.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(59400, 320)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jOJq0B6qsKKE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Import libraries and classes for pipelines\n",
        "\n",
        "import category_encoders as ce\n",
        "import numpy as np\n",
        "from sklearn.impute import SimpleImputer\n",
        "from sklearn.model_selection import cross_val_score\n",
        "from sklearn.pipeline import make_pipeline\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.model_selection import GridSearchCV, RandomizedSearchCV\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UtDM5WWWsgCA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Create target group and drop target from features columns\n",
        "\n",
        "target = 'status_group'\n",
        "features = train.columns.drop(target)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J_mLZWWNsw45",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Isolate train dataframes based on features and target groups \n",
        "\n",
        "X_train = train[features]\n",
        "y_train = train[target]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XFu1_ah0s61l",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Make a pipeline for RandomForestRegressor cross-validation\n",
        "\n",
        "pipeline = make_pipeline(\n",
        "    ce.OrdinalEncoder(), \n",
        "    SimpleImputer(strategy='mean'), \n",
        "    RandomForestClassifier(n_jobs=-1)\n",
        ")\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tzVMiJASuqL1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "param_distributions = {\n",
        "    'randomforestclassifier__bootstrap': [False,True],\n",
        "    'randomforestclassifier__n_estimators': [100,120,140,160,200,210],\n",
        "    'randomforestclassifier__min_samples_leaf': [1,2,5,7,9,10],\n",
        "    'randomforestclassifier__random_state': [0,42],\n",
        "    'randomforestclassifier__criterion': ['gini','entropy']\n",
        "}\n",
        "\n",
        "search = RandomizedSearchCV (\n",
        "    pipeline, \n",
        "    param_distributions=param_distributions,\n",
        "    n_iter=5,\n",
        "    cv=5,\n",
        "    scoring='accuracy',\n",
        "    verbose=10,\n",
        "    return_train_score=True,\n",
        "    n_jobs=-1\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GZubkBZFzOSf",
        "colab_type": "code",
        "outputId": "52ba573f-38e7-41db-9ca6-921a1c7c0ab7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 853
        }
      },
      "source": [
        "# Apply search to training data \n",
        "\n",
        "search.fit(X_train,y_train)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Fitting 5 folds for each of 5 candidates, totalling 25 fits\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 2 concurrent workers.\n",
            "[Parallel(n_jobs=-1)]: Done   1 tasks      | elapsed:   57.4s\n",
            "[Parallel(n_jobs=-1)]: Done   4 tasks      | elapsed:  2.0min\n",
            "[Parallel(n_jobs=-1)]: Done   9 tasks      | elapsed:  4.3min\n",
            "/usr/local/lib/python3.6/dist-packages/joblib/externals/loky/process_executor.py:706: UserWarning: A worker stopped while some jobs were given to the executor. This can be caused by a too short worker timeout or by a memory leak.\n",
            "  \"timeout or by a memory leak.\", UserWarning\n",
            "[Parallel(n_jobs=-1)]: Done  14 tasks      | elapsed:  5.8min\n",
            "[Parallel(n_jobs=-1)]: Done  21 tasks      | elapsed:  9.3min\n",
            "[Parallel(n_jobs=-1)]: Done  25 out of  25 | elapsed: 11.1min remaining:    0.0s\n",
            "[Parallel(n_jobs=-1)]: Done  25 out of  25 | elapsed: 11.1min finished\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "RandomizedSearchCV(cv=5, error_score=nan,\n",
              "                   estimator=Pipeline(memory=None,\n",
              "                                      steps=[('ordinalencoder',\n",
              "                                              OrdinalEncoder(cols=None,\n",
              "                                                             drop_invariant=False,\n",
              "                                                             handle_missing='value',\n",
              "                                                             handle_unknown='value',\n",
              "                                                             mapping=None,\n",
              "                                                             return_df=True,\n",
              "                                                             verbose=0)),\n",
              "                                             ('simpleimputer',\n",
              "                                              SimpleImputer(add_indicator=False,\n",
              "                                                            copy=True,\n",
              "                                                            fill_value=None,\n",
              "                                                            missing_values=nan,\n",
              "                                                            strategy='mean',\n",
              "                                                            verbose=0)...\n",
              "                   param_distributions={'randomforestclassifier__bootstrap': [False,\n",
              "                                                                              True],\n",
              "                                        'randomforestclassifier__criterion': ['gini',\n",
              "                                                                              'entropy'],\n",
              "                                        'randomforestclassifier__min_samples_leaf': [1,\n",
              "                                                                                     2,\n",
              "                                                                                     5,\n",
              "                                                                                     7,\n",
              "                                                                                     9,\n",
              "                                                                                     10],\n",
              "                                        'randomforestclassifier__n_estimators': [100,\n",
              "                                                                                 120,\n",
              "                                                                                 140,\n",
              "                                                                                 160,\n",
              "                                                                                 200,\n",
              "                                                                                 210],\n",
              "                                        'randomforestclassifier__random_state': [0,\n",
              "                                                                                 42]},\n",
              "                   pre_dispatch='2*n_jobs', random_state=None, refit=True,\n",
              "                   return_train_score=True, scoring='accuracy', verbose=10)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CI0HJqWtzH_L",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Check best parameters \n",
        "X_test = test\n",
        "y_pred = search.predict(X_test)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RZfFgQM5tQ90",
        "colab_type": "code",
        "outputId": "52377e56-6878-4963-f339-70ef06524517",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "prediction = pd.DataFrame(y_pred)\n",
        "prediction.rename(columns={0:'status_group'}, inplace=True)\n",
        "prediction.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>status_group</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>non functional</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>functional</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>functional</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>non functional</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>functional</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "     status_group\n",
              "0  non functional\n",
              "1      functional\n",
              "2      functional\n",
              "3  non functional\n",
              "4      functional"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a0KcFDKEtexl",
        "colab_type": "code",
        "outputId": "b40a4a34-0f60-41ad-ef51-9549421e92ab",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 360
        }
      },
      "source": [
        "train.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>amount_tsh</th>\n",
              "      <th>funder</th>\n",
              "      <th>gps_height</th>\n",
              "      <th>installer</th>\n",
              "      <th>longitude</th>\n",
              "      <th>latitude</th>\n",
              "      <th>wpt_name</th>\n",
              "      <th>num_private</th>\n",
              "      <th>basin</th>\n",
              "      <th>subvillage</th>\n",
              "      <th>region</th>\n",
              "      <th>region_code</th>\n",
              "      <th>district_code</th>\n",
              "      <th>lga</th>\n",
              "      <th>ward</th>\n",
              "      <th>population</th>\n",
              "      <th>public_meeting</th>\n",
              "      <th>scheme_management</th>\n",
              "      <th>scheme_name</th>\n",
              "      <th>permit</th>\n",
              "      <th>construction_year</th>\n",
              "      <th>extraction_type</th>\n",
              "      <th>extraction_type_group</th>\n",
              "      <th>extraction_type_class</th>\n",
              "      <th>management</th>\n",
              "      <th>management_group</th>\n",
              "      <th>payment</th>\n",
              "      <th>payment_type</th>\n",
              "      <th>water_quality</th>\n",
              "      <th>quality_group</th>\n",
              "      <th>quantity</th>\n",
              "      <th>quantity_group</th>\n",
              "      <th>source</th>\n",
              "      <th>source_type</th>\n",
              "      <th>source_class</th>\n",
              "      <th>waterpoint_type</th>\n",
              "      <th>waterpoint_type_group</th>\n",
              "      <th>status_group</th>\n",
              "      <th>year_recorded</th>\n",
              "      <th>years</th>\n",
              "      <th>years_MISSING</th>\n",
              "      <th>water/person</th>\n",
              "      <th>pop/year</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>6000.0</td>\n",
              "      <td>Roman</td>\n",
              "      <td>1390.0</td>\n",
              "      <td>Roman</td>\n",
              "      <td>34.938093</td>\n",
              "      <td>-9.856322</td>\n",
              "      <td>none</td>\n",
              "      <td>0</td>\n",
              "      <td>Lake Nyasa</td>\n",
              "      <td>Mnyusi B</td>\n",
              "      <td>Iringa</td>\n",
              "      <td>11</td>\n",
              "      <td>5</td>\n",
              "      <td>Ludewa</td>\n",
              "      <td>Mundindi</td>\n",
              "      <td>109.0</td>\n",
              "      <td>True</td>\n",
              "      <td>VWC</td>\n",
              "      <td>Roman</td>\n",
              "      <td>False</td>\n",
              "      <td>1999.0</td>\n",
              "      <td>gravity</td>\n",
              "      <td>gravity</td>\n",
              "      <td>gravity</td>\n",
              "      <td>vwc</td>\n",
              "      <td>user-group</td>\n",
              "      <td>pay annually</td>\n",
              "      <td>annually</td>\n",
              "      <td>soft</td>\n",
              "      <td>good</td>\n",
              "      <td>enough</td>\n",
              "      <td>enough</td>\n",
              "      <td>spring</td>\n",
              "      <td>spring</td>\n",
              "      <td>groundwater</td>\n",
              "      <td>communal standpipe</td>\n",
              "      <td>communal standpipe</td>\n",
              "      <td>functional</td>\n",
              "      <td>2011</td>\n",
              "      <td>12.0</td>\n",
              "      <td>False</td>\n",
              "      <td>55.045872</td>\n",
              "      <td>9.083333</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.0</td>\n",
              "      <td>Grumeti</td>\n",
              "      <td>1399.0</td>\n",
              "      <td>GRUMETI</td>\n",
              "      <td>34.698766</td>\n",
              "      <td>-2.147466</td>\n",
              "      <td>Zahanati</td>\n",
              "      <td>0</td>\n",
              "      <td>Lake Victoria</td>\n",
              "      <td>Nyamara</td>\n",
              "      <td>Mara</td>\n",
              "      <td>20</td>\n",
              "      <td>2</td>\n",
              "      <td>Serengeti</td>\n",
              "      <td>Natta</td>\n",
              "      <td>280.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Other</td>\n",
              "      <td>NaN</td>\n",
              "      <td>True</td>\n",
              "      <td>2010.0</td>\n",
              "      <td>gravity</td>\n",
              "      <td>gravity</td>\n",
              "      <td>gravity</td>\n",
              "      <td>wug</td>\n",
              "      <td>user-group</td>\n",
              "      <td>never pay</td>\n",
              "      <td>never pay</td>\n",
              "      <td>soft</td>\n",
              "      <td>good</td>\n",
              "      <td>insufficient</td>\n",
              "      <td>insufficient</td>\n",
              "      <td>rainwater harvesting</td>\n",
              "      <td>rainwater harvesting</td>\n",
              "      <td>surface</td>\n",
              "      <td>communal standpipe</td>\n",
              "      <td>communal standpipe</td>\n",
              "      <td>functional</td>\n",
              "      <td>2013</td>\n",
              "      <td>3.0</td>\n",
              "      <td>False</td>\n",
              "      <td>0.003571</td>\n",
              "      <td>93.333333</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>25.0</td>\n",
              "      <td>Lottery Club</td>\n",
              "      <td>686.0</td>\n",
              "      <td>World vision</td>\n",
              "      <td>37.460664</td>\n",
              "      <td>-3.821329</td>\n",
              "      <td>Kwa Mahundi</td>\n",
              "      <td>0</td>\n",
              "      <td>Pangani</td>\n",
              "      <td>Majengo</td>\n",
              "      <td>Manyara</td>\n",
              "      <td>21</td>\n",
              "      <td>4</td>\n",
              "      <td>Simanjiro</td>\n",
              "      <td>Ngorika</td>\n",
              "      <td>250.0</td>\n",
              "      <td>True</td>\n",
              "      <td>VWC</td>\n",
              "      <td>Nyumba ya mungu pipe scheme</td>\n",
              "      <td>True</td>\n",
              "      <td>2009.0</td>\n",
              "      <td>gravity</td>\n",
              "      <td>gravity</td>\n",
              "      <td>gravity</td>\n",
              "      <td>vwc</td>\n",
              "      <td>user-group</td>\n",
              "      <td>pay per bucket</td>\n",
              "      <td>per bucket</td>\n",
              "      <td>soft</td>\n",
              "      <td>good</td>\n",
              "      <td>enough</td>\n",
              "      <td>enough</td>\n",
              "      <td>dam</td>\n",
              "      <td>dam</td>\n",
              "      <td>surface</td>\n",
              "      <td>communal standpipe multiple</td>\n",
              "      <td>communal standpipe</td>\n",
              "      <td>functional</td>\n",
              "      <td>2013</td>\n",
              "      <td>4.0</td>\n",
              "      <td>False</td>\n",
              "      <td>0.100000</td>\n",
              "      <td>62.500000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.0</td>\n",
              "      <td>Unicef</td>\n",
              "      <td>263.0</td>\n",
              "      <td>UNICEF</td>\n",
              "      <td>38.486161</td>\n",
              "      <td>-11.155298</td>\n",
              "      <td>Zahanati Ya Nanyumbu</td>\n",
              "      <td>0</td>\n",
              "      <td>Ruvuma / Southern Coast</td>\n",
              "      <td>Mahakamani</td>\n",
              "      <td>Mtwara</td>\n",
              "      <td>90</td>\n",
              "      <td>63</td>\n",
              "      <td>Nanyumbu</td>\n",
              "      <td>Nanyumbu</td>\n",
              "      <td>58.0</td>\n",
              "      <td>True</td>\n",
              "      <td>VWC</td>\n",
              "      <td>NaN</td>\n",
              "      <td>True</td>\n",
              "      <td>1986.0</td>\n",
              "      <td>submersible</td>\n",
              "      <td>submersible</td>\n",
              "      <td>submersible</td>\n",
              "      <td>vwc</td>\n",
              "      <td>user-group</td>\n",
              "      <td>never pay</td>\n",
              "      <td>never pay</td>\n",
              "      <td>soft</td>\n",
              "      <td>good</td>\n",
              "      <td>dry</td>\n",
              "      <td>dry</td>\n",
              "      <td>machine dbh</td>\n",
              "      <td>borehole</td>\n",
              "      <td>groundwater</td>\n",
              "      <td>communal standpipe multiple</td>\n",
              "      <td>communal standpipe</td>\n",
              "      <td>non functional</td>\n",
              "      <td>2013</td>\n",
              "      <td>27.0</td>\n",
              "      <td>False</td>\n",
              "      <td>0.017241</td>\n",
              "      <td>2.148148</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.0</td>\n",
              "      <td>Action In A</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Artisan</td>\n",
              "      <td>31.130847</td>\n",
              "      <td>-1.825359</td>\n",
              "      <td>Shuleni</td>\n",
              "      <td>0</td>\n",
              "      <td>Lake Victoria</td>\n",
              "      <td>Kyanyamisa</td>\n",
              "      <td>Kagera</td>\n",
              "      <td>18</td>\n",
              "      <td>1</td>\n",
              "      <td>Karagwe</td>\n",
              "      <td>Nyakasimbi</td>\n",
              "      <td>NaN</td>\n",
              "      <td>True</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>True</td>\n",
              "      <td>NaN</td>\n",
              "      <td>gravity</td>\n",
              "      <td>gravity</td>\n",
              "      <td>gravity</td>\n",
              "      <td>other</td>\n",
              "      <td>other</td>\n",
              "      <td>never pay</td>\n",
              "      <td>never pay</td>\n",
              "      <td>soft</td>\n",
              "      <td>good</td>\n",
              "      <td>seasonal</td>\n",
              "      <td>seasonal</td>\n",
              "      <td>rainwater harvesting</td>\n",
              "      <td>rainwater harvesting</td>\n",
              "      <td>surface</td>\n",
              "      <td>communal standpipe</td>\n",
              "      <td>communal standpipe</td>\n",
              "      <td>functional</td>\n",
              "      <td>2011</td>\n",
              "      <td>NaN</td>\n",
              "      <td>True</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   amount_tsh        funder  gps_height  ... years_MISSING  water/person   pop/year\n",
              "0      6000.0         Roman      1390.0  ...         False     55.045872   9.083333\n",
              "1         0.0       Grumeti      1399.0  ...         False      0.003571  93.333333\n",
              "2        25.0  Lottery Club       686.0  ...         False      0.100000  62.500000\n",
              "3         0.0        Unicef       263.0  ...         False      0.017241   2.148148\n",
              "4         0.0   Action In A         NaN  ...          True           NaN        NaN\n",
              "\n",
              "[5 rows x 43 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QAn7tgaCty2D",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "prediction = pd.DataFrame(y_pred, columns=['status_group']).to_csv('Kristine_submission_2.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F0b1kYxttXiB",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from google.colab import files\n",
        "files.download('Kristine_submission_2.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LwbYqdTDtWvx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Check scores for different 5 fold values \n",
        "\n",
        "k = 5\n",
        "scores = cross_val_score(pipeline, X_train, y_train, cv=k,\n",
        "                         scoring='accuracy')\n",
        "\n",
        "print(f'Accuracy for {k} folds is:', scores)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q4whGMRMvSG3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "scores.mean()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tXPqRYbGkC1G",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print('Model Hyperparameters:')\n",
        "print(pipeline.named_steps['randomforestclassifier'])"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}