DataCheckpoint.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# UC San Diego: Data Science in Practice - Data Checkpoint\n",
    "### Summer Session I 2023 | Instructor : C. Alex Simpkins Ph.D.\n",
    "\n",
    "## Power Outages Data Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Names\n",
    "\n",
    "- Wenhua Tang\n",
    "- Qianyu Peng\n",
    "- Vicky Li\n",
    "- Yao Liu\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='research_question'></a>\n",
    "# Research Question"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* One sentence that describes the question you address in your project. Make sure what you’re measuring (variables) to answer your question is clear!\n",
    "\n",
    "Is there a significant difference in the number of affected customers when the outage is induced by a disruption in system operability or a disruption in non-system operability?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dataset(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Fill in your dataset information here*<br>\n",
    "\n",
    "- Dataset name: Major Power Outage Risks in the U.S.\n",
    "- Link to the dataset: https://engineering.purdue.edu/LASCI/research-data/outages/outagerisks\n",
    "- Number of observations: 1534\n",
    "- This dataset contains information about significant power outages that occurred in various states across the continental U.S. In addition to the outages, the data includes details about their locations, the weather conditions in those regions, land usage, electricity consumption patterns, and economic information related to the affected states.\n",
    "- Since this dataset contains the variables we need for the research question, such as \"affected customers\" and \"system operability,\" we believe it is suitable for our project."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Wrangling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Explain steps taken to pull the data you need into Python.\n",
    "- First, we downloaded the data from the given url into our local computer\n",
    "- Then, we imported and installed all necessary packages into the Jupyter Notebook for further use.\n",
    "- Lastly, we use `pd.read_excel()` to read the data and select only the useful rows from the raw dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# set up\n",
    "import matplotlib.pyplot as plt \n",
    "import numpy as np\n",
    "import os\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "# !pip install pandas openpyxl"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Major power outage events in the continental U.S.</th>\n",
       "      <th>Unnamed: 1</th>\n",
       "      <th>Unnamed: 2</th>\n",
       "      <th>Unnamed: 3</th>\n",
       "      <th>Unnamed: 4</th>\n",
       "      <th>Unnamed: 5</th>\n",
       "      <th>Unnamed: 6</th>\n",
       "      <th>Unnamed: 7</th>\n",
       "      <th>Unnamed: 8</th>\n",
       "      <th>Unnamed: 9</th>\n",
       "      <th>...</th>\n",
       "      <th>Unnamed: 47</th>\n",
       "      <th>Unnamed: 48</th>\n",
       "      <th>Unnamed: 49</th>\n",
       "      <th>Unnamed: 50</th>\n",
       "      <th>Unnamed: 51</th>\n",
       "      <th>Unnamed: 52</th>\n",
       "      <th>Unnamed: 53</th>\n",
       "      <th>Unnamed: 54</th>\n",
       "      <th>Unnamed: 55</th>\n",
       "      <th>Unnamed: 56</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>2011</td>\n",
       "      <td>7</td>\n",
       "      <td>Minnesota</td>\n",
       "      <td>MN</td>\n",
       "      <td>MRO</td>\n",
       "      <td>East North Central</td>\n",
       "      <td>-0.3</td>\n",
       "      <td>normal</td>\n",
       "      <td>...</td>\n",
       "      <td>73.27</td>\n",
       "      <td>15.28</td>\n",
       "      <td>2279</td>\n",
       "      <td>1700.5</td>\n",
       "      <td>18.2</td>\n",
       "      <td>2.14</td>\n",
       "      <td>0.6</td>\n",
       "      <td>91.592666</td>\n",
       "      <td>8.407334</td>\n",
       "      <td>5.478743</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>NaN</td>\n",
       "      <td>2</td>\n",
       "      <td>2014</td>\n",
       "      <td>5</td>\n",
       "      <td>Minnesota</td>\n",
       "      <td>MN</td>\n",
       "      <td>MRO</td>\n",
       "      <td>East North Central</td>\n",
       "      <td>-0.1</td>\n",
       "      <td>normal</td>\n",
       "      <td>...</td>\n",
       "      <td>73.27</td>\n",
       "      <td>15.28</td>\n",
       "      <td>2279</td>\n",
       "      <td>1700.5</td>\n",
       "      <td>18.2</td>\n",
       "      <td>2.14</td>\n",
       "      <td>0.6</td>\n",
       "      <td>91.592666</td>\n",
       "      <td>8.407334</td>\n",
       "      <td>5.478743</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NaN</td>\n",
       "      <td>3</td>\n",
       "      <td>2010</td>\n",
       "      <td>10</td>\n",
       "      <td>Minnesota</td>\n",
       "      <td>MN</td>\n",
       "      <td>MRO</td>\n",
       "      <td>East North Central</td>\n",
       "      <td>-1.5</td>\n",
       "      <td>cold</td>\n",
       "      <td>...</td>\n",
       "      <td>73.27</td>\n",
       "      <td>15.28</td>\n",
       "      <td>2279</td>\n",
       "      <td>1700.5</td>\n",
       "      <td>18.2</td>\n",
       "      <td>2.14</td>\n",
       "      <td>0.6</td>\n",
       "      <td>91.592666</td>\n",
       "      <td>8.407334</td>\n",
       "      <td>5.478743</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>2012</td>\n",
       "      <td>6</td>\n",
       "      <td>Minnesota</td>\n",
       "      <td>MN</td>\n",
       "      <td>MRO</td>\n",
       "      <td>East North Central</td>\n",
       "      <td>-0.1</td>\n",
       "      <td>normal</td>\n",
       "      <td>...</td>\n",
       "      <td>73.27</td>\n",
       "      <td>15.28</td>\n",
       "      <td>2279</td>\n",
       "      <td>1700.5</td>\n",
       "      <td>18.2</td>\n",
       "      <td>2.14</td>\n",
       "      <td>0.6</td>\n",
       "      <td>91.592666</td>\n",
       "      <td>8.407334</td>\n",
       "      <td>5.478743</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>NaN</td>\n",
       "      <td>5</td>\n",
       "      <td>2015</td>\n",
       "      <td>7</td>\n",
       "      <td>Minnesota</td>\n",
       "      <td>MN</td>\n",
       "      <td>MRO</td>\n",
       "      <td>East North Central</td>\n",
       "      <td>1.2</td>\n",
       "      <td>warm</td>\n",
       "      <td>...</td>\n",
       "      <td>73.27</td>\n",
       "      <td>15.28</td>\n",
       "      <td>2279</td>\n",
       "      <td>1700.5</td>\n",
       "      <td>18.2</td>\n",
       "      <td>2.14</td>\n",
       "      <td>0.6</td>\n",
       "      <td>91.592666</td>\n",
       "      <td>8.407334</td>\n",
       "      <td>5.478743</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 57 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Major power outage events in the continental U.S. Unnamed: 1 Unnamed: 2   \n",
       "6                                                NaN          1       2011  \\\n",
       "7                                                NaN          2       2014   \n",
       "8                                                NaN          3       2010   \n",
       "9                                                NaN          4       2012   \n",
       "10                                               NaN          5       2015   \n",
       "\n",
       "   Unnamed: 3 Unnamed: 4 Unnamed: 5 Unnamed: 6          Unnamed: 7 Unnamed: 8   \n",
       "6           7  Minnesota         MN        MRO  East North Central       -0.3  \\\n",
       "7           5  Minnesota         MN        MRO  East North Central       -0.1   \n",
       "8          10  Minnesota         MN        MRO  East North Central       -1.5   \n",
       "9           6  Minnesota         MN        MRO  East North Central       -0.1   \n",
       "10          7  Minnesota         MN        MRO  East North Central        1.2   \n",
       "\n",
       "   Unnamed: 9  ... Unnamed: 47 Unnamed: 48 Unnamed: 49 Unnamed: 50   \n",
       "6      normal  ...       73.27       15.28        2279      1700.5  \\\n",
       "7      normal  ...       73.27       15.28        2279      1700.5   \n",
       "8        cold  ...       73.27       15.28        2279      1700.5   \n",
       "9      normal  ...       73.27       15.28        2279      1700.5   \n",
       "10       warm  ...       73.27       15.28        2279      1700.5   \n",
       "\n",
       "   Unnamed: 51 Unnamed: 52 Unnamed: 53 Unnamed: 54 Unnamed: 55 Unnamed: 56  \n",
       "6         18.2        2.14         0.6   91.592666    8.407334    5.478743  \n",
       "7         18.2        2.14         0.6   91.592666    8.407334    5.478743  \n",
       "8         18.2        2.14         0.6   91.592666    8.407334    5.478743  \n",
       "9         18.2        2.14         0.6   91.592666    8.407334    5.478743  \n",
       "10        18.2        2.14         0.6   91.592666    8.407334    5.478743  \n",
       "\n",
       "[5 rows x 57 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_excel('outage.xlsx') # read data\n",
    "coloum_name_lst = list(df.iloc[4]) # get the actual column name\n",
    "coloum_name_lst.remove('variables') # remove unnecessary columns\n",
    "coloum_name_lst.remove('OBS')\n",
    "df = df.iloc[6:1539] # select only useful raws \n",
    "df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Cleaning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Describe your data cleaning steps here.\n",
    "- We first dropped the unrelated columns from the dataset\n",
    "- Then, we got the actual columns from the dataset and replaced the default columns with the desired ones.\n",
    "- For better analysis, we sorted the dataset by `Year` and then reset the index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# drop unrelated columns\n",
    "df = df.drop(\n",
    "    columns = ['Major power outage events in the continental U.S.','Unnamed: 1'])\n",
    "\n",
    "#replace default column names with new column names\n",
    "df.columns = coloum_name_lst \n",
    "\n",
    "#sort rows based on years\n",
    "df = (df.sort_values('YEAR')\n",
    "      .reset_index()\n",
    "      .drop(columns = ['index']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "YEAR                       0.000000\n",
       "MONTH                      0.005219\n",
       "U.S._STATE                 0.000000\n",
       "POSTAL.CODE                0.000000\n",
       "NERC.REGION                0.000000\n",
       "CLIMATE.REGION             0.003262\n",
       "ANOMALY.LEVEL              0.005219\n",
       "CLIMATE.CATEGORY           0.005219\n",
       "OUTAGE.START.DATE          0.005219\n",
       "OUTAGE.START.TIME          0.005219\n",
       "OUTAGE.RESTORATION.DATE    0.037182\n",
       "OUTAGE.RESTORATION.TIME    0.037182\n",
       "CAUSE.CATEGORY             0.000000\n",
       "CAUSE.CATEGORY.DETAIL      0.307241\n",
       "HURRICANE.NAMES            0.953033\n",
       "dtype: float64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the proportion of missing values in each column\n",
    "df.isna().mean().head(15) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the missingness in `HURRICANE.NAMES` is approximately 0.96, we have decided not to drop the NaN values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH</th>\n",
       "      <th>U.S._STATE</th>\n",
       "      <th>POSTAL.CODE</th>\n",
       "      <th>NERC.REGION</th>\n",
       "      <th>CLIMATE.REGION</th>\n",
       "      <th>ANOMALY.LEVEL</th>\n",
       "      <th>CLIMATE.CATEGORY</th>\n",
       "      <th>OUTAGE.START.DATE</th>\n",
       "      <th>OUTAGE.START.TIME</th>\n",
       "      <th>...</th>\n",
       "      <th>POPPCT_URBAN</th>\n",
       "      <th>POPPCT_UC</th>\n",
       "      <th>POPDEN_URBAN</th>\n",
       "      <th>POPDEN_UC</th>\n",
       "      <th>POPDEN_RURAL</th>\n",
       "      <th>AREAPCT_URBAN</th>\n",
       "      <th>AREAPCT_UC</th>\n",
       "      <th>PCT_LAND</th>\n",
       "      <th>PCT_WATER_TOT</th>\n",
       "      <th>PCT_WATER_INLAND</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>North Carolina</td>\n",
       "      <td>NC</td>\n",
       "      <td>SERC</td>\n",
       "      <td>Southeast</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>66.09</td>\n",
       "      <td>11.21</td>\n",
       "      <td>1367.2</td>\n",
       "      <td>1043.7</td>\n",
       "      <td>73.5</td>\n",
       "      <td>9.48</td>\n",
       "      <td>2.11</td>\n",
       "      <td>90.336127</td>\n",
       "      <td>9.663873</td>\n",
       "      <td>7.52894</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2000</td>\n",
       "      <td>6</td>\n",
       "      <td>California</td>\n",
       "      <td>CA</td>\n",
       "      <td>WECC</td>\n",
       "      <td>West</td>\n",
       "      <td>-0.7</td>\n",
       "      <td>cold</td>\n",
       "      <td>2000-06-14 00:00:00</td>\n",
       "      <td>13:13:00</td>\n",
       "      <td>...</td>\n",
       "      <td>94.95</td>\n",
       "      <td>5.22</td>\n",
       "      <td>4303.7</td>\n",
       "      <td>2124.1</td>\n",
       "      <td>12.7</td>\n",
       "      <td>5.28</td>\n",
       "      <td>0.59</td>\n",
       "      <td>95.164177</td>\n",
       "      <td>4.835823</td>\n",
       "      <td>1.730658</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2000</td>\n",
       "      <td>5</td>\n",
       "      <td>Texas</td>\n",
       "      <td>TX</td>\n",
       "      <td>TRE</td>\n",
       "      <td>South</td>\n",
       "      <td>-0.7</td>\n",
       "      <td>cold</td>\n",
       "      <td>2000-05-02 00:00:00</td>\n",
       "      <td>04:00:00</td>\n",
       "      <td>...</td>\n",
       "      <td>84.7</td>\n",
       "      <td>9.35</td>\n",
       "      <td>2435.3</td>\n",
       "      <td>1539.9</td>\n",
       "      <td>15.2</td>\n",
       "      <td>3.35</td>\n",
       "      <td>0.58</td>\n",
       "      <td>97.258336</td>\n",
       "      <td>2.742036</td>\n",
       "      <td>2.090873</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2000</td>\n",
       "      <td>5</td>\n",
       "      <td>Texas</td>\n",
       "      <td>TX</td>\n",
       "      <td>SPP</td>\n",
       "      <td>South</td>\n",
       "      <td>-0.7</td>\n",
       "      <td>cold</td>\n",
       "      <td>2000-05-24 00:00:00</td>\n",
       "      <td>10:15:00</td>\n",
       "      <td>...</td>\n",
       "      <td>84.7</td>\n",
       "      <td>9.35</td>\n",
       "      <td>2435.3</td>\n",
       "      <td>1539.9</td>\n",
       "      <td>15.2</td>\n",
       "      <td>3.35</td>\n",
       "      <td>0.58</td>\n",
       "      <td>97.258336</td>\n",
       "      <td>2.742036</td>\n",
       "      <td>2.090873</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2000</td>\n",
       "      <td>3</td>\n",
       "      <td>Texas</td>\n",
       "      <td>TX</td>\n",
       "      <td>TRE</td>\n",
       "      <td>South</td>\n",
       "      <td>-1.1</td>\n",
       "      <td>cold</td>\n",
       "      <td>2000-03-18 00:00:00</td>\n",
       "      <td>16:00:00</td>\n",
       "      <td>...</td>\n",
       "      <td>84.7</td>\n",
       "      <td>9.35</td>\n",
       "      <td>2435.3</td>\n",
       "      <td>1539.9</td>\n",
       "      <td>15.2</td>\n",
       "      <td>3.35</td>\n",
       "      <td>0.58</td>\n",
       "      <td>97.258336</td>\n",
       "      <td>2.742036</td>\n",
       "      <td>2.090873</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1528</th>\n",
       "      <td>2016</td>\n",
       "      <td>7</td>\n",
       "      <td>North Carolina</td>\n",
       "      <td>NC</td>\n",
       "      <td>SERC</td>\n",
       "      <td>Southeast</td>\n",
       "      <td>-0.3</td>\n",
       "      <td>normal</td>\n",
       "      <td>2016-07-07 00:00:00</td>\n",
       "      <td>05:53:00</td>\n",
       "      <td>...</td>\n",
       "      <td>66.09</td>\n",
       "      <td>11.21</td>\n",
       "      <td>1367.2</td>\n",
       "      <td>1043.7</td>\n",
       "      <td>73.5</td>\n",
       "      <td>9.48</td>\n",
       "      <td>2.11</td>\n",
       "      <td>90.336127</td>\n",
       "      <td>9.663873</td>\n",
       "      <td>7.52894</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1529</th>\n",
       "      <td>2016</td>\n",
       "      <td>2</td>\n",
       "      <td>New York</td>\n",
       "      <td>NY</td>\n",
       "      <td>NPCC</td>\n",
       "      <td>Northeast</td>\n",
       "      <td>2</td>\n",
       "      <td>warm</td>\n",
       "      <td>2016-02-07 00:00:00</td>\n",
       "      <td>11:30:00</td>\n",
       "      <td>...</td>\n",
       "      <td>87.87</td>\n",
       "      <td>5.21</td>\n",
       "      <td>4161.4</td>\n",
       "      <td>1700</td>\n",
       "      <td>54.6</td>\n",
       "      <td>8.68</td>\n",
       "      <td>1.26</td>\n",
       "      <td>86.38255</td>\n",
       "      <td>13.61745</td>\n",
       "      <td>3.645862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1530</th>\n",
       "      <td>2016</td>\n",
       "      <td>5</td>\n",
       "      <td>New York</td>\n",
       "      <td>NY</td>\n",
       "      <td>NPCC</td>\n",
       "      <td>Northeast</td>\n",
       "      <td>0.6</td>\n",
       "      <td>warm</td>\n",
       "      <td>2016-05-31 00:00:00</td>\n",
       "      <td>07:30:00</td>\n",
       "      <td>...</td>\n",
       "      <td>87.87</td>\n",
       "      <td>5.21</td>\n",
       "      <td>4161.4</td>\n",
       "      <td>1700</td>\n",
       "      <td>54.6</td>\n",
       "      <td>8.68</td>\n",
       "      <td>1.26</td>\n",
       "      <td>86.38255</td>\n",
       "      <td>13.61745</td>\n",
       "      <td>3.645862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1531</th>\n",
       "      <td>2016</td>\n",
       "      <td>5</td>\n",
       "      <td>Missouri</td>\n",
       "      <td>MO</td>\n",
       "      <td>SERC</td>\n",
       "      <td>Central</td>\n",
       "      <td>0.6</td>\n",
       "      <td>warm</td>\n",
       "      <td>2016-05-24 00:00:00</td>\n",
       "      <td>08:00:00</td>\n",
       "      <td>...</td>\n",
       "      <td>70.44</td>\n",
       "      <td>13.83</td>\n",
       "      <td>2053.5</td>\n",
       "      <td>1510.1</td>\n",
       "      <td>26.6</td>\n",
       "      <td>2.99</td>\n",
       "      <td>0.8</td>\n",
       "      <td>98.615634</td>\n",
       "      <td>1.384366</td>\n",
       "      <td>1.384366</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1532</th>\n",
       "      <td>2016</td>\n",
       "      <td>3</td>\n",
       "      <td>Washington</td>\n",
       "      <td>WA</td>\n",
       "      <td>WECC</td>\n",
       "      <td>Northwest</td>\n",
       "      <td>1.6</td>\n",
       "      <td>warm</td>\n",
       "      <td>2016-03-01 00:00:00</td>\n",
       "      <td>15:00:00</td>\n",
       "      <td>...</td>\n",
       "      <td>84.05</td>\n",
       "      <td>9.08</td>\n",
       "      <td>2380</td>\n",
       "      <td>1487.9</td>\n",
       "      <td>16.7</td>\n",
       "      <td>3.57</td>\n",
       "      <td>0.62</td>\n",
       "      <td>93.208786</td>\n",
       "      <td>6.791214</td>\n",
       "      <td>2.405397</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1533 rows × 55 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      YEAR MONTH      U.S._STATE POSTAL.CODE NERC.REGION CLIMATE.REGION   \n",
       "0     2000   NaN  North Carolina          NC        SERC      Southeast  \\\n",
       "1     2000     6      California          CA        WECC           West   \n",
       "2     2000     5           Texas          TX         TRE          South   \n",
       "3     2000     5           Texas          TX         SPP          South   \n",
       "4     2000     3           Texas          TX         TRE          South   \n",
       "...    ...   ...             ...         ...         ...            ...   \n",
       "1528  2016     7  North Carolina          NC        SERC      Southeast   \n",
       "1529  2016     2        New York          NY        NPCC      Northeast   \n",
       "1530  2016     5        New York          NY        NPCC      Northeast   \n",
       "1531  2016     5        Missouri          MO        SERC        Central   \n",
       "1532  2016     3      Washington          WA        WECC      Northwest   \n",
       "\n",
       "     ANOMALY.LEVEL CLIMATE.CATEGORY    OUTAGE.START.DATE OUTAGE.START.TIME   \n",
       "0              NaN              NaN                  NaN               NaN  \\\n",
       "1             -0.7             cold  2000-06-14 00:00:00          13:13:00   \n",
       "2             -0.7             cold  2000-05-02 00:00:00          04:00:00   \n",
       "3             -0.7             cold  2000-05-24 00:00:00          10:15:00   \n",
       "4             -1.1             cold  2000-03-18 00:00:00          16:00:00   \n",
       "...            ...              ...                  ...               ...   \n",
       "1528          -0.3           normal  2016-07-07 00:00:00          05:53:00   \n",
       "1529             2             warm  2016-02-07 00:00:00          11:30:00   \n",
       "1530           0.6             warm  2016-05-31 00:00:00          07:30:00   \n",
       "1531           0.6             warm  2016-05-24 00:00:00          08:00:00   \n",
       "1532           1.6             warm  2016-03-01 00:00:00          15:00:00   \n",
       "\n",
       "      ... POPPCT_URBAN POPPCT_UC POPDEN_URBAN POPDEN_UC POPDEN_RURAL   \n",
       "0     ...        66.09     11.21       1367.2    1043.7         73.5  \\\n",
       "1     ...        94.95      5.22       4303.7    2124.1         12.7   \n",
       "2     ...         84.7      9.35       2435.3    1539.9         15.2   \n",
       "3     ...         84.7      9.35       2435.3    1539.9         15.2   \n",
       "4     ...         84.7      9.35       2435.3    1539.9         15.2   \n",
       "...   ...          ...       ...          ...       ...          ...   \n",
       "1528  ...        66.09     11.21       1367.2    1043.7         73.5   \n",
       "1529  ...        87.87      5.21       4161.4      1700         54.6   \n",
       "1530  ...        87.87      5.21       4161.4      1700         54.6   \n",
       "1531  ...        70.44     13.83       2053.5    1510.1         26.6   \n",
       "1532  ...        84.05      9.08         2380    1487.9         16.7   \n",
       "\n",
       "     AREAPCT_URBAN AREAPCT_UC   PCT_LAND PCT_WATER_TOT PCT_WATER_INLAND  \n",
       "0             9.48       2.11  90.336127      9.663873          7.52894  \n",
       "1             5.28       0.59  95.164177      4.835823         1.730658  \n",
       "2             3.35       0.58  97.258336      2.742036         2.090873  \n",
       "3             3.35       0.58  97.258336      2.742036         2.090873  \n",
       "4             3.35       0.58  97.258336      2.742036         2.090873  \n",
       "...            ...        ...        ...           ...              ...  \n",
       "1528          9.48       2.11  90.336127      9.663873          7.52894  \n",
       "1529          8.68       1.26   86.38255      13.61745         3.645862  \n",
       "1530          8.68       1.26   86.38255      13.61745         3.645862  \n",
       "1531          2.99        0.8  98.615634      1.384366         1.384366  \n",
       "1532          3.57       0.62  93.208786      6.791214         2.405397  \n",
       "\n",
       "[1533 rows x 55 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# EDA Planning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the EDA section, we plan to explore the distribution of power outage start times to identify any potential patterns. In addition, we plan to plot a line chart of the number of outages in each year to observe any trends."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}