stats/ab-test.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Click [here](https://morihosseini.medium.com/unlocking-the-power-of-a-b-testing-65f3f3ee67d1) to access the associated Medium article."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A/B Testing\n",
    "\n",
    "We’ll start by importing the necessary libraries for this tutorial: numpy, pandas, scipy, and matplotlib. If you don’t have these libraries installed, you can install them by running the following commands in your command prompt:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python3 -m venv .venv\n",
    "\n",
    "!source .venv/bin/activate  # for macOS and Linux\n",
    "# !.venv\\Scripts\\activate.bat  # for Windows\n",
    "\n",
    "!pip install -q numpy pandas scipy matplotlib"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once you have the libraries installed, you can import them in your Python script as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from scipy.stats import ttest_ind\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we’ll load the data that we will be using for our A/B test. For the purpose of this tutorial, we’ll be using a simulated dataset that contains information about 10000 website visitors (5000 for the control group and 5000 for the treatment group) and whether they made a purchase or not. We’ll load this dataset using pandas, and then split it into our control and treatment groups:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a simulated dataset with 5000 observations in each of the control\n",
    "# and treatment groups\n",
    "control_size = treatment_size = 5000\n",
    "np.random.seed(777)  # Set the seed for reproducibility\n",
    "data = pd.DataFrame(\n",
    "    {\n",
    "        \"group\": [\"control\"] * control_size + [\"treatment\"] * treatment_size,\n",
    "        \"purchased\": np.random.randint(0, 2, control_size + treatment_size),\n",
    "    }\n",
    ")\n",
    "\n",
    "# Split the data into the control and treatment groups\n",
    "control = data[data[\"group\"] == \"control\"]\n",
    "treatment = data[data[\"group\"] == \"treatment\"]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have our data loaded and split into our groups, we can start analyzing the results of our A/B test. In the next step, we’ll start by calculating the conversion rate for each group.\n",
    "\n",
    "## Performing t-test\n",
    "\n",
    "The conversion rate is the percentage of users who completed the desired action, in this case, making a purchase. We can calculate the conversion rate by dividing the number of purchases by the total number of visitors in each group."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Control conversion rate: 0.5088\n",
      "Treatment conversion rate: 0.4912\n"
     ]
    }
   ],
   "source": [
    "# Calculate the conversion rate for the control group\n",
    "control_conversion_rate = control[\"purchased\"].mean()\n",
    "\n",
    "# Calculate the conversion rate for the treatment group\n",
    "treatment_conversion_rate = treatment[\"purchased\"].mean()\n",
    "\n",
    "print(\"Control conversion rate:\", control_conversion_rate)\n",
    "print(\"Treatment conversion rate:\", treatment_conversion_rate)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use these conversion rates to determine whether the changes made in the treatment group had a positive impact on the desired outcome.\n",
    "\n",
    "Now, we perform a hypothesis test to determine whether the observed difference in conversion rates between the control and treatment groups is statistically significant.\n",
    "\n",
    "The null hypothesis for our test is that there is no difference in conversion rates between the control and treatment groups. The alternative hypothesis is that there is a difference in conversion rates between the control and treatment groups.\n",
    "\n",
    "We’ll use a two-sample t-test to compare the means of the control and treatment groups. The t-test will calculate a p-value, which represents the probability of observing the difference in conversion rates between the two groups if the null hypothesis is true. If the p-value is less than our significance level (usually 0.05), we can reject the null hypothesis and conclude that the difference in conversion rates is statistically significant."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t-statistic: 1.7600966160767242\n",
      "p-value: 0.07842200109645418\n"
     ]
    }
   ],
   "source": [
    "# Perform a two-sample t-test\n",
    "t_stat, p_val = ttest_ind(control[\"purchased\"], treatment[\"purchased\"])\n",
    "\n",
    "print(\"t-statistic:\", t_stat)\n",
    "print(\"p-value:\", p_val)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next step, we’ll visualize the results of our A/B test using a histogram and a confidence interval.\n",
    "\n",
    "## Analyzing the test results\n",
    "\n",
    "We’ll start by plotting histograms, using the matplotlib library, for the control and treatment groups to visualize the distribution of our data.\n",
    "\n",
    "This code bellow will create a histogram for each group and display them on the same plot. The alpha parameter controls the transparency of each histogram, and the label parameter adds a legend to the plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGwCAYAAAC3qV8qAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAtI0lEQVR4nO3de1hU9aL/8c+ADshdTG4nFMxMMUSDoikzTRLN3PXkOeUlL0V6NKxDpJZnd9RsF+WltCTduxtWdrJOZW0tETF1a3gJo9TUUvHgPgpWBiOkgMDvj36u3Wy1wMDhi+/X88zzMGt9Z63vgifn3Zo1M7a6uro6AQAAGMTD3RMAAABoKAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMZp5e4JNJXa2lodPnxY/v7+stls7p4OAACoh7q6Oh0/flwRERHy8Dj3eZYWGzCHDx9WZGSku6cBAADOw6FDh3TppZeec32LDRh/f39JP/8CAgIC3DwbAABQH06nU5GRkdbz+Lm02IA5/bJRQEAAAQMAgGF+6/IPLuIFAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGCcVu6egJE+zXD3DBqu3zR3zwAAcA7P5Xzj7ik02EM3d3Hr/jkDAwAAjNOggMnIyNDVV18tf39/hYSE6Pbbb9fevXtdxvTt21c2m83lNmHCBJcxRUVFGjx4sHx8fBQSEqIpU6bo1KlTLmPWrVunq666Sl5eXurcubOysrLO7wgBAECL06CAWb9+vVJTU7V582bl5OSourpaAwYMUEVFhcu4cePG6ciRI9Zt9uzZ1rqamhoNHjxYVVVV+uyzz7RkyRJlZWVp+vTp1pjCwkINHjxY/fr1U0FBgdLS0nTfffcpOzv7dx4uAABoCRp0DcyqVatc7mdlZSkkJET5+fnq06ePtdzHx0dhYWFn3cbq1av19ddfa82aNQoNDVXPnj31xBNP6JFHHtHMmTNlt9u1ePFiRUdHa968eZKkbt26aePGjXruueeUnJzc0GMEAAAtzO+6BqasrEySFBwc7LJ86dKluuSSS3TllVdq2rRp+umnn6x1eXl5io2NVWhoqLUsOTlZTqdTu3btssYkJSW5bDM5OVl5eXnnnEtlZaWcTqfLDQAAtEzn/S6k2tpapaWl6frrr9eVV15pLR8xYoQ6duyoiIgIffXVV3rkkUe0d+9evf/++5Kk4uJil3iRZN0vLi7+1TFOp1MnTpxQmzZtzphPRkaGHn/88fM9HAAAYJDzDpjU1FTt3LlTGzdudFk+fvx46+fY2FiFh4erf//+2r9/vy677LLzn+lvmDZtmtLT0637TqdTkZGRTbY/AADgPuf1EtKkSZO0YsUKffrpp7r00kt/dWxiYqIkad++fZKksLAwlZSUuIw5ff/0dTPnGhMQEHDWsy+S5OXlpYCAAJcbAABomRoUMHV1dZo0aZI++OADrV27VtHR0b/5mIKCAklSeHi4JMnhcGjHjh06evSoNSYnJ0cBAQGKiYmxxuTm5rpsJycnRw6HoyHTBQAALVSDAiY1NVVvvvmm3nrrLfn7+6u4uFjFxcU6ceKEJGn//v164oknlJ+fr4MHD+qjjz7S6NGj1adPH/Xo0UOSNGDAAMXExGjUqFH68ssvlZ2drccee0ypqany8vKSJE2YMEEHDhzQ1KlTtWfPHr344ot655139NBDDzXy4QMAABM1KGAWLVqksrIy9e3bV+Hh4dZt2bJlkiS73a41a9ZowIAB6tq1qx5++GENHTpUf/3rX61teHp6asWKFfL09JTD4dDdd9+t0aNHa9asWdaY6OhorVy5Ujk5OYqLi9O8efP08ssv8xZqAAAgSbLV1dXVuXsSTcHpdCowMFBlZWWNfj1M3iuTG3V7F4IjZa67pwAAOAeeV/6hvs/ffBcSAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjNOggMnIyNDVV18tf39/hYSE6Pbbb9fevXtdxpw8eVKpqalq166d/Pz8NHToUJWUlLiMKSoq0uDBg+Xj46OQkBBNmTJFp06dchmzbt06XXXVVfLy8lLnzp2VlZV1fkcIAABanAYFzPr165WamqrNmzcrJydH1dXVGjBggCoqKqwxDz30kP7617/q3Xff1fr163X48GHdcccd1vqamhoNHjxYVVVV+uyzz7RkyRJlZWVp+vTp1pjCwkINHjxY/fr1U0FBgdLS0nTfffcpOzu7EQ4ZAACYzlZXV1d3vg/+7rvvFBISovXr16tPnz4qKytT+/bt9dZbb+lf//VfJUl79uxRt27dlJeXp2uvvVaffPKJbr31Vh0+fFihoaGSpMWLF+uRRx7Rd999J7vdrkceeUQrV67Uzp07rX0NGzZMpaWlWrVqVb3m5nQ6FRgYqLKyMgUEBJzvIZ5V3iuTG3V7F4IjZa67pwAAOAeeV/6hvs/fv+samLKyMklScHCwJCk/P1/V1dVKSkqyxnTt2lUdOnRQXl6eJCkvL0+xsbFWvEhScnKynE6ndu3aZY355TZOjzm9jbOprKyU0+l0uQEAgJbpvAOmtrZWaWlpuv7663XllVdKkoqLi2W32xUUFOQyNjQ0VMXFxdaYX8bL6fWn1/3aGKfTqRMnTpx1PhkZGQoMDLRukZGR53toAACgmTvvgElNTdXOnTv19ttvN+Z8ztu0adNUVlZm3Q4dOuTuKQEAgCbS6nweNGnSJK1YsUIbNmzQpZdeai0PCwtTVVWVSktLXc7ClJSUKCwszBqzdetWl+2dfpfSL8f88zuXSkpKFBAQoDZt2px1Tl5eXvLy8jqfwwEAAIZp0BmYuro6TZo0SR988IHWrl2r6Ohol/Xx8fFq3bq1cnNzrWV79+5VUVGRHA6HJMnhcGjHjh06evSoNSYnJ0cBAQGKiYmxxvxyG6fHnN4GAAC4uDXoDExqaqreeustffjhh/L397euWQkMDFSbNm0UGBiolJQUpaenKzg4WAEBAXrggQfkcDh07bXXSpIGDBigmJgYjRo1SrNnz1ZxcbEee+wxpaamWmdQJkyYoIULF2rq1Km69957tXbtWr3zzjtauXJlIx8+AAAwUYPOwCxatEhlZWXq27evwsPDrduyZcusMc8995xuvfVWDR06VH369FFYWJjef/99a72np6dWrFghT09PORwO3X333Ro9erRmzZpljYmOjtbKlSuVk5OjuLg4zZs3Ty+//LKSk5Mb4ZABAIDpGnQGpj4fGePt7a3MzExlZmaec0zHjh318ccf/+p2+vbtqy+++KIh0wMAABcJvgsJAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcRocMBs2bNCQIUMUEREhm82m5cuXu6wfO3asbDaby23gwIEuY44dO6aRI0cqICBAQUFBSklJUXl5ucuYr776SjfccIO8vb0VGRmp2bNnN/zoAABAi9TggKmoqFBcXJwyMzPPOWbgwIE6cuSIdfvv//5vl/UjR47Url27lJOToxUrVmjDhg0aP368td7pdGrAgAHq2LGj8vPzNWfOHM2cOVN/+ctfGjpdAADQArVq6AMGDRqkQYMG/eoYLy8vhYWFnXXd7t27tWrVKm3btk0JCQmSpBdeeEG33HKL5s6dq4iICC1dulRVVVV69dVXZbfb1b17dxUUFOjZZ591CZ1fqqysVGVlpXXf6XQ29NAAAIAhmuQamHXr1ikkJERXXHGFJk6cqB9++MFal5eXp6CgICteJCkpKUkeHh7asmWLNaZPnz6y2+3WmOTkZO3du1c//vjjWfeZkZGhwMBA6xYZGdkUhwYAAJqBRg+YgQMH6vXXX1dubq6eeeYZrV+/XoMGDVJNTY0kqbi4WCEhIS6PadWqlYKDg1VcXGyNCQ0NdRlz+v7pMf9s2rRpKisrs26HDh1q7EMDAADNRINfQvotw4YNs36OjY1Vjx49dNlll2ndunXq379/Y+/O4uXlJS8vrybbPgAAaD6a/G3UnTp10iWXXKJ9+/ZJksLCwnT06FGXMadOndKxY8es62bCwsJUUlLiMub0/XNdWwMAAC4eTR4wf//73/XDDz8oPDxckuRwOFRaWqr8/HxrzNq1a1VbW6vExERrzIYNG1RdXW2NycnJ0RVXXKG2bds29ZQBAEAz1+CAKS8vV0FBgQoKCiRJhYWFKigoUFFRkcrLyzVlyhRt3rxZBw8eVG5urm677TZ17txZycnJkqRu3bpp4MCBGjdunLZu3apNmzZp0qRJGjZsmCIiIiRJI0aMkN1uV0pKinbt2qVly5ZpwYIFSk9Pb7wjBwAAxmpwwHz++efq1auXevXqJUlKT09Xr169NH36dHl6euqrr77SH/7wB3Xp0kUpKSmKj4/X3/72N5frU5YuXaquXbuqf//+uuWWW9S7d2+Xz3gJDAzU6tWrVVhYqPj4eD388MOaPn36Od9CDQAALi4Nvoi3b9++qqurO+f67Ozs39xGcHCw3nrrrV8d06NHD/3tb39r6PQAAMBFgO9CAgAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEaHDAbNmzQkCFDFBERIZvNpuXLl7usr6ur0/Tp0xUeHq42bdooKSlJ3377rcuYY8eOaeTIkQoICFBQUJBSUlJUXl7uMuarr77SDTfcIG9vb0VGRmr27NkNPzoAANAiNThgKioqFBcXp8zMzLOunz17tp5//nktXrxYW7Zska+vr5KTk3Xy5ElrzMiRI7Vr1y7l5ORoxYoV2rBhg8aPH2+tdzqdGjBggDp27Kj8/HzNmTNHM2fO1F/+8pfzOEQAANDStGroAwYNGqRBgwaddV1dXZ3mz5+vxx57TLfddpsk6fXXX1doaKiWL1+uYcOGaffu3Vq1apW2bdumhIQESdILL7ygW265RXPnzlVERISWLl2qqqoqvfrqq7Lb7erevbsKCgr07LPPuoTOL1VWVqqystK673Q6G3poAADAEI16DUxhYaGKi4uVlJRkLQsMDFRiYqLy8vIkSXl5eQoKCrLiRZKSkpLk4eGhLVu2WGP69Okju91ujUlOTtbevXv1448/nnXfGRkZCgwMtG6RkZGNeWgAAKAZadSAKS4uliSFhoa6LA8NDbXWFRcXKyQkxGV9q1atFBwc7DLmbNv45T7+2bRp01RWVmbdDh069PsPCAAANEsNfgmpufLy8pKXl5e7pwEAAC6ARj0DExYWJkkqKSlxWV5SUmKtCwsL09GjR13Wnzp1SseOHXMZc7Zt/HIfAADg4tWoARMdHa2wsDDl5uZay5xOp7Zs2SKHwyFJcjgcKi0tVX5+vjVm7dq1qq2tVWJiojVmw4YNqq6utsbk5OToiiuuUNu2bRtzygAAwEANDpjy8nIVFBSooKBA0s8X7hYUFKioqEg2m01paWn605/+pI8++kg7duzQ6NGjFRERodtvv12S1K1bNw0cOFDjxo3T1q1btWnTJk2aNEnDhg1TRESEJGnEiBGy2+1KSUnRrl27tGzZMi1YsEDp6emNduAAAMBcDb4G5vPPP1e/fv2s+6ejYsyYMcrKytLUqVNVUVGh8ePHq7S0VL1799aqVavk7e1tPWbp0qWaNGmS+vfvLw8PDw0dOlTPP/+8tT4wMFCrV69Wamqq4uPjdckll2j69OnnfAs1AAC4uNjq6urq3D2JpuB0OhUYGKiysjIFBAQ06rbzXpncqNu7EBwpc909BQDAOfC88g/1ff7mu5AAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgnEYPmJkzZ8pms7ncunbtaq0/efKkUlNT1a5dO/n5+Wno0KEqKSlx2UZRUZEGDx4sHx8fhYSEaMqUKTp16lRjTxUAABiqVVNstHv37lqzZs0/dtLqH7t56KGHtHLlSr377rsKDAzUpEmTdMcdd2jTpk2SpJqaGg0ePFhhYWH67LPPdOTIEY0ePVqtW7fWU0891RTTBQAAhmmSgGnVqpXCwsLOWF5WVqZXXnlFb731lm666SZJ0muvvaZu3bpp8+bNuvbaa7V69Wp9/fXXWrNmjUJDQ9WzZ0898cQTeuSRRzRz5kzZ7fammDIAADBIk1wD8+233yoiIkKdOnXSyJEjVVRUJEnKz89XdXW1kpKSrLFdu3ZVhw4dlJeXJ0nKy8tTbGysQkNDrTHJyclyOp3atWvXOfdZWVkpp9PpcgMAAC1TowdMYmKisrKytGrVKi1atEiFhYW64YYbdPz4cRUXF8tutysoKMjlMaGhoSouLpYkFRcXu8TL6fWn151LRkaGAgMDrVtkZGTjHhgAAGg2Gv0lpEGDBlk/9+jRQ4mJierYsaPeeecdtWnTprF3Z5k2bZrS09Ot+06nk4gBAKCFavK3UQcFBalLly7at2+fwsLCVFVVpdLSUpcxJSUl1jUzYWFhZ7wr6fT9s11Xc5qXl5cCAgJcbgAAoGVq8oApLy/X/v37FR4ervj4eLVu3Vq5ubnW+r1796qoqEgOh0OS5HA4tGPHDh09etQak5OTo4CAAMXExDT1dAEAgAEa/SWkyZMna8iQIerYsaMOHz6sGTNmyNPTU8OHD1dgYKBSUlKUnp6u4OBgBQQE6IEHHpDD4dC1114rSRowYIBiYmI0atQozZ49W8XFxXrssceUmpoqLy+vxp4uAAAwUKMHzN///ncNHz5cP/zwg9q3b6/evXtr8+bNat++vSTpueeek4eHh4YOHarKykolJyfrxRdftB7v6empFStWaOLEiXI4HPL19dWYMWM0a9asxp4qAAAwVKMHzNtvv/2r6729vZWZmanMzMxzjunYsaM+/vjjxp4aAABoIfguJAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxGv2D7AC4T01Njaqrq909DTQyu90uDw/+fxP4JQIGaAHq6upUXFx8xje9o2Xw8PBQdHS07Ha7u6cCNBsEDNACnI6XkJAQ+fj4yGazuXtKaCS1tbU6fPiwjhw5og4dOvC3Bf4/AgYwXE1NjRUv7dq1c/d00ATat2+vw4cP69SpU2rdurW7pwM0C7yoChju9DUvPj4+bp4Jmsrpl45qamrcPBOg+SBggBaClxZaLv62wJkIGAAAYBwCBgAaaN26dbLZbLzrC3AjLuIFWrDncr65oPt76OYu5/W44uJiPfnkk1q5cqX+7//+TyEhIerZs6fS0tLUv3//Rplb37591bNnT82fP79RtgfAvQgYAG518OBBXX/99QoKCtKcOXMUGxur6upqZWdnKzU1VXv27Llgc6mrq1NNTY1ateKfRqC54yUkAG51//33y2azaevWrRo6dKi6dOmi7t27Kz09XZs3b5YkFRUV6bbbbpOfn58CAgJ05513qqSkxNrGzJkz1bNnT73xxhuKiopSYGCghg0bpuPHj0uSxo4dq/Xr12vBggWy2Wyy2Ww6ePCg9VLQJ598ovj4eHl5eWnjxo2qrKzUgw8+qJCQEHl7e6t3797atm2bW34/AM6OgAHgNseOHdOqVauUmpoqX1/fM9YHBQWptrZWt912m44dO6b169crJydHBw4c0F133eUydv/+/Vq+fLlWrFihFStWaP369Xr66aclSQsWLJDD4dC4ceN05MgRHTlyRJGRkdZjH330UT399NPavXu3evTooalTp+q9997TkiVLtH37dnXu3FnJyck6duxY0/5CANQb50kBuM2+fftUV1enrl27nnNMbm6uduzYocLCQis6Xn/9dXXv3l3btm3T1VdfLennT6zNysqSv7+/JGnUqFHKzc3Vk08+qcDAQNntdvn4+CgsLOyMfcyaNUs333yzJKmiokKLFi1SVlaWBg0aJEl66aWXlJOTo1deeUVTpkxp1N8BgPPDGRgAblNXV/ebY3bv3q3IyEiXMyYxMTEKCgrS7t27rWVRUVFWvEhSeHi4jh49Wq95JCQkWD/v379f1dXVuv76661lrVu31jXXXOOyPwDuRcAAcJvLL79cNputUS7U/eeP2LfZbKqtra3XY8/28hWA5o2AAeA2wcHBSk5OVmZmpioqKs5YX1paqm7duunQoUM6dOiQtfzrr79WaWmpYmJi6r0vu91er4/iv+yyy2S327Vp0yZrWXV1tbZt29ag/QFoWgQMALfKzMxUTU2NrrnmGr333nv69ttvtXv3bj3//PNyOBxKSkpSbGysRo4cqe3bt2vr1q0aPXq0brzxRpeXfn5LVFSUtmzZooMHD+r7778/59kZX19fTZw4UVOmTNGqVav09ddfa9y4cfrpp5+UkpLSWIcN4HfiIl6gBTvfD5a7kDp16qTt27frySef1MMPP6wjR46offv2io+P16JFi2Sz2fThhx/qgQceUJ8+feTh4aGBAwfqhRdeaNB+Jk+erDFjxigmJkYnTpxQYWHhOcc+/fTTqq2t1ahRo3T8+HElJCQoOztbbdu2/b2HC6CR2OrqcxWdgZxOpwIDA1VWVqaAgIBG3XbeK5MbdXsXgiNlrrungCZy8uRJFRYWKjo6Wt7e3u6eDpoAf+OWj+eVf6jv8zcvIQEAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4fBcS0JJ9mnFh99dvWr2H2my2X10/Y8YMzZw583dO6Exjx45VaWmpli9f3ujbPh9ZWVlKS0tTaWmpu6cCGIWAAeAWR44csX5etmyZpk+frr1791rL/Pz8rJ/r6upUU1OjVq34JwvAz3gJCYBbhIWFWbfAwEDZbDbr/p49e+Tv769PPvlE8fHx8vLy0saNG1VbW6uMjAxFR0erTZs2iouL0//8z/9Y26ypqVFKSoq1/oorrtCCBQus9TNnztSSJUv04YcfymazyWazad26dTp48KBsNpveeecd3XDDDWrTpo2uvvpqffPNN9q2bZsSEhLk5+enQYMG6bvvvnM5jpdfflndunWTt7e3unbtqhdffNFad3q777//vvr16ycfHx/FxcUpLy9PkrRu3Trdc889Kisrs+bTFGedgJaI/50B0Gw9+uijmjt3rjp16qS2bdsqIyNDb775phYvXqzLL79cGzZs0N1336327dvrxhtvVG1trS699FK9++67ateunT777DONHz9e4eHhuvPOOzV58mTt3r1bTqdTr732miQpODhYhw8flvTzy1bz589Xhw4ddO+992rEiBHy9/fXggUL5OPjozvvvFPTp0/XokWLJElLly7V9OnTtXDhQvXq1UtffPGFxo0bJ19fX40ZM8Y6jj/+8Y+aO3euLr/8cv3xj3/U8OHDtW/fPl133XWaP3++y9mnX555AnBuBAyAZmvWrFm6+eabJUmVlZV66qmntGbNGjkcDklSp06dtHHjRv35z3/WjTfeqNatW+vxxx+3Hh8dHa28vDy98847uvPOO+Xn56c2bdqosrJSYWFhZ+xv8uTJSk5OliT9x3/8h4YPH67c3Fxdf/31kqSUlBRlZWVZ42fMmKF58+bpjjvusPb39ddf689//rNLwEyePFmDBw+WJD3++OPq3r279u3bp65du7qcfQJQfwQMgGYrISHB+nnfvn366aefrKA5raqqSr169bLuZ2Zm6tVXX1VRUZFOnDihqqoq9ezZs17769Gjh/VzaGioJCk2NtZl2dGjRyVJFRUV2r9/v1JSUjRu3DhrzKlTpxQYGHjO7YaHh0uSjh49qq5du9ZrXgDORMAAaLZ8fX2tn8vLyyVJK1eu1L/8y7+4jPPy8pIkvf3225o8ebLmzZsnh8Mhf39/zZkzR1u2bKnX/lq3bm39fPpdUv+8rLa21mU+L730khITE1224+np+ZvbPb0dAOeHgAFghJiYGHl5eamoqEg33njjWcds2rRJ1113ne6//35r2f79+13G2O121dTU/O75hIaGKiIiQgcOHNDIkSPPezuNNR/gYkPAADCCv7+/Jk+erIceeki1tbXq3bu3ysrKtGnTJgUEBGjMmDG6/PLL9frrrys7O1vR0dF64403tG3bNkVHR1vbiYqKUnZ2tvbu3at27dqd8XJPQzz++ON68MEHFRgYqIEDB6qyslKff/65fvzxR6Wnp9drG1FRUSovL1dubq7i4uLk4+MjHx+f854TcLEgYICWrAEfLGeCJ554Qu3bt1dGRoYOHDigoKAgXXXVVfrP//xPSdK///u/64svvtBdd90lm82m4cOH6/7779cnn3xibWPcuHFat26dEhISVF5erk8//VRRUVHnNZ/77rtPPj4+mjNnjqZMmSJfX1/FxsYqLS2t3tu47rrrNGHCBN1111364YcfmuwD/ICWxlZXV1fn7kk0BafTqcDAQJWVlSkgIKBRt533yuRG3d6F4EiZ6+4poImcPHlShYWFio6Olre3t7ungybA37jl43nlH+r7/M0H2QEAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAC1EC70eH+JvC5wNAQMY7vSnvP70009ungmaSlVVlaQzP+EXuJjxOTCA4Tw9PRUUFGR9R4+Pj4/1cfUwX21trb777jv5+PioVSv+yQZO478GoAU4/U3GpyMGLYuHh4c6dOhAmAK/QMAALYDNZlN4eLhCQkJUXV3t7umgkdntdnl48Io/8EsEDNCCeHp6cp0EgItCs076zMxMRUVFydvbW4mJidq6dau7pwQAAJqBZhswy5YtU3p6umbMmKHt27crLi5OycnJvMYPAACab8A8++yzGjdunO655x7FxMRo8eLF8vHx0auvvuruqQEAADdrltfAVFVVKT8/X9OmTbOWeXh4KCkpSXl5eWd9TGVlpSorK637ZWVlkn7+VsvGVnGi8rcHNTNN8XsAADQOnlfO3O5vfYBjswyY77//XjU1NQoNDXVZHhoaqj179pz1MRkZGXr88cfPWB4ZGdkkczTOAwvdPQMAQEvSxM8rx48fV2Bg4DnXN8uAOR/Tpk1Tenq6db+2tlbHjh1Tu3btGvWzE5xOpyIjI3Xo0CEFBAQ02nYBADBFUz4X1tXV6fjx44qIiPjVcc0yYC655BJ5enqqpKTEZXlJSYn1gV3/zMvLS15eXi7LgoKCmmqKCggIIGAAABe1pnou/LUzL6c1y4t47Xa74uPjlZubay2rra1Vbm6uHA6HG2cGAACag2Z5BkaS0tPTNWbMGCUkJOiaa67R/PnzVVFRoXvuucfdUwMAAG7WbAPmrrvu0nfffafp06eruLhYPXv21KpVq864sPdC8/Ly0owZM854uQoAgItFc3gutNX91vuUAAAAmplmeQ0MAADAryFgAACAcQgYAABgHAIGAAAYh4BpoMzMTEVFRcnb21uJiYnaunWru6cEAMAFsWHDBg0ZMkQRERGy2Wxavny52+ZCwDTAsmXLlJ6erhkzZmj79u2Ki4tTcnKyjh496u6pAQDQ5CoqKhQXF6fMzEx3T4W3UTdEYmKirr76ai1c+PMXWNXW1ioyMlIPPPCAHn30UTfPDgCAC8dms+mDDz7Q7bff7pb9cwamnqqqqpSfn6+kpCRrmYeHh5KSkpSXl+fGmQEAcPEhYOrp+++/V01NzRmfBBwaGqri4mI3zQoAgIsTAQMAAIxDwNTTJZdcIk9PT5WUlLgsLykpUVhYmJtmBQDAxYmAqSe73a74+Hjl5uZay2pra5WbmyuHw+HGmQEAcPFptt9G3Rylp6drzJgxSkhI0DXXXKP58+eroqJC99xzj7unBgBAkysvL9e+ffus+4WFhSooKFBwcLA6dOhwQefC26gbaOHChZozZ46Ki4vVs2dPPf/880pMTHT3tAAAaHLr1q1Tv379zlg+ZswYZWVlXdC5EDAAAMA4XAMDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BA6BZ69u3r9LS0tw9jTM013kBFwsCBkC9jR07VjabTTabTXa7XZ07d9asWbN06tQpd08NwEWGL3ME0CADBw7Ua6+9psrKSn388cdKTU1V69atNW3atAZvq6qqSna7vQlmCaCl4wwMgAbx8vJSWFiYOnbsqIkTJyopKUkfffTRWV9Suf322zV27FjrflRUlJ544gmNHj1aAQEBGj9+vCRp06ZN6tu3r3x8fNS2bVslJyfrxx9/tB5XW1urqVOnKjg4WGFhYZo5c6bLfp599lnFxsbK19dXkZGRuv/++1VeXm6t/9///V8NGTJEbdu2la+vr7p3766PP/7YWr9z504NGjRIfn5+Cg0N1ahRo/T9999b6ysqKjR69Gj5+fkpPDxc8+bNa4TfJIDfg4AB8Lu0adNGVVVV9R4/d+5cxcXF6YsvvtB//dd/qaCgQP3791dMTIzy8vK0ceNGDRkyRDU1NdZjlixZIl9fX23ZskWzZ8/WrFmzlJOTY6338PDQ888/r127dmnJkiVau3atpk6daq1PTU1VZWWlNmzYoB07duiZZ56Rn5+fJKm0tFQ33XSTevXqpc8//1yrVq1SSUmJ7rzzTuvxU6ZM0fr16/Xhhx9q9erVWrdunbZv3/57fm0AfideQgJwXurq6pSbm6vs7Gw98MAD2rZtW70ed9NNN+nhhx+27o8YMUIJCQl68cUXrWXdu3d3eUyPHj00Y8YMSdLll1+uhQsXKjc3VzfffLMkuZz5iYqK0p/+9CdNmDDB2mZRUZGGDh2q2NhYSVKnTp2s8QsXLlSvXr301FNPWcteffVVRUZG6ptvvlFERIReeeUVvfnmm+rfv7+kn4Pq0ksvrdfxAmgaBAyABlmxYoX8/PxUXV2t2tpajRgxQjNnztTgwYPr9fiEhASX+wUFBfq3f/u3X31Mjx49XO6Hh4fr6NGj1v01a9YoIyNDe/bskdPp1KlTp3Ty5En99NNP8vHx0YMPPqiJEydq9erVSkpK0tChQ61tfvnll/r000+tMzK/tH//fp04cUJVVVVKTEy0lgcHB+uKK66o1/ECaBq8hASgQfr166eCggJ9++23OnHihPXyjoeHh+rq6lzGVldXn/F4X19fl/tt2rT5zX22bt3a5b7NZlNtba0k6eDBg7r11lvVo0cPvffee8rPz1dmZqYkWS9t3XfffTpw4IBGjRqlHTt2KCEhQS+88IIkqby8XEOGDFFBQYHL7dtvv1WfPn3q+VsBcKERMAAaxNfXV507d1aHDh3UqtU/TuK2b99eR44cse7X1NRo586dv7m9Hj16KDc397znk5+fr9raWs2bN0/XXnutunTposOHD58xLjIyUhMmTND777+vhx9+WC+99JIk6aqrrtKuXbsUFRWlzp07u9x8fX112WWXqXXr1tqyZYu1rR9//FHffPPNec8ZwO9HwABoFDfddJNWrlyplStXas+ePZo4caJKS0t/83HTpk3Ttm3bdP/99+urr77Snj17tGjRIpd3Af2azp07q7q6Wi+88IIOHDigN954Q4sXL3YZk5aWpuzsbBUWFmr79u369NNP1a1bN0k/X+B77NgxDR8+XNu2bdP+/fuVnZ2te+65RzU1NfLz81NKSoqmTJmitWvXaufOnRo7dqw8PPjnE3An/gsE0CjuvfdejRkzRqNHj9aNN96oTp06qV+/fr/5uC5dumj16tX68ssvdc0118jhcOjDDz90Obvza+Li4vTss8/qmWee0ZVXXqmlS5cqIyPDZUxNTY1SU1PVrVs3DRw4UF26dLEu8I2IiNCmTZtUU1OjAQMGKDY2VmlpaQoKCrIiZc6cObrhhhs0ZMgQJSUlqXfv3oqPj2/gbwhAY7LV/fOL1gAAAM0cZ2AAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAY5/8BuxnJHebn8RoAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot histograms for the control and treatment groups\n",
    "plt.hist(control[\"purchased\"], alpha=0.5, label=\"Control\")\n",
    "plt.hist(treatment[\"purchased\"], alpha=0.5, label=\"Treatment\")\n",
    "plt.legend()\n",
    "plt.xlabel(\"Purchased\")\n",
    "plt.xticks([0, 1])\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we’ll plot a confidence interval for the difference in means between the control and treatment groups. The confidence interval will give us a range of values that the true difference in means is likely to fall within."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAGKCAYAAADkAf55AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAApQElEQVR4nO3df3DU9YH/8VfWkAWKu9tgkk3sIjDtmMTWUhcIS7UKiSYDd6fDOmO89E40EvUSeiY5NVwV77xz6NcfjaJHMxaVMsAcpThX+XHRQBRGXYFbLnUIJL326IEJG7jG7Ioxycbs9w+/fOp+8wPSsobk/XzMfMbh83m/P/v++E+e+ex+NkmxWCwmAAAAg9jGegEAAABfNgIIAAAYhwACAADGIYAAAIBxCCAAAGAcAggAABiHAAIAAMYhgAAAgHGSx3oBl6qBgQG1t7fr8ssvV1JS0lgvBwAAXIBYLKaPP/5YWVlZstmGv89DAA2jvb1dHo9nrJcBAAD+CCdPntTXvva1YY8TQMO4/PLLJX3+P9DhcIzxagAAwIWIRCLyeDzWz/HhEEDDOPe2l8PhIIAAABhnzvfxFT4EDQAAjEMAAQAA4xBAAADAOAQQAAAwDgEEAACMQwABAADjEEAAAMA4BBAAADAOAQQAAIxDAAEAAOMQQAAAwDgEEAAAMA4BBAAAjEMAATBKd1+/Ztbs0syaXeru6x/r5QAYIwQQAAAwDgEEAACMQwABAADjEEAAAMA4BBAAADAOAQQAAIxDAAEAAOMQQAAAwDgEEAAAMA4BBAAAjEMAAQAA4xBAAADAOF9KAP3Lv/yLZs6cqcmTJysvL08HDx4ccfy2bduUnZ2tyZMn61vf+pZ2794ddzwWi2n16tXKzMzUlClTVFBQoP/6r/+KG9PZ2amSkhI5HA65XC6Vlpbq7NmzF/3aAADA+JPwANq6dauqqqr0+OOP6/Dhw/r2t7+twsJCnT59esjx7733nu68806VlpbqP//zP3Xbbbfptttu05EjR6wxTz31lNauXau6ujodOHBAX/nKV1RYWKienh5rTElJiZqbm9XQ0KCdO3dq//79KisrS/TlAgCAcSApFovFEvkCeXl5mjdvnl588UVJ0sDAgDwej1auXKmamppB4++44w598skn2rlzp7VvwYIFmjNnjurq6hSLxZSVlaXq6mr93d/9nSQpHA4rIyNDGzZsUHFxsY4dO6bc3FwdOnRIc+fOlSTV19dryZIl+vDDD5WVlXXedUciETmdToXDYTkcjovxvwLAJaC7r1+5q9+QJB19olBTU5LHeEUALqYL/fmd0DtAfX19CgaDKigo+MML2mwqKChQIBAYck4gEIgbL0mFhYXW+OPHjysUCsWNcTqdysvLs8YEAgG5XC4rfiSpoKBANptNBw4cGPJ1e3t7FYlE4jYAADAxJTSA/vd//1efffaZMjIy4vZnZGQoFAoNOScUCo04/tx/zzcmPT097nhycrJSU1OHfd01a9bI6XRam8fjucCrBAAA4w1Pgf0/q1atUjgctraTJ0+O9ZIAAECCJDSArrjiCl122WXq6OiI29/R0SG32z3kHLfbPeL4c/8935j//0PW/f396uzsHPZ17Xa7HA5H3AYAACamhAZQSkqKvF6v9u7da+0bGBjQ3r175fP5hpzj8/nixktSQ0ODNX7WrFlyu91xYyKRiA4cOGCN8fl86urqUjAYtMY0NjZqYGBAeXl5F+36AADA+JTwxx+qqqp01113ae7cuZo/f76ee+45ffLJJ7r77rslSX/913+tK6+8UmvWrJEk/e3f/q1uvPFGPfvss1q6dKn+9V//Vf/xH/+hl156SZKUlJSkBx98UP/8z/+sb3zjG5o1a5Yee+wxZWVl6bbbbpMk5eTkqKioSCtWrFBdXZ2i0agqKipUXFx8QU+AAQCAiS3hAXTHHXfozJkzWr16tUKhkObMmaP6+nrrQ8wnTpyQzfaHG1ELFy7Uli1b9Oijj+rv//7v9Y1vfEP/9m//pm9+85vWmIcffliffPKJysrK1NXVpeuvv1719fWaPHmyNWbz5s2qqKhQfn6+bDab/H6/1q5dm+jLBQAA40DCvwdovOJ7gICJie8BAia2S+J7gAAAAC5FBBAAADAOAQQAAIxDAAEAAOMQQAAAwDgEEAAAMA4BBAAAjEMAAQAA4xBAAADAOAQQAAAwDgEEAACMQwABAADjEEAAAMA4BBAAADAOAQQAAIxDAAEAAOMQQAAAwDgEEAAAMA4BBAAAjEMAAQAA4xBAAADAOAQQAAAwDgEEAACMQwABAADjEEAAAMA4BBAAADAOAQQAAIxDAAEAAOMQQAAAwDgEEAAAMA4BBAAAjEMAAQAA4yQ0gDo7O1VSUiKHwyGXy6XS0lKdPXt2xDk9PT0qLy/X9OnTNW3aNPn9fnV0dMSNOXHihJYuXaqpU6cqPT1dDz30kPr7+63jb7/9tpKSkgZtoVAoIdcJAADGl4QGUElJiZqbm9XQ0KCdO3dq//79KisrG3FOZWWlduzYoW3btmnfvn1qb2/XsmXLrOOfffaZli5dqr6+Pr333nv62c9+pg0bNmj16tWDztXa2qpTp05ZW3p6+kW/RgAAMA7FEuTo0aMxSbFDhw5Z+/793/89lpSUFGtraxtyTldXV2zSpEmxbdu2WfuOHTsWkxQLBAKxWCwW2717d8xms8VCoZA15ic/+UnM4XDEent7Y7FYLPbWW2/FJMU++uijP3r94XA4JikWDof/6HMAuPR80huNXfXIzthVj+yMfdIbHevlALjILvTnd8LuAAUCAblcLs2dO9faV1BQIJvNpgMHDgw5JxgMKhqNqqCgwNqXnZ2tGTNmKBAIWOf91re+pYyMDGtMYWGhIpGImpub4843Z84cZWZm6uabb9a777474np7e3sViUTiNgAAMDElLIBCodCgt5ySk5OVmpo67GdxQqGQUlJS5HK54vZnZGRYc0KhUFz8nDt+7pgkZWZmqq6uTtu3b9f27dvl8Xh000036fDhw8Oud82aNXI6ndbm8XhGdb0AAGD8GHUA1dTUDPkB4y9uLS0tiVjrBbv66qt13333yev1auHChXrllVe0cOFC1dbWDjtn1apVCofD1nby5MkvccUAAODLlDzaCdXV1Vq+fPmIY2bPni23263Tp0/H7e/v71dnZ6fcbveQ89xut/r6+tTV1RV3F6ijo8Oa43a7dfDgwbh5554SG+68kjR//ny98847wx632+2y2+0jXhcAAJgYRh1AaWlpSktLO+84n8+nrq4uBYNBeb1eSVJjY6MGBgaUl5c35Byv16tJkyZp79698vv9kj5/kuvEiRPy+XzWeZ988kmdPn3aeoutoaFBDodDubm5w66nqalJmZmZo7pWAAAwMY06gC5UTk6OioqKtGLFCtXV1SkajaqiokLFxcXKysqSJLW1tSk/P18bN27U/Pnz5XQ6VVpaqqqqKqWmpsrhcGjlypXy+XxasGCBJOmWW25Rbm6u/uqv/kpPPfWUQqGQHn30UZWXl1t3cJ577jnNmjVL11xzjXp6erR+/Xo1NjbqzTffTNTlAgCAcSRhASRJmzdvVkVFhfLz82Wz2eT3+7V27VrreDQaVWtrq7q7u619tbW11tje3l4VFhZq3bp11vHLLrtMO3fu1AMPPCCfz6evfOUruuuuu/TEE09YY/r6+lRdXa22tjZNnTpV1157rfbs2aNFixYl8nIBAMA4kRSLxWJjvYhLUSQSkdPpVDgclsPhGOvlALhIuvv6lbv6DUnS0ScKNTUlob8HAviSXejPb/4WGAAAMA4BBAAAjEMAAQAA4xBAAADAOAQQAAAwDgEEAACMQwABAADjEEAAAMA4BBAAADAOAQQAAIxDAAEAAOMQQAAAwDgEEAAAMA4BBAAAjEMAAQAA4xBAAADAOAQQAAAwDgEEAACMQwABAADjEEAAAMA4BBAAADAOAQQAAIxDAAEAAOMQQAAAwDgEEAAAMA4BBAAAjEMAAQAA4xBAAADAOAQQAAAwDgEEAACMQwABAADjEEAAAMA4CQugzs5OlZSUyOFwyOVyqbS0VGfPnh1xTk9Pj8rLyzV9+nRNmzZNfr9fHR0dcWN+8IMfyOv1ym63a86cOUOe54MPPtANN9ygyZMny+Px6KmnnrpYlwUAACaAhAVQSUmJmpub1dDQoJ07d2r//v0qKysbcU5lZaV27Nihbdu2ad++fWpvb9eyZcsGjbvnnnt0xx13DHmOSCSiW265RVdddZWCwaCefvpp/cM//INeeumli3JdAABg/EtOxEmPHTum+vp6HTp0SHPnzpUkvfDCC1qyZImeeeYZZWVlDZoTDof18ssva8uWLVq8eLEk6dVXX1VOTo7ef/99LViwQJK0du1aSdKZM2f0wQcfDDrP5s2b1dfXp1deeUUpKSm65ppr1NTUpB//+MfnDTAAAGCGhNwBCgQCcrlcVvxIUkFBgWw2mw4cODDknGAwqGg0qoKCAmtfdna2ZsyYoUAgMKrX/t73vqeUlBRrX2FhoVpbW/XRRx8NO6+3t1eRSCRuAwAAE1NCAigUCik9PT1uX3JyslJTUxUKhYadk5KSIpfLFbc/IyNj2DnDnScjI2PQOc4dG86aNWvkdDqtzePxXPBrAgCA8WVUAVRTU6OkpKQRt5aWlkStNaFWrVqlcDhsbSdPnhzrJQEAgAQZ1WeAqqurtXz58hHHzJ49W263W6dPn47b39/fr87OTrnd7iHnud1u9fX1qaurK+4uUEdHx7BzhjvP///k2Ll/j3Qeu90uu91+wa8DAADGr1EFUFpamtLS0s47zufzqaurS8FgUF6vV5LU2NiogYEB5eXlDTnH6/Vq0qRJ2rt3r/x+vySptbVVJ06ckM/nu+A1+nw+/fCHP1Q0GtWkSZMkSQ0NDbr66qv11a9+9YLPAwAAJq6EfAYoJydHRUVFWrFihQ4ePKh3331XFRUVKi4utp4Aa2trU3Z2tg4ePChJcjqdKi0tVVVVld566y0Fg0Hdfffd8vl81hNgkvSb3/xGTU1NCoVC+vTTT9XU1KSmpib19fVJkv7yL/9SKSkpKi0tVXNzs7Zu3arnn39eVVVVibhUAAAwDiXkMXjp88fRKyoqlJ+fL5vNJr/fbz3CLknRaFStra3q7u629tXW1lpje3t7VVhYqHXr1sWd995779W+ffusf3/nO9+RJB0/flwzZ86U0+nUm2++qfLycnm9Xl1xxRVavXo1j8ADAABLUiwWi431Ii5FkUhETqdT4XBYDodjrJcD4CLp7utX7uo3JElHnyjU1JSE/R4IYAxc6M9v/hYYAAAwDgEEAACMQwABAADjEEAAAMA4BBAAADAOAQQAAIxDAAEAAOMQQAAAwDgEEAAAMA4BBAAAjEMAAQAA4xBAAADAOAQQAAAwDgEEAACMQwABAADjEEAAAMA4BBAAADAOAQQAAIxDAAEAAOMQQAAAwDgEEAAAMA4BBAAAjEMAAQAA4xBAAADAOAQQAAAwDgEEAACMQwABAADjEEAAAMA4BBAAADAOAQQAAIxDAAEAAOMQQAAAwDgJC6DOzk6VlJTI4XDI5XKptLRUZ8+eHXFOT0+PysvLNX36dE2bNk1+v18dHR1xY37wgx/I6/XKbrdrzpw5g87xu9/9TklJSYO2999//2JeHgAAGMcSFkAlJSVqbm5WQ0ODdu7cqf3796usrGzEOZWVldqxY4e2bdumffv2qb29XcuWLRs07p577tEdd9wx4rn27NmjU6dOWZvX6/2TrgcAAEwcyYk46bFjx1RfX69Dhw5p7ty5kqQXXnhBS5Ys0TPPPKOsrKxBc8LhsF5++WVt2bJFixcvliS9+uqrysnJ0fvvv68FCxZIktauXStJOnPmjD744INh1zB9+nS53e6LfWkAAGACSMgdoEAgIJfLZcWPJBUUFMhms+nAgQNDzgkGg4pGoyooKLD2ZWdna8aMGQoEAqNew1/8xV8oPT1d119/vV5//fXzju/t7VUkEonbAADAxJSQAAqFQkpPT4/bl5ycrNTUVIVCoWHnpKSkyOVyxe3PyMgYds5Qpk2bpmeffVbbtm3Trl27dP311+u22247bwStWbNGTqfT2jwezwW/JgAAGF9GFUA1NTVDfsD4i1tLS0ui1npBrrjiClVVVSkvL0/z5s3Tj370I33/+9/X008/PeK8VatWKRwOW9vJkye/pBUDAIAv26g+A1RdXa3ly5ePOGb27Nlyu906ffp03P7+/n51dnYO+7kct9utvr4+dXV1xd0F6ujo+JM/y5OXl6eGhoYRx9jtdtnt9j/pdQAAwPgwqgBKS0tTWlraecf5fD51dXUpGAxaT181NjZqYGBAeXl5Q87xer2aNGmS9u7dK7/fL0lqbW3ViRMn5PP5RrPMQZqampSZmfknnQMAAEwcCXkKLCcnR0VFRVqxYoXq6uoUjUZVUVGh4uJi6wmwtrY25efna+PGjZo/f76cTqdKS0tVVVWl1NRUORwOrVy5Uj6fz3oCTJJ+85vf6OzZswqFQvr000/V1NQkScrNzVVKSop+9rOfKSUlRd/5znckSa+99ppeeeUVrV+/PhGXCgAAxqGEBJAkbd68WRUVFcrPz5fNZpPf77ceYZekaDSq1tZWdXd3W/tqa2utsb29vSosLNS6deviznvvvfdq37591r/Phc7x48c1c+ZMSdI//dM/6X/+53+UnJys7Oxsbd26VbfffnuiLhUAAIwzSbFYLDbWi7gURSIROZ1OhcNhORyOsV4OgIuku69fuavfkCQdfaJQU1MS9nsggDFwoT+/+VtgAADAOAQQAAAwDgEEAACMQwABAADjEEAAAMA4BBAAADAOAQQAAIxDAAEAAOMQQAAAwDgEEAAAMA4BBMAonw384a//HDzeGfdvAOYggAAYo/7IKRX8+A9/THn5q4d0/f9pVP2RU2O4KgBjgQACYIT6I6f0wKbD6oj0xu0PhXv0wKbDRBBgGAIIwIT32UBM/7jjqIZ6s+vcvn/ccZS3wwCDEEAAJryDxzt1Ktwz7PGYpFPhHh083vnlLQrAmCKAAEx4pz8ePn7+mHEAxj8CCMCEl3755Is6DsD4RwABmPDmz0pVpnOykoY5niQp0zlZ82elfpnLAjCGCCAAE95ltiQ9/ue5kjQogs79+/E/z9VltuESCcBEQwABMELRNzP1k+9fp3SHPW6/2zlZP/n+dSr6ZuYYrQzAWEge6wUAwJel6JuZ+u7Xr9C3/uFNSdKGu+fphm+kcecHMBB3gAAY5YuxM39WKvEDGIoAAgAAxiGAAACAcQggAABgHAIIAAAYhwACAADGIYAAAIBxCCAAAGAcAggAABiHAAIAAMYhgAAAgHESGkCdnZ0qKSmRw+GQy+VSaWmpzp49O+Kcnp4elZeXa/r06Zo2bZr8fr86Ojqs47/61a905513yuPxaMqUKcrJydHzzz8/6Dxvv/22rrvuOtntdn3961/Xhg0bLvblAQCAcSqhAVRSUqLm5mY1NDRo586d2r9/v8rKykacU1lZqR07dmjbtm3at2+f2tvbtWzZMut4MBhUenq6Nm3apObmZv3whz/UqlWr9OKLL1pjjh8/rqVLl2rRokVqamrSgw8+qHvvvVdvvPFGwq4VAACMH0mxWCyWiBMfO3ZMubm5OnTokObOnStJqq+v15IlS/Thhx8qKytr0JxwOKy0tDRt2bJFt99+uySppaVFOTk5CgQCWrBgwZCvVV5ermPHjqmxsVGS9Mgjj2jXrl06cuSINaa4uFhdXV2qr6+/oPVHIhE5nU6Fw2E5HI5RXTuAS1d3X79yV3/+y9DRJwo1NSV5jFcE4GK60J/fCbsDFAgE5HK5rPiRpIKCAtlsNh04cGDIOcFgUNFoVAUFBda+7OxszZgxQ4FAYNjXCofDSk1NjXvtL55DkgoLC0c8R29vryKRSNwGAAAmpoQFUCgUUnp6ety+5ORkpaamKhQKDTsnJSVFLpcrbn9GRsawc9577z1t3bo17q21UCikjIyMQeeIRCL69NNPhzzPmjVr5HQ6rc3j8ZzvEgEAwDg16gCqqalRUlLSiFtLS0si1jrIkSNHdOutt+rxxx/XLbfc8ieda9WqVQqHw9Z28uTJi7RKAABwqRn1m9/V1dVavnz5iGNmz54tt9ut06dPx+3v7+9XZ2en3G73kPPcbrf6+vrU1dUVdxeoo6Nj0JyjR48qPz9fZWVlevTRRwed54tPjp07h8Ph0JQpU4Z8bbvdLrvdPuJ1AQCAiWHUAZSWlqa0tLTzjvP5fOrq6lIwGJTX65UkNTY2amBgQHl5eUPO8Xq9mjRpkvbu3Su/3y9Jam1t1YkTJ+Tz+axxzc3NWrx4se666y49+eSTQ7727t274/Y1NDTEnQMAAJgrYZ8BysnJUVFRkVasWKGDBw/q3XffVUVFhYqLi60nwNra2pSdna2DBw9KkpxOp0pLS1VVVaW33npLwWBQd999t3w+n/UE2JEjR7Ro0SLdcsstqqqqUigUUigU0pkzZ6zXvv/++/Xf//3fevjhh9XS0qJ169bp5z//uSorKxN1uQAAYBxJ6PcAbd68WdnZ2crPz9eSJUt0/fXX66WXXrKOR6NRtba2qru729pXW1urP/uzP5Pf79f3vvc9ud1uvfbaa9bxX/ziFzpz5ow2bdqkzMxMa5s3b541ZtasWdq1a5caGhr07W9/W88++6zWr1+vwsLCRF4uAAAYJxL2PUDjHd8DBExMfA8QMLGN+fcAAQAAXKoIIAAAYBwCCAAAGIcAAgAAxiGAAACAcQggAABgHAIIAAAYhwACAADGIYAAAIBxCCAAAGAcAggAABiHAAIAAMYhgAAAgHEIIAAAYBwCCAAAGIcAAgAAxiGAAACAcQggAABgHAIIAAAYhwACAADGIYAAAIBxCCAAAGAcAggAABiHAAIAAMYhgAAAgHEIIAAAYBwCCAAAGIcAAgAAxiGAAACAcQggAABgHAIIAAAYhwACAADGSWgAdXZ2qqSkRA6HQy6XS6WlpTp79uyIc3p6elReXq7p06dr2rRp8vv96ujosI7/6le/0p133imPx6MpU6YoJydHzz//fNw53n77bSUlJQ3aQqFQQq4TAACML8mJPHlJSYlOnTqlhoYGRaNR3X333SorK9OWLVuGnVNZWaldu3Zp27Ztcjqdqqio0LJly/Tuu+9KkoLBoNLT07Vp0yZ5PB699957Kisr02WXXaaKioq4c7W2tsrhcFj/Tk9PT8yFAgCAcSUpFovFEnHiY8eOKTc3V4cOHdLcuXMlSfX19VqyZIk+/PBDZWVlDZoTDoeVlpamLVu26Pbbb5cktbS0KCcnR4FAQAsWLBjytcrLy3Xs2DE1NjZK+vwO0KJFi/TRRx/J5XL9UeuPRCJyOp0Kh8NxEQVgfOvu61fu6jckSUefKNTUlIT+HgjgS3ahP78T9hZYIBCQy+Wy4keSCgoKZLPZdODAgSHnBINBRaNRFRQUWPuys7M1Y8YMBQKBYV8rHA4rNTV10P45c+YoMzNTN998s3UHaTi9vb2KRCJxGwAAmJgSFkChUGjQW07JyclKTU0d9rM4oVBIKSkpg+7aZGRkDDvnvffe09atW1VWVmbty8zMVF1dnbZv367t27fL4/Hopptu0uHDh4dd75o1a+R0Oq3N4/Fc4JUCAIDxZtQBVFNTM+QHjL+4tbS0JGKtgxw5ckS33nqrHn/8cd1yyy3W/quvvlr33XefvF6vFi5cqFdeeUULFy5UbW3tsOdatWqVwuGwtZ08efLLuAQAADAGRv3md3V1tZYvXz7imNmzZ8vtduv06dNx+/v7+9XZ2Sm32z3kPLfbrb6+PnV1dcXdBero6Bg05+jRo8rPz1dZWZkeffTR8657/vz5euedd4Y9brfbZbfbz3seAAAw/o06gNLS0pSWlnbecT6fT11dXQoGg/J6vZKkxsZGDQwMKC8vb8g5Xq9XkyZN0t69e+X3+yV9/iTXiRMn5PP5rHHNzc1avHix7rrrLj355JMXtO6mpiZlZmZe0FgAADCxJezxh5ycHBUVFWnFihWqq6tTNBpVRUWFiouLrSfA2tralJ+fr40bN2r+/PlyOp0qLS1VVVWVUlNT5XA4tHLlSvl8PusJsCNHjmjx4sUqLCxUVVWV9dmgyy67zAqz5557TrNmzdI111yjnp4erV+/Xo2NjXrzzTcTdbkAAGAcSejzn5s3b1ZFRYXy8/Nls9nk9/u1du1a63g0GlVra6u6u7utfbW1tdbY3t5eFRYWat26ddbxX/ziFzpz5ow2bdqkTZs2Wfuvuuoq/e53v5Mk9fX1qbq6Wm1tbZo6daquvfZa7dmzR4sWLUrk5QIAgHEiYd8DNN7xPUDAxMT3AAET25h/DxAAAMCligACAADGIYAAAIBxCCAAAGAcAggAABiHAAIAAMYhgAAAgHEIIAAAYBwCCAAAGIcAAgAAxiGAAACAcQggAABgHAIIAAAYhwACAADGIYAAAIBxCCAAAGAcAggAABiHAAIAAMYhgAAAgHEIIAAAYBwCCAAAGIcAAgAAxiGAAACAcQggAABgHAIIAAAYhwACAADGIYAAAIBxCCAAAGAcAggAABiHAAIAAMYhgAAAgHEIIAAAYJyEBlBnZ6dKSkrkcDjkcrlUWlqqs2fPjjinp6dH5eXlmj59uqZNmya/36+Ojg7r+O9//3sVFRUpKytLdrtdHo9HFRUVikQiced5++23dd1118lut+vrX/+6NmzYkIhLBAAA41BCA6ikpETNzc1qaGjQzp07tX//fpWVlY04p7KyUjt27NC2bdu0b98+tbe3a9myZX9YsM2mW2+9Va+//rp+/etfa8OGDdqzZ4/uv/9+a8zx48e1dOlSLVq0SE1NTXrwwQd177336o033kjYtQIAgPEjKRaLxRJx4mPHjik3N1eHDh3S3LlzJUn19fVasmSJPvzwQ2VlZQ2aEw6HlZaWpi1btuj222+XJLW0tCgnJ0eBQEALFiwY8rXWrl2rp59+WidPnpQkPfLII9q1a5eOHDlijSkuLlZXV5fq6+svaP2RSEROp1PhcFgOh2NU1w7g0tXd16/c1Z//MnT0iUJNTUke4xUBuJgu9Od3wu4ABQIBuVwuK34kqaCgQDabTQcOHBhyTjAYVDQaVUFBgbUvOztbM2bMUCAQGHJOe3u7XnvtNd14441xr/3Fc0hSYWHhsOeQpN7eXkUikbgNAABMTAkLoFAopPT09Lh9ycnJSk1NVSgUGnZOSkqKXC5X3P6MjIxBc+68805NnTpVV155pRwOh9avXx93noyMjEHniEQi+vTTT4d87TVr1sjpdFqbx+O50EsFAADjzKgDqKamRklJSSNuLS0tiVhrnNraWh0+fFi//OUv9dvf/lZVVVV/0vlWrVqlcDhsbefeTgMAABPPqN/8rq6u1vLly0ccM3v2bLndbp0+fTpuf39/vzo7O+V2u4ec53a71dfXp66urri7QB0dHYPmuN1uud1uZWdnKzU1VTfccIMee+wxZWZmyu12xz05du4cDodDU6ZMGfK17Xa77Hb7iNcFAAAmhlEHUFpamtLS0s47zufzqaurS8FgUF6vV5LU2NiogYEB5eXlDTnH6/Vq0qRJ2rt3r/x+vySptbVVJ06ckM/nG/a1BgYGJH3+OZ5zr7179+64MQ0NDSOeAwAAmCNhjz/k5OSoqKhIK1asUF1dnaLRqCoqKlRcXGw9AdbW1qb8/Hxt3LhR8+fPl9PpVGlpqaqqqpSamiqHw6GVK1fK5/NZT4Dt3r1bHR0dmjdvnqZNm6bm5mY99NBD+u53v6uZM2dKku6//369+OKLevjhh3XPPfeosbFRP//5z7Vr165EXS4AABhHEvr85+bNm1VRUaH8/HzZbDb5/X6tXbvWOh6NRtXa2qru7m5rX21trTW2t7dXhYWFWrdunXV8ypQp+ulPf6rKykr19vbK4/Fo2bJlqqmpscbMmjVLu3btUmVlpZ5//nl97Wtf0/r161VYWJjIywUAAONEwr4HaLzje4CAiYnvAQImtjH/HiAAAIBLFQEEAACMQwABAADjEEAAAMA4BBAAADAOAQQAAIxDAAEAAOMQQAAAwDgEEAAAMA4BBAAAjEMAAQAA4xBAAADAOAQQAAAwDgEEAACMQwABAADjEEAAAMA4BBAAADAOAQQAAIxDAAEAAOMQQAAAwDgEEAAAMA4BBAAAjEMAAQAA4xBAAADAOAQQAAAwDgEEAACMQwABAADjEEAAAMA4BBAAADAOAQQAAIxDAAEAAOMkNIA6OztVUlIih8Mhl8ul0tJSnT17dsQ5PT09Ki8v1/Tp0zVt2jT5/X51dHRYx3//+9+rqKhIWVlZstvt8ng8qqioUCQSsca8/fbbSkpKGrSFQqGEXSsAABg/EhpAJSUlam5uVkNDg3bu3Kn9+/errKxsxDmVlZXasWOHtm3bpn379qm9vV3Lli37w4JtNt166616/fXX9etf/1obNmzQnj17dP/99w86V2trq06dOmVt6enpF/0aAQDA+JOcqBMfO3ZM9fX1OnTokObOnStJeuGFF7RkyRI988wzysrKGjQnHA7r5Zdf1pYtW7R48WJJ0quvvqqcnBy9//77WrBggb761a/qgQcesOZcddVV+pu/+Rs9/fTTg86Xnp4ul8uVmAsEAADjVsLuAAUCAblcLit+JKmgoEA2m00HDhwYck4wGFQ0GlVBQYG1Lzs7WzNmzFAgEBhyTnt7u1577TXdeOONg47NmTNHmZmZuvnmm/Xuu++OuN7e3l5FIpG4DQAATEwJC6BQKDToLafk5GSlpqYO+1mcUCiklJSUQXdtMjIyBs258847NXXqVF155ZVyOBxav369dSwzM1N1dXXavn27tm/fLo/Ho5tuukmHDx8edr1r1qyR0+m0No/HM8orBgAA48WoA6impmbIDxh/cWtpaUnEWuPU1tbq8OHD+uUvf6nf/va3qqqqso5dffXVuu++++T1erVw4UK98sorWrhwoWpra4c936pVqxQOh63t5MmTCb8GAAAwNkb9GaDq6motX758xDGzZ8+W2+3W6dOn4/b39/ers7NTbrd7yHlut1t9fX3q6uqKuwvU0dExaI7b7Zbb7VZ2drZSU1N1ww036LHHHlNmZuaQ554/f77eeeedYddst9tlt9tHvC4AADAxjDqA0tLSlJaWdt5xPp9PXV1dCgaD8nq9kqTGxkYNDAwoLy9vyDler1eTJk3S3r175ff7JX3+JNeJEyfk8/mGfa2BgQFJn3+OZzhNTU3DxhEAADBLwp4Cy8nJUVFRkVasWKG6ujpFo1FVVFSouLjYegKsra1N+fn52rhxo+bPny+n06nS0lJVVVUpNTVVDodDK1eulM/n04IFCyRJu3fvVkdHh+bNm6dp06apublZDz30kL773e9q5syZkqTnnntOs2bN0jXXXKOenh6tX79ejY2NevPNNxN1uQAAYBxJWABJ0ubNm1VRUaH8/HzZbDb5/X6tXbvWOh6NRtXa2qru7m5rX21trTW2t7dXhYWFWrdunXV8ypQp+ulPf6rKykr19vbK4/Fo2bJlqqmpscb09fWpurpabW1tmjp1qq699lrt2bNHixYtSuTlAgCAcSIpFovFxnoRl6JIJCKn06lwOCyHwzHWywFwkXT39St39RuSpKNPFGpqSkJ/DwTwJbvQn9/8LTAAAGAcAggAABiHAAIAAMYhgAAAgHEIIAAAYBwCCAAAGIcAAgAAxiGAAACAcQggAABgHAIIAAAYhwACAADGIYAAAIBx+CuAAIwyNSVZv/vR0rFeBoAxxh0gAABgHAIIAAAYhwACAADGIYAAAIBxCCAAAGAcAggAABiHAAIAAMYhgAAAgHEIIAAAYBwCCAAAGIcAAgAAxiGAAACAcQggAABgHAIIAAAYhwACAADGSR7rBVyqYrGYJCkSiYzxSgAAwIU693P73M/x4RBAw/j4448lSR6PZ4xXAgAARuvjjz+W0+kc9nhS7HyJZKiBgQG1t7fr8ssvV1JS0lgvB8BFFIlE5PF4dPLkSTkcjrFeDoCLKBaL6eOPP1ZWVpZstuE/6UMAATBOJBKR0+lUOBwmgABD8SFoAABgHAIIAAAYhwACYBy73a7HH39cdrt9rJcCYIzwGSAAAGAc7gABAADjEEAAAMA4BBAAADAOAQQAAIxDAAEAAOMQQAAAwDgEEAAAMA4BBAAAjPN/AQahI/dzionUAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Calculate the mean difference between the control and treatment groups\n",
    "mean_difference = treatment_conversion_rate - control_conversion_rate\n",
    "\n",
    "# Calculate the standard error of the mean difference\n",
    "se = np.sqrt(\n",
    "    (control[\"purchased\"].var() / len(control))\n",
    "    + (treatment[\"purchased\"].var() / len(treatment))\n",
    ")\n",
    "\n",
    "# Calculate the margin of error for a 95% confidence interval\n",
    "me = 1.96 * se\n",
    "\n",
    "# Calculate the lower and upper bounds of the confidence interval\n",
    "lower_bound = mean_difference - me\n",
    "upper_bound = mean_difference + me\n",
    "\n",
    "# Plot the confidence interval\n",
    "plt.errorbar(0, mean_difference, yerr=me, fmt=\"o\")\n",
    "plt.xlim(-0.5, 0.5)\n",
    "plt.xticks([0], [\"\"])\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This plot shows the mean difference between the two groups as a dot, with the confidence interval as vertical lines above and below the dot. If the confidence interval does not include zero, we can conclude that the difference in means between the two groups is statistically significant&mdash;which is clearly not the case here.\n",
    "\n",
    "## Drawing conclusions from the test results\n",
    "\n",
    "Now that we have visualized the results of our A/B test, let’s interpret our findings and draw conclusions.\n",
    "\n",
    "Based on our hypothesis test and visualization, we can conclude whether the changes made in the treatment group had a statistically significant impact on the conversion rate compared to the control group. If the p-value from our hypothesis test is less than our significance level (usually 0.05), and the confidence interval does not include zero, we can conclude that the difference in conversion rates between the control and treatment groups is statistically significant and that the changes made in the treatment group had a positive impact on the desired outcome.\n",
    "\n",
    "If the p-value is greater than our significance level, or the confidence interval includes zero, we cannot conclude that the difference in conversion rates between the control and treatment groups is statistically significant, and we cannot claim that the changes made in the treatment group had a positive impact on the desired outcome.\n",
    "\n",
    "Based on our experiment here, the p-value is ~0.08 > 0.05 and the confidence interval includes zero, therefore, the positive impact of the modifications made in the treatment group on the desired outcome cannot be claimed.\n",
    "\n",
    "It’s important to note that A/B testing is not a one-time process, and it may take several iterations to achieve statistically significant results. It’s also important to consider the practical significance of the observed difference in conversion rates, as even a statistically significant difference may not be meaningful in the context of the overall business goals."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.2"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}