OPEN_RPA_PUBLIC
/
ORPA-pyOpenRPA
2
0
Fork 1
Code Issues 2 Pull Requests Packages Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'c2d82ec6d7'
${ noResults }
ORPA-pyOpenRPA/Resources/WPy64-3720/notebooks/docs/Eight_Schools_Tensorflow_Pr...

692 lines
23 KiB
Raw Blame History

{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "-knK4sZodDZg"
},
"source": [
"##### Copyright 2018 The TensorFlow Authors.\n",
"\n",
"Licensed under the Apache License, Version 2.0 (the \"License\");"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "zAM8G9A4dF4R"
},
"outputs": [],
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\"); { display-mode: \"form\" }\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "cPw5xFcq1kpw"
},
"source": [
"<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/probability/blob/master/tensorflow_probability/examples/jupyter_notebooks/Eight_Schools.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://github.com/tensorflow/probability/blob/master/tensorflow_probability/examples/jupyter_notebooks/Eight_Schools.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
" </td>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "5MzjGu_O7HwY"
},
"source": [
"# Eight schools\n",
"\n",
"The eight schools problem ([Rubin 1981](https://www.jstor.org/stable/1164617)) considers the effectiveness of SAT coaching programs conducted in parallel at eight schools. It has become a classic problem ([Bayesian Data Analysis](http://www.stat.columbia.edu/~gelman/book/), [Stan](https://github.com/stan-dev/rstan/wiki/RStan-Getting-Started)) that illustrates the usefulness of hierarchical modeling for sharing information between exchangeable groups.\n",
"\n",
"The Edward2 implemention below is an adaptation of an Edward 1.0 [tutorial](https://github.com/blei-lab/edward/blob/master/notebooks/eight_schools.ipynb). "
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "1hFtob1DsFqm"
},
"source": [
"<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/probability/blob/master/tensorflow_probability/examples/jupyter_notebooks/Eight_Schools.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://github.com/tensorflow/probability/blob/master/tensorflow_probability/examples/jupyter_notebooks/Eight_Schools.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
" </td>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "TNuvn0Ih4D_R"
},
"source": [
"# Imports"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "vznFo-cU7Pc_"
},
"outputs": [],
"source": [
"# installed by defaultf now\n",
"# !pip install -q tensorflow-probability"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "XMTEI6ep4D_S"
},
"outputs": [],
"source": [
"from __future__ import absolute_import\n",
"from __future__ import division\n",
"from __future__ import print_function\n",
"\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"import seaborn as sns\n",
"\n",
"import tensorflow as tf\n",
"import tensorflow_probability as tfp\n",
"from tensorflow_probability import edward2 as ed\n",
"import warnings\n",
"\n",
"plt.style.use(\"ggplot\")\n",
"warnings.filterwarnings('ignore')"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "cIbNcemwwO2y"
},
"source": [
"# The Data\n",
"\n",
"From Bayesian Data Analysis, section 5.5 (Gelman et al. 2013):\n",
"\n",
"> *A study was performed for the Educational Testing Service to analyze the effects of special coaching programs for SAT-V (Scholastic Aptitude Test-Verbal) in each of eight high schools. The outcome variable in each study was the score on a special administration of the SAT-V, a standardized multiple choice test administered by the Educational Testing Service and used to help colleges make admissions decisions; the scores can vary between 200 and 800, with mean about 500 and standard deviation about 100. The SAT examinations are designed to be resistant to short-term efforts directed specifically toward improving performance on the test; instead they are designed to reflect knowledge acquired and abilities developed over many years of education. Nevertheless, each of the eight schools in this study considered its short-term coaching program to be very successful at increasing SAT scores. Also, there was no prior reason to believe that any of the eight programs was more effective than any other or that some were more similar in effect to each other than to any other.*\n",
"\n",
"\n",
"For each of the eight schools ($J = 8$), we have an estimated treatment effect $y_j$ and a standard error of the effect estimate $\\sigma_j$. The treatment effects in the study were obtained by a linear regression on the treatment group using PSAT-M and PSAT-V scores as control variables. As there was no prior belief that any of the schools were more or less similar or that any of the coaching programs would be more effective, we can consider the treatment effects as [exchangeable](https://en.wikipedia.org/wiki/Exchangeable_random_variables)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 516
},
"colab_type": "code",
"id": "rSngqHwAKv_j",
"outputId": "db4a11bc-9946-4be9-a362-3b65f29a08ca"
},
"outputs": [],
"source": [
"num_schools = 8 # number of schools\n",
"treatment_effects = np.array(\n",
" [28, 8, -3, 7, -1, 1, 18, 12], dtype=np.float32) # treatment effects\n",
"treatment_stddevs = np.array(\n",
" [15, 10, 16, 11, 9, 11, 10, 18], dtype=np.float32) # treatment SE\n",
"\n",
"fig, ax = plt.subplots()\n",
"plt.bar(range(num_schools), treatment_effects, yerr=treatment_stddevs)\n",
"plt.title(\"8 Schools treatment effects\")\n",
"plt.xlabel(\"School\")\n",
"plt.ylabel(\"Treatment effect\")\n",
"fig.set_size_inches(10, 8)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "S6Yj8WEDwI3L"
},
"source": [
"# Model\n",
"\n",
"To capture the data, we use a hierarchical normal model. It follows the generative process,\n",
"\n",
"\\begin{align*}\n",
"\\mu &\\sim \\text{Normal}(\\text{loc}{=}0,\\ \\text{scale}{=}10) \\\\\n",
"\\log\\tau &\\sim \\text{Normal}(\\text{loc}{=}5,\\ \\text{scale}{=}1) \\\\\n",
"\\text{for } & i=1\\ldots 8:\\\\\n",
"& \\theta_i \\sim \\text{Normal}\\left(\\text{loc}{=}\\mu,\\ \\text{scale}{=}\\tau \\right) \\\\\n",
"& y_i \\sim \\text{Normal}\\left(\\text{loc}{=}\\theta_i,\\ \\text{scale}{=}\\sigma_i \\right) \n",
"\\end{align*}\n",
"\n",
"where $\\mu$ represents the prior average treatment effect and $\\tau$ controls how much variance there is between schools. The $y_i$ and $\\sigma_i$ are observed. As $\\tau \\rightarrow \\infty$, the model approaches the no-pooling model, i.e., each of the school treatment effect estimates are allowed to be more independent. As $\\tau \\rightarrow 0$, the model approaches the complete-pooling model, i.e., all of the school treatment effects are closer to the group average $\\mu$. To restrict the standard deviation to be positive, we draw $\\tau$ from a lognormal distribution (which is equivalent to drawing $log(\\tau)$ from a normal distribution).\n",
"\n",
"Following [Diagnosing Biased Inference with Divergences](http://mc-stan.org/users/documentation/case-studies/divergences_and_bias.html), we transform the model above into an equivalent non-centered model:\n",
"\n",
"\\begin{align*}\n",
"\\mu &\\sim \\text{Normal}(\\text{loc}{=}0,\\ \\text{scale}{=}10) \\\\\n",
"\\log\\tau &\\sim \\text{Normal}(\\text{loc}{=}5,\\ \\text{scale}{=}1) \\\\\n",
"\\text{for } & i=1\\ldots 8:\\\\\n",
"& \\theta_i' \\sim \\text{Normal}\\left(\\text{loc}{=}0,\\ \\text{scale}{=}1 \\right) \\\\\n",
"& \\theta_i = \\mu + \\tau \\theta_i' \\\\\n",
"& y_i \\sim \\text{Normal}\\left(\\text{loc}{=}\\theta_i,\\ \\text{scale}{=}\\sigma_i \\right) \n",
"\\end{align*}\n",
"\n",
"We reify this model as an Edward2 program:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "EiEtvl1zokAG"
},
"outputs": [],
"source": [
"def schools_model(num_schools, treatment_stddevs):\n",
" avg_effect = ed.Normal(loc=0., scale=10., name=\"avg_effect\") # `mu` above\n",
" avg_stddev = ed.Normal(\n",
" loc=5., scale=1., name=\"avg_stddev\") # `log(tau)` above\n",
" school_effects_standard = ed.Normal(\n",
" loc=tf.zeros(num_schools),\n",
" scale=tf.ones(num_schools),\n",
" name=\"school_effects_standard\") # `theta_prime` above\n",
" school_effects = avg_effect + tf.exp(\n",
" avg_stddev) * school_effects_standard # `theta` above\n",
" treatment_effects = ed.Normal(\n",
" loc=school_effects, scale=treatment_stddevs,\n",
" name=\"treatment_effects\") # `y` above\n",
" return treatment_effects\n",
"\n",
"log_joint = ed.make_log_joint_fn(schools_model)\n",
"\n",
"\n",
"def target_log_prob_fn(avg_effect, avg_stddev, school_effects_standard):\n",
" \"\"\"Unnormalized target density as a function of states.\"\"\"\n",
" return log_joint(\n",
" num_schools=num_schools,\n",
" treatment_stddevs=treatment_stddevs,\n",
" avg_effect=avg_effect,\n",
" avg_stddev=avg_stddev,\n",
" school_effects_standard=school_effects_standard,\n",
" treatment_effects=treatment_effects)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "jnVK-1yH9WCY"
},
"source": [
"# Bayesian Inference\n",
"\n",
"Given data, we perform Hamiltonian Monte Carlo (HMC) to calculate the posterior distribution over the model's parameters."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"colab_type": "code",
"id": "-66vCUVrQRnb",
"outputId": "c6d7f3d0-073b-4d6c-9d7f-55826134d596"
},
"outputs": [],
"source": [
"num_results = 5000\n",
"num_burnin_steps = 3000\n",
"\n",
"states, kernel_results = tfp.mcmc.sample_chain(\n",
" num_results=num_results,\n",
" num_burnin_steps=num_burnin_steps,\n",
" current_state=[\n",
" tf.zeros([], name='init_avg_effect'),\n",
" tf.zeros([], name='init_avg_stddev'),\n",
" tf.ones([num_schools], name='init_school_effects_standard'),\n",
" ],\n",
" kernel=tfp.mcmc.HamiltonianMonteCarlo(\n",
" target_log_prob_fn=target_log_prob_fn,\n",
" step_size=0.4,\n",
" num_leapfrog_steps=3))\n",
"\n",
"avg_effect, avg_stddev, school_effects_standard = states\n",
"\n",
"with tf.Session() as sess:\n",
" [\n",
" avg_effect_,\n",
" avg_stddev_,\n",
" school_effects_standard_,\n",
" is_accepted_,\n",
" ] = sess.run([\n",
" avg_effect,\n",
" avg_stddev,\n",
" school_effects_standard,\n",
" kernel_results.is_accepted,\n",
" ])\n",
"\n",
"school_effects_samples = (\n",
" avg_effect_[:, np.newaxis] +\n",
" np.exp(avg_stddev_)[:, np.newaxis] * school_effects_standard_)\n",
"num_accepted = np.sum(is_accepted_)\n",
"print('Acceptance rate: {}'.format(num_accepted / num_results))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 729
},
"colab_type": "code",
"id": "2-iMMOcFvE03",
"outputId": "3d6e2b56-d76e-41e4-f60e-29332149d350"
},
"outputs": [],
"source": [
"fig, axes = plt.subplots(8, 2, sharex='col', sharey='col')\n",
"fig.set_size_inches(12, 10)\n",
"for i in range(num_schools):\n",
" axes[i][0].plot(school_effects_samples[:,i])\n",
" axes[i][0].title.set_text(\"School {} treatment effect chain\".format(i))\n",
" sns.kdeplot(school_effects_samples[:,i], ax=axes[i][1], shade=True)\n",
" axes[i][1].title.set_text(\"School {} treatment effect distribution\".format(i))\n",
"axes[num_schools - 1][0].set_xlabel(\"Iteration\")\n",
"axes[num_schools - 1][1].set_xlabel(\"School effect\")\n",
"fig.tight_layout()\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 153
},
"colab_type": "code",
"id": "l4t9XLxSszBe",
"outputId": "45b3857b-1abd-4dff-e177-692ce11a7aa3"
},
"outputs": [],
"source": [
"print(\"E[avg_effect] = {}\".format(avg_effect_.mean()))\n",
"print(\"E[avg_stddev] = {}\".format(avg_stddev_.mean()))\n",
"print(\"E[school_effects_standard] =\")\n",
"print(school_effects_standard_[:, ].mean(0))\n",
"print(\"E[school_effects] =\")\n",
"print(school_effects_samples[:, ].mean(0))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "Wxp1uFW6RWMW"
},
"outputs": [],
"source": [
"# Compute the 95% interval for school_effects\n",
"school_effects_low = np.array([\n",
" np.percentile(school_effects_samples[:, i], 2.5) for i in range(num_schools)\n",
"])\n",
"school_effects_med = np.array([\n",
" np.percentile(school_effects_samples[:, i], 50) for i in range(num_schools)\n",
"])\n",
"school_effects_hi = np.array([\n",
" np.percentile(school_effects_samples[:, i], 97.5)\n",
" for i in range(num_schools)\n",
"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 516
},
"colab_type": "code",
"id": "yY-qBFTotd3F",
"outputId": "e4d9fc9b-1d40-47f6-ae34-ff8281155b42"
},
"outputs": [],
"source": [
"fig, ax = plt.subplots(nrows=1, ncols=1, sharex=True)\n",
"ax.scatter(np.array(range(num_schools)), school_effects_med, color='red', s=60)\n",
"ax.scatter(\n",
" np.array(range(num_schools)) + 0.1, treatment_effects, color='blue', s=60)\n",
"\n",
"avg_effect = avg_effect_.mean()\n",
"\n",
"plt.plot([-0.2, 7.4], [avg_effect, avg_effect], 'k', linestyle='--')\n",
"\n",
"ax.errorbar(\n",
" np.array(range(8)),\n",
" school_effects_med,\n",
" yerr=[\n",
" school_effects_med - school_effects_low,\n",
" school_effects_hi - school_effects_med\n",
" ],\n",
" fmt='none')\n",
"\n",
"ax.legend(('avg_effect', 'Edward2/HMC', 'Observed effect'), fontsize=14)\n",
"\n",
"plt.xlabel('School')\n",
"plt.ylabel('Treatment effect')\n",
"plt.title('Edward2 HMC estimated school treatment effects vs. observed data')\n",
"fig.set_size_inches(10, 8)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "2dV93ZSzGSIm"
},
"source": [
"We can observe the shrinkage toward the group `avg_effect` above."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 51
},
"colab_type": "code",
"id": "LcljZ1prD91d",
"outputId": "c3eed6e1-5b00-4ddf-c40c-1c045c4f6b0c"
},
"outputs": [],
"source": [
"print(\"Inferred posterior mean: {0:.2f}\".format(\n",
" np.mean(school_effects_samples[:,])))\n",
"print(\"Inferred posterior mean se: {0:.2f}\".format(\n",
" np.std(school_effects_samples[:,])))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "vWPCzgk7IMgt"
},
"source": [
"# Criticism\n",
"\n",
"To get the posterior predictive distribution, i.e., a model of new data $y^*$ given the observed data $y$:\n",
"\n",
"$$ p(y^*|y) \\propto \\int_\\theta p(y^* | \\theta)p(\\theta |y)d\\theta$$\n",
"\n",
"we \"intercept\" the values of the random variables in the model to set them to the mean of the posterior distribution and sample from that model to generate new data $y^*$."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "6eV4Cx0HQeMU"
},
"outputs": [],
"source": [
"def interceptor(rv_constructor, *rv_args, **rv_kwargs):\n",
" \"\"\"Replaces prior on effects with empirical posterior mean from MCMC.\"\"\"\n",
" name = rv_kwargs.pop(\"name\")\n",
" if name == \"avg_effect\":\n",
" rv_kwargs[\"value\"] = np.mean(avg_effect_, 0)\n",
" elif name == \"avg_stddev\":\n",
" rv_kwargs[\"value\"] = np.mean(avg_stddev_, 0)\n",
" elif name == \"school_effects_standard\":\n",
" rv_kwargs[\"value\"] = np.mean(school_effects_standard_, 0)\n",
" return rv_constructor(*rv_args, **rv_kwargs)\n",
"\n",
"\n",
"with ed.interception(interceptor):\n",
" posterior = schools_model(\n",
" num_schools=num_schools, treatment_stddevs=treatment_stddevs)\n",
"\n",
"with tf.Session() as sess:\n",
" posterior_predictive = sess.run(\n",
" posterior.distribution.sample(sample_shape=(5000)))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 742
},
"colab_type": "code",
"id": "y3c8W--fPmph",
"outputId": "489e1285-2dcb-456e-ca2d-43b9138a1801"
},
"outputs": [],
"source": [
"fig, axes = plt.subplots(4, 2, sharex=True, sharey=True)\n",
"fig.set_size_inches(12, 10)\n",
"fig.tight_layout()\n",
"for i, ax in enumerate(axes):\n",
" sns.kdeplot(posterior_predictive[:, 2*i], ax=ax[0], shade=True)\n",
" ax[0].title.set_text(\n",
" \"School {} treatment effect posterior predictive\".format(2*i))\n",
" sns.kdeplot(posterior_predictive[:, 2*i + 1], ax=ax[1], shade=True)\n",
" ax[1].title.set_text(\n",
" \"School {} treatment effect posterior predictive\".format(2*i + 1))\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "ATOOfzg0HMII"
},
"outputs": [],
"source": [
"# The mean predicted treatment effects for each of the eight schools.\n",
"prediction = posterior_predictive.mean(axis=0)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "MkwASzOLSgbs"
},
"source": [
"We can look at the residuals between the treatment effects data and the predictions of the model posterior. These correspond with the plot above which shows the shrinkage of the estimated effects toward the population average."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 51
},
"colab_type": "code",
"id": "ulqqNf_AHMBm",
"outputId": "1c49d020-30a1-4f06-baa0-6403c7f6970b"
},
"outputs": [],
"source": [
"treatment_effects - prediction"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "0KMqrBaGRo4S"
},
"source": [
"Because we have a distribution of predictions for each school, we can consider the distribution of residuals as well."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "7j9RAYhIRDDz"
},
"outputs": [],
"source": [
"residuals = treatment_effects - posterior_predictive"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 742
},
"colab_type": "code",
"id": "zW1RKYtBRIhd",
"outputId": "64c06680-22b0-4eab-d3a3-b0efe39d0376"
},
"outputs": [],
"source": [
"fig, axes = plt.subplots(4, 2, sharex=True, sharey=True)\n",
"fig.set_size_inches(12, 10)\n",
"fig.tight_layout()\n",
"for i, ax in enumerate(axes):\n",
" sns.kdeplot(residuals[:, 2*i], ax=ax[0], shade=True)\n",
" ax[0].title.set_text(\n",
" \"School {} treatment effect residuals\".format(2*i))\n",
" sns.kdeplot(residuals[:, 2*i + 1], ax=ax[1], shade=True)\n",
" ax[1].title.set_text(\n",
" \"School {} treatment effect residuals\".format(2*i + 1))\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "PIReUYcT0CEZ"
},
"source": [
"# Acknowledgements\n",
"\n",
"This tutorial was originally written in Edward 1.0 ([source](https://github.com/blei-lab/edward/blob/master/notebooks/eight_schools.ipynb)). We thank all contributors to writing and revising that version."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "g7cgoQ1XyqGv"
},
"source": [
"# References\n",
"1. Donald B. Rubin. Estimation in parallel randomized experiments. Journal of Educational Statistics, 6(4):377-401, 1981.\n",
"2. Andrew Gelman, John Carlin, Hal Stern, David Dunson, Aki Vehtari, and Donald Rubin. Bayesian Data Analysis, Third Edition. Chapman and Hall/CRC, 2013."
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [
"cPw5xFcq1kpw",
"TNuvn0Ih4D_R",
"cIbNcemwwO2y",
"S6Yj8WEDwI3L",
"jnVK-1yH9WCY",
"vWPCzgk7IMgt",
"PIReUYcT0CEZ",
"g7cgoQ1XyqGv"
],
"name": "Eight_Schools.ipynb",
"provenance": [],
"toc_visible": true,
"version": "0.3.2"
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.8"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Reference in new issue View Git Blame Copy Permalink