#!/usr/bin/env python
# coding: utf-8

# # Lecture 1: Probability and Counting
# 
# ## Stat 110, Prof. Joe Blitzstein, Harvard University
# 
# 
# ----

# ## Definitions
# 
# We start with some basic definitions:
# 
# #### Definition: sample space
# 
# > A __sample space__ is the set of all possible outcomes of an experiment.
# 
# 
# 
# #### Definition: event
# 
# > An __event__ is a subset of the sample space.
# 
# 
# #### Definition: naïve definition of probability
# > Under the __naïve definition of probability__, the probability of a given event $A$ occurring is expressed as
# >
# > \begin\{align\}
# >   P(A) &= \frac{ \text{# favorable outcomes}}{\text{# possible outcomes}}
# > \end\{align\}
# >
# > assuming all outcomes are equally likely in a finite sample space.
# 
# ----

# ## Counting
# 
# With the __multiplication rule__, if we have an experiment with $n_1$ possible outcomes; and we have a 2<sup>nd</sup> experiment with $n_2$ possible outcomes; ..., and for the r<sup>th</sup> experiment there are $n_r$ possible outcomes; then overall there are $n_1 n_2 ... n_r$ possible outcomes (product).
# 
# Let's say you are ordering ice cream. You can either get a cone or a cup, and the ice cream comes in three flavors. The order of choice here does not matter, and the total number of choices is $2 \times 3 = 3 \times 2 = 6$. This can be represented with a very simple [probability tree][1].
# 
# [1]: https://en.wikipedia.org/wiki/Tree_diagram_(probability_theory)

# ![title](images/L0101.png)

# The __binomial coefficient__  is defined as 
# 
# \begin{align}
#   \binom{n}{k} =
#     \begin{cases}
#       \frac{n!}{(n-k)!k!} & \quad \text{if } 0 \le k \le n \\
#       0 & \quad \text{if } k \gt n
#     \end{cases}
# \end{align}
# 
# This expresses the number of ways you could choose a subset of size $k$ from $n$ items, where order doesn't matter.
# 
# ----

# ## Sampling
# 
# Choose $k$ objects out of $n$
# 
# |           | ordered | unordered |
# |-----------|:---------:|:-----------:|
# | __w/ replacement__   | $n^k$     |   ???     |
# | __w/o replacement__  | $n(n-1)(n-2) \ldots (n-k+1)$ | $\binom{n}{k}$  |
# 
# 
# * __ordered, w/ replacement__: there are $n$ choices for each $k$, so this follows from the multiplication rule.
# * __ordered, w/out replacement__: there are $n$ choices for the 1<sup>st</sup> position; $n-1$ for the 2<sup>nd</sup>; $n-2$ for the 3<sup>rd</sup>; and $n-k+1$ for the $k$<sup>th</sup>.
# * __unordered, w/ replacement__: ???
# * __unordered, w/out replacement__: the binomial coefficient; think of choosing a hand from a deck of cards.
# 
# Out of the 4 ways of choosing $k$ objects out of $n$, the case of __unordered, with replacement__ is perhaps not as clear-cut and easy to grasp as the other three. Move on to Lecture 2.
# 
# -----

# View [Lecture 1: Probability and Counting | Statistics 110](http://bit.ly/2vSEEeI) on YouTube.