Statistical Inference/Probability Theory

Set Theory

Sample Space

The set, S, of all possible outcomes of a particular experiment is called the sample space for the experiment. [Definition 1.1.1]

Event

An event is any collection of possible outcomes of an experiment, that is, any subset of S (including S itself). [Definition 1.1.2]

Let A be an event, a subset of S. We say the event A occurs if the outcome of the experiment is in the set A. When speaking of probabilities, we generally speak of the probability of an event, rather than a set. But we may use the terms interchangeably.

Base relationships

We first need to define formally the following two relationships, which allow us to order and equate sets:

Containment:

<amsmath>A \subset B \Leftrightarrow x \in A \Rightarrow x \in B</amsmath> 

Equality:

<amsmath>A = B \Leftrightarrow A \subset B \land B \subset A</amsmath>

Elementary operations

Given any two events (or sets) A and B , we have the following elementary set operations:

Union: The union of A and B, written <amsmath>A \cup B</amsmath>, is the set of elements that belong to either A or B or both:

<amsmath>A \cup B = \{ x : x \in A \lor x \in B \}</amsmath>

Intersection: The intersection of A and B, written <amsmath>A \cap B</amsmath>, is the set of elements that belong to both A and B:

<amsmath>A \cap B = \{ x : x \in A \land x \in B \}</amsmath>

Complementation: The complement of A, written <amsmath>\overline{A}</amsmath>, is the set of all elements that are not in A:

<amsmath>\overline{A} = \{ x : x \notin A \}</amsmath>

Event operations

The elementary set operations can be combined: for any three events, A, B, and C, defined on a sample space S, the following relationships hold [Theorem 1.1.4].

Commutativity

• <amsmath>A \cup B = B \cup A</amsmath>
• <amsmath>A \cap B = B \cap A</amsmath>

Associativity

• <amsmath>A \cup (B \cup C) = (A \cup B) \cup C</amsmath>
• <amsmath>A \cap (B \cap C) = (A \cap B) \cap C</amsmath>

Distributive Laws

• <amsmath>A \cap (B \cup C) = (A \cap B) \cup (A \cap C)</amsmath>
• <amsmath>A \cup (B \cap C) = (A \cup B) \cap (A \cup C)</amsmath>

DeMorgan Laws

• <amsmath>\overline{A \cup B} = \overline{A} \cap \overline{B}</amsmath>
• <amsmath>\overline{A \cap B} = \overline{A} \cup \overline{B}</amsmath>

Disjoint events

Two events A and B are disjoint (or mutually exclusive) [Definition 1.1.5] if <amsmath>A \cap B = \emptyset</amsmath>.

The events A1, A2, ... are pairwise disjoint (or mutually exclusive) if <amsmath>A_i \cap A_j = \emptyset, \forall_{i\ne j}</amsmath>.

Disjoint sets are sets with no points in common. If we draw a Venn diagram for two disjoint sets, the sets do not overlap. The collection <amsmath>A_i = [i, i + 1), i = 0,1,2, \dots</amsmath> consists of pairwise disjoint sets. Note further that <amsmath>\cup^\infty_{i=0} A_i = [0, \infty)</amsmath>.

Event space partitions

If A1, A2,... are pairwise disjoint and <amsmath>\cup^\infty_{i=0} A_i = S</amsmath>, then the collection A1, A2, . . . forms a partition of S. [Definition 1.1.6]

The sets <amsmath>A_i = [i, i + 1)</amsmath> form a partition of <amsmath>[0, \infty)</amsmath>. In general, partitions are very useful, allowing us to divide the sample space into small, nonoverlapping pieces.