Introduction to Statistics

Experiments, Outcomes and Events

The easiest way to think of probability is in terms of experiments and their potential outcomes. Many examples can be drawn from everyday experience: On the drive home from work, you can encounter a flat tire, or have an uneventful drive; the outcome of an election can include either a win by candidate A, B, or C, or a runoff.

Definition: The entire collection of possible outcomes from an experiment is termed the sample space, indicated as <math>\Omega</math>

The simplest (albeit uninteresting) example would be an experiment with only one possible outcome, say <math>A</math>. If we remember our set theory from elementary school, we can expressed the sample space as follows:

<math>\Omega = \{ A \} </math>

A more interesting example is the result of rolling a six sided dice. The sample space for this experiment is:

<math>\Omega = \{ 1,2,3,4,5,6 \}</math>

We may be interested in events in an experiment.

Definition: An event is some subset of outcomes from the sample space

In the dice example, events of interest might include
a) the outcome is an even number
b) the outcome is less than three

These events can be expressed in terms of the possible outcomes from the experiment:
a) : <math> \{2,4,6\} </math> b) <math> \{ 1,2 \}</math>

We can borrow definitions from set theory to express events in terms of outcomes. Here is a refresher of some terminology, and some new terms that will be important later:
<math> \cup </math> represents the Union of two events
[[Image:[1]]]
<math> \cap </math> represents the Interection of two events
[[Image:[2]]]
<math>\{\cdots\}^{c}</math> represents the complement of an event. For instance, "the outcome is an even number" is the complement of "the outcome is an odd number" in the dice example.
<math>\varnothing</math> or <math>\{\}</math> represent an impossible event
<math>\Omega</math> represents a certain event
<math>A</math> and <math>B</math> are called disjoint events if <math>A\cap B = \varnothing</math>

Probability

Now that we know what events are, we should think a bit about a way to express the likelihood of an event occuring. The classical definition of probability comes from the following. If we can perform our experiment over and over in a way that is repeatable, we can count the number of times that the experiment gives rise to event <math>A</math>. We also keep track of the number of times that we perform the same experiment. If we repeat the experiment a large enough number of times, we can express the probability of event <math>A</math> as follows:
<math>P(A) = \frac{N_{A}}{N} </math>
where <math>N_{A}</math> is the number of times event <math>A</math> occurred, and <math>N</math> is the number of times the experiment was repeated. As <math>N</math> approaches infinity, the fraction above approaches the true probability of the event <math>A</math>. The value of <math>P(A)</math> is clearly between 0 and 1. If our event is the certain event <math>\Omega</math>, then for each time we perform the experiment, the event <math>\Omega</math> is observed; <math>N_{\Omega} = N</math> and <math>P(\Omega)=1</math>. If our event is the impossible event <math>\varnothing</math>, we know <math>N_{\varnothing}=0</math> and <math> P(\varnothing) = 0</math>.

If <math>A</math> and <math>B</math> are disjoint events, then whenever event <math>A</math> is observed, then it is impossible for event <math>B</math> to be observed simultaneously. Then
<math>N(A\cup B) = N(A) + N(B)</math>
Given our definition of probability, we can arrive at the following:
<math>P(A\cup B) = P(A) + P(B)</math>

At this point it's worth remembering that all events are not disjoint events. I was originally confused by events and outcomes, and this was the source of many misunderstandings. For events that are not disjoint, we end up with the following probability definition.
<math> P(A\cup B) = P(A) + P(B) - P(A\cap B)</math>
How can we see this from example? Well, let's consider drawing from a deck of cards. I'll define two events: "drawing a Queen", and "drawing a Spade".