# Probability

Probability involves the analysis of random phenomena. The chance of an event occurring in a group of possible events is the probability of the event occurring in the random phenomena.

## Contents |

## Definitions

A **random experiment** is an experiment with an outcome that can not be predicted with certainty.

A **sample space**, S, is the set of all possible outcomes from a random experiment.

An **event** is the a sample point or subset of the sample space.

Sample space of the random experiment of rolling a die:

- S = {1,2,3,4,5,6}

An event of the sample space, where the sample points are odd numbers:

- E = {1,3,5}

For events, they can be manipulated to fit the required subset of the sample space using the following:

**Union**, E_{1} U E_{2}, is the events that occur in either E_{1} **or** E_{2}.

**Intersection**, E_{1} E_{2}, is the events that occur in both E_{1} **and** E_{2}.

**Complement**, E^{c}, is the events that do not occur in E.

## Probability Theory

**P(E)** is the probability of E, an event, to occur.

The axioms of probability theory includes:

- The probability of an event occurring should satisfy the following:
- 0 <= P(E) <= 1

- The probability of the sample space occurring is the following:
- P(S) = 1

- The probability of union of all mutually exclusive events is equal to the sum of the probabilities of those events.

The **frequentist** definition of probability states that the probability of an event is the proportion of times the event occurs if the experiment is repeated independently infinitely many times.

- P(E) = lim (n -> infinity) (Number of times E occurs)/n where n is the frequency of experiments

Another definition is the number of required events divided by the total number of event possible.

- P(E) = (Number of E) / (Number of E in S)

### Important Probability Rules

**Multiplication Rule:** If the events occur in a sequence, one after the other, multiplication of the probability of each step provides the final probability of the sequence occuring.

- n
_{1}x n_{2}x n_{3}x_{4}... x n_{k}

**Permutations:** This provides the amount of ways a subset of events can occur in different orders.

- n! = n x (n - 1) x (n - 2) x ... x 2 x 1

**Combinations:** This provides amount of ways a subset of events can be taken out of a sample space without repeating subsets (not considering its order).

### Conditional Probability

Conditional probability is the probability with a new condition added. In this case, it is the probability that E_{1} is in the subset of event in E_{2}

The probability of E_{1} given E_{2} means the probability of E_{1} occurring if E_{2} has occurred.

### Independent and Mutually Exclusive Events

Independence of an event occurring means that the probability of one event occurring does not affect a subsequent event from occurring.

For two events, this means P(E_{1} | E_{2}) = P(E_{1}) x P(E_{2}).

Events being mutually exclusive means that they can not occur at the same time. A good example is a coin toss, where the event can not be both heads and tails.

### Partition

A partition is a sequence of events that are a subsets of a sample space, and are mutually exclusive.

#### Law of Total Probability

For an event A in a partition {E_{1}, E_{2}, ... , E_{n}}, it can be written as the sum of the intersections of A and E_{i}.

#### Bayes' Second Rule

For any event A and E such that 0 < P(E) < 1, the following holds true:

- P(A) = P(A | E)P(E) + P(A | E
^{c})(1 - P(E))

Putting the law of total porbability into Bayes' first rule gives:

- P(E | A) = (P(A | E)P(E)) / (P(A | E)P(E) + P(A | E
^{c})(1 - P(E)))

## End

This is the end of this topic. Click here to go back to the main subject page for Numerical Methods & Statistics.