# Probability

Probability involves the analysis of random phenomena. The chance of an event occurring in a group of possible events is the probability of the event occurring in the random phenomena.

## Definitions

A random experiment is an experiment with an outcome that can not be predicted with certainty.
A sample space, S, is the set of all possible outcomes from a random experiment.
An event is the a sample point or subset of the sample space.

Sample space of the random experiment of rolling a die:

S = {1,2,3,4,5,6}

An event of the sample space, where the sample points are odd numbers:

E = {1,3,5}

For events, they can be manipulated to fit the required subset of the sample space using the following:
Union, E1 U E2, is the events that occur in either E1 or E2.
Intersection, E1 E2, is the events that occur in both E1 and E2.
Complement, Ec, is the events that do not occur in E.

## Probability Theory

P(E) is the probability of E, an event, to occur.
The axioms of probability theory includes:

The probability of an event occurring should satisfy the following:
0 <= P(E) <= 1
The probability of the sample space occurring is the following:
P(S) = 1
The probability of union of all mutually exclusive events is equal to the sum of the probabilities of those events. The frequentist definition of probability states that the probability of an event is the proportion of times the event occurs if the experiment is repeated independently infinitely many times.

P(E) = lim (n -> infinity) (Number of times E occurs)/n where n is the frequency of experiments

Another definition is the number of required events divided by the total number of event possible.

P(E) = (Number of E) / (Number of E in S)

### Important Probability Rules

P(Ec) = 1 - P(E)
P(E1 U E2) = P(E1) + P(E2) - P(E1 E2)

Multiplication Rule: If the events occur in a sequence, one after the other, multiplication of the probability of each step provides the final probability of the sequence occuring.

n1 x n2 x n3 x4 ... x nk

Permutations: This provides the amount of ways a subset of events can occur in different orders.

n! = n x (n - 1) x (n - 2) x ... x 2 x 1

Combinations: This provides amount of ways a subset of events can be taken out of a sample space without repeating subsets (not considering its order). where n is the set of elements to be chosen from, and r is the size of the required combination.

### Conditional Probability

Conditional probability is the probability with a new condition added. In this case, it is the probability that E1 is in the subset of event in E2
The probability of E1 given E2 means the probability of E1 occurring if E2 has occurred. ### Independent and Mutually Exclusive Events

Independence of an event occurring means that the probability of one event occurring does not affect a subsequent event from occurring.
For two events, this means P(E1 | E2) = P(E1) x P(E2).
Events being mutually exclusive means that they can not occur at the same time. A good example is a coin toss, where the event can not be both heads and tails.

### Partition

A partition is a sequence of events that are a subsets of a sample space, and are mutually exclusive.

#### Law of Total Probability

For an event A in a partition {E1, E2, ... , En}, it can be written as the sum of the intersections of A and Ei.

P(A) = P(A E1) + P(A E2) + ... + P(A En)

#### Bayes' Second Rule

For any event A and E such that 0 < P(E) < 1, the following holds true:

P(A) = P(A | E)P(E) + P(A | Ec)(1 - P(E))

Putting the law of total porbability into Bayes' first rule gives:

P(E | A) = (P(A | E)P(E)) / (P(A | E)P(E) + P(A | Ec)(1 - P(E)))

## End

This is the end of this topic. Click here to go back to the main subject page for Numerical Methods & Statistics.