Bayes theorem

Bayes theorem
In probability theory and statistics, Bayes s theorem (alternatively Bayes s law or Bayes s rule) is a theorem with two distinct interpretations. In the Bayesian interpretation, it expresses how a subjective degree of belief should rationally change to account for evidence. In the frequentist interpretation, it relates inverse representations of the probabilities concerning two events. In the Bayesian interpretation, Bayes theorem is fundamental to Bayesian statistics, and has applications in fields including science, engineering, medicine and law. The application of Bayes theorem to update beliefs is called Bayesian inference.

Bayes theorem is named for Thomas Bayes (/?be?z/ (1701 – 1761)), who first suggested using the theorem to update beliefs. However, his work was published posthumously. His ideas gained limited exposure until they were independently rediscovered and further developed by Laplace, who first published the modern formulation in his 1812 Théorie analytique des probabilités. Until the second half of the 20th century, the Bayesian interpretation attracted widespread dissent[citation needed] from the mathematics community who generally held frequentist views,[citation needed] rejecting Bayesianism as unscientific. However, it is now widely accepted. This may have been due to the development of computing, which enabled the successful application of Bayesianism to many complex problems.[1]


Introductory example

If someone told you they had a nice conversation in the train, the probability it was a woman they spoke with is 50%. If they told you the person they spoke to was going to visit a quilt exhibition, it is far more likely than 50% it is a woman. Call the event "they spoke to a woman", and the event "a visitor of the quilt exhibition". Then: , but with the knowledge of the updated value is that may be calculated with Bayes formula as:

in which (man) is the complement of . As and , the updated value will be quite close to 1.

Statement and interpretation

Mathematically, Bayes theorem gives the relationship between the probabilities of and , and , and the conditional probabilities of given and given , and . In its most common form, it is:

The meaning of this statement depends on the interpretation of probability ascribed to the terms:

Bayesian interpretation

Main article: Bayesian probability

In the Bayesian (or epistemological) interpretation, probability measures a degree of belief. Bayes theorem then links the degree of belief in a proposition before and after accounting for evidence. For example, suppose somebody proposes that a biased coin is twice as likely to land heads than tails. Degree of belief in this might initially be 50%. The coin is then flipped a number of times to collect evidence. Belief may rise to 70% if the evidence supports the proposition.

For proposition and evidence ,
, the prior, is the initial degree of belief in .
, the posterior, is the degree of belief having accounted for .
represents the support provides for .

For more on the application of Bayes theorem under the Bayesian interpretation of probability, see Bayesian inference.