## Saturday, August 7, 2010

### Our friend Bayes -- Part I

Our friend Bayes, Thomas Bayes, late of the 18th century, an Englishman, was a mathematician and a pastor who's curiosity led him to ponder the nature of random events.

There was already a body of knowledge about probabilities by his time, so curious Bayes went at probability in a different way. Until Bayes came along, probability was a matter of frequency:
"How many times did an event happen/how many times could an event happen". In other words, "actual/opportunity".

To apply this definition in practice, certain, or "calibrated", information is needed about the opportunity, and of course actual outcomes are needed, often several trials of actual outcomes.

Bayes' Insight
Recognizing the practicalities of obtaining the requisite information, brother Bayes decided, more or less, to look backward from actual observations to ascertain and understand conditions that influenced the actual outcomes, and might influence future outcomes.

So Bayes developed his own definition of probability that is not frequency and trials oriented, but it does require an actual observation. Bayes’ definition of probability, somewhat paraphrased, is that probability is...
The ratio of expected value before an event happens to the actual observed value at the time the event happens.

This way of looking at probability is really a bet on an outcome based on [mostly subjective] evaluations of circumstances that might lead to that outcome. It's a ratio of values, rather than a frequency ratio.

Bayes' Theorem
He developed a widely known explanation of his ideas [first published after his death] that have become known as Bayes' Theorem. Used quantitatively [rather qualitatively as Bayes himself reasoned], Bayesian reasoning begins with an observation and works backward through a set of mathematical functions to arrive at the underlying probabilities.

To use his theorem, information about two probabilistic events is needed:

One event, call it 'A', must be independent of outcomes, but otherwise has some influence over outcomes. For example, 'A' could be the weather. The weather seems to go its own way most of the time. Specifically 'good weather' is the event 'A+', and 'bad weather' is the event 'A-'.

The second event, call it 'B', is hypothesized to have some dependency on 'A'. [This is Bayes' 'bet' on the future value] For example, project test results in some cases could be weather dependent. Specifically, 'B+' is the event 'good test result' and 'B-' is a bad test result;  test results could depend on the weather, but not the other way 'round.

Project Questions
Now situation we have described raises some interesting questions:
• What is the likelihood of B+, given A+?
• What are the prospects for B+ if A+ doesn't happen?
• Is there a way to estimate the likelihood of B+ or B- given any condition of A?
• Can we validate that B indeed depends on A?

Bayes' Grid
Curious Bayes [or those who came after him] realized that a "Bayes' Grid", a 2x2 matrix, could help sort out functional relationships between the 'A' space and the 'B' space. Bayes' Grid is a device that simplifies the reasoning, provides a visualization of the relationships, and avoids dealing directly with equations of probabilities.

Since there's a lot detail behind Bayes' Grid, we'll take up those details in Part II of this series.

Photo credit: Wikipedia Bookmark this on Delicious