Today’s post describes how a (semi) normal person can use Naive Bayes Classifiers (NBC) to navigate normal life. It is a natural part of life to make guesses, fill in the blanks. Our senses can only transmit so much information. So what happens when you get incomplete information? First, though, what is an NBC?
Assume you have some objects or situations that are all mutually exclusive and which are characterized by a set of attributes. An NBC is an acyclic graph in which there are nodes that describe the object and the attributes and which are connected. Each attribute value is connected to a given object by a conditional probability. Taken together, the graph is a large joint probability table of all possible attribute values and all possible objects. As an example, consider identifying birds. Perhaps you think of them as having attributes such as size, color, song, region.
Say we have the following species: [crow, robin, hummingbird, raven]. Similarly there are series of values for each of the four attributes. If the color set is [brown, red, yellow, black, green], then if the species = hummingbird, we might say that there is a 75% chance that it is green and 25% chance that it is red if the region is the Pacific Northwest, reflecting the distribution of two different types. In if the region is California, the probabilities might be different. So, the NBC works by identifying the species of a particular bird given the set of attribute values. In other words, using Bayes Theorem, the NBC works backward from the available data to calculate the probabilities of the bird being of each of the possible species. What happens if you have incomplete information? Say you see a big black bird–is it a raven or crow? They both have the attributes of being big and black. You would probably want to hear their song to distinguish them. Or knowing the region, you might be able to come up with a better guess–their regions overlap but are not identical.
Hopefully, at this point it is clear that there are times when we are using our knowledge of a situation (known as “prior probability) to help us come to a conclusion when our knowledge is incomplete. Is that weird noise in the house due to the house settling, your brother sneaking upstairs, or an axe murderer outside your door? However, in general people don’t need to do this sort of reasoning all that often, and waiting a moment will give you complete information (brother’s door closing, the axe flashing in the moonlight…).
I am interested in this topic for several reasons. For one, I think it is interesting to be aware of our decision making and how we navigate the world. I also have more personal reasons for thinking of these issues. First, I have condition known as Auditory Processing Disorder (APD) which makes it difficult for me to place sounds in the right order (perhaps similar to dyslexics). For example, I was 10 or so before I was told that the street I lived on (Silver Lane) was not the same as Civil Lane. Second, I also suffer from a moderate form of facial agnosia, which means I have difficulty identifying faces. Both of these situations mean that I have incomplete information about spoken words and in some interpersonal relations.
I have found that I do some version of an NBC in this situations. Words, in general, are not so hard because of the context (= prior probabilities). If I cannot distinguish a certain word, I can quickly identify the correct one based on the topic of conversation and the preceding and following words. For example, if I clearly heard the following word “war”, I am likely to conclude the preceding word was “civil”, not “silver”. This becomes more difficult in noisy situations where the chances of not being able to identify any given word increases. When I lived overseas, it was very difficult for me to follow conversations and to learn words from hearing them spoken. In English, a speaker’s accent causes the same confusion due to reduced information. Clearly, I have found that the more context, i.e. attributes are definitively known, then the better I am able to reason backwards and classify the word(s) correctly. Foreign language is hard because I don’t have as many known attributes and even fewer probabilities.
I bring up facial agnosia because the set of attributes span a much larger range of possibilities. For example, there are characteristics of the face and hair, the attached body, the social situation, the physical location, etc. A situation occurred the other day which actually is one of the reasons I am writing this post. I needed to talk to a colleague. One the placard on the door, there were three names listed: two women and one man. When I opened the door, one desk was empty, one had a woman at it and the third a man. The only attributes of the person I could remember was that she was female, she was neither very young nor old, her hair was of some vaguely dark color and she was of European ancestry (by appearance). The two people looked at me and I asked by name if the person was there. Since I could not identify her face and I had no information about the missing person other than she was female, I couldn’t know if the woman was who I was seeking. The woman did exhibit the attributes I knew of, but they were so common, that the classification probability was very uncertain. There was a bit of uncomfortable silence until she finally said that she was that person. She clearly knew who I was and I clearly did not recognize her–not a great social success.
So, I hope that I have demonstrated the great utility of NBC and that we are all a bit smarter than perhaps we think we are since we can do these arcane calculations in our heads! (Of course, sometimes we get them wrong to our great embarrassment)
Bayes' blog 