Naïve Bayes Classifier

One application of Bayesian techniques is a Naïve Bayes classifier. Naïve Bayes is based on Bayes’ rule (Russell, Norvig, Canny, Malik, & Edwards, 2003, pp. 497-499).  Bayes is useful to determine the probability of an effect given a cause.  Naïve Bayes assumes that multiple causes are conditionally independent (even when they are not) to simplify the computations.  This allows multiple causes to be considered for a single effect in a highly efficient manner.  Naïve Bayes is ann established technique in classification (McCallum & Nigam, 1998).  It is often used as a comparison or baseline when evaluating other methods.

Paltoglou and Thelwall (2012) compared the effectiveness of a lexicon based sentiment analysis algorithm to that of a Bayes classifier (and other approaches) when processing social media posts.  Sentiment analysis attempts to assign a positive or negative rating to text.  It is relevant to social media due to the large quantity of user generated content.  That content may contain positive or negative reviews of products or services, making it an important research area.

Paltoglou and Thelwall (2012) utilized an unsupervised approach which leveraged an emotional dictionary from the Linguistic Inquiry and Word Count software.  This approach was selected because it does not require a training set (also referred to as a reference corpus).  The words in a given string are parsed and compared to the emotional dictionary.  The emotional dictionary contained phrases/words and associated positive and negative scores.  The score can be reversed if negation is involved.  An example of negation is “not good” which has the opposite value of “good.”

Not surprisingly, the proposed approach outperformed Naïve Bayes classification.  That said, Naïve Bayes classification performed relatively well.  Its results were typically within 10% of alternative approaches.  In several cases, it outperformed the lexicon-based approach.

While Naïve Bayes classification is not a new concept, it is a reliable one.  The results are often used as a baseline for comparison of other approaches.  While the fundamental assumption that the various causes are independent may not be true, the results are often useful.

References

McCallum, A., & Nigam, K. (1998). A comparison of event models for naive bayes text classification. Paper presented at the AAAI-98 workshop on learning for text categorization.

Paltoglou, G., & Thelwall, M. (2012). Twitter, MySpace, Digg: Unsupervised sentiment analysis in social media. ACM Trans. Intell. Syst. Technol., 3(4), 1-19. doi:10.1145/2337542.2337551

Russell, S. J., Norvig, P., Canny, J. F., Malik, J. M., & Edwards, D. D. (2003). Artificial intelligence: a modern approach (Vol. 2). http://aima.cs.berkeley.edu/: Prentice hall Upper Saddle River.