naive-bayes-classifier

A simple C++ implementation of the Naive Bayes Classifier for educational purposes. This project demonstrates how Bayes' Theorem can be used to classify an email as Spam or Not Spam using predefined probabilities.

Overview

Naive Bayes is a probabilistic machine learning algorithm based on Bayes' Theorem. It predicts the most likely class by calculating the probability of each class given the observed features.

The algorithm assumes that all features are independent of one another, which is why it is called Naive.

The classifier computes:

P(C|X) ∝ P(C) · P(X_1|C) · P(X_2|C) · ... · P(X_n|C)

Where:

• C = Class (Spam or Not Spam)
• X = Input features (words in the email)

---

Example

Email:

Win a free iPhone!

Given:

Probability	Value
P(Spam)	0.4
P("win" | Spam)	0.8
P("free" | Spam)	0.7
P(Not Spam)	0.6
P("win" | Not Spam)	0.1
P("free" | Not Spam)	0.2

Spam Score:

0.4 × 0.8 × 0.7 = 0.224

Not Spam Score:

0.6 × 0.1 × 0.2 = 0.012

Since 0.224 > 0.012, the prediction is:

SPAM

---

Features

• Beginner-friendly C++ implementation
• Demonstrates Bayes' Theorem step-by-step
• Simple spam classification example
• Well-commented and easy to modify
• No external libraries required

---

Project Structure

.
├── naive_bayes.cpp
└── README.md

---

Compilation

Using g++:

g++ naive_bayes.cpp -o naive_bayes

Run the program:

Windows

naive_bayes.exe

Linux / macOS

./naive_bayes

---

Sample Output

=== Naive Bayes Spam Classifier ===

Email contains: win free

Spam Score      = 0.224
Not Spam Score  = 0.012

Prediction: SPAM

---

How It Works

1. Define the prior probabilities for each class.
2. Define the likelihood of each word given each class.
3. Multiply the prior by the likelihoods.
4. Compare the resulting scores.
5. Choose the class with the highest probability.

---

Advantages

• Simple to understand
• Fast to compute
• Excellent for text classification
• Works well with small datasets

---

Limitations

• Assumes feature independence
• Performance may decrease when features are highly correlated

---

Learning Objectives

This project helps you understand:

• Bayes' Theorem
• Naive Bayes Classification
• Prior Probability
• Likelihood
• Posterior Probability
• Basic Machine Learning Concepts
• Probabilistic Classification

---

License

This project is released under the MIT License.