gaussian-naive-bayes

A simple educational implementation of the Gaussian Naive Bayes Classifier written in C++. This project demonstrates how Bayes' Theorem and the Gaussian (Normal) Distribution work together to classify continuous numerical data.

The example classifies a flower as Setosa or Versicolor using its petal length.

---

What is Gaussian Naive Bayes?

Gaussian Naive Bayes is a probabilistic machine learning algorithm that predicts the most likely class for continuous numerical data.

Unlike Multinomial or Bernoulli Naive Bayes, Gaussian Naive Bayes assumes that every feature follows a Gaussian (Normal) Distribution.

Instead of counting occurrences, it asks:

"How likely is this value under each class's bell curve?"

The prediction is based on Bayes' Theorem:

P(C|X) ∝ P(C) · P(X_1|C) · P(X_2|C) · ... · P(X_n|C)

Each likelihood is computed using the Gaussian Probability Density Function:

P(X|C) = (1 / sqrt(2πσ²)) exp(-((X-μ)²)/(2σ²))

where

• μ = Mean
• σ² = Variance

---

Example

Observed petal length:

1.6 cm

Setosa

Parameter	Value
Prior	0.50
Mean	1.50
Variance	0.04

Likelihood:

≈ 1.7603

Score:

0.50 × 1.7603 = 0.8802

---

Versicolor

Parameter	Value
Prior	0.50
Mean	4.50
Variance	0.25

Likelihood:

≈ 0

Score:

≈ 0

Prediction:

SETOSA

---

Features

• Beginner-friendly implementation
• Step-by-step Gaussian probability calculation
• Demonstrates Bayes' Theorem
• Uses continuous numerical features
• No external libraries required
• Well-commented and easy to understand

---

Project Structure

.
├── gaussian_naive_bayes.cpp
└── README.md

---

Compilation

Compile using g++:

g++ gaussian_naive_bayes.cpp -o gaussian_naive_bayes

Run the program.

Windows

gaussian_naive_bayes.exe

Linux / macOS

./gaussian_naive_bayes

---

Sample Output

=====================================
 Gaussian Naive Bayes Demonstration
=====================================

Observed Petal Length = 1.600000 cm

----- Setosa -----
Prior              = 0.500000
Mean               = 1.500000
Variance           = 0.040000
Likelihood         = 1.760327
Final Score        = 0.880163

----- Versicolor -----
Prior              = 0.500000
Mean               = 4.500000
Variance           = 0.250000
Likelihood         = 0.000000
Final Score        = 0.000000

=============================
Prediction: SETOSA
=============================

---

How the Algorithm Works

1. Read the observed feature value.
2. Store the mean and variance for each class.
3. Compute the Gaussian likelihood using the Normal Distribution.
4. Multiply the likelihood by the class prior.
5. Compare the scores.
6. Predict the class with the highest probability.

---

Formula Used

Gaussian Probability Density Function:

P(X|C) = (1 / sqrt(2πσ²)) exp(-((X-μ)²)/(2σ²))

Naive Bayes Classification:

P(C|X) ∝ P(C) · P(X|C)

---

Advantages

• Very fast classification
• Excellent baseline machine learning algorithm
• Handles continuous numerical features naturally
• Requires relatively little training data
• Easy to implement and understand

---

Limitations

• Assumes features are independent
• Assumes every feature follows a Gaussian distribution
• Performance may decrease for highly correlated or non-normal data

---

Common Applications

• Iris Flower Classification
• Medical Diagnosis
• Credit Risk Assessment
• Sensor Data Analysis
• Fault Detection
• Quality Control
• Pattern Recognition

---

Learning Objectives

This project demonstrates:

• Bayes' Theorem
• Gaussian (Normal) Distribution
• Probability Density Function (PDF)
• Mean and Variance
• Posterior Probability
• Statistical Classification
• Supervised Machine Learning
• Gaussian Naive Bayes

---

Requirements

• C++11 or later
• GCC / g++
• Any standard C++ compiler

---

License

This project is licensed under the MIT License.