Naive Bayes Cheat Sheet
A reference for Naive Bayes covering Gaussian, Multinomial, and Bernoulli variants in scikit-learn, Bayes' theorem, and Laplace smoothing.
1 PageBeginnerMar 15, 2026
GaussianNB
For continuous, normally distributed features.
python
from sklearn.naive_bayes import GaussianNBmodel = GaussianNB()model.fit(X_train, y_train)y_pred = model.predict(X_test)proba = model.predict_proba(X_test) # assumes features are normal per class
MultinomialNB for Text
A classic pipeline for text classification.
python
from sklearn.feature_extraction.text import CountVectorizerfrom sklearn.naive_bayes import MultinomialNBfrom sklearn.pipeline import make_pipelinetext_clf = make_pipeline( CountVectorizer(stop_words='english'), MultinomialNB(alpha=1.0) # alpha: Laplace/Lidstone smoothing)text_clf.fit(train_texts, train_labels)predictions = text_clf.predict(test_texts)
BernoulliNB
For binary/boolean feature vectors.
python
from sklearn.naive_bayes import BernoulliNBmodel = BernoulliNB(alpha=1.0, binarize=0.0) # binary/boolean feature presencemodel.fit(X_train, y_train)
Key Concepts
Core theory behind Naive Bayes.
- Bayes' theorem- P(y given x) is proportional to P(x given y) times P(y): combines likelihood and prior into a posterior
- Conditional independence- The 'naive' assumption that features are independent given the class; rarely true but works well in practice
- Laplace smoothing (alpha)- Prevents zero probabilities for feature/class combinations unseen during training
- GaussianNB- Assumes continuous features follow a normal distribution within each class
- MultinomialNB- Best for discrete counts, such as word frequencies in text classification
- BernoulliNB- Best for binary/boolean features, such as word presence or absence
ComplementNB for Imbalanced Text
A variant designed specifically to correct MultinomialNB's bias toward the majority class on skewed text datasets.
python
from sklearn.naive_bayes import ComplementNBfrom sklearn.feature_extraction.text import TfidfVectorizerfrom sklearn.pipeline import make_pipeline# ComplementNB estimates parameters from the COMPLEMENT of each class,# which stabilizes weights when class frequencies are highly skewedclf = make_pipeline( TfidfVectorizer(sublinear_tf=True, min_df=2), ComplementNB(alpha=1.0, norm=True),)clf.fit(train_texts, train_labels)
Incremental Learning with partial_fit
Train Naive Bayes on data streams or datasets too large to fit in memory, one batch at a time.
python
from sklearn.naive_bayes import MultinomialNBimport numpy as npclf = MultinomialNB(alpha=1.0)all_classes = np.unique(y) # must be supplied on the first callfor X_batch, y_batch in stream_of_batches(batch_size=2000): clf.partial_fit(X_batch, y_batch, classes=all_classes)# Later batches only need X_batch, y_batch (classes is remembered)clf.partial_fit(X_next_batch, y_next_batch)
Manual Log-Space Posterior Computation
Avoid floating-point underflow by working in log-probabilities, exactly as scikit-learn does internally.
python
import numpy as npdef predict_log_proba_gaussian(x, means, vars_, log_priors): # log N(x; mu, sigma^2) summed over independent features log_likelihood = -0.5 * np.sum( np.log(2 * np.pi * vars_) + ((x - means) ** 2) / vars_, axis=1 ) log_joint = log_likelihood + log_priors # log P(x|y) + log P(y) log_norm = np.logaddexp.reduce(log_joint) # log-sum-exp for stability return log_joint - log_norm # log posterior per class# Equivalent to model.predict_log_proba(x) for a fitted GaussianNB
CategoricalNB for Discrete Features
Model categorical (non-ordinal, non-count) features directly without one-hot encoding them into a sparse count matrix.
python
from sklearn.naive_bayes import CategoricalNBfrom sklearn.preprocessing import OrdinalEncoder# CategoricalNB expects non-negative integer category codes, not raw stringsX_encoded = OrdinalEncoder(dtype=int).fit_transform(X_categorical)model = CategoricalNB(alpha=1.0, min_categories=None)model.fit(X_encoded, y_train)# feature_log_prob_[i] holds log P(feature_i=category | class) per classprint(model.feature_log_prob_[0].shape)
Advanced Concepts
Internals and practical caveats beyond the three standard variants.
- feature_log_prob_- Fitted attribute holding log P(feature | class); inspecting it directly reveals which features most separate classes without needing predict()
- class_prior override- Pass class_prior=[...] to replace the empirical P(y) estimate, useful when training data class balance doesn't match deployment reality
- Zero-frequency problem- Without smoothing, any unseen feature/class combination collapses the entire product of likelihoods to zero regardless of other evidence
- alpha as Bayesian prior- Laplace/Lidstone smoothing is equivalent to placing a symmetric Dirichlet prior over the multinomial parameters; alpha=1 is add-one smoothing, alpha<1 is a weaker prior
- Poor probability calibration- Because independence rarely holds, predicted probabilities are often pushed toward 0 or 1 even when the classifier's rankings/decisions are accurate
- ComplementNB- Estimates weights from all classes EXCEPT the target class, which reduces the bias MultinomialNB shows toward classes with more training examples
- GaussianNB.var_smoothing- Adds a small fraction of the largest feature variance to all variances, preventing division-by-zero on near-constant features
Pro Tip
Naive Bayes is a fast, strong baseline for text classification despite its unrealistic independence assumption — but its predicted probabilities are often poorly calibrated even when the predicted class label is correct, so don't trust predict_proba() outputs at face value.
Was this cheat sheet helpful?
Explore Topics
#NaiveBayes#NaiveBayesCheatSheet#DataScience#Beginner#GaussianNB#MultinomialNBForText#BernoulliNB#KeyConcepts#MachineLearning#CheatSheet#SkillVeris
