Naive Bayes for Text
Naive Bayes is the classic text model. It is naive because it assumes words are independent, which is false, but it still works well for spam, sentiment, and topic classification and it trains in seconds.
You will pair it with a text vectorizer that turns sentences into counts. CountVectorizer or TfidfVectorizer makes each word a feature. The model then learns how often spam uses “lottery” versus ham.
It is a strong baseline: try Naive Bayes first, then see if a heavier model is worth it. Often it is not.
From text to numbers
Vectorize first, then classify. Use a pipeline so the vocabulary is learned from train only.
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
pipe = Pipeline([("tfidf", TfidfVectorizer()), ("nb", MultinomialNB())])
pipe.fit(train_texts, y_train)
pipe.predict(["free lottery now"] )When to use it
Use it for short text, small data, and fast iteration. For long documents or context, try linear SVM or transformers.
pipe.score(test_texts, y_test)Syntax
from sklearn.naive_bayes import MultinomialNB
from sklearn.feature_extraction.text import TfidfVectorizer
pipe = Pipeline([("tfidf", TfidfVectorizer()), ("nb", MultinomialNB())])
pipe.fit(X_train_text, y_train)TF-IDF vs CountVectorizer matters
CountVectorizer counts, Tfidf downweights common words like “the”. On spam data Tfidf often gains 5-10% F1. This GitHub scikit-learn issue is the top “why my Naive Bayes is bad” fix.
TfidfVectorizer(stop_words="english", max_features=5000) # try vs CountVectorizer()Good to know
- Naive means it assumes words are independent.
- Pair with TfidfVectorizer for text.
- Great baseline, trains in seconds.