Machine Learning & AI · Chapter 4.10 · Pro level

Capstone: Spam Detector End-to-End + Responsible AI + Next Steps

कैपस्टोन: Spam Detector शुरू से आख़िर तक + Responsible AI + आगे का रास्ता

Final project: a spam detector like Gmail's, built end-to-end in pure Python. 1) Data — messages with the labels spam or ham (ham = a normal message). 2) Clean — lowercase the text and split it into words. 3) Train — count how often each word appears in spam and in ham. 4) Predict — a new message gets a spam score and a ham score, and the bigger side wins. 5) Evaluate — test on NEW messages and compute the accuracy. Real projects do the same with scikit-learn: CountVectorizer turns text into word counts and MultinomialNB learns from them. Responsible AI — a pro also asks three questions. Bias: if the training data has only English spam, Hinglish spam slips through — unfair data makes an unfair AI. Privacy: never collect people's chats, phone numbers or Aadhaar details without consent. Hallucination: chatbots (like ai_assistant in this lab) can confidently invent facts, so always verify. Next steps: practise in Google Colab (free Python in the browser with real scikit-learn), follow the official scikit-learn tutorials and join beginner competitions on Kaggle. Remember: model, nlp_model, NeuralNetwork and ai_assistant were learning helpers of this lab — the code in this chapter is 100% real Python.

आख़िरी प्रोजेक्ट: Gmail जैसा spam detector, pure Python में शुरू से आख़िर तक। 1) Data — spam या ham (ham = सामान्य मैसेज) लेबल वाले मैसेज। 2) सफ़ाई — टेक्स्ट को छोटे अक्षरों में करके शब्दों में तोड़ो। 3) Training — गिनो कि हर शब्द spam में और ham में कितनी बार आता है। 4) Prediction — नए मैसेज का spam score और ham score निकलता है, जो बड़ा वही जीतता है। 5) जाँच — नए मैसेज पर टेस्ट करके accuracy निकालो। असली प्रोजेक्ट scikit-learn से यही करते हैं: CountVectorizer टेक्स्ट को शब्दों की गिनती में बदलता है और MultinomialNB उससे सीखता है। Responsible AI — एक pro तीन सवाल भी पूछता है। Bias: अगर training data में सिर्फ़ English spam है, तो Hinglish spam बच निकलेगा — अनुचित data से अनुचित AI बनता है। Privacy: लोगों की चैट, फ़ोन नंबर या Aadhaar जानकारी बिना इजाज़त कभी मत इकट्ठा करो। Hallucination: chatbots (जैसे इस लैब का ai_assistant) पूरे भरोसे से झूठे तथ्य गढ़ सकते हैं, इसलिए हमेशा जाँचो। आगे का रास्ता: Google Colab में अभ्यास करो (ब्राउज़र में मुफ़्त Python, असली scikit-learn के साथ), scikit-learn के official tutorials पढ़ो और Kaggle पर शुरुआती competitions में हिस्सा लो। याद रखो: model, nlp_model, NeuralNetwork और ai_assistant इस लैब के सीखने वाले helpers थे — इस चैप्टर का कोड 100% असली Python है।

Open this chapter in the live classroom Machine Learning & AI course outline

In the classroom you can run and edit this code in your browser, get hints and mark the chapter as done. क्लास में यह code ब्राउज़र में चलाकर बदल सकते हो।

Words and symbols in this chapter

text.lower().split(): Cleaning: "FREE Prize" → ["free", "prize"].; सफ़ाई: "FREE Prize" → ["free", "prize"]।
spam_count[w] = spam_count.get(w, 0) + 1: Training: count each word (get(w, 0) gives 0 for a new word).; Training: हर शब्द गिनो (नए शब्द के लिए get(w, 0) 0 देता है)।
sum(spam_count.get(w, 0) for w in words(text)): The spam score of a new message.; नए मैसेज का spam score।
correct / len(test): Accuracy on messages the model never saw.; उन मैसेज पर accuracy जो model ने कभी नहीं देखे।
Bias / Privacy / Hallucination: Unfair data, personal data, invented answers — check all three before trusting an AI.; अनुचित data, निजी data, गढ़े हुए जवाब — AI पर भरोसा करने से पहले तीनों जाँचो।

Ready example

# STEP 1 - Data: messages with labels
train = [
    ("win a free iphone now", "spam"),
    ("claim your lottery prize today", "spam"),
    ("free recharge click this link", "spam"),
    ("congratulations you won cash", "spam"),
    ("kal school mein maths test hai", "ham"),
    ("mummy ne dinner pe bulaya", "ham"),
    ("cricket practice at 5 pm", "ham"),
    ("please share the homework notes", "ham"),
]

# STEP 2 - Clean: lowercase + split into words
def words(text):
    return text.lower().replace("!", "").split()

# STEP 3 - Train: count every word in spam and in ham
spam_count = {}
ham_count = {}
for text, label in train:
    for w in words(text):
        if label == "spam":
            spam_count[w] = spam_count.get(w, 0) + 1
        else:
            ham_count[w] = ham_count.get(w, 0) + 1

# STEP 4 - Predict: which side do the words belong to more?
def predict(text):
    spam_score = sum(spam_count.get(w, 0) for w in words(text))
    ham_score = sum(ham_count.get(w, 0) for w in words(text))
    return "spam" if spam_score > ham_score else "ham"

# STEP 5 - Evaluate on NEW messages the model never saw
test = [
    ("FREE prize click now!", "spam"),
    ("maths homework kal tak", "ham"),
    ("you won a free recharge", "spam"),
    ("dinner at 8 pm", "ham"),
]
correct = 0
for text, real in test:
    guess = predict(text)
    if guess == real:
        correct += 1
    print(f"{text:<26} -> {guess}")
print("Accuracy:", correct / len(test))

What you will see: Memory boxes show the word-count dictionaries; the terminal shows every test message with its prediction and the accuracy.

मेमोरी डिब्बों में शब्द-गिनती वाली dictionaries दिखेंगी; terminal में हर test मैसेज, उसकी prediction और accuracy छपेगी।

Line by line

train = [ ("win a free iphone now", "spam"), ... ]: Labelled examples = the dataset.; लेबल वाले उदाहरण = dataset।
for text, label in train: ...: Training = counting words for each label.; Training = हर लेबल के लिए शब्द गिनना।
def predict(text): ...: Compares the spam score with the ham score.; Spam score की तुलना ham score से करता है।
print("Accuracy:", correct / len(test)): 4 of 4 new messages correct → 1.0.; 4 में से 4 नए मैसेज सही → 1.0।

Common mistake

✗ Wrong

def words(text):
    return text.split()   # "FREE" aur "free" alag gine jayenge

✓ Right

def words(text):
    return text.lower().split()

Clean the text before counting: without .lower(), "FREE" and "free" are different words and spam slips through.

गिनने से पहले टेक्स्ट साफ़ करो: .lower() के बिना "FREE" और "free" अलग शब्द बन जाते हैं और spam बच निकलता है।

Your task

Finish the spam filter: write predict() (compare the spam and ham scores) and the evaluation loop over test that prints message -> guess and counts the correct answers. Expected: 4 predictions and Accuracy: 1.0.

Spam filter पूरा करो: predict() लिखो (spam और ham score की तुलना) और test पर जाँच वाला लूप लिखो जो message -> guess छापे और सही जवाब गिने। उम्मीद: 4 predictions और Accuracy: 1.0।

In predict() write spam_score = sum(spam_count.get(w, 0) for w in words(text)) and the same for ham_score, then return "spam" if spam_score > ham_score, else return "ham".
predict() में spam_score = sum(spam_count.get(w, 0) for w in words(text)) और वैसे ही ham_score लिखो, फिर spam_score > ham_score हो तो return "spam", वरना return "ham"।
Write for text, real in test: → guess = predict(text) → if guess == real: correct += 1 → print(text, "->", guess).
for text, real in test: → guess = predict(text) → if guess == real: correct += 1 → print(text, "->", guess) लिखो।
Run — you should see 4 lines with -> and Accuracy: 1.0.
Run करो — -> वाली 4 लाइनें और Accuracy: 1.0 दिखना चाहिए।

Starter code for the task

train = [
    ("win a free iphone now", "spam"),
    ("claim your lottery prize today", "spam"),
    ("free recharge click this link", "spam"),
    ("kal school mein maths test hai", "ham"),
    ("mummy ne dinner pe bulaya", "ham"),
    ("cricket practice at 5 pm", "ham"),
]
test = [
    ("free lottery prize click now", "spam"),
    ("maths test kal hai", "ham"),
    ("win free recharge today", "spam"),
    ("dinner at 8 pm", "ham"),
]

def words(text):
    return text.lower().split()

spam_count = {}
ham_count = {}
for text, label in train:
    for w in words(text):
        if label == "spam":
            spam_count[w] = spam_count.get(w, 0) + 1
        else:
            ham_count[w] = ham_count.get(w, 0) + 1

def predict(text):
    # TODO 1: spam_score = sum(spam_count.get(w, 0) for w in words(text))
    # TODO 2: ham_score = sum(ham_count.get(w, 0) for w in words(text))
    # TODO 3: agar spam_score > ham_score to "spam" return karo, warna "ham"
    return "ham"

correct = 0
# TODO 4: for text, real in test:  har message ka guess = predict(text) nikalo,
#         guess == real ho to correct += 1, aur print(text, "->", guess)
print("Accuracy:", correct / len(test))

Do this task in the classroom: your code is checked automatically and you get hints if something is missing.

Recap

End-to-end ML = data → clean → train → predict → evaluate on new data.
Responsible AI: look for bias in the data, protect privacy and verify AI answers — they can hallucinate.
Next: Google Colab + real scikit-learn (CountVectorizer, MultinomialNB) + Kaggle datasets and competitions.

शुरू से आख़िर तक ML = data → सफ़ाई → training → prediction → नए data पर जाँच।
Responsible AI: data में bias देखो, privacy बचाओ और AI के जवाब जाँचो — वे hallucinate कर सकते हैं।
आगे: Google Colab + असली scikit-learn (CountVectorizer, MultinomialNB) + Kaggle के datasets और competitions।

Quick quiz

A chatbot confidently gives you a fact with a made-up source. What is this called?

कोई chatbot पूरे भरोसे से एक तथ्य और गढ़ा हुआ स्रोत बता दे। इसे क्या कहते हैं?

Overfitting
Hallucination
Feature scaling
Compilation

Show the answer

Answer: B (Hallucination)

Generative AI can produce text that sounds right but is false — always verify important answers.

Generative AI ऐसा टेक्स्ट बना सकता है जो सही लगे पर झूठा हो — ज़रूरी जवाब हमेशा जाँचो।

Practice problems for this chapter

Mini Project: SMS Spam Detector from Scratch (+ Responsible AI) · Pro · Mini project
मिनी प्रोजेक्ट: शुरू से SMS Spam Detector (+ Responsible AI)

Solve them in the Practice Zone, where each answer is checked automatically and two hints are available.