09. Logistic Regression¶

Learn how Logistic Regression solves binary classification problems by estimating probabilities and making intelligent decisions based on data.

🎯 Learning Objectives¶

After completing this chapter, you will be able to:

• Understand why Linear Regression cannot solve classification problems
• Explain the concept of Logistic Regression
• Understand the Sigmoid Function
• Learn how probability-based classification works
• Understand the Decision Boundary
• Build a Logistic Regression model using Scikit-Learn

📖 Overview¶

Although its name contains the word Regression, Logistic Regression is primarily a classification algorithm.

Instead of predicting continuous numerical values, Logistic Regression predicts the probability that an observation belongs to a particular class.

It is one of the most widely used supervised Machine Learning algorithms for binary classification problems such as fraud detection, disease diagnosis, spam detection, customer churn prediction, and credit approval.

🧠 Core Concepts¶

Logistic Regression predicts the probability of an event occurring.

Examples include:

• Will a customer churn?
• Is this email spam?
• Is a transaction fraudulent?
• Will a patient develop a disease?
• Will a loan default?

Instead of predicting numbers, Logistic Regression predicts probabilities between 0 and 1, which are then converted into class labels.

🏗️ Logistic Regression Workflow¶

flowchart LR

A[Historical Data]

--> B[Logistic Regression]

--> C[Probability]

--> D[Classification]

📘 Why Not Linear Regression?¶

Linear Regression predicts continuous values.

For classification problems, predictions must belong to predefined classes.

Example:

A Linear Regression model may predict values less than 0 or greater than 1, making it unsuitable for probability estimation.

Logistic Regression overcomes this limitation by transforming predictions into probabilities.

📈 Sigmoid Function¶

The Sigmoid Function converts any real-valued input into a probability between 0 and 1.

Characteristics:

• Smooth S-shaped curve
• Outputs values between 0 and 1
• Easy to interpret as probabilities
• Ideal for binary classification

🏗️ Sigmoid Curve¶

flowchart LR

LinearOutput

--> SigmoidFunction

--> Probability

--> Classification

📊 Probability Interpretation¶

📌 Decision Boundary¶

Once the probability is calculated, a threshold determines the final prediction.

The most common threshold is:

0.5

If:

Probability ≥ 0.5

↓

Positive Class

Otherwise

↓

Negative Class

The threshold can be adjusted depending on the business requirements.

🏗️ Classification Process¶

flowchart LR

Features

--> LogisticRegression

--> Probability

Probability --> Threshold

Threshold --> Positive

Threshold --> Negative

📈 Linear Regression vs Logistic Regression¶

🌍 Real-World Applications¶

Logistic Regression is widely used for binary classification.

🚗 Case Study¶

Customer Churn Prediction¶

A telecom company wants to identify customers who are likely to leave.

Input Features:

• Monthly Charges
• Contract Type
• Internet Usage
• Customer Tenure
• Support Tickets

↓

Logistic Regression

↓

Probability of Customer Churn

↓

Business Action

• High probability → Retention campaign
• Low probability → No action required

💻 Implementation Example¶

from sklearn.linear_model import LogisticRegression

model = LogisticRegression()

model.fit(X_train, y_train)

predictions = model.predict(X_test)

probabilities = model.predict_proba(X_test)

prediction = model.predict(customer)

if prediction == 1:
    print("Customer will churn")
else:
    print("Customer will stay")

🏢 Enterprise Perspective¶

Logistic Regression remains one of the most trusted algorithms for enterprise classification problems because it is:

• Easy to interpret
• Computationally efficient
• Fast to train
• Highly explainable
• Suitable for probability estimation

Many organizations begin with Logistic Regression before evaluating more advanced classification algorithms such as Decision Trees, Random Forests, Gradient Boosting, or Neural Networks.

💡 Best Practices¶

• Use Logistic Regression for binary classification problems.
• Scale numerical features when appropriate.
• Remove highly correlated features.
• Evaluate probability thresholds based on business requirements.
• Compare Logistic Regression with more complex classifiers before deployment.

⚠️ Common Mistakes¶

• Using Logistic Regression for regression problems.
• Assuming the default threshold is optimal.
• Ignoring class imbalance.
• Evaluating models using only accuracy.
• Using poorly engineered features.

📌 Key Takeaways¶

• Logistic Regression is a supervised classification algorithm.
• It predicts probabilities rather than continuous values.
• The Sigmoid Function converts predictions into values between 0 and 1.
• Decision thresholds determine the final class prediction.
• Logistic Regression is simple, interpretable, and widely used in production systems.

📚 Further Reading¶

The next chapter explores how regression models are trained, optimized, and evaluated using Gradient Descent, cost functions, and regression performance metrics.

➡️ Next Chapter¶

10. Training and Evaluation Regression Models