> For the complete documentation index, see [llms.txt](https://machine-learning.paperspace.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://machine-learning.paperspace.com/wiki/logistic-regression.md).

# Logistic Regression

![Source: Technology of Computing](/files/-LwAVc1EdfmPMge5dlYC)

Logistic regression is a machine learning algorithm used for classification problems.  The term logistic is derived from the cost function (logistic function) which is a type of **sigmoid function** known for its characteristic S-shaped curve.  A logistic regression model predicts probability values which are mapped to two (binary classification) or more (multiclass classification) classes.

![Source: Analytics India Magazine](/files/-LwAYhg8-OqxqG7aWKhN)

![Formula of a sigmoid function](/files/-LwAgUAGfE21Zw1J-gnb)

Where:

* 1 = the curve's maximum value
* *S(z)*  = output between 0 and 1 (probability estimate)
* *z* = the input
* *e* = base of natural log (also known as Euler's number)

In multiclass classification with logistic regression, a **softmax function** is used instead of the sigmoid function.  

Like [linear regression](/wiki/linear-regression.md), [gradient descent](/wiki/gradient-descent.md) is typically used to optimize the values of the coefficients (each input value or column) by iteratively minimizing the loss of the model during training.  

The **decision boundary** is the acceptable threshold at which a probability can be mapped to a discrete class e.g. pass/fail or vegan/vegetarian/omnivore.  

The cost function in logistic regression is more complex than linear regression. For example, mean squared error would yield a non-convex function with many local minimums, making it difficult to optimize with gradient descent. **Cross entropy**, also called **log loss** is the cost function used with logistic regression.

**Regularization** is a technique used to prevent [overfitting ](/wiki/overfitting-vs-underfitting.md)by penalizing signals that provide too much explanatory power to a single feature.  Regularization is extremely important in logistic regression.

[Accuracy](/wiki/accuracy-and-loss.md#accuracy), a [model evaluation metric](/wiki/metrics-in-machine-learning.md), is used to measure how accurate a model's predictions are -- this is expressed as the number of true classifications divided by the total.

### Linear vs Logistic Regression

**Linear regression** predictions are continuous (e.g. test scores from 0-100).

**Logistic regression** predictions classify items where only specific values or classes are allowed (e.g. binary classification or multiclass classification). The model provides a probability score (confidence) with each prediction.