Long Short-Term Memory (LSTM) - AI Wiki

Gradient Platform Docs Get Started Free Contact Sales

Search…

Artificial Intelligence Wiki

Topics

Accuracy and Loss

Activation Function

AI Chips for Training and Inference

Artificial General Intelligence (AGI)

AUC (Area under the ROC Curve)

Automated Machine Learning (AutoML)

CI/CD for Machine Learning

Comparison of ML Frameworks

Confusion Matrix

Convolutional Neural Network (CNN)

Datasets and Machine Learning

Data Science vs Machine Learning vs Deep Learning

Distributed Training (TensorFlow, MPI, & Horovod)

Generative Adversarial Network (GAN)

Epochs, Batch Size, & Iterations

Features, Feature Engineering, & Feature Stores

Gradient Boosting

Gradient Descent

Hyperparameter Optimization

Interpretability

Jupyter Notebooks

Linear Regression

Logistic Regression

Long Short-Term Memory (LSTM)

Machine Learning Operations (MLOps)

Managing Machine Learning Models

Metrics in Machine Learning

Machine Learning Models Explained

Model Deployment (Inference)

Model Drift & Decay

Overfitting vs Underfitting

Recurrent Neural Network (RNN)

Reproducibility in Machine Learning

Serverless ML: FaaS and Lambda

Structured vs Unstructured Data

Supervised, Unsupervised, & Reinforcement Learning

Tensor Processing Unit (TPU)

Transfer Learning

Weights and Biases

Powered By GitBook

Long Short-Term Memory (LSTM)

We recommend reading the RNN article before diving in to LSTMs.
Long Short Term Memory networks (LSTMs) are a special kind of recurrent neural network (RNN) capable of learning long-term dependencies in sequence data.  LSTMs were created to overcome the inability to retain information for long periods of time, which is an inherent limitation in RNNs.
The following image illustrates the structure of a vanilla RNN:
​
LSTMs have four activation functions which are used to save pertinent information to be used in later stages of training.  
Source: Christopher Olah
The horizontal line running across the cells illustrates the cell state, a channel for information to flow across the entire chain.  The cell state can be updated if the information is deemed pertinent (regulated by gates in each cell).  With this novel refinement to RNNs, LSTMs were explicitly designed to capture, retain, and re-use key information over long sequences.  

Topics - Previous

Logistic Regression

Machine Learning Operations (MLOps)

Last modified 2yr ago

Copy link