Academic
  • Introduction
  • Artificial Intelligence
    • Introduction
    • AI Concepts, Terminology, and Application Areas
    • AI: Issues, Concerns and Ethical Considerations
  • Biology
    • Scientific Method
    • Chemistry of Life
    • Water, Acids, Bases
    • Properties of carbon
    • Macromolecules
    • Energy and Enzymes
    • Structure of a cell
    • Membranes and transport
    • Cellular respiration
    • Cell Signaling
    • Cell Division
    • Classical and molecular genetics
    • DNA as the genetic material
    • Central dogma
    • Gene regulation
  • Bioinformatics
    • Bioinformatics Overview
  • Deep Learning
    • Neural Networks and Deep Learning
      • Introduction
      • Logistic Regression as a Neural Network
      • Python and Vectorization
      • Shallow Neural Network
      • Deep Neural Network
    • Improving Deep Neural Networks
      • Setting up your Machine Learning Application
      • Regularizing your Neural Network
      • Setting up your Optimization Problem
      • Optimization algorithms
      • Hyperparameter, Batch Normalization, Softmax
    • Structuring Machine Learning Projects
    • Convolutional Neural Networks
      • Introduction
    • Sequence Models
      • Recurrent Neural Networks
      • Natural Language Processing & Word Embeddings
      • Sequence models & Attention mechanism
  • Linear Algebra
    • Vectors and Spaces
      • Vectors
      • Linear combinations and spans
      • Linear dependence and independence
      • Subspaces and the basis for a subspace
      • Vector dot and cross products
      • Matrices for solving systems by elimination
      • Null space and column space
    • Matrix transformations
      • Functions and linear transformations
      • Linear transformation examples
      • Transformations and matrix multiplication
      • Inverse functions and transformations
      • Finding inverses and determinants
      • More Determinant Depth
  • Machine Learning
    • Introduction
    • Linear Regression
      • Model and Cost Function
      • Parameter Learning
      • Multivariate Linear Regression
      • Computing Parameters Analytically
      • Octave
    • Logistic Regression
      • Classification and Representation
      • Logistic Regression Model
    • Regularization
      • Solving the Problem of Overfitting
    • Neural Networks
      • Introduction of Neural Networks
      • Neural Networks - Learning
    • Improve Learning Algorithm
      • Advice for Applying Machine Learning
      • Machine Learning System Design
    • Support Vector Machine
      • Large Margin Classification
      • Kernels
      • SVM in Practice
  • NCKU - Artificial Intelligence
    • Introduction
    • Intelligent Agents
    • Solving Problems by Searching
    • Beyond Classical Search
    • Learning from Examples
  • NCKU - Computer Architecture
    • First Week
  • NCKU - Data Mining
    • Introduction
    • Association Analysis
    • FP-growth
    • Other Association Rules
    • Sequence Pattern
    • Classification
    • Evaluation
    • Clustering
    • Link Analysis
  • NCKU - Machine Learning
    • Probability
    • Inference
    • Bayesian Inference
    • Introduction
  • NCKU - Robotic Navigation and Exploration
    • Kinetic Model & Vehicle Control
    • Motion Planning
    • SLAM Back-end (I)
    • SLAM Back-end (II)
    • Computer Vision / Multi-view Geometry
    • Lie group & Lie algebra
    • SLAM Front-end
  • Python
    • Numpy
    • Pandas
    • Scikit-learn
      • Introduction
      • Statistic Learning
  • Statstics
    • Quantitative Data
    • Modeling Data Distribution
    • Bivariate Numerical Data
    • Probability
    • Random Variables
    • Sampling Distribution
    • Confidence Intervals
    • Significance tests
Powered by GitBook
On this page
  • Recurrent Neural Networks
  • Natural Language Processing & Word Embeddings
  • Sequence models & Attention mechanism

Was this helpful?

  1. Deep Learning

Sequence Models

在 Sequence models 中

將會學到如何建立 natural language, audio, 或其他 sequence data 的模型

因為有 deep learning，sequence algorithms 才能夠快速進步

進步的專案有

  • speech recognition

  • music synthesis

  • chatbots

  • machine translation

  • natural language understanding

  • and many others...

在這堂課還會學到

  • Recurrent Neural Networks (RNNs) 和常見的變形，如 GRUs 和 LSTMs

  • 能夠將 sequence models 套用到 natural language problems，包含 text synthesis

  • 能夠將 sequence models 套用到 audio applications，包含 speech recognition 和 music synthesis

這將是 deep learning 的第五堂課 !

另外本堂課 deeplearning.ai 還有和 NVIDIA 的 Deep Learning Institute 進行合作

所以在 assignment 中可以碰到實際的 machine translation

是能夠接近 cutting-edge 的 industry-relevant content 的機會

Recurrent Neural Networks

介紹什麼是 RNNs，RNNs 這類型的網路 (LSTM, GRU, BRNNs) 能夠有效處理 temporal data

  • Sequence models

  • RNNs model

  • Backpropogation through time

  • Language model and sequence generation

  • Sampling novel sequences

  • Vanishing gradients with RNNs

  • Gated Recurrent Unit (GRU)

  • Long Short Term Memory (LSTM)

  • Bidirectional RNN

  • Deep RNNs

Natural Language Processing & Word Embeddings

RNNs 和 NLP 能夠非常有效的搭配在一起

利用 word vector 和 embedding layers 來訓練 RNNs 能夠產生相當多的產業

例如 sentiment analysis, named entity recognition, machine translation

  • Word Representation

  • Word embeddings

  • Embedding matrix

  • Word2Vec

  • Negative Sampling

  • GloVe word vectors

  • Sentiment Classification

  • Debiasing word embeddings

Sequence models & Attention mechanism

我們可以利用 attention mechanism 來強化 sequence models

這個 algorithm 告訴 model 該 focus 在哪一段 sequence of inputs

  • Basic Models

  • Beam Search

  • Error analysis

  • Bleu Score

  • Attention Model

  • Speech recognition

  • Trigger Word Detection

PreviousIntroductionNextRecurrent Neural Networks

Last updated 5 years ago

Was this helpful?