> For the complete documentation index, see [llms.txt](https://sejkai.gitbook.io/academic/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://sejkai.gitbook.io/academic/deep-learning/sequence-models.md). # Sequence Models 在 Sequence models 中 將會學到如何建立 natural language, audio, 或其他 sequence data 的模型 因為有 deep learning，sequence algorithms 才能夠快速進步 進步的專案有 * speech recognition * music synthesis * chatbots * machine translation * natural language understanding * and many others... 在這堂課還會學到 * Recurrent Neural Networks (RNNs) 和常見的變形，如 GRUs 和 LSTMs * 能夠將 sequence models 套用到 natural language problems，包含 text synthesis * 能夠將 sequence models 套用到 audio applications，包含 speech recognition 和 music synthesis 這將是 deep learning 的第五堂課 ! 另外本堂課 deeplearning.ai 還有和 NVIDIA 的 Deep Learning Institute 進行合作 所以在 assignment 中可以碰到實際的 machine translation 是能夠接近 cutting-edge 的 industry-relevant content 的機會 ## Recurrent Neural Networks 介紹什麼是 RNNs，RNNs 這類型的網路 (LSTM, GRU, BRNNs) 能夠有效處理 temporal data * Sequence models * RNNs model * Backpropogation through time * Language model and sequence generation * Sampling novel sequences * Vanishing gradients with RNNs * Gated Recurrent Unit (GRU) * Long Short Term Memory (LSTM) * Bidirectional RNN * Deep RNNs ## Natural Language Processing & Word Embeddings RNNs 和 NLP 能夠非常有效的搭配在一起 利用 word vector 和 embedding layers 來訓練 RNNs 能夠產生相當多的產業 例如 sentiment analysis, named entity recognition, machine translation * Word Representation * Word embeddings * Embedding matrix * Word2Vec * Negative Sampling * GloVe word vectors * Sentiment Classification * Debiasing word embeddings ## Sequence models & Attention mechanism 我們可以利用 attention mechanism 來強化 sequence models 這個 algorithm 告訴 model 該 focus 在哪一段 sequence of inputs * Basic Models * Beam Search * Error analysis * Bleu Score * Attention Model * Speech recognition * Trigger Word Detection --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://sejkai.gitbook.io/academic/deep-learning/sequence-models.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.