Improving Deep Neural Networks

在 Improving deep neural networks 中

將學習到 deep learning 中的 "magic"

而不只是當成 black box 一樣使用他

知道如何控制 performance, 並且有系統性的取得 good results

• 了解業界在 build dl application 的 best-practices

• 能夠使用 common neural network tricks

  • initialization, L2, dropout regularization, batch normalization, gradient checking

• 能夠應用各種優化演算法

  • mini-batch gradient descent, momentum, RMSprop, Adam, check convergence

• 知道如何設定 new best-practices

  • train/dev/test sets

  • analyze bias/variance

• 會用 TensorFlow

這將是 Deep Learning 的第二堂課 !

Practical aspects of Deep Learning

• Setting up your Machine Learning Application

  • train / dev / test sets

  • bias / variance

• Regularizing your neural network

  • regularization

  • dropout

  • L2

• Setting up your optimization problem

  • normalizing inputs

  • vanishing / exploding gradients

  • weight initialization

  • gradient checking

Optimization algorithms

• mini-batch gradient descent

• exponentially weighted averages

• bias correction

• momentum

• RMSprop

• Adam

• learning rate decay

Hyperparameter tuning, Batch Normalization and Programming Frameworks

• Hyperparameter tuning

  • appropriate scale to pick hyperparameter

  • pandas, caviar practice

• Batch normalization

  • normalize activations

  • fitting batch norm

• Multi-class classification

  • softmax regression

  • softmax classifier

• Introduction to programming frameworks

  • deep learning frameworks

  • tensorflow