TensorFlow and Deep Learning Tutorials

Google's Deep Learning Tutorials

• TensorFlow Official Deep Learning Tutorial [中文].
• MLP with Dropout TensorFlow [中文] TensorLayer [中文]
• Autoencoder TensorLayer [中文]
• Convolutional Neural Network TensorFlow [中文] TensorLayer [中文]
• Recurrent Neural Network TensorFlow [中文] TensorLayer [中文]
• Deep Reinforcement Learning TensorLayer [中文]
• Sequence to Sequence TensorFlow TensorLayer[中文]
• Word Embedding TensorFlow [中文] TensorLayer [中文]

Deep Learning Reading List

• MIT Deep Learning Book
• Karpathy Blog
• Stanford UFLDL Tutorials
• Colah's Blog - Word Embedding [中文]
• Colah's Blog - Understand LSTN [门函数]

Tutorial index

0 - Prerequisite

• Introduction to Machine Learning (notebook)
• Introduction to MNIST Dataset (notebook)

1 - Introduction

• Hello World (notebook) (code)
• Basic Operations (notebook) (code)

2 - Basic Models

• Nearest Neighbor (notebook) (code)
• Linear Regression (notebook) (code)
• Logistic Regression (notebook) (code)

3 - Neural Networks

• Multilayer Perceptron (notebook) (code)
• Convolutional Neural Network (notebook) (code)
• Recurrent Neural Network (LSTM) (notebook) (code)
• Bidirectional Recurrent Neural Network (LSTM) (notebook) (code)
• Dynamic Recurrent Neural Network (LSTM) (code)
• AutoEncoder (notebook) (code)

4 - Utilities

• Save and Restore a model (notebook) (code)
• Tensorboard - Graph and loss visualization (notebook) (code)
• Tensorboard - Advanced visualization (code)

5 - Multi GPU

• Basic Operations on multi-GPU (notebook) (code)

Dataset

Some examples require MNIST dataset for training and testing. Don't worry, this dataset will automatically be downloaded when running examples (with input_data.py). MNIST is a database of handwritten digits, for a quick description of that dataset, you can check this notebook.

Official Website: http://yann.lecun.com/exdb/mnist/

Selected Repositories

• jtoy/awesome-tensorflow
• nlintz/TensorFlow-Tutoirals
• adatao/tensorspark
• ry/tensorflow-resnet

Tricks

• Tricks to use TensorLayer

Examples

Basics

• Multi-layer perceptron (MNIST). A multi-layer perceptron implementation for MNIST classification task, see tutorial_mnist_simple.py here.

Computer Vision

• Denoising Autoencoder (MNIST). A multi-layer perceptron implementation for MNIST classification task, see tutorial_mnist.py here.
• Stacked Denoising Autoencoder and Fine-Tuning (MNIST). A multi-layer perceptron implementation for MNIST classification task, see tutorial_mnist.py here.
• Convolutional Network (MNIST). A Convolutional neural network implementation for classifying MNIST dataset, see tutorial_mnist.py here.
• Convolutional Network (CIFAR-10). A Convolutional neural network implementation for classifying CIFAR-10 dataset, see tutorial_cifar10.py here.
• VGG 16 (ImageNet). A Convolutional neural network implementation for classifying ImageNet dataset, see tutorial_vgg16.py here.
• VGG 19 (ImageNet). A Convolutional neural network implementation for classifying ImageNet dataset, see tutorial_vgg19.py here.

Natural Language Processing

• Recurrent Neural Network (LSTM). Apply multiple LSTM to PTB dataset for language modeling, see tutorial_ptb_lstm.py here.
• Word Embedding - Word2vec. Train a word embedding matrix, see tutorial_word2vec_basic.py here.
• Restore Embedding matrix. Restore a pre-train embedding matrix, see tutorial_generate_text.py here.
• Text Generation. Generates new text scripts, using LSTM network, see tutorial_generate_text.py here.
• Machine Translation (WMT). Translate English to French. Apply Attention mechanism and Seq2seq to WMT English-to-French translation data, see tutorial_translate.py here.

Reinforcement Learning

• Deep Reinforcement Learning - Pong Game. Teach a machine to play Pong games, see tutorial_atari_pong.py here.

Useful Links

• Tricks to use TensorLayer