Theory, Papers of Deep Learning DL

2017-03-10. Category & Tags: Deep Learning, DL, Theory

MIXED TUTORIAL + TRIKCS #

Google: TF, DL, Tutorial, Tricks, etc
The Ultimate Course and Book list to be an expert in Mathematics and Programming (Content: Discrete Mathematics, Linear Algebra, Calculus, Probability, Cryptography, Geometry and Statistics. 45 Mathematics Courses)
Deep Learning and Reinforcement Learning Summer School 2018 - Yoshua Bengio etc.
深度学习课程笔记 by Tess Ferrandez, recommended by Andrew Ng

PAPERS, TERMS & DEFINATIONS #

一文读懂自注意力机制：8 大步骤图解+代码 ref: Illustrated: Self-Attention – Step-by-step guide to self-attention with illustrations and code
[典型/经典 DL/NLP 模型] 读完这 45 篇论文，“没人比我更懂 AI 了”
如何开启深度学习之旅？这三大类 125 篇论文为你导航
Github DL papers DeepLearningBook/DeepLearningPapers.md
Papers Milestone AndrewYuan.github.io/
DL without PhD By Google Cloud NEXT2017. backup evernote
浅析 Hinton 最近提出的 Capsule 计划
一起读懂传说中的经典： Restricted Boltzmann Machine(RBM) 受限波尔兹曼机
RWA in ostmeyer2017machine ( 代码实现了一个用于处理连续数据的新型 RNN 模型，该模型计算每个预处理步骤的复现加权平均值（RWA）。使用这种方法，模型可以沿着序列方向，在任何位置形成直接连接。这与仅使用预处理的传统 RNN 架构相反。 )
[DL 基本理论] 邹晓艺专栏 |Deep Learning 系列笔记 2013
TinyBERT：模型小 7 倍，速度快 8 倍，华中科大、华为出品. tip: 1~2min with a single NVIDIA K80 GPU. (see also FastBert in /dl-do-it)

ToC:
一 ~ 三、概述, 背景, 人脑视觉机理
四、关于特征: 特征表示的粒度; 初级（浅层）特征表示; 结构性特征表示; 需要有多少个特征？
五 ~ 七、Deep Learning 的基本思想 (vs. Shallow Learning), Neural Network
八、Deep learning 训练过程: 传统神经网络 vs. deep learning
九、Deep Learning 的常用模型或者方法: AutoEncoder 自动编码器; Sparse Coding 稀疏编码; Restricted Boltzmann Machine(RBM) 受限波尔兹曼机; Deep BeliefNetworks 深信度网络; Convolutional Neural Networks 卷积神经网络
十、总结与展望
十一、参考文献和 Deep Learning 学习资源

EBOOK #

邱锡鹏教授 2020 神经网络与深度学习 #

Keywords: CNN, RNN, Attention, Gaussian Mix., RBM, DBN, GAN, RL, TF, PyTorch: equation, pseudocode, exercise …
PDF, PPT, code, exercise & solution, etc.

python data science handbook - 2017 #

[easy to find common examples]
Jupyter Notebooks for the Python Data Science Handbook

stanford deep learning tutorial #

This tutorial will teach you the main ideas of Unsupervised Feature Learning and Deep Learning. By working through it, you will also get to implement several feature learning/deep learning algorithms, get to see them work for yourself, and learn how to apply/adapt these ideas to new problems.

mathematics for computer science – mit 麻省理工的英文原版教材 #

tensorflow cook book #

Nick McClure - TensorFlow Machine Learning Cookbook: code, (similar simple cn cases)

datascience from scratch - 2014 #

In this book, you’ll learn how many of the most fundamental data science tools and algorithms work by implementing them from scratch.

周志华机器学习 2016 #

入门的书籍，书中的例子十分形象且简单易懂。

the deep learning textbook - 2015 #

It helps students and practitioners enter the field of machine learning in general and deep learning in particular.

TOOLS #

VISDOM: DATA VISUALIZATION #

For Torch/Numpy, by Facebook. github
A flexible tool for creating, organizing, and sharing visualizations of live, rich data. Supports Torch and Numpy.’

TALKS & PRESENTATIONS #

< machine learning: testing and error metrics > #

model selection & improvement.
by Luis Serrano Udacity @ youtube

weight Initialization in deep learning #

by 夏飞@google LeiPhone

GAN FAQ #

Generative Adversarial Networks (GANs), Some Open Questions/problems by Sanjeev Arora. See their next blog for solutions.

See sections below for GAN details.

多源数据融合与时空数据挖掘 #

微软研究院资深主任研究员郑宇
（一） (evernote)

Practical Deep Learning For Coders Part 1 #

Taught by Jeremy Howard (Kaggle’s #1 competitor 2 years running, and founder of Enlitic).
fast.ai

CNN #

CNN, ImageNet: CNN 浅析和历年 ImageNet 冠军模型解析
 How do Convolutional Neural Networks work?
cn
Insights into AlexNet
CNN explained, with python, analyticsvidhya 2016
CNN 图像分割简史：从 R-CNN 到 Mask R-CNN

cnn visualization/comprehensibility #

一位中国博士把整个 CNN 都给可视化了，可交互有细节，每次卷积 ReLU 池化都清清楚楚来源：量子位 2020-05
Netscope CNN Analyzer 2017-11: A web-based tool for visualizing and analyzing convolutional neural network architectures (or technically, any directed acyclic graph). Currently supports Caffe’s prototxt format.
深度学习调参入门，有哪些技巧？ AI 科技评论 2017-03
TensorBoard in “Insights into AlexNet”

OTHER KEYWORDS in GITHUB #

“awesome deep learning”
“awesom deep learning papers”
“图解 TensorFlow 架构与设计”

邓仰东专栏｜机器学习的那些事儿 #

目录
1.绪论
1.1.概述
1.2 机器学习简史
1.3 机器学习改变世界：基于 GPU 的机器学习实例
1.3.1 基于深度神经网络的视觉识别
1.3.2 AlphaGO
1.3.3 IBM Waston
1.4 机器学习方法分类和本书组织
1.3 机器学习改变世界：基于 GPU 的机器学习实例
机器学习技术正在不断取得举世瞩目的成就，这一节会介绍三个机器学习的成功案例，让大家体会机器学习技术怎样解决极度挑战性的实际问题。
1.3.1 基于深度神经网络的视觉识别