Coral Edge Accelerator
#
- ML accelerator: Edge TPU ASIC (application-specific integrated circuit) designed by Google. Provides high-performance ML inferencing for TensorFlow Lite Models.
- USB 3.1 (gen 1) port and cable (SuperSpeed, 5Gb/s transfer speed)
- Dimensions: 30 x 65 x 8mm
- Price: USD74.99 in 2019, USD ＄ 60 in 2021, ￥ 740 @JD.com
- 0.5 watts/TOPS
- MAX 4 TOPS (int8), 2W.
- TF Lite + Debian only.
- Not available world wide
Intel NCS2 Neural Compute Stick 2
#
- Processor: Intel Movidius Myriad X Vision Processing Unit (VPU)
- USB 3.0 Type-A
- Dimensions: 72.5 x 27 x 14mm
- Price: USD ＄ 87.99 (2019), USD ＄ 72 (2021), ￥ 570 @JD.com
- MAX 4 TOPS, 7 W.
- Any model (TF, PyTorch, Caffe …)
ref
...
“if you plan to use deep learning extensively (>150 hrs/mo), building your own deep learning workstation might be the right move.” [medium]
- Baidu AI Studio (only for PaddlePaddle)
- Paperspace (cooperating with fast.ai)
- Google Colab (cooperating with fast.ai)
- vast.ai (C2C/P2P sharing, very cheap, a lot of time to init/load/unload)
- Kaggle (max 6h, good GPU but complex steps to use)
- MS Azure
- Amazon
- FloydHub (special CLI interface)
ref:
Install (w/ GPU)
#
win10
#
Install NV GPU driver and compatible CUDA version first, or install using pip together.
See PyTorch doc’s selector to find a compatible CUDA version.
Then use the cmd given by the selector to install PyTorch:
pip source:
#
pip install pytorch torchvision torchaudio cudatoolkit=11.1
pip binary:
#
pip3 install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio===0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
Tip: the torch...whl file is > 3GB, which can be pre-downloaded via IDM etc., then:
...
Note: tested with Ubuntu 16.04.1 using /root, for newer Ubuntu version (>= 17.04), check here.
Installation & Self-Tests
#
Use the installation script here.
//(Sunny only added conditional USE_CUDNN=1, the rest is the same as: ref. You may wanna set USE_CUDNN to 0, if no GPU is used).
Timing: 15min if everything goes well, while downloading speed 1~8MB/s.
Hello World (Mnist)
#
prepare data:
#
./data/mnist/get_mnist.sh # will download into ./data/mnist/
./examples/mnist/create_mnist.sh # will create lmdb, compute image mean
train
#
# < 1min on TitanX
./build/tools/caffe train -solver examples/mnist/lenet_solver.prototxt -gpu 0
updatedb && \
locate lenet_iter
//by default, -gpu device index is -1 (no gpu)
//The training can start from any training snapshot:
caffe train -solver=lenet_solver.prototxt -snapshot=lenet_iter_350.solverstate
(see the end of this article)
...
See also:
Install
#
Summary: install CUDA first, then TF.
ref TF 1.0 doc
ref nvidia doc, until step 3
requirements
#
- 64-bit Linux
- Python 2.7 or 3.3+ (3.5 used in this blog)
- NVIDIA CUDA 7.5 (8.0 if Pascal GPU)
- NVIDIA cuDNN >v4.0 (v5.1 recommended)
- NVIDIA GPU with compute capability >3.0
steps
#
1.Manually download “cuDNN v6.0 Library for Linux”.
2.Bash auto Install CUDA in Ubuntu 16.04.2, can combine (&& \) with code below.
3.Install cuDNN, PIP:
...
see also: /dl-do-it
MIXED TUTORIAL + TRIKCS
#
- Google: TF, DL, Tutorial, Tricks, etc
- The Ultimate Course and Book list to be an expert in Mathematics and Programming (Content: Discrete Mathematics, Linear Algebra, Calculus, Probability, Cryptography, Geometry and Statistics. 45 Mathematics Courses)
- Deep Learning and Reinforcement Learning Summer School 2018 - Yoshua Bengio etc.
- 深度学习课程笔记 by Tess Ferrandez, recommended by Andrew Ng
PAPERS, TERMS & DEFINATIONS
#
ToC:
一 ~ 三、概述, 背景, 人脑视觉机理
四、关于特征: 特征表示的粒度; 初级（浅层）特征表示; 结构性特征表示; 需要有多少个特征？
五 ~ 七、Deep Learning 的基本思想 (vs. Shallow Learning), Neural Network
八、Deep learning 训练过程: 传统神经网络 vs. deep learning
九、Deep Learning 的常用模型或者方法: AutoEncoder 自动编码器; Sparse Coding 稀疏编码; Restricted Boltzmann Machine(RBM) 受限波尔兹曼机; Deep BeliefNetworks 深信度网络; Convolutional Neural Networks 卷积神经网络
十、总结与展望
十一、参考文献和 Deep Learning 学习资源
...
