Advanced Technologies for Resource-constrained Deep Learning
窮人的深度學習：有限資源下的深度學習進階技術
National Taiwan University Prof. Shou-De Lin
計畫主持人：國立臺灣大學 資訊工程學系林守德教授
深度學習受限於兩大現實的條件： 1.許多先進的深度神經網路模型包含了大量的參數，當中所產生的沈重運算量造成訓練上的負擔，難以在邊際設備上運作。 2.深度學習需要大量的標註資料才能夠訓練，而標註資料本身的蒐集是極為耗費人力金錢與時間的。 基於以上兩個議題，在本計畫中，我們將著手於研究並開發學習「稀疏」類神經網路的方法，並期望藉由導入模型的稀疏性來減輕計算量但又不失去深度學習網路原先的傑出表現。我們也將會把其他各種類神經學習網路（如CNN，RNN，memory network等）納入稀疏化加速的目標，使得未來的深度學習有機會在edge device上面應用。
In this project, we aim at addressing the following three concerns for deep learning: first, the training of deep learning models requires heavy computation power, which confines the usage from those who can hardly access high-end computation resources. Second, a trained deep learning model usually contains millions or even trillions of parameters, which can prohibit it from being executed on light-weight end devices. Third, deep learning usually demands significant amount of labelled data to train the model, while in modern era the labelled data are generated in a much slower path than the unlabeled one.
