Short description: Demand for deep-learning compute has continued to grow, as computation errors and accuracy issues remain among stochastic computing (SC) methods. In order to increase adoptability, proposed solutions have often been exchanged for loss of runtime or training performance. To address this issue of accuracy, Prof. Gupta et. al. proposes Range-Extended Stochastic Computing (REX-SC). This method increases SC computation accuracy without trading-off the performance level. REX-SC has been demonstrated to improve SC accuracy by 3-8%, reduce energy consumption by 3.6X, and improve training performance of SC models by 2-9X, compared to standard methods. The proposed method offers a solution to previous drawbacks in SC, offering an invaluable innovation to the field of deep learning.
Summary:
UCLA researchers in the Department of Electrical and Computer Engineering have developed Range-Extended Stochastic Computing (REX-SC) to increase compute efficiency and accuracy of neural networks.
Background:
Demand for deep-learning computational power has continued to grow, generating a proportional increase in the cost of such applications. Recent development of stochastic computing (SC) methods has offered higher compute efficiency. However, computation errors and accuracy issues remain a primary obstacle for widespread adoption of SC protocols. In order to increase adoptability, proposed solutions have often been exchanged for loss of runtime or training performance. In addition, resource-constrained devices have further limitations on their capability to utilize deep neural networks. In order to meet the growing demands of SC-based neural networks, there is a clear and pressing need to improve SC computational efficiency without sacrificing accuracy and performance.
Innovation:
To address the performance and accuracy issues of current SC methods, a team of researchers led by Professor Gupta propose Range-Extended Stochastic Computing (REX-SC). This method increases SC computation accuracy without trading-off the performance level. REX-SC has been demonstrated to improve SC accuracy by 3-8%, reduce energy consumption by 3.6X, and improve training performance of SC models by 2-9X, compared to standard methods. The proposed method offers a solution to previous drawbacks in SC, offering an invaluable innovation to the field of deep-learning based neural networks.
Potential Applications:
- Neural network training
- Image recognition
- Voice recognition
- Machine translation
Advantages:
- Compact computation unit
- Improved data reuse
- Reduced memory loss
- Improved performance
- Decreased cost
Related Publications: Not published.
Development to Date: First description of the invention completed.