RE•WORK Deep Learning Summit

Deep neural networks reach state-of-the-art performance for wide range of Natural Language Processing, Computer Vision and Speech applications. Yet, one of the biggest challenges is running these complex networks on devices with tiny memory footprint and low computational capacity such as mobile phones, smart watches and Internet of Things. In this talk, I will introduce novel on-device Self-Governing Neural Networks (SGNNs), which learn compact projection vectors with local sensitive hashing. The key advantage of SGNNs over existing work is that they surmount the need for pre-trained word embeddings and complex networks with huge parameters. I will showcase results from extensive evaluations on wide range of natural language tasks such as dialog act classification and user intent prediction. Our findings show that SGNNs are effective at capturing low-dimensional semantic text representations, while maintaining high accuracy and improving over state-of-the-art results.