Speaker:Rangeet Pan

Title: On Decomposing a Deep Neural Network into Modules

Abstract: Deep learning is being incorporated in many modern software systems. Deep learning approaches train a deep neural network (DNN) model using training examples and then use the DNN model for prediction. While the structure of a DNN model as layers is observable, the model is treated in its entirety as a monolithic component. To change the logic implemented by the model, e.g. to add/remove logic that recognizes inputs belonging to a certain class, or to replace the logic with an alternative, the training examples need to be changed and the DNN needs to be retrained using the new set of examples. We argue that decomposing a DNN into DNN modules— akin to decomposing a monolithic software code into modules—can bring the benefits of modularity to deep learning. In this work, we develop a methodology for decomposing DNNs for multi-class problems into DNN modules. For four canonical problems, namely MNIST, EMNIST, FMNIST, and KMNIST, we demonstrate that such decomposition enables the reuse of DNN modules to create different DNNs, enables replacement of one DNN module in a DNN with another without needing to retrain. The DNN models formed by composing DNN modules are at least as good as traditional monolithic DNNs in terms of test accuracy for our problems.

Bio: Rangeet Pan is a third-year Ph.D. student at the Department of Computer Science at Iowa State University. His Ph.D. advisor is Dr. Hridesh Rajan. He worked as a research intern at Microsoft Research during summer 2020 with Dr. Nachi Nagappan, Dr. Shuvendu Lahiri, Dr. Vu Le, and Dr. Sumit Gulwani. His research interests include program analysis, machine learning, and software engineering. His works are focused on the deep learning bugs, their characteristics, their fixing patterns, and how deep neural network models can be decomposed into smaller modules to enable reusability and replaceability. He has published works at ESEC/FSE and ICSE. He has won the 2nd prize at the ACM Student Research Competition at ICSE 2020.

After the presentation, there will be a short time for discussion and questions afterwards.

Please contact Bridgette Hare for virtual conference instructions