Hridesh Rajan, Kingland Professor and Chair of the Department of Computer Science gave an invited talk at the SEMLA symposium. The Software Engineering for Machine Learning Applications (SEMLA) international symposium aims at bringing together leading researchers and practitioners in software engineering and machine learning to reflect on and discuss the challenges and implications of engineering complex data-intensive software systems.
The abstract of the talk follows, “A class of machine learning algorithms known as deep learning that has received much attention in academia and industry. Deep learning has a large number of important societal applications, from self-driving cars to question-answering systems such as Siri and Alexa. A deep learning algorithm uses multiple layers of transformation functions to convert inputs to outputs, each layer learning higher-level of abstractions in the data successively. The availability of large datasets has made it feasible to train deep learning models. Since the layers are organized in the form of a network, such models are also referred to as deep neural networks (DNN). While the jury is still out on the impact of deep learning on the overall understanding of software’s behavior, a significant uptick in its usage and applications in wide-ranging areas and safety-critical systems, e.g., autonomous driving, aviation system, medical analysis, etc., combine to warrant research on software engineering practices in the presence of deep learning. One challenge is to enable the reuse and replacement of the parts of a DNN that has the potential to make DNN development more reliable. This talk will describe a comprehensive approach to systematically investigate the decomposition of deep neural networks into modules to enable reuse, replacement, and independent evolution of those modules. A module is an independent part of a software system that can be tested, validated, or utilized without a major change to the rest of the system. Allowing the reuse of DNN modules is expected to reduce energy- and data-intensive training efforts to construct DNN models. Allowing replacement is expected to help replace faulty functionality in DNN models without needing costly retraining steps. Our preliminary work has shown that it is possible to decompose fully connected neural networks and CNN models into modules and conceptualize the notion of modules. A serious problem facing the current software development workforce is that deep learning is widely utilized in our software systems, but scientists and practitioners do not yet have a clear handle on critical problems such as explainability of DNN models, DNN reuse, replacement, independent testing, and independent development. There was no apparent need to investigate the notions of modularity as neural network models trained before the deep learning era were mostly small, trained on small datasets, and were mostly used as experimental features. The notion of DNN modules developed by our work is helping make significant advances on a number of open challenges in this area. DNN modules enable the reuse of already trained DNN modules in another context. Viewing a DNN as a composition of DNN modules instead of a black box enhances the explainability of a DNN’s behavior. More modular deep learning will thus have a large positive impact on the productivity of these programmers, the understandability and maintainability of the DNN models that they deploy, and the scalability and correctness of software systems that they produce.”
Hridesh Rajan is the Kingland Professor and Chair in the Department of Computer Science at Iowa State University (ISU), where he has been since August 2005. He served as the Professor-In-Charge of the ISU Data Science program from 2017-Oct 2019. He has held visiting positions at the University of Bristol, Harvard University, and the University of Texas, Austin. Prof. Rajan earned his Ph.D. in Computer Science from the University of Virginia. He is a AAAS Fellow, ACM Distinguished Scientist and a Fulbright Scholar. He has also been recognized by the US National Science Foundation (NSF) with a CAREER award in 2009, by the Iowa State University College of LAS with an Early Achievement in Research Award in 2010, a Big-12 Fellowship in 2012, a ACM Senior Membership in 2014, an exemplary mentor for Junior Faculty award in 2017, a Kingland Endowed Professorship in 2017, and early achievement in departmental leadership award in 2022. Prof. Rajan specializes in data science, programming languages and software engineering. He is credited with giving the definitive treatment for how to modularly reason about crosscutting concerns, and for the design and implementation of the Boa infrastructure for large-scale analysis of open source software and its evolution. Prof. Rajan served as an associate editor for the IEEE Transactions on Software Engineering and as an associate editor for the ACM SIGSOFT Software Engineering Notes. He served as the general chair of SPLASH 2020 and SPLASH 2021, the ACM SIGPLAN conference on Systems, Programming, Languages, and Applications: Software for Humanity.