An AI-Assisted Framework for Comprehensive Testing of Image Classification Models

Standard evaluation of deep learning-based image classification models typically relies on accuracy metrics measured against fixed test datasets. While useful, such methods fall short in revealing model robustness, latent failure modes, and dependencies among learned features. To address these gaps, this project introduces an AI-assisted testing framework that augments traditional evaluation through a combination of complementary strategies random testing, stress testing, AI-guided test case generation, and feature-based testing.

The framework begins by examining the model and its associated dataset to surface underlying assumptions and potential sources of bias. It then subjects the model to a range of conditions including input perturbations and feature-specific scenarios to expose weaknesses that conventional testing methods tend to overlook. Central to the framework is the integration of large language models (LLMs), which generate human-readable diagnostic summaries, surface recurring failure patterns, and inform the construction of targeted test cases.

Empirical results show that stress testing effectively uncovers significant robustness vulnerabilities, while feature-based testing yields richer insight into how the model behaves internally. Together, these strategies form a more holistic and adaptive evaluation pipeline, equipping practitioners with the understanding needed to assess and enhance the reliability of image classification systems.

Committee: Simanta Mitra (major professor)