Multimodal Artificial Intelligence for Agricultural Decision Support

This thesis explores how AI and foundation models can be applied to agricultural problems, from 3D plant reconstruction to scalable disease diagnosis. We first evaluate Neural Radiance Fields for 3D reconstruction of plants in field conditions, proposing an early stopping technique that halves training time with minimal quality loss. We then develop AgEval, a benchmark for evaluating vision-language models on 12 agricultural tasks, showing that few-shot prompting improves F1 scores from 46.24% to 73.37% and that strategic example selection matters. Building on this finding, we introduce an assisted few-shot learning approach that uses image encoders to select better examples, raising average F1 to 80.45%. We then present PestIDBot, a system that pairs specialist vision models with region-aware retrieval-augmented generation to deliver location-specific pest management guidance across North America, Africa, and India, while abstaining under uncertainty. Finally, we address the data scarcity problem for disease diagnosis with SAGE, which compiles approximately 613K images across 55 crops, automatically generates source-cited symptom knowledge, and builds an agentic diagnostic pipeline that requires no task-specific training and improves directly with better foundation models. Across these studies, the thesis moves from specialized model development toward systems that leverage foundation models with structured domain knowledge at increasing scale.

Committee: Soumik Sarkar (major professor), Baskar Ganapathysubramanian, Robyn Lutz, Aditya Balu, and Adarsh Krishnamurthy