From Tokens to Throughput: Measuring the ROI of AI in Instructional Design
This presentation presents an in-depth analysis of how Artificial Intelligence (AI) is reshaping Interactive Multimedia Instruction (IMI) content development for virtual training. Using a mixed-methods comparative analysis of production metrics and stakeholder feedback, we examine the impact of AI on Instructional Systems Designers’ (ISDs) workflows, emphasizing measurable gains in productivity and cost-effectiveness without sacrificing quality. The study draws data from over 140 personnel in the Navy Ready Relevant Learning (RRL) program. Our research demonstrates that custom AI platforms and tools, featuring Retrieval-Augmented Generation (RAG), Large Language Models (LLMs), and enhanced data synthesis, significantly accelerate content development. Machine Learning (ML) tools enhance consistency, reduce manual formatting burdens, and support crafting instructional assessment questions. By comparing high- and low-usage periods during implementation, we show that increased AI engagement strongly correlates with ISD task completion rates. To quantify the value of productivity gains, we used a cost-adjusted framework that compares token usage costs with labor hours saved. With hundreds of thousands of AI-enabled inquiries processed to date, the productivity elasticity analysis shows that a 1 percent increase in AI usage yields a significant increase in completed tasks, highlighting the scalable and compounding impact of AI integration on output efficiency. Qualitative findings reinforce that AI can be a productivity multiplier augmenting human-led design. These results confirm the transformative value of AI in the IMI development lifecycle. AI-enhanced workflows are positioned as a strategic enabler of scalable, cost-efficient, and high-quality training for all stakeholders in virtual training development.
Professional
