Menu
Patrick Hall
Patrick Hall

Patrick Hall

Assistant Professor
George Washington School of Business
Bio

Patrick Hall is an Assistant Professor at the George Washington School of Business, where he focuses on applied and responsible machine learning. Professor Hall conducts research in support of the NIST AI Risk Management Framework, earning a Department of Commerce Gold Medal for his efforts. He is the Board President of the AI Incident Database, and advises various firms at the intersection of AI, safety, and regulation. His expertise has been sought in the New York Times and on NPR, and his work has been featured in WIRED and Fortune. Before academia, he held technical, customer-facing, and leadership roles at machine learning companies, helping bring some of the first responsible AI applications to market.


Measuring AI Performance and Risk in the Real World

Organizations need to know their AI serves customers, meets policy, and stays reliable in the real world. This talk presents a practical, business-ready way to measure AI that blends familiar benchmarks with the signals leaders actually need—security, safety, customer impact, operational efficiency, and governance readiness. I’ll outline where over reliance on benchmarks falls short in production, provide an overview of pragmatic, real-world measurement approaches, and then offer a detailed examination of the NIST Assessing Risks and Impacts of AI (ARIA) challenge and its measurement tree method.

Measurement trees turn real-world signals into decision-ready results: build a transparent structure; collect field-relevant evidence (structured red-teaming, user feedback, incident reviews, targeted experiments, and even a few benchmarks); quantify variance; then roll results up for executives without losing drill-down for builders. I’ll also introduce CoRIx (Contextual Robustness Index)—an example measurement tree that fuses benchmarks, red-teaming outcomes, and field testing into a single, auditable index, with error bars and traceability baked in.

Attendees will learn how to (1) scope a measurement plan tied to business outcomes, (2) track what matters beyond accuracy (helpfulness, guardrail adherence, low-value/irrelevant content, latency/efficiency, accessibility), (3) incorporate user data, and (4) communicate results with traceability so findings can be trusted, audited, and used to improve systems over time.

Professional
Back To Top

The Institute for Operations Research and the Management Sciences

5521 Research Park Drive, Suite 200
Catonsville, MD 21228 USA

phone 1 443-757-3500

phone 2 800-4INFORMS (800-446-3676)

fax 443-757-3515

email

Get the Latest Updates

Copyright INFORMS. All Rights Reserved