AAAI 2025 Tutorial on User-Driven Capability Assessment of Taskable AI Systems
by @pulkit_verma and @siddharthsrivastava6272
This tutorial covers approaches for assessing the safety and functionality of AI systems designed to learn continuously and complete tasks in a user’s environment. AI systems are increasingly interacting with non-expert users, leading to growing calls for better safety assessment and regulation by users, governments, and industry. While recent AI developments have made it easier to develop taskable AI systems, ensuring their safety presents unique challenges. Unlike traditional engineered systems where limited functionality yields safety, taskable AI systems are designed to adapt to user-specific tasks and environments, invalidating conventional approaches to safety assurance. These challenges cannot be addressed by simply extending existing verification and validation paradigms.
The tutorial covers novel capability discovery and assessment techniques. Specifically, we address three main areas: (i) why conventional verification and validation approaches fall short, (ii) specific requirements and promising research directions for formal assessment of AI systems, and (iii) solutions developed for restricted settings.
For more details and updates, please refer to https://aair-lab.github.io/aia2025-tutorial