The Foundation’s first event of 2025 was co-hosted with H2O.ai, featuring Agus Sudjianto, a renowned expert in Model Risk Management (MRM) for AI models.
Over 40 industry practitioners and experts gathered at this closed-door roundtable, bringing together an impressive collective experience of over 500 years in model building and testing.
Key takeaways from the discussion:
Embracing Acceptable Risk: The focus of MRM and AI Governance should be on helping organisations “take” acceptable AI risks rather than eliminating risk entirely.
Use Case Level Testing: For companies implementing Generative AI, validation is the most useful and practical when applied to the individual use case with a defined purpose and scope, rather than the underlying foundation models.
Evaluating the Evaluators: Trusting LLM evaluations blindly is unwise. Evaluations need as much careful design and calibration, as the application itself.
Human-AI Alignment in Testing: Automated testing frameworks should incorporate human calibration, explainable assessments, and probabilistic methods to align machine-generated evaluations with human judgments.
Throughout the year, the Foundation will continue to bring together the best of Singapore and the world on AI testing across various industries. Look forward to our upcoming efforts in incorporating emerging best practices into our open-source Project Moonshot library, and advancing the field of AI testing and validation.
Related Events
No posts found!
Don't miss out on the next exciting event from the Foundation!
Be part of the growing community shaping the future of AI safety and innovation.
Thank you for completing the form. Your submission was successful.
Preview all the questions
1
Your organisation’s background – Could you briefly share your organisation’s background (e.g. sector, goods/services offered, customers), AI solution(s) that has/have been developed/used/deployed in your organisation, and what it is used for (e.g. product recommendation, improving operation efficiency)?
2
Your AI Verify use case – Could you share the AI model and use case that was tested with AI Verify? Which version of AI Verify did you use?
3
Your reasons for using AI Verify – Why did your organisation decide to use AI Verify?
4
Your experience with AI Verify – Could you share your journey in using AI Verify? For example, preparation work for the testing, any challenges faced, and how were they overcome? How did you find the testing process? Did it take long to complete the testing?
5
Your key learnings and insights – Could you share key learnings and insights from the testing process? For example, 2 to 3 key learnings from the testing process? Any actions you have taken after using AI Verify?
6
Your thoughts on trustworthy AI – Why is demonstrating trustworthy AI important to your organisation and to any other organisations using AI systems? Would you recommend AI Verify? How does AI Verify help you demonstrate trustworthy AI?
Enter your name and email address below to download the Discussion Paper by Aicadium and IMDA.
Disclaimer: By proceeding, you agree that your information will be shared with the authors of the Discussion Paper.