The ATxAI 2023 conference, part of the ATxSummit, was held on 7 June 2023 at Capella Singapore. This global flagship conference on AI governance brought together respected visionaries, experts, and policymakers in a series of discussions on building trustworthy AI, opportunities and challenges.
The conference was packed. Many attendees were eager to hear the insights from 19 international and local speakers across 4 panels, an industry showcase and a fireside chat.
The first panel session on Generative AI – Mitigate Risks to Maximise Opportunities shared about the opportunities of generative AI. The esteemed panellists agreed on the need for objective evaluation frameworks for AI models. Michael Sellitto stressed the current lack of good ways to evaluate models, and called for collaborative efforts among industry, government, academia, and civil society to develop rigorous and repeatable evaluation processes. Kathy Baxter and Professor Yan Shui Cheng further pointed out the challenges with subjective evaluation when dealing with content generation, and Michael Zeller emphasised the need for independent third-party evaluation of models on agreed-upon metrics.
Collaboration emerged as a resounding theme as the panellists emphasised the significance of sharing best practices and fostering common standards in AI development and regulation. In particular, Kathy Baxter said that ethics and security should not be treated as proprietary advantages but should be shared widely, pointing to initiatives like the National Institute of Standards and Technology’s (NIST) AI Risk Management Framework and the AI Verify Foundation as essential steps in this direction. Michael Zeller echoed these sentiments, underscoring the importance of the AI Verify Foundation as a critical first step that requires the unified efforts of academia, industry, and government to establish sensible regulations and instil confidence in harnessing the power of large AI models in the future.
Watch panel 1 here:
The second panel session, titled “Are Current AI Governance Approaches Around the World Effective?” delved into the accountability of generative AI models. The discussion revolved around the need to identify roles and establish clear responsibilities between developers and deployers of generative AI models. The unique nature of generative AI, the context and the players involved such as platforms, creators, and end-consumers, necessitated a comprehensive understanding of the value chain and alignment.
The panellists also deep-dived into the use of data on the internet that was previously placed without the expectation that it would be freely used to train foundation models or to generate output using generative AI. The panellists debated whether this could lead to commercial exploitation of talent and privacy violations. Kay Firth-Butterfield cited a recent US case law that addressed the fair use of licensed or publicly available creative output. The courts favoured creators who could prove commercial impact or loss because of misuse of their work. This development sparked intriguing discussions on how copyright and intellectual property laws should apply to where generative AI using existing content to create new output or images.
The panel unanimously agreed that there should be greater transparency around data used to train the foundation model, how it operates as well as what generative AI models should or should not be used for. Such transparency empower users to make informed decisions and ensures responsible usage of these AI models in alignment with their intended purpose.
Watch panel 2 here:
We also relieved key highlights when IBM, SIA and UBA shared their first-hand experience in piloting the AI Verify Minimum Viable Product (MVP).
Headlining the AI Verify Industry Showcase was Anup Kumar from IBM who described AI Verify as a practical toolkit, comparing it to “eating your recipe” as the AI Verify MVP had utilised several of IBM’s open-source tools. IBM said that the toolkit was able to generate a report on the performance of their AI model in just 15 minutes. He also affirmed the importance of crowd-sourcing international efforts for AI testing. This was highlighted to the fore when IBM piloted AI Verify with a credit risk use case.
Helen Wang, representing UBS, shared her experience in using AI Verify to review the bank’s internal AI development process. Testing AI Verify with an internal income prediction model, UBS found the process checklist to be comprehensive and relevant. Even non-technical users found the toolkit user-friendly, with a single click generating a report complete with detailed explanations, definitions and illustrations that shed light on possible underlying bias. The process checklist within AI Verify provided a comprehensive view of AI principles around the world from key regulators, think tanks and research institutions.
Patrick Chua from SIA advocated AI Verify as an excellent starting point for organisations looking to conduct self-assessments on their AI systems. SIA piloted AI Verify with Joey, and successfully recalibrated its AI development process by introducing relevant AI principles at different stages. The experience with AI Verify empowered SIA to ensure responsible AI practices throughout their organisation.
ATxAI ended with a networking lunch for speakers and participants.
No posts found!
Your organisation’s background – Could you briefly share your organisation’s background (e.g. sector, goods/services offered, customers), AI solution(s) that has/have been developed/used/deployed in your organisation, and what it is used for (e.g. product recommendation, improving operation efficiency)?
Your AI Verify use case – Could you share the AI model and use case that was tested with AI Verify? Which version of AI Verify did you use?
Your experience with AI Verify – Could you share your journey in using AI Verify? For example, preparation work for the testing, any challenges faced, and how were they overcome? How did you find the testing process? Did it take long to complete the testing?
Your key learnings and insights – Could you share key learnings and insights from the testing process? For example, 2 to 3 key learnings from the testing process? Any actions you have taken after using AI Verify?
