SIA piloted AI Verify testing framework on Joey:
Introducing Joey, SIA’s first chatbot developed in-house for HR-related queries. Launched in April 2021, Joey uses Smart Search technology to provide fast and accurate responses, improving efficiency, accuracy, and accessibility in HR processes by:
Early testers in beta testing
SIA was among the early users of AI Verify, participating in its beta testing even before the Minimum Viable Product was launched. Recognising the importance of responsible AI practices, SIA saw AI Verify as a strong foundation to kickstart its AI testing and aimed to:
When SIA first started its journey in testing AI Verify, it was not without challenges as the organisation had to spend a substantial amount of time to:
Despite the limitations in technical testing tools, it did not stop SIA from pursuing further tests. SIA reviewed and tested its AI initiatives using the testing framework and process checks where possible. While the testing process was rigorous and intensive, SIA felt it was critical to streamline the implementation of responsible AI with its AI development process. SIA made several improvements, including:
Your organisation’s background – Could you briefly share your organisation’s background (e.g. sector, goods/services offered, customers), AI solution(s) that has/have been developed/used/deployed in your organisation, and what it is used for (e.g. product recommendation, improving operation efficiency)?
Your AI Verify use case – Could you share the AI model and use case that was tested with AI Verify? Which version of AI Verify did you use?
Your experience with AI Verify – Could you share your journey in using AI Verify? For example, preparation work for the testing, any challenges faced, and how were they overcome? How did you find the testing process? Did it take long to complete the testing?
Your key learnings and insights – Could you share key learnings and insights from the testing process? For example, 2 to 3 key learnings from the testing process? Any actions you have taken after using AI Verify?