Responsible AI with fmeval - an open source library to evaluate LLMs

Mia Chang

Friday 13:20 in Europium2

The term "Responsible AI" has seen a threefold increase in search interest compared to 2020 across the globe. As developers, the questions like "How can we build large language model-enabled applications that are responsible and accountable to its users?" encountered in the conversation more often than before. And the discussion is further compounded by concerns surrounding uncertainty, bias, explainability, and other ethical considerations.

In this session, the speaker will guide you through fmeval, an open-source library designed to evaluate Large Language Models (LLMs) across a range of tasks. The library provides notebooks that you can integrate into your daily development process, enabling you to identify, measure, and mitigate potential responsible AI issues throughout your system development lifecycle.

Target Audience: Machine Learning Engineers/Data Scientists, AI/ML Researchers, Software Developers, AI/ML Project Managers, Solutions Architectures.

Mia Chang

Mia Chang is a GenAI/ML Specialist Solutions Architect for Amazon Web Services. She shares best practices for running GenAI/ML workloads through customer engagements, public speaking, blog posts, and authoring books. She works in a multi-culture environment with customers in EMEA, which brings her to see technology with different culture lens. In her free time, Mia spends time mentoring aspired data scientists, and she enjoys traveling, hiking, board games, and meditation.