FairBench

A comprehensive AI fairness exploration framework. Part of the MAI-BIAS ecosystem.

What's your task?

⚡ Quickly compute a fairness metric: jump to quickstart.
🤖 Investigate LLM fairness: jump to assessing LLMs.
📚 Exploring the library for principled bias analysis: go through the overview in order, and then read the documentation. Consider starring or submitting issues at GitHub.

Try it in your browser

▶ Run 🔄 Clear powered by pyodide # some sensitive attribute with values M,F,O, binary target # labels y, and predicted ones yhat (see docs for multiclass) s = ["M","F","M","F","M","F","M", "O", "O"] y = [1, 1, 0, 0, 1, 0, 1, 1, 0] yhat = [1, 1, 1, 0, 0, 0, 0, 1, 0] # one of the patterns to unpack sensitive attributes into dimensions s = fb.Dimensions(fb.categories @ s) # a fairness report for base measures (all applicable to the # arguments) computed and compared across all groups report = fb.reports.pairwise(predictions=yhat, labels=y, sensitive=s) # show in an environment (default: Console) & focus on max differences # report.help() shows what to focus on report.show(env=fb.export.ConsoleTable(legend=False)) report.maxdiff.show(env=fb.export.Console(ansiplot=False))