Caveats and recommendations
• This is a generic list of caveats that apply to all measures.
• Non-quantitative criteria may also impact perceived fairness.
• Choose carefully the criteria on when measures are considered close to their ideal values.
• A single measure cannot decide whether a system is fair or biased without further investigation. It can at best indicate the absense of a particular bias. However, different measures are often at odds with each other, even when they have similar optima.
• Consult with stakeholders to determine on which social and legal criteria systems should follow. This translates to choosing measures appropriate for the operating context.