Because the algorithm, trained on historical data, has learned to favour certain names, backgrounds, or even word choices, perpetuating biases that neither the applicant nor the employer intended.

This scenario is not hypothetical. It is a growing reality as generative artificial intelligence (GenAI) and other automated tools become increasingly embedded in recruitment and workplace decision-making. As South African employers embrace these technologies, understanding and addressing algorithmic bias is not only a matter of compliance and reputation, but also of fairness and social responsibility.

Understanding algorithmic bias in GenAI

Algorithmic bias refers to systematic and repeatable responses in a computer system that create unfair outcomes, such as privileging one group over another. In the context of GenAI, these biases often arise because the algorithms are trained on large datasets that reflect historical patterns, including past prejudices and inequalities.

How does it occur?

Bias can be introduced at multiple stages: in the data used to train the AI, in the design of the algorithms themselves, or through the assumptions and decisions made by the humans who build and deploy these systems. For example, if an AI recruitment tool is trained on resumes or job applications from a period when a company predominantly hired men, it may learn to favour male candidates or deem a certain gender to be occupationally desirable by default.

Why is it problematic?

Algorithmic bias can lead to discrimination against individuals based on race, gender, age, disability, or other protected characteristics. This not only undermines equity and the promotion of a diverse and representative workplace but can also expose employers to legal liability and reputational harm. Importantly, because AI systems can process applications at scale, they can replicate and amplify biases more quickly and broadly than an individual human decision-maker.

Examples of AI bias affecting the workplace

The impact of algorithmic bias is not theoretical. Several high-profile cases and research studies have highlighted the tangible consequences for individuals and organisations.

Almost a decade ago, Amazon developed an AI system to automate job application screening, hoping to remove human bias. The tool was trained on historical hiring data, which was dominated by male applicants. The system learned to downgrade applications or resumes that included the word “women” or referenced all-women’s colleges, prioritising male candidates. Amazon eventually scrapped the tool, but the learnings remain relevant.

Recent research from the University of Washington found that the GenAI models under study, when ranking job applications, consistently favoured names associated with White men over those associated with Black men or women. In over three million comparisons, White-associated names were preferred 85% of the time, while Black male-associated names were never preferred over White male-associated names. This demonstrates how AI can reinforce both racial and gender disparities in hiring.

Separately, UN Women has highlighted how AI systems, when trained on biased data or data that is not adequately contextualised, can reinforce stereotypes and limit opportunities for women. For example, language models may associate “nurse” with women and “scientist” with men, or voice assistants may default to female voices, reinforcing traditional gender roles. In healthcare, AI tools have been shown to focus on male symptoms, leading to misdiagnoses for women.

These examples underscore that GenAI tools are limited by the data on which they are trained, and algorithmic bias is not an abstract risk, but a tangible issue with direct consequences for individuals and organisations.

Implications for employers deploying AI tools

The use of GenAI and other automated decision-making tools in the workplace brings both opportunities and risks. Other than potentially excluding high-value candidates, employers should be aware of the following legal and ethical implications.

Legal liability

South African law, including the Employment Equity Act 55 of 1998, prohibits unfair discrimination in employment on grounds such as race, gender, age, and disability, among others. If an AI tool used in recruitment or other employment decisions results in discriminatory outcomes, employers may be held liable.

Reputational risk

Public awareness of AI bias is growing. Organisations found to be using biased AI tools may face negative publicity, loss of trust among employees and clients, and damage to their brand and reputation.

Beyond legal compliance, there is often a public expectation that employers will use technology responsibly and transparently. This includes ensuring that AI systems do not perpetuate or exacerbate existing inequalities, and that affected individuals have avenues for recourse.

Regulatory developments

While South Africa does not yet have specific regulations governing AI bias or the deployment of AI more broadly, global trends indicate that such frameworks are likely to emerge. The European Union and the US are already moving towards stricter oversight of AI in employment. Proactive compliance can position South African employers ahead of the curve.

Practical steps employers can take

Employers can take concrete actions to mitigate the risk of algorithmic bias in GenAI and other AI tools. Some of these are outlined below.

Audit and assess AI tools

• Conduct regular audits of AI systems used in recruitment, promotion, performance and other employment decisions to identify and address potential biases.
• Request transparency from technology vendors regarding the data and algorithms used in their systems.

Diversify data and teams

• Ensure that training data for AI systems is diverse and representative of all relevant groups, including on the basis of race, gender, age, and abilities.
• Involve diverse teams in the development, selection, and oversight of AI tools to reduce blind spots.

Implement human oversight

Employers should not rely solely on automated decisions. They should ensure that human review is integrated into key stages of the hiring and employment process, especially where AI flags or rejects candidates.

Establish clear policies and accountability

• Develop and communicate clear policies on the use of AI in employment and the workplace, including procedures for raising and addressing concerns about bias. Workplace policies on AI should provide clear, detailed guidance that addresses the diverse ways in which AI may be leveraged, including for completing tasks, conducting research, supporting administrative processes, and managing and processing data, among other applications.
• Assign responsibility for AI oversight to specific individuals or committees within the organisation.

Engage in ongoing training and awareness

Employers should provide training for HR professionals, managers, and decision-makers on the risks of AI bias and best practices for fair and inclusive hiring.

Conclusion

As generative AI and other automated tools become integral to the modern workplace, South African employers must be proactive in understanding and addressing algorithmic bias. By taking practical steps to ensure fairness and compliance, organisations can protect themselves from legal and reputational harm, foster a more inclusive workplace, and contribute to a more equitable society.