Why do AI systems make unfair decisions? Because they learn patterns from data, and data records the world as it was, including past discrimination, gaps and shortcuts. A model that screens CVs or scores loan applicants can repeat those patterns at scale without anyone intending it, so fairness has to be tested deliberately rather than assumed.
This explainer covers the main sources of bias, where it tends to appear, how a small business or a school can check a system before relying on it, and why “fair” turns out to have more than one definition.
Where does AI bias come from?
A machine learning model is a pattern-finder. Show it thousands of past hiring decisions and it learns which features tend to go with “hired”. If past managers favoured candidates from certain schools, or penalised career gaps that mostly affect women returning after having children, the model learns that too and applies it consistently to every new application.
A widely reported example came from a large technology company that experimented with an automated CV-screening tool and found it downgraded applications mentioning women’s activities, because it had learned from years of past hiring that was mostly male. The project was dropped. The lesson was not that one company was careless, but that bias arrives through perfectly ordinary historical data.
Missing groups, proxies and language: other roots of bias
Bias also comes from who is missing. Research on commercial face analysis systems found noticeably higher error rates for darker-skinned women than for lighter-skinned men, because the training images were unbalanced. Speech recognition can show a similar effect with accents: a system trained mainly on some regional voices may make more mistakes with others, which matters for a language with strong regional variation such as Vietnamese.
Then there are proxies. Even if a model is never given gender or ethnicity, it may infer them from postcode, first name, phone model, shopping times or school. Deleting a sensitive column rarely removes the bias; it just makes it harder to see.
Large language models add another layer. Trained largely on internet text, much of it in English, they can reproduce stereotypes about jobs, regions or groups, and they may perform less well in Vietnamese than in English. Providers apply filters and tuning that reduce obvious problems, but that does not guarantee neutral output.
Where does bias show up? Common patterns by application
Bias looks different in each application. These are patterns reported repeatedly, not guaranteed outcomes for every system.
| Area | How bias can appear | What to check |
|---|---|---|
| Recruitment screening | Ranks down CVs that resemble groups underrepresented in past hires; penalises career gaps or certain schools. | Compare pass rates by gender and age group on a test set; review a sample of rejections by hand. |
| Credit scoring | Relies on proxies such as phone type or location that track income or region rather than repayment behaviour. | Ask which variables are used; compare approval rates across regions for similar repayment histories. |
| Face recognition | Higher error rates for groups underrepresented in training images, worse in poor lighting. | Test with your own users in your own lighting; always keep a non-biometric fallback. |
| Speech recognition | More mistakes with some accents, older voices or noisy settings. | Pilot with staff and customers from different regions before rolling out. |
| Content moderation | Flags slang, dialect or minority languages as harmful more often. | Review a sample of removed posts; offer a human appeal. |
| Generative AI | Stereotyped images or text about professions, genders or regions. | Ask for diversity explicitly in prompts; review outputs before publishing. |
The common thread: test on people who resemble your real users, and keep a way for a human to correct the machine.
Questions to ask before trusting an AI decision system
What data was it trained on? Ask the vendor where the training data came from, which countries and which period. Data from another market may not fit Vietnamese users.
Has it been tested on different groups? Ask for error rates broken down by gender, age, region or language, not one overall accuracy figure. A high average can hide poor results for a minority.
What exactly does it decide? Is it ranking, recommending or making the final call? A system that only recommends, with a person deciding, carries less risk.
Can a person override it? There should be a clear way for staff to correct a result and for the people affected to ask for a human review.
Can it explain a result? Even a simple explanation, such as the main factors behind a score, helps catch errors and is increasingly expected by regulators and customers.
Which inputs might be proxies? Look for location, name, device or school features that may stand in for characteristics that should not influence the decision.
How is it monitored after launch? Performance drifts as the population changes. Ask how often results are re-checked and who is responsible for doing it.
What happens to the data of the people assessed? Fairness and privacy go together. Know how long applicant or customer data is kept and who can see it.
Why does “fair” have more than one meaning?
Once you start measuring, a surprise appears: fairness can be defined in several reasonable ways that cannot all be satisfied at the same time. Equal approval rates across groups, equal error rates, and equally accurate risk scores are each defensible, but when groups differ in their underlying data, improving one measure often worsens another.
That means fairness is a decision people have to make, not a setting a vendor can switch on. A school using AI to flag students who may need support might accept more false alarms so that nobody is missed; a lender might choose differently. Whatever the choice, it should be explicit, written down and reviewed.
Practical safeguards matter more than perfect metrics. Keep a human involved in decisions that affect someone’s job, money or education. Tell people when an automated system is part of the process. Provide a simple way to appeal. And test again whenever the data or the model changes.
For small organisations without data scientists, the most useful step is often the cheapest: before adopting a tool, run it on a few dozen real past cases where you already know the right answer, covering people from different backgrounds, and look at where it goes wrong. If the vendor cannot support that kind of test, that tells you something too.
Frequently asked questions
Can AI be completely unbiased?
No system is perfectly neutral, because data and design choices always reflect some point of view. The realistic goal is to measure bias, reduce harmful patterns and keep humans able to correct decisions.
If we remove gender from the data, is the model fair?
Not necessarily. Other variables such as names, schools or career gaps can act as proxies. Fairness has to be checked by testing outcomes, not by deleting columns.
Do chatbots show bias in Vietnamese?
They can. Models trained mostly on English text may reproduce stereotypes and perform less well in Vietnamese, including with regional expressions. Review anything AI writes about people, regions and jobs before using it.
Who is responsible when an AI system discriminates?
Generally the organisation that uses the system to make decisions remains responsible, not the algorithm. Contracts may share some responsibility with the vendor; for specific legal questions, consult a lawyer.
How can a small company test an AI tool for bias?
Run it on past cases where the right answer is known, covering different genders, ages and regions, and compare error rates between groups. Keep a human review step for important decisions.
