Using AI ethically is a set of decisions, not a values statement.
This page is a standalone educational resource. It doesn't sit inside Horizon, though if something here maps onto a live problem in your organisation, we'll say so.
92m → 170m
17.7% → 46.5%
4% → 6%
7
Why this is a practical problem
Organisations don't fail ethically on purpose. They fail by swinging between two errors.
The seven principles
Each one, in evidence
Does this system treat comparable people comparably, and can you prove it?
A widely used US healthcare algorithm assigned Black patients the same risk score as White patients who were, on average, considerably sicker. The bias wasn't in the code's intent; it came from using healthcare cost as a proxy for health need, which quietly encoded years of unequal access to care into the model's "objective" output.
Obermeyer, Powers, Vogeli & Mullainathan, Dissecting racial bias in an algorithm used to manage the health of populations, Science, 2019Warning signs
- Nobody can say what proxy the model is actually optimising for
- Outcomes haven't been checked by group, only in aggregate
- The training data reflects a historical decision nobody has re-examined
What good looks like
- Outcomes are audited by demographic group on a fixed schedule
- The proxy variable is named and challenged before deployment
- There's a route to correct the model when a disparity is found
Can the person affected by this decision get a real answer to why?
People's trust in algorithmic systems isn't stable; it collapses sharply the first time they see the system err, often more than it would after an equivalent human mistake, and the opacity of "black box" reasoning is one of the consistent drivers. A lack of explainability doesn't just create legal exposure; it actively erodes the willingness of your own staff to use the tool.
Dietvorst, Simmons & Massey, Algorithm aversion, Journal of Experimental Psychology: General, 2015; Burton, Stein & Jensen, 2020Warning signs
- The system flagged it is the end of the explanation, not the start
- Staff can't describe in plain language what the model weighs
- Vendor documentation is the only source of explanation available
What good looks like
- A plain-language explanation exists for every consequential output
- People affected by a decision can ask for and receive that explanation
- Explainability was a selection criterion, not an afterthought
When this system gets it wrong, whose job is it to have caught that?
Human in the loop isn't a fix by itself. Research on automation bias shows that once a system has been correct a number of times, human reviewers stop critically evaluating its output and start rubber-stamping it, meaning the human safeguard quietly disappears exactly when it's needed most. NIST's AI Risk Management Framework treats this as a governance design problem, not a training problem: oversight has to be structured across the whole lifecycle (govern, map, measure, manage), not assigned to one tired reviewer at the end of the process.
Parasuraman & Manzey, Complacency and Bias in Human Use of Automation, 2010; NIST AI Risk Management Framework (AI RMF 1.0), 2023Warning signs
- The named accountable owner hasn't overridden the system in months
- Oversight is one person's part-time responsibility, not a process
- No one has asked what override actually looks like in practice
What good looks like
- Override rates are tracked; a rate of zero is itself a red flag
- Accountability sits with a role, documented, not an individual's goodwill
- Oversight is resourced and rotated to prevent reviewer fatigue
Does the person whose data this is know it's being used this way, and did they agree to it?
Regulators have converged on disclosure as a minimum bar, not an aspiration. Under the EU AI Act, providers deploying a chatbot or AI assistant must inform users they're interacting with AI, a low bar that a large share of deployed systems still don't clear voluntarily. UNESCO's global ethics recommendation goes further, treating privacy protection as a "do no harm" precondition for legitimate AI use, not a bolt-on compliance task.
EU AI Act, Article 50 transparency obligations (applying from 2 August 2026); UNESCO Recommendation on the Ethics of Artificial Intelligence, 2021Warning signs
- Staff paste client or personal data into public AI tools without a policy
- Nobody outside IT knows what data a deployed model was trained on
- Users interact with an AI system without being told it's AI
What good looks like
- A written policy on what data can and can't go into which tools
- AI interactions are disclosed to the people experiencing them
- Data retention and deletion rights are defined before rollout, not after a complaint
Can someone say "I think the AI got this wrong" without it costing them?
Psychological safety, the shared belief that it's safe to speak up, is what determines whether people actually raise concerns about a flawed process, AI-driven or not. Where a new system also threatens someone's sense of professional identity or competence, the research on social identity shows people defend the identity first and the process second, which is precisely when good challenge goes quiet.
Edmondson, "Psychological Safety and Learning Behaviour in Work Teams," Administrative Science Quarterly, 1999; Tajfel & Turner, Social Identity Theory, 1979Warning signs
- Concerns about the AI system are raised privately, never in the room
- People describe the tool as something being "done to" them
- Challenging the system's output is treated as a competence gap, not useful input
What good looks like
- There's a named, low-friction route to flag a bad AI-driven decision
- Leaders visibly change course in response to that feedback sometimes
- Adoption is framed as augmenting the role, not replacing the person's judgement
Have you actually asked what running this at scale costs, beyond the licence fee?
Generative AI's electricity and water demands are real and measurable, even though most vendors don't report them. US data centre electricity demand, much of it now AI-driven, rose from roughly 4% of national demand in 2022 to a projected 6% by 2026, and independent modelling projects tens of millions of tonnes of additional US carbon emissions from AI servers by 2030. None of that shows up in a per-seat licensing cost.
US Government Accountability Office, "Generative AI's Environmental and Human Effects," 2025; Nature Sustainability, "Environmental impact and net-zero pathways for sustainable AI servers in the USA," 2025Warning signs
- Environmental impact has never come up in a vendor selection conversation
- Usage is scaling with no view of the underlying resource cost
- Sustainability reporting excludes AI tooling by default
What good looks like
- Vendor environmental disclosures are requested, not assumed unavailable
- Scale of use is a deliberate decision, not a side effect of convenience
- AI usage is included in existing sustainability reporting, not carved out
Have the people whose roles this change affects been told the truth about it, early enough to act on it?
The global data doesn't support either extreme in the debate. The World Economic Forum's 2025 survey of over 1,000 employers projects both significant displacement and significant job creation by 2030, but roughly 40% of employers also say they expect to reduce headcount specifically where AI can automate tasks, and entry-level roles are disproportionately exposed. The ethical question isn't whether change happens; it's whether it's communicated honestly and early.
World Economic Forum, The Future of Jobs Report 2025Warning signs
- Redundancy conversations happen without a stated reskilling plan
- "Efficiency" language is used to avoid naming headcount impact
- Entry-level roles are quietly cut with no next-generation talent plan
What good looks like
- Workforce impact is modelled and shared before decisions are finalised
- A genuine reskilling pathway exists, not a one-off webinar
- Entry-level talent pipelines are protected deliberately, not left to erode
