Executive Summary

More and more, an algorithm judges first — in hiring, and in deciding who qualifies for welfare. According to figures obtained by Korea's National Assembly, 351 job seekers were rejected over the past two years by AI hiring assessments run at public institutions across all 17 metropolitan cities and provinces. This article asks what a person on the receiving end of such a decision can actually do now: how far the individual's right to opt out of an automated decision, and to be told why it was made, has really come.

The right itself is already on the books. Korea's Personal Information Protection Act (PIPA) Article 37-2 has, since 15 March 2024, granted the right to refuse a fully automated decision and to demand an explanation of it, and the EU built the same protection through GDPR Article 22 and AI Act Article 86. The catch is that every one of these rights sits on the narrow ledge of "solely automated." A single, even nominal, human touch is usually enough for the law to conclude the right does not apply.

Then the timetable slipped as well. In May 2026 the EU pushed the application of its high-risk AI obligations — which cover hiring and welfare — back by 16 months, to December 2027. Between a right being written into law and that right actually firing, there is still a gap. What closes it is the data infrastructure that lets the basis of a decision be pulled back out and examined after the fact.

Key figures

351

rejected by AI hiring tests

Over 2 years, public bodies across 17 regions

15 Mar 2024

PIPA Article 37-2 takes effect

New right to refuse and to demand explanation

16 months

EU high-risk AI delay

Hiring and welfare pushed to Dec 2027

₩1.2tn

Australia's Robodebt refund

Automated welfare debts ruled unlawful

1

The algorithm decides, and you find out later

A hiring process with no human in the interview room is no longer strange. A natural-language model reads the résumé first; aptitude tests and video interviews convert expressions and answers into scores. According to documents obtained by the National Assembly's Public Administration and Security Committee, over the past two years 351 job seekers were rejected by AI hiring assessments at public institutions across all 17 metropolitan cities and provinces. Most of those who received a rejection go on to prepare their next application without ever knowing why they scored the way they did.

Welfare is no different. Systems already run that analyze administrative data — power and water shut-offs, unpaid health-insurance premiums — to flag "households in crisis," and on the other side sit algorithms that automatically hunt for benefit fraud. Either way, the individual learns that a judgment has been made about them only after the fact, and usually only in the form of a notice of the outcome.

So the question narrows to one thing. Whether you were passed over for a job or dropped from a benefits roll, can the person on the receiving end demand to know why it was decided that way and refuse the decision? This question stands on the far side of the data story Pebblous has been telling. Where the focus used to be the supplier's duty to make the data good enough, it now turns to the data subject's right: can the individual question the decision that data produced?

The conditions for AI-ready data have so far been a matter of input quality. Is the training data accurate, representative, traceable? But for decisions that touch a life directly — hiring, welfare — one more condition attaches. When the person on the receiving end asks why, can you answer? If data quality is the supplier's language, the right to explanation is the language of the individual who received the decision that data made.

2

The right is already in the law

The right to question is already written into the laws of several countries. Korea amended its Personal Information Protection Act in 2023 to add Article 37-2, in force since 15 March 2024. It does two things. If a decision made by a fully automated system materially affects a person's rights or obligations, that person may refuse the decision (the right to opt out); and when an automated decision has been made, they may demand an explanation of it (the right to explanation). Absent a legitimate ground, the controller must either not apply the decision or have a human review it again.

The same structure exists across borders. EU GDPR Article 22 grants the right not to be subject to a significant decision based solely on automated processing, and where such decisions are exceptionally allowed, it requires safeguards of human intervention, the right to express a view, and the right to contest. The EU AI Act, which entered into force in 2024, goes a step further in Article 86: where a decision based on the output of a high-risk AI system adversely affects a person's health, safety, or fundamental rights, that person has the right to a clear and meaningful explanation of the decision's main elements. Korea's AI Basic Act likewise requires "meaningful explanation" for high-impact AI under Article 34.

Korea · PIPA Article 37-2

Right to refuse a fully automated decision plus a right to explanation. In force since 15 Mar 2024. Automated decisions are permitted in principle, but where the conditions are met, the subject gains rights to respond.

EU · GDPR Article 22

Right not to be subject to a solely automated decision. A prohibition-by-default structure. Where exceptions apply, it requires human intervention, the right to express a view, and the right to contest.

EU · AI Act Article 86

Right to an explanation of a high-risk AI decision. The main elements that actually influenced the decision must be explained clearly and meaningfully.

Korea · AI Basic Act Article 34

Duty on high-impact AI operators to give "meaningful explanation" of the basis, key criteria, and an overview of the training data. In force 22 Jan 2026.

List the provisions and it looks as though individuals are already thickly protected. Yet these four articles share one soft spot. They all stand on the threshold of "solely" automated. The right switches on only when a decision was made entirely automatically, and the moment a human intervenes even slightly, most of the protection falls outside the trigger.

Korea's own Personal Information Protection Commission has publicly acknowledged this limit. The right to opt out is confined to fully automated decisions, it noted, and once a human is suitably involved in the decision-making, exercising that right becomes effectively difficult. The regulator that designed the protection was the first to point out that its trigger will rarely be met in practice.

3

Sidestepping the threshold — hiring

How the threshold gets sidestepped shows up most sharply in hiring. In 2026, an applicant rejected by a 100% AI hiring process gained the ability, under PIPA Article 37-2, to ask the employer why they were turned down. But for that right to switch on, the decision has to be "fully automated." When even AI interview vendors say none of them run an entire hiring process through AI alone, most employers keep AI as an aid and lay a human name on top at the end. The rule exists, but its trigger runs against how the work is actually done.

Where the line falls is being gauged by looking at U.S. case law. If a system extracts and structures skills, experience, and education data from a résumé, scores job fit, and then reaches a decision automatically, it is likely to count as high-impact under Korean law too. Conversely, functions that operate apart from the substance of the judgment — checking for missing documents, detecting duplicate applications, scheduling interviews — steer clear of a high-impact finding. How far the AI is involved in the "judgment" becomes the line where protection does or does not switch on.

3.1Where two U.S. lawsuits diverge

Two live U.S. lawsuits show two different strategies for attacking AI hiring. Mobley v. Workday argues that a hiring algorithm indirectly discriminated by age. In March 2026 a federal judge allowed the disparate-impact claim under the age-discrimination statute to proceed, rejecting Workday's defense that the claim did not reach applicants. It is the route of contesting bias head-on.

Kistler v. Eightfold AI opens a different door. Its core is not a claim that the algorithm was biased, but that its very existence was hidden. The allegation is that it scraped the personal data of more than a billion people, scored applicants from 0 to 5, and filtered out low scorers before any human looked. The plaintiff frames it as a consumer-protection violation — that the system effectively produced a "consumer report" without the disclosure that the Fair Credit Reporting Act requires. Rather than the heavy burden of proving bias, it takes the lighter route of showing the process was never disclosed.

Gavel in a U.S. courtroom, where Mobley v. Workday and Kistler v. Eightfold AI are being litigated
▲ Lawsuits over AI hiring algorithms are working through U.S. courts on two different theories: bias and undisclosed screening | Source: Wikimedia Commons (Joe Gratz, CC0)

The contrast between the two suits is telling. Proving bias is hard, but the fact that you were never told which algorithm filtered you out, on which data, is comparatively easy to contest. The substance of the right to question narrows, in the end, to whether the procedure behind a decision can be transparently reconstructed.

4

Sidestepping the threshold — welfare

In welfare, several countries have already piled up their failures. Three cases show how automated welfare decisions can collapse an individual's procedural rights.

What each of the three took from people differs slightly. Australia's Robodebt reversed the burden of proof, forcing recipients to prove for themselves that they owed nothing. The Netherlands' SyRI never disclosed which data it used to suspect whom, leaving no ground to contest. Michigan's MiDAS, on a flaw in its method of mechanically dividing quarterly income, branded blameless recipients as fraudsters. The point of failure differs, but all three share one thing: the person who received the decision had no channel to trace back its basis.

Australia · Robodebt

An automated system averaged a recipient's annual income into weekly figures to calculate overpayments, then dropped onto the individual the burden of proving they owed nothing. In 2019 a court ruled it unlawful and the government refunded roughly ₩1.2 trillion. The detailed record left by the Royal Commission became the standard case study of automated-welfare failure.

Netherlands · SyRI

A system that combined multiple government databases to predict the likelihood of welfare and tax fraud. In 2020 the District Court of The Hague ruled it unlawful on the ground that the government never transparently disclosed which data and which algorithm it used. It is a rare ruling in which opacity itself became the core basis for illegality.

United States · Michigan MiDAS

By mechanically splitting quarterly income across 13 weeks to detect unemployment-benefit fraud, it wrongly flagged roughly 48,000 people in 2013 alone. A 2024 class-action settlement awarded plaintiffs US$20 million.

A Centrelink branch office in Australia, where Robodebt's automated debt notices were issued
▲ A branch of Centrelink, Australia's welfare agency. Automated debt calculations reached recipients through counters like this one | Source: Wikimedia Commons (Bidgee, CC BY 3.0)

Korea's crisis-household detection AI treats these cases as cautionary lessons. At a welfare seminar in July 2026, the point was stressed that even where AI screens for households in crisis, the final judgment is kept with a human who goes through initial and in-depth counseling and a site visit — and Robodebt was named explicitly as the lesson to avoid. There is also ongoing discussion of building a model that explains, in natural language, the grounds for selecting a given household.

But the same structural trap that shadows hiring settles over this too. The principle that "the final judgment rests with a human" is a matter of policy caution and, at the same time, a way of clearing the fully-automated threshold in PIPA Article 37-2. The moment a human intervenes, the decision legally falls outside the trigger for the rights to opt out and to explanation. That caution and evasion come out of the same design is the trickiest part of this threshold.

5

The right is written, the clock runs back

Even in the cases where a right does clear the threshold and fire, if the rollout of the infrastructure it leans on is delayed, the right stays on paper. The EU has just created exactly that situation. On 7 May 2026 the Council, Parliament, and Commission reached provisional agreement on the "Digital Omnibus," the first amendment since the AI Act's adoption, and pushed the application of Annex III high-risk AI obligations — which include hiring and welfare-eligibility determinations — from 2 August 2026 to 2 December 2027, a 16-month delay.

The European Parliament building in Brussels, where the EU AI Act amendment was negotiated
▲ The European Parliament building in Brussels. The "Digital Omnibus" agreed here in May 2026 pushed back high-risk AI obligations by 16 months | Source: Wikimedia Commons (Steven Lek, CC BY-SA 4.0)

The weight of this delay does not lie in a schedule simply slipping. The Article 86 right to explanation functions in practice only when deployer obligations such as risk management, logging, and human oversight are operating. An explanation is possible only if a record remains of what a decision was based on. That those obligations are deferred until December 2027 means the floor the individual's right to question would stand on gets laid that much later. At the very moment the right was written into law, the clock that supports it was pushed back 16 months.

Korea walked the other way. It brought the AI Basic Act into force in January 2026 and, in July of that year, enacted the implementing decree carrying operators' obligations. That said, a grace period of at least a year comes with it, so the provisions do not translate straight into strong enforcement. In both regions the same fact holds: there is a lag between writing a right down and filling in the infrastructure that lets that right actually work.

6

Provenance is not only for auditors

Beyond the law and the rollout clock, the right to explanation has a third trap: it may be technically impossible to reconstruct. The hard part is not writing the explanation. It is recovering, after the fact, which features, weights, and thresholds the model actually used at the moment of decision. If no log was kept, then however firmly the right sits in the law, there is no material from which to build an accurate answer.

Here the place of data provenance shifts. Until now, provenance was an after-the-fact record for regulators and auditors — supplier-facing infrastructure by which an operator attests to where the training data came from and what processing it went through. The Pebblous work on tracing and erasing training data at the token level stood in that same line. Was the data good, and can it be removed?

The right to explanation over hiring and welfare decisions moves that question one square over. Provenance is also the infrastructure that lets you answer when the individual who received the decision asks why. Unless you record, at the moment of decision, which data made this person's score by which path, and which factors actually influenced that judgment, no amount of later asking can reconstruct it. Just as a right evaporates before the single sentence "it was not fully automated," an explanation evaporates before the fact that no record was kept.

Rows of server racks inside a data center, representing the infrastructure needed to record decision-time data
▲ Reconstructing the features, weights, and thresholds behind a past decision requires infrastructure that captured them at the moment the decision was made | Source: Wikimedia Commons (Carl Lender, CC BY 2.0)

If the condition for AI-ready data has so far been input quality, then for decisions that touch a life — hiring, welfare — that condition extends to the reconstructability of the moment of decision. For the right to question to guarantee anything real, alongside the law's threshold and the rollout's clock there has to be data infrastructure that keeps the basis of a decision. A right begins in a statute, but the place that right gets its answer is, in the end, the data.

R

References

Statutes & official documents

Case law & litigation

Industry & press commentary