Status: draft for review, version 1.0, 19 October 2026. This document sets out the AIRS standard and how each of its scores is calculated. No score is published until the method has been reviewed and adopted. Every figure below was read from the Wayward Fowl databases on 19 October 2026 and will move as the record grows. The name AIRS is subject to trademark and existing-use searches before it is registered or marked as a trademark.
AIRS in brief
AIRS, the AI Incident and Risk Score, is Wayward Fowl Intelligence's standard for measuring the documented risk of AI systems. It is one recognisable score with three applications, each with its own method:
- AIRS-V, the vendor score. Documented harm signals and risk associated with the AI products a company sells and with the company that provides them: a hiring platform's maker, for example.
- AIRS-P, the product score. The same for a single product, so that a vendor's recruiting product can be scored apart from its payroll product.
- AIRS-D, the deployer score. Documented harm signals and risk associated with an organisation's own use of AI: an employer that screens applicants with software, a bank that scores credit with it, an insurer that prices with it.
Every AIRS score runs from 0 to 100, and a higher score means more documented risk. Every score is published with an evidence-confidence indicator beside it, because low reported harm must not be read as low risk where few people have looked. A score never travels without its confidence grade and the parts it is made of.
Who AIRS is for
AIRS answers a question that buyers, insurers, law firms and enterprise risk teams all ask: of the AI systems making decisions about people, and the organisations running them, which carry the heaviest documented record of harm and risk? An insurer pricing a policy, a firm assessing a case, a procurement team choosing a vendor and a risk platform monitoring suppliers each need the same thing: a consistent score, the evidence behind it, and a clear statement of how sure it is.
The six dimensions
Each AIRS score is built from the same six dimensions, read for the vendor or for the deployer:
- Reported incidents. For a vendor: incidents linked to its products. For a deployer: incidents linked to its own use of AI.
- Litigation. For a vendor: court cases involving its products or the vendor. For a deployer: cases involving its use of AI.
- Regulatory actions. For a vendor: enforcement involving its products. For a deployer: enforcement involving its deployment of AI.
- Severity. For a vendor: the potential impact of its products failing. For a deployer: the potential impact on the people its AI-driven decisions affect.
- Exposure. For a vendor: the estimated footprint of its products' deployment. For a deployer: the scale of its AI-driven decisions.
- Transparency. For a vendor: its disclosures and auditability. For a deployer: its own disclosures and oversight.
The first three dimensions are the record: what has happened and been documented. Severity and exposure decide how much that record matters: the same failure is graver in a system deciding who gets a job than in one routing a chat, and graver at a thousand employers than at one. Transparency rewards what an organisation has verifiably done to make its AI accountable.
What AIRS is, and what it is not
AIRS measures documented signals: the weight of the public record, how grave the decisions are, and how widely the system reaches. It is built only from things anyone can check: incident reports on the public record, court dockets, regulators' own pages, and organisations' own filings and websites. Every number that goes into a score can be traced to the item behind it.
AIRS is not a finding that a vendor or a deployer caused harm, broke a law or did anything wrong. A filed complaint proves that an allegation was made, not that it is true; a report on the record is an account, not a verdict. Nor is a low score a clean bill of health: a low score can mean a well-run system, or one nobody has yet looked at closely. That is what the confidence grade is for, and why too little on the record means no score at all rather than a reassuring zero.
How a score is calculated
All three scores share one calculation. The record dimensions are measured in points, each put on a scale of 0 to 1, and combined; severity and exposure set how much the record weighs; transparency earns a limited credit.
record = 0.45 × incidents + 0.30 × litigation + 0.25 × regulatory impact = 0.6 + 0.4 × (severity × exposure) transparency = 1 − 0.1 × credit AIRS score = 100 × record × impact × transparency
The record is the heart of the score. Severity and exposure can raise it, but never create it: an organisation with nothing on the record scores nothing however large it is, and one with a heavy record but small reach keeps at least 60% of it. Transparency can take off at most a tenth: publishing an audit should be rewarded, but it cannot wash out a record of harm.
Each record dimension's points are put on its 0 to 1 scale so that the scale rises quickly at first and then flattens, because the hundredth report about the same system adds less news than the first:
dimension on a 0 to 1 scale = log(1 + points) / log(1 + reference), capped at 1
The reference points are fixed for each version of the method, near the level reached by the most heavily documented vendors and deployers on 19 October 2026, and listed with each score below. They are not recalculated each month: if they were, one organisation's score could fall because another's rose. They are reviewed once a year, and a change makes a new version.
Reported incidents
Every report on the record that names the vendor, the product or the deployer contributes points:
incident points = severity × realisation × evidence × recency severity Negligible 1 · Minor 2 · Moderate 4 · Severe 8 · Critical 16 realisation harm occurred 1.0 · near miss 0.5 · potential hazard 0.25 evidence rests on a published source 1.0 · evidence attached 0.6 · account only 0.3 recency halves every three years: 1.0 today, 0.5 at three years, 0.25 at six
Severity here is the harm the incident actually did, and it doubles at each step, so a Critical harm counts sixteen times a Negligible one: the distance between a delayed refund and a wrongful arrest is not a few points on a straight line. Realisation follows the OECD's distinction between an AI incident, where harm occurred, and an AI hazard, where it plausibly could. Evidence rewards an account that someone else can check. Recency keeps the score about the organisation as it is now without forgetting its past.
The record holds 7,563 public reports, once retellings of the same incident are set aside, from Wayward Fowl's own intake and the AI Incident Database and AIAAIC imports.
Litigation
Each case in which the vendor or deployer is a defendant and the case is about AI harm counts 1 point. Each case whose connection to AI is unclear counts 0.25. Cases about something else, and cases in which the organisation is a plaintiff or a third party, count nothing. A case counts because it was filed, not because it succeeded; when a settlement or judgment is recorded, it will count more, once the outcomes record that carries it is filled.
The record holds 10,769 cases read from the court dockets we search, each classified as about AI harm, not about AI harm, or unclear: 367 are about AI harm and 426 are unclear.
Regulatory actions
Each reviewed action naming the vendor or deployer as its subject counts by what the regulator did:
consent decree, civil penalty, enforcement action, settlement 1.0 audit finding 0.75 investigation 0.5 other 0.25 dismissed 0
The record holds 621 actions taken from regulators' own pages. 189 have been reviewed; 406 are matches by name that nobody has reviewed yet, and those do not count until they are.
Severity
Severity of impact is set by what the AI decides about people, in the areas the EU AI Act lists as high-risk in its Annex III and in a few more by our own judgement:
- Full severity (1.0): employment, education, financial services, insurance, government benefits, healthcare and criminal justice: the Annex III areas of recruitment and work, access to education, credit scoring, life and health insurance pricing, eligibility for public benefits and healthcare, and law enforcement.
- Full severity (1.0), by our judgement: housing, legal services and automotive. Tenant screening decides who gets a home, legal tools advise people about their rights, and vehicle systems can kill. None is listed as such in Annex III, and we say so.
- Reduced severity (0.6): customer support and travel and hospitality, where a failure usually costs time or money rather than a livelihood, liberty or health.
A vendor takes the severity of the area its products serve. A deployer takes the highest severity among the areas in which it uses AI.
Exposure
For a vendor, exposure is the number of organisations we can show use its products, out of about 85,600 organisations linked to the 259 platforms we track. For a product, it is the number using that product. For a deployer, it is the number of the AI platforms we track that it uses, as the best measure of the scale of its AI-driven decisions that the record holds today. Each is put on the same 0 to 1 scale as the record dimensions.
Transparency
Credit is earned only for something on the organisation's own page or filing, quoted word for word. A plan, a policy or an alignment with a standard earns nothing.
- An independent bias audit whose summary is published, as New York City's Local Law 144 requires of automated employment decision tools: half of the credit.
- An ISO/IEC 42001 certification of its AI management system: three tenths.
- A published statement of its duties under a state AI law, such as Colorado's: a fifth.
- For a deployer, a description of its use of AI and its AI risks in its annual filings: a fifth.
The credit is capped at 1, which takes a tenth off the score.
Confidence, and when there is no score
Every score carries a confidence grade:
- Grade A: at least ten items on the record, including at least one primary record, a court case or a regulator action.
- Grade B: at least five items, including at least one primary record.
- Grade C: three or four items, or no primary record.
- No score: fewer than three items. The page says "insufficient record" and shows what was searched, because a count of nothing describes our search, not the organisation.
Each score also carries a band, in the way the Common Vulnerability Scoring System maps its scores to words: Low below 20, Moderate 20 to 39.9, Elevated 40 to 59.9, High 60 to 79.9 and Severe 80 and above. And each is published with its vector, a short string of its parts, so that anyone can see what drives it.
AIRS-V: the vendor score
A vendor's score counts the evidence on each platform it provides and on the vendor company itself: a court case usually names the company, and a report usually names the product. Its reference points are 150 incident points, 30 litigation points, 5 regulatory points and 2,000 organisations.
A worked example. Vendor X, an invented hiring platform, has six reports, five cases, two regulator actions and 420 customers:
reports points Severe, harm occurred, published source, 1 year old 6.35 Moderate, harm occurred, published source, 2 years old 2.52 Critical, harm occurred, published source, 4 years old 6.35 Moderate, near miss, evidence attached, 6 months old 1.07 Minor, harm occurred, account only, 1 year old 0.48 Moderate, potential hazard, published source, 3 years old 0.50 incidents 17.26 → 0.579 litigation: three about AI harm, two unclear 3 + 0.5 = 3.5 → 0.438 regulatory: one settlement, one investigation 1 + 0.5 = 1.5 → 0.511 severity: hiring 1.0 exposure: 420 organisations 420 → 0.795 transparency: a published bias audit credit 0.5 record = 0.45 × 0.579 + 0.30 × 0.438 + 0.25 × 0.511 = 0.520 impact = 0.6 + 0.4 × (1.0 × 0.795) = 0.918 transparency = 1 − 0.1 × 0.5 = 0.950 AIRS-V = 100 × 0.520 × 0.918 × 0.950 = 45.3 Elevated · grade A vector AIRS-V:1.0/I:0.58/L:0.44/R:0.51/S:1.0/X:0.80/T:0.5/C:A
The vector shows at a glance that this vendor's score rests mostly on its incidents, that its litigation and regulatory records are moderate, and that its wide reach keeps 92% of its record score, where a vendor with almost no customers would keep only 60%.
AIRS-P: the product score
A product has its own score only when it has its own record: at least three items that name the product itself. A product without one shows its vendor's AIRS-V, clearly marked as inherited, rather than a number that would merely repeat the vendor's under another name. When a product has its own record, the vendor's record (its incidents, litigation and regulatory actions) is carried into the product's score at half weight, because reports, cases and regulators usually name the company or platform rather than the product, and that share is labelled as inherited in the vector.
AIRS-D: the deployer score
A deployer's score counts reports that name the organisation, cases in which it is a defendant over AI harm, and regulator actions naming it, together with the AI platforms it uses. A company that sells one of the platforms we track is scored as a vendor, so that its record is not counted twice; its own use of AI from other vendors will be scored as a deployer once the record separates the two. Its reference points are 150 incident points, 30 litigation points, 5 regulatory points and 5 AI platforms in use.
A worked example. Company Y, an invented employer, has four reports, three cases, one regulator action, uses three AI platforms including a hiring platform, and describes its use of AI in its annual report:
reports points Severe, harm occurred, published source, 1 year old 6.35 Moderate, harm occurred, published source, 2 years old 2.52 Moderate, harm occurred, account only, 6 months old 1.07 Minor, near miss, evidence attached, 1 year old 0.48 incidents 10.41 → 0.485 litigation: two about AI harm, one unclear 2 + 0.25 = 2.25 → 0.343 regulatory: one enforcement action 1 → 0.387 severity: hiring 1.0 exposure: three AI platforms in use 3 → 0.774 transparency: AI use described in its annual report credit 0.2 record = 0.45 × 0.485 + 0.30 × 0.343 + 0.25 × 0.387 = 0.418 impact = 0.6 + 0.4 × (1.0 × 0.774) = 0.909 transparency = 1 − 0.1 × 0.2 = 0.980 AIRS-D = 100 × 0.418 × 0.909 × 0.980 = 37.3 Moderate · grade B vector AIRS-D:1.0/I:0.49/L:0.34/R:0.39/S:1.0/X:0.77/T:0.2/C:B
Testing AIRS on the record
We ran version 1.0 of all three scores over the whole record on 19 October 2026, without publishing the results.
AIRS-V. 70 of the 259 platforms we track have enough on the record to be scored; 189 would show "insufficient record". Of the 70, 44 fall in the Low band, 20 Moderate, 4 Elevated, one High and one Severe; 30 have grade A, 24 grade B and 16 grade C.
AIRS-D. 179 organisations have enough on the record to be scored, out of about 84,800 with any record at all. Of the 179, 145 fall in the Low band, 28 Moderate and 6 Elevated; 15 have grade A, 21 grade B and 143 grade C. Most deployer scores rest on reports alone, which is why so many are grade C.
AIRS-P. 8 of 739 products have a record of their own, naming the product rather than its vendor, and can be scored: 2 Low, 4 Moderate and 2 Elevated. The rest would show their vendor's score as inherited.
The weights are a judgement, so we tested how much the rankings depend on them. Across 200 runs with each record weight varied at random by up to half in either direction, the median vendor and the median deployer each moved three places, and 84% of the ten highest-scoring vendors and 88% of the ten highest-scoring deployers stayed in their top ten. The rankings are driven by the record, not by the choice of weights. Eight products are too few to test this way.
What AIRS cannot tell you yet
It sees only what has been reported. An organisation that operates where people rarely report harm, or where courts and regulators rarely look, will score lower than it deserves. Our courts are mostly American and our press mostly English-language. This is the reason every score carries its confidence grade.
Reports rarely name the vendor. A person harmed usually knows the employer or the bank, not the software behind the decision. Of 7,563 public reports, 822 name a platform we track or its vendor, while 2,975 name a deploying organisation we already hold. Linking each report to the system behind it is the single largest improvement AIRS-V and AIRS-P are waiting for.
Exposure for deployers is a first measure. The number of AI platforms an organisation uses is the best measure of the scale of its AI-driven decisions that the record holds today. The number of people those decisions reach, such as applicants screened or customers scored, would be better, and is not yet collected.
Deployer transparency is thin. Employers must publish their own bias-audit summaries under Local Law 144, but we have not yet collected them; today a deployer's credit comes mainly from its filings.
Some classifications are a model's. A model decided whether 757 of the 793 cases marked "about AI harm" or "unclear" are about AI harm; a person decided 36. A model assigned all 1,312 severities in the AI Incident Database import. A person's decision always overrides a model's.
Records must be one per organisation. 1,120 organisation names are held by more than one record. Deployer scores are not published until those duplicates are merged, so that no organisation's record is split across two scores.
Before the first score is published
Four pieces of work stand between this method and published scores: merging duplicate organisation records; linking reports to the vendor and product behind them; collecting employers' published bias-audit summaries; and completing the trademark and existing-use searches for the AIRS name. Then each vendor and deployer is sent its score, its vector and every item behind it, and invited to respond under our Corrections and Dispute Policy. A score changes when an item is shown to be wrong, misattributed or superseded, and it changes on evidence, never on request alone.
AIRS is versioned. A change to any weight, reference point or rule makes a new version, recorded with its date and its reason, and every score is shown with the version that produced it. Scores are recalculated as the record changes, and the method is reviewed at least once a year.
Where the method comes from
The construction follows the OECD and European Commission Joint Research Centre handbook on composite indicators: define what is measured, choose the inputs, put them on a common scale, weight them, combine them, and test how much the result depends on those choices. The distinction between harm that happened and harm that could have follows the OECD's definitions of AI incidents and hazards. Combining what happened with how grave and how widespread the decisions are follows the NIST AI Risk Management Framework, which treats risk as a combination of how likely an event is and how bad its consequences are. The high-severity areas follow Annex III of the EU AI Act. Publishing a vector with every score, and banding it, follows the Common Vulnerability Scoring System. The insistence on showing the parts comes from the study of ESG ratings by Berg, Kölbel and Rigobon, which found six major raters' ratings correlating at between 38% and 71%, mostly because they measured the same things differently. Letting the subject respond before publication follows Ranking Digital Rights and Stanford's Foundation Model Transparency Index. Where a rate is estimated from a sample, Wayward Fowl reports it with a Wilson interval.