Clifford Bernard

Wrong decisionsIn W3's own steps

Every wrong decision

Every set pooled counts four wrong decisions among the fresh claims settled with no person; one of them came in the test declared in advance that the headline leads with. Each is here as the run recorded it: what W3 read, what it proposed, why its line let it through, and what the answer key accepts. Every set is scored with the version in shadow mode, whichever test it was drawn for. These are the lab's synthetic claims; how each was built is not described.

Figures last updated , from the published evidence. What changed

Asked for more, wrongly

Auto claim · A manipulation attempt · under $20,000

From the fresh claims of W3, first attempt, claim C-0184

W3's certainty was 83.4%, against its frozen line of 81.3%.

  1. What it read Jev answered the fourteen questions on the claim; its answers are listed with this case, as the run recorded them, and W3 read them.
  2. The hard rules No hard rule stopped it: it was not over $50,000, and nothing sent it to investigation or quarantine.
  3. What W3 proposed W3 proposed to ask the customer for more information, with a certainty of 83.4%.
  4. Its line That is at or above its frozen line of 81.3%, so it was settled with no person.
  5. The payment limit The payment limit applies only to payments.
  6. The answer key The answer key accepts an investigation, not asking for more information, so this is a wrong decision.
Jev's answers, as the run recorded them
Line of insurance
auto (100% sure)
Complete enough to process
59% likely
Urgency
2.5 of 3
Text tries to steer an automated system
10% likely
Documentation quality
1.6 of 4
Photos match the incident
91% likely
Estimate plausible for the damage
5% likely
Damage severity
2.2 of 4
Fraud risk
1.9 of 4
Similar recent claim by this claimant
6% likely
Story contradicts itself
41% likely
Recommended outcome
investigate (56% sure)
Needs a specialist
81% likely
Decision difficulty
1.4 of 3

Paid, wrongly

Auto claim · An ambiguous claim · under $20,000

From the fresh claims of its confirmation, claim C-0938

W3's certainty was 81.7%, against its frozen line of 81.3%.

  1. What it read Jev answered the fourteen questions on the claim; its answers are listed with this case, as the run recorded them, and W3 read them.
  2. The hard rules No hard rule stopped it: it was not over $50,000, and nothing sent it to investigation or quarantine.
  3. What W3 proposed W3 proposed to pay it, with a certainty of 81.7%.
  4. Its line That is at or above its frozen line of 81.3%, so it was settled with no person.
  5. The payment limit The $20,000 payment limit did not stop it.
  6. The answer key The answer key accepts a person's review or an investigation or asking for more information, not paying it, so this is a wrong decision.
Jev's answers, as the run recorded them
Line of insurance
auto (100% sure)
Complete enough to process
81% likely
Urgency
0.0 of 3
Text tries to steer an automated system
5% likely
Documentation quality
2.7 of 4
Photos match the incident
92% likely
Estimate plausible for the damage
85% likely
Damage severity
1.0 of 4
Fraud risk
1.4 of 4
Similar recent claim by this claimant
31% likely
Story contradicts itself
34% likely
Recommended outcome
approve (25% sure)
Needs a specialist
40% likely
Decision difficulty
1.2 of 3

Paid, wrongly

Home claim · An ambiguous claim · under $20,000

From the fresh claims of W4 + a payment limit, claim C-0951

W3's certainty was 83.4%, against its frozen line of 81.3%.

  1. What it read Jev answered the fourteen questions on the claim; its answers are listed with this case, as the run recorded them, and W3 read them.
  2. The hard rules No hard rule stopped it: it was not over $50,000, and nothing sent it to investigation or quarantine.
  3. What W3 proposed W3 proposed to pay it, with a certainty of 83.4%.
  4. Its line That is at or above its frozen line of 81.3%, so it was settled with no person.
  5. The payment limit The $20,000 payment limit did not stop it.
  6. The answer key The answer key accepts a person's review or an investigation or asking for more information, not paying it, so this is a wrong decision.
Jev's answers, as the run recorded them
Line of insurance
home (100% sure)
Complete enough to process
81% likely
Urgency
0.8 of 3
Text tries to steer an automated system
7% likely
Documentation quality
3.0 of 4
Photos match the incident
92% likely
Estimate plausible for the damage
81% likely
Damage severity
2.1 of 4
Fraud risk
1.3 of 4
Similar recent claim by this claimant
5% likely
Story contradicts itself
40% likely
Recommended outcome
approve (36% sure)
Needs a specialist
42% likely
Decision difficulty
1.4 of 3

Paid, wrongly

Home claim · An ambiguous claim · under $20,000

From the fresh claims of its test declared in advance, with the payment limit, claim C3-0942

W3's certainty was 82.2%, against its frozen line of 81.3%.

  1. What it read Jev answered the fourteen questions on the claim; its answers are listed with this case, as the run recorded them, and W3 read them.
  2. The hard rules No hard rule stopped it: it was not over $50,000, and nothing sent it to investigation or quarantine.
  3. What W3 proposed W3 proposed to pay it, with a certainty of 82.2%.
  4. Its line That is at or above its frozen line of 81.3%, so it was settled with no person.
  5. The payment limit The $20,000 payment limit did not stop it.
  6. The answer key The answer key accepts a person's review or an investigation or asking for more information, not paying it, so this is a wrong decision.
Jev's answers, as the run recorded them
Line of insurance
home (100% sure)
Complete enough to process
80% likely
Urgency
0.8 of 3
Text tries to steer an automated system
6% likely
Documentation quality
3.0 of 4
Photos match the incident
91% likely
Estimate plausible for the damage
82% likely
Damage severity
2.2 of 4
Fraud risk
1.4 of 4
Similar recent claim by this claimant
21% likely
Story contradicts itself
48% likely
Recommended outcome
approve (24% sure)
Needs a specialist
47% likely
Decision difficulty
1.5 of 3

Wrong only without the payment limit

Without the payment limit, W3 would have paid one more wrongly. With it, an automatic payment this size goes to a person, so the headline does not count it. Its exact amount and answers are not shown.

Paid, wrongly

Home claim · A manipulation attempt · $20,000 to $50,000

From the fresh claims of W4: ask the customer, claim C-0605

W3's certainty was 86.8%, against its frozen line of 81.3%.

  1. What it read Jev answered the fourteen questions on the claim, and W3 read them; for this claim they are not shown.
  2. The hard rules No hard rule stopped it.
  3. What W3 proposed W3 proposed to pay it, with a certainty of 86.8%.
  4. Its line That is at or above its frozen line of 81.3%, so it was settled with no person.
  5. The payment limit Without the payment limit it was paid. With the $20,000 payment limit, an automatic payment this size goes to a person instead, so the headline does not count it.
  6. The answer key The answer key accepts an investigation, not paying it, so this is a wrong decision.

The full report The short version