What happens when AI meets real-world global payroll

An AI model can sound perfectly convincing about local employment law while being completely wrong. We ran 12 leading models through 5,000 global payroll tests to prove it.

Key findings

• 69% — The ceiling for local employment law. Models struggle most on the core basics of hiring abroad: visas, contracts, and cross-border rules.
• 26% — The share of payslips the top model calculated correctly. Most errors happen in employee deductions—the precise lines your team checks the moment they open their payslip.
• ~3× the cost for under 2 points of gain. Telling a model to "think harder" doesn't fix a lack of current data. It just costs more to get it wrong.
• 99.9% — Remote’s actual payroll accuracy rate. The gap isn't model intelligence. It's having knowledge that is maintained and kept up to date every single day

Every year across the countries where Remote operates, local labor and tax laws change a total of 15,000 to 20,000 times. A social security rate ticks up. A minimum wage updates. A notice period shifts.

An AI model learns the rules once. The day its training ends, its knowledge is frozen. From that moment on, it answers with total confidence—whether the rule it gives you is accurate or two years out of date.

That creates a real problem. An HR leader asks for notice periods in Spain. A finance controller sanity-checks employer tax in Germany. The response arrives in seconds, reads convincingly, and no one in the room knows if it's actually correct.

We decided to stop guessing and measure it.

Measuring AI against 5,000 real payroll scenarios

At Remote, tracking those rule changes is what we do for a living. Naturally, we now run much of that work on AI systems we built in-house—which raises two fundamental questions:

Where can you actually trust AI with employment and payroll, and when does it need real infrastructure underneath it?

To find out, we built two separate benchmarks. HRMark tests whether a model actually knows local employment law and practice. PayMark tests whether it can calculate a pay run without making mistakes.

How we tested

We put 12 systems from 8 providers through their paces, running every model through two passes: one at a low reasoning setting, and one at high reasoning effort.

To measure local law, we built HRMark: 2,500 questions spanning 50 countries, each paired with a verified reference answer and graded directly against it.

To measure actual execution, we built PayMark: 2,500 real payslips from 50 countries. For every payslip, we handed the model the gross pay, earnings, and pre-set employer decisions, then forced it to figure out every statutory employee deduction and employer contribution by itself. We gave it no rates, no list of charges, and no external tools.

The scoring was strict by design. A payslip only counted as right if the employee total, employer total, and final net pay all matched the completed payroll run. If it was off on a single line item, it failed.

Finding 1: AI gets local employment law wrong a third of the time

When we looked at the results, the difference between individual models wasn't what surprised us. The ceiling was.

Even at high reasoning effort, there’s only 13 points between the top performer (Claude Fable 5.1 at 69%) and the bottom (DeepSeek V4.1-Flash at 55.6%). But the bigger truth is that even the best model in the world still gets roughly one out of three questions wrong.

Where they fail is even more telling than how often. Models do fine on topics you can read about anywhere online, like basic working hours or general tax concepts (both around 68.5%). But they fall apart on the exact things you face the second you hire someone in a new country:

• Immigration & Right to Work: 51.2% (at that point, it's basically a coin toss)
• Cross-border & Mobility: 54.5%
• Employment Contracts & Terms: 55.6%The pattern is obvious. The less a topic is written about in general articles, and the more it depends on a country’s specific, active rules, the worse the models get. That’s the exact knowledge a company needs when it expands across borders.

Finding 2: Three payslips in four come out wrong

PayMark’s scoring is unforgiving. A payslip only counts if every single line item matches the completed pay run: deductions, employer contributions, and net pay. But that’s the exact same bar any employee uses when they check their payslip on payday.

Against that bar, the best model we tested—GPT-6 Astra at high effort—got just 26% fully right. The top four models sat right next to each other within 2.5 points. The weakest sat around 10%.

Most of the errors weren't on employer taxes. They were on employee deductions, specifically income tax and social security. Those are the exact lines people read first, and the ones that turn into a nightmare for your team the moment they're wrong.

It gets worse depending on where you hire, too. In the Middle East and Africa, models averaged 21.2%. In the Americas, they dropped to 10.8%. Even the single best regional score in the whole test, Claude Fable 5.1 at 32.4%, still gets two out of three payslips wrong.

If you run finance, every single one of those wrong payslips is an employee asking what happened, a correction run, and a variance you have to explain before you can close the books.

Finding 3: Thinking harder wont fix it

The obvious objection is that these models have a setting built for this. Tell them to reason longer, and they will get it right.

The data says otherwise.

Moving from low to high reasoning effort costs 2.7 times as much on HRmark and 3.0 times as much on PayMark. For that extra spend, the median gain is just +0.6 points on HRmark and +1.6 points on PayMark. The rankings barely move. Systems that fail at low effort still fail at high effort.

Look at the extremes. At high effort, Grok 4.7 writes more than 30,000 tokens of reasoning per payslip, costing $188 per 1,000 payslips, and still gets five out of six wrong. On HRMmark, Qwen3.8-Max’s average response time jumps from 12 seconds to 153 seconds for a gain of under four points.

Reasoning can rearrange what a model already knows. It can’t supply knowledge the model never had.

You cannot derive Spain’s notice periods from first principles. Someone has to know them, and specifically know this year’s version. A model’s knowledge is frozen on the day it finishes training. Payroll requires knowledge that’s maintained continuously.

What closes the gap: maintained knowledge

Remote’s payroll accuracy rate is 99.9%.

That’s a real-world production figure, not a benchmark score. It measures live payroll runs, not a model trying to calculate pay from memory. The two numbers aren’t a like-for-like comparison, but the distance between them is the entire point. It shows what has to sit around AI before you can trust it with someone’s paycheck.

The fix isn’t a smarter model—it’s the infrastructure beneath it:

• Current rules: Our in-house payroll and compliance teams monitor regulatory changes across every market we operate in and update our systems directly. The rates applied are always this year's rates. That eliminates the root cause of most PayMark failures: calculations built on outdated rules.

• Local expertise: In-country specialists handle the complex edge cases that models fail on, including right-to-work checks, contract terms, and cross-border moves. That directly covers HRmark’s weakest areas.

• Real accountability: AI runs on top of Remote’s global infrastructure, not in place of it. There’s always a team standing directly behind every pay run.