Your manager rated you a 2.
Your work says 4.

Bias lives in groups

When I thought about where bias comes from, it kept coming back to groups. Your gender. Your age. How long you've been there. Where you work, what kind of contract you're on, your department. And your manager.

So the check compares ratings across all of those, and across managers, and flags the group that's furthest out. Then it sets that group aside and checks the rest again, so one low group shows up as one flag, not five.

The maths decides. The AI just writes the note

I didn't want the AI judging anybody. The maths does the filtering: it works out who actually needs a second look and who doesn't, so the AI has way less to do.

It's also more private that way. The AI isn't going through everybody's data. The company chooses what gets passed, and the AI only sees what it needs to write one note.

The evidence is the stuff the manager doesn't control: measured objectives count the most, then peer feedback and past cycles. The employee's own progress update counts for less, because it's their own figure.

The AI doesn't judge. It suggests.

A human has to be in the loop

Say a manager writes that someone isn't a culture fit, but the person is excellent at what they do. The note can point out that the rating is about character, not about the work.

But some companies hold culture fit really high, and maybe the manager knows the person isn't trainable. A human has more context than the app, because a lot of things happen outside the application. So nothing gets decided for you: every flagged rating needs a decision from a person before the review can be published.

Five people isn't enough

Small companies have small groups. If there are only three women in a department, an average of three people doesn't tell you anything, and it's small enough that you could work out who got what.

So any group under five is hidden and left out of the comparison. And if hiding it would still let you work out the missing numbers by subtracting, the next smallest group gets hidden too.

The note shouldn't read like a court transcript

The first versions quoted exactly what the manager and the employee wrote. It didn't feel natural to read. So now it passes on what each person was saying, with enough context, in three or four sentences, with the exact figures in it. And it says what the evidence suggests, not what someone's “record” says.

Names never go to the model. Everyone becomes a placeholder before the prompt, and the names only come back on screen. Notes are saved and reused until the inputs change, so it isn't paying for the same note twice.

The first time it got one right

The first time it flagged someone correctly in the demo data, I was so happy. Like, is there a human in this computer actually doing this? It felt unreal.

Of course, I'm a badass engineer, so there's that too. Haha.