HackerRank Open-Sourced Its ATS and Nobody Can Agree What It Scores
HackerRank open-sourced its ATS and devs are stress-testing it on their own resumes — with wildly inconsistent results.
The signal: HackerRank open-sourced its Applicant Tracking System and the HN crowd immediately turned it into a resume scoring circus — same resume, same tool, scores jumping between 74, 88, and 90 depending on who’s running it.
Why it matters: If you’re building hiring pipelines or any product that scores human-generated documents, this is a live stress test of what non-determinism looks like in production. Resume scoring with variance that wide isn’t a feature — it’s a liability, and your users will notice before you do.
The pattern I’m watching: Open-sourcing evaluation tooling is becoming a legitimacy play — “trust us, look at the code” — but the real question is always reproducibility. Age verification systems are heading the same direction: the tech gets open, the outcomes stay opaque.
What I’d do with this: Fork it, run your own resume through it five times, and screenshot the variance — that’s your benchmark for whether it’s production-ready. If you’re building anything that scores people, lock your randomness seeds and version your prompts like they’re contracts.