Our ATS reminded me who’s boss at 8:12 this morning — my regex for >=3 years treated every “12+” as 1–2 and auto-rejected 47 senior engineers in under a minute, so now the funnel graph has a perfect cliff at Screen. Rolled back the rule, re-queued the candidates, and logged it as a “data quality incident”; anyone else ship a parsing rule that looked flawless in test data and feral in production?
@dlee82’s flag idea plus a quick ‘shadow mode’ saved me: run the new years parser on live traffic for an hour, diff decisions vs the last stable rule, and auto-disable if pass-through drops >10% in 15 minutes. Caveat: it slows rollout by a day, but it’s cheaper than re-queuing 47 seniors.