About IT Challenge Coins
Militaries hand out challenge coins for acts of valor. IT has its own kind of bravery: leaving the load-bearing comment alone, owning up to the one-line change that took down production, accepting the promotion that doubled your workload. These coins recognize those moments.
Give one to a teammate, keep one on your desk, or slide one across the table at the next retro. And if you've ever wondered where the idea came from, it came from a day like this one.
One Day in the Life of Dale Mercer
Dale is fictional, a composite of every engineer we've worked beside. Every one of these things has happened to somebody. Most of them have happened to all of us.
3:07 AM: The Summoning
The phone didn't ring so much as detonate. Dale Mercer, Senior Systems Engineer at Halcyon Mutual Insurance (a company we made up) for fourteen years, came awake the way only on-call people do: already sitting up, already reaching, already composing the sentence "I'm looking into it" before his eyes had opened.
The alert said CRITICAL: customer-portal UNREACHABLE. It did not say why. Alerts never say why. They are town criers
who shout "the castle is on fire" and then refuse to tell you which tower.
He padded down the hall in socks, past the room where his daughter slept, and opened the laptop on the kitchen counter. The glow lit the room the color of a hospital corridor. The portal servers were up. The load balancer was up. The database was up. Everything was up, and nothing worked.
He checked the firewall. He checked the certificates. He restarted the web tier, because sometimes the old magic still works. He said, out loud, to the empty kitchen, "It's not DNS." He checked something else. He said, a little quieter, "There's no way it's DNS."
At 4:41 AM he found it: someone had decommissioned an old domain controller the previous afternoon, and with it went a DNS forwarder that, according to no documentation anywhere, the portal had been quietly depending on since 2017. He pointed the portal at the right server, waited for a TTL that felt longer than his marriage, and watched the dashboard turn green one tile at a time.
"It was DNS," he whispered, and went back to bed for ninety minutes of the worst sleep available to humankind.
Coins earned: Knights of the 3 AM Page · The DNS Memorial Guard
6:52 AM: The Gauntlet of the Inbox
Dale read email in the school drop-off line, which is the closest thing an IT engineer has to a morning commute meditation.
The first message was from the CEO. It was urgent. It needed him to purchase six gift cards immediately and reply with the codes, because she was "in a meeting and can't talk." She had also, apparently, changed her email address overnight to a free webmail account and forgotten how commas work.
Dale reported it to security, blocked the sender, and sent a short, gentle note to the whole company: "If our CEO ever asks you for gift cards, it isn't our CEO." Within the hour, two people in Accounts Payable thanked him. One of them admitted she'd already been to the store and was standing in the checkout line.
Coin earned: The Phishing Defense Fleet
7:45 AM: The Beast on the Third Floor
He had not yet reached his desk when Marjorie from Claims intercepted him at the elevator. She did not say good morning. She said, "It's doing it again."
There is a printer on the third floor of Halcyon Mutual. Nobody knows who bought it. It has outlived three leases, two office managers and one merger. It speaks only in error codes, and the codes are lies. That morning it reported a paper jam in Tray 2. The printer does not have a Tray 2.
Dale approached it the way you approach a dog you don't know. He opened every door. He pulled out a crumpled, half-printed page from a meeting agenda dated March of last year. He closed every door. He turned it off. He counted to thirty, because the counting matters, nobody knows why, and nobody is willing to find out. He turned it on.
It printed Marjorie's report, then forty-one copies of the March agenda, then stopped, satisfied, as if it had made its point.
Coins earned: Printer Pacification Corps · Ancient Order of Have You Tried Turning It Off and On Again
9:00 AM: The Round Table
The weekly status meeting had been scheduled for thirty minutes. It was, as always, sixty. Eleven people attended. Three spoke. One presented a slide deck whose second slide was titled "Agenda" and whose final slide, forty-six slides later, was titled "Next Steps: Schedule Follow-Up Meeting."
Near the end, a project manager turned to Dale and asked how long it would take to move the claims system to the cloud. Every head in the room turned toward him, the way a herd turns toward a sound in the grass.
Dale knew the honest answer. The honest answer involved a 2009 vendor library, four undocumented integrations and a batch job nobody had dared touch since flip phones were cool. The honest answer was "nobody can know that yet."
What he said was, "Probably two weeks."
He has regretted it every single day since. It is now month seven.
Coins earned: Knights of the Meeting That Could Have Been an Email · The Order of the Two-Week Estimate · The Fellowship of the Scope Creep
10:15 AM: The Excavation
Back at his desk, Dale opened a ticket that had been waiting for him with the patience of a curse: a small change to how the billing system rounded premiums. One line. Maybe two.
The billing system was older than two of his coworkers. Its source code read like an archaeological dig: layer upon layer of developers long departed, each leaving fossils of their era. There were comments in it written by people who had retired. There were comments written by people who had, as far as anyone could tell, simply walked into the sea.
Deep in a file called BillingUtilsFinal2.cs, he found it. A single, empty line of code, and above it, a comment in all
capital letters:
// DO NOT REMOVE. I DON'T KNOW WHY. - R.Dale had been here long enough to know who R was, and that R had once kept a system alive through an entire holiday weekend using nothing but a cron job and spite. He respected R. He left the line exactly where it was, the way you leave a stone in a pyramid that you suspect is holding up the pyramid.
Then he made his one-line change three files away, ran the tests, and watched the nightly invoice job crash in a way that had absolutely nothing to do with rounding.
He spent the next two hours learning that his change had shifted the order in which two modules loaded, and that the second module had been relying, for eleven years, on a bug in the first module to suppress a different bug in a third. He had touched it, and it broke. He fixed it properly, wrote down everything he'd learned, and added a comment of his own beneath R's:
// R was right. Here's why: see docs/billing-load-order.md - D.Coins earned: The Order of the Load-Bearing Comment · I Touched It and It Broke · Legion of the Legacy Codebase · The Order of the Documentation
12:20 PM: The Certificate That Time Forgot
Dale had taken exactly one bite of a turkey sandwich when the support line lit up like a switchboard in an old war movie. Agents couldn't reach the partner API. Customers couldn't reach the agents. Somewhere, a vice president had noticed.
The error was one every engineer recognizes in their bones: certificate has expired.
It had been a three-year certificate. Someone had diligently set a renewal reminder. The reminder had gone, diligently, to the inbox of the person who'd set it, who had left the company two years ago to open a bakery.
Dale renewed it, installed it, restarted the services and watched the errors drain away. Then, instead of finishing his sandwich, he spent his lunch writing a script that checks every certificate the company owns and emails a shared team mailbox sixty days before any of them expires. The sandwich did not survive. The company did.
Coin earned: The Order of the Expired Certificate
1:30 PM: The Storm
It began innocently. Someone in Facilities sent a note to the all-company distribution list about a parking lot repaving. All 2,400 employees.
Someone replied all: "Please remove me from this list."
Someone replied all to that: "Me too."
Someone replied all to that, asking everyone to please stop replying all, and in doing so became part of the storm they sought to end. Within eleven minutes there were three hundred messages. The mail server's queue began to climb like floodwater. Somebody, somewhere, replied all with a GIF.
Dale didn't send a single email. He quietly restricted who could post to the list, purged the queue, and went back to work. The storm passed the way storms do. People still talk about it. They call it "Parking Lot Tuesday," even though it was a Friday.
Coin earned: Regiment of the Reply-All Storm
2:45 PM: The Spreadsheet Beneath the World
A routine audit request arrived: list every system that stores customer data. Dale knew most of them by heart. Then, tracing a strange nightly connection from the finance subnet, he found the one he didn't know.
It lived on a shared drive. It was called Master_Rates_FINAL_v3_USE_THIS_ONE.xlsx. It had 940,000 rows, nineteen tabs,
a macro written by a summer intern in 2015, and it was, he slowly realized, the actual source of truth for every premium the company had
quoted in eight years. The billing system he'd spent the morning in? It imported this file every night at 2 AM.
He sat very still for a long moment. Then he made a backup. Then he made another backup somewhere else. Then he opened a ticket titled, with great restraint, "Discussion: spreadsheet."
Coins earned: Society of the Spreadsheet Database · The Shadow IT Brigade
3:30 PM: The Silent Counsel
A junior developer, Priya, appeared at the edge of his cubicle with the expression of someone who has been staring at the same bug for six hours. She wanted to know if he had a minute. He did not. He said yes anyway, because someone had once said yes to him.
She started explaining the problem. Dale listened. He nodded. He did not say a word. Halfway through her third sentence she stopped, stared at the wall, said "Oh. Oh. It's an off-by-one," and walked away without saying goodbye.
On his desk sat a small rubber duck wearing a tiny crown, a gift from a team he'd led years ago. Dale looked at the duck. The duck looked back. They had both done their jobs.
Coins earned: The Order of the Rubber Duck · The Order of the Off-By-One
4:55 PM: The Charge
It was Friday. I should have mentioned that. It had been Friday all along.
At 4:55 PM, the release manager pinged the team channel. The fix for the claims form (the one customers had been complaining about for a month, the one marketing had promised in a press release) was ready. Tested. Approved. Should they ship it now, or wait until Monday?
Everyone in IT knows the rule. You do not deploy on a Friday. You certainly do not deploy at 4:55 PM on a Friday. The rule is written in no handbook, because it doesn't need to be. It is written in the scars.
Dale looked at the change. He looked at the tests. He looked at the rollback plan, which he had written himself, twice. He thought about the customers who'd spend another weekend fighting a broken form.
"Ship it," he said. "I'll watch it."
It worked on staging. It worked in production. For eleven minutes. Then a validation rule that existed only in production (put there by hand, years ago, by someone who'd forgotten to tell anyone) began rejecting every claim filed from the state of Ohio.
Dale rolled it back in four minutes. He found the rule in twenty. He fixed it properly, redeployed at 6:10 PM, and watched the logs until Ohio was whole again. Nobody outside the team ever knew. That's the job. When it goes right, it looks like nothing happened.
Coins earned: The Order of Friday Deploys · Brotherhood of "It Works On My Machine"
6:30 PM: The Near Miss
The building was empty. Dale had one last task: clean up some test records a contractor had left in the database. He opened a query window,
typed DELETE FROM policies, and reached for the semicolon.
Then he stopped.
He looked at the top of the window. The connection label didn't say TEST. It said PROD.
He sat there for a long time with his hand frozen above the keyboard, listening to the hum of the server room down the hall, feeling his
heart in his ears. Then he closed the window without saving, opened a new one against the test server, added a WHERE clause,
and ran it there. Then he went and checked that last night's backups had actually completed, because now he needed to know.
They had. He exhaled for what felt like the first time since 3:07 AM.
Coins earned: The Order of the Fat Finger · Society of the Production Database
7:40 PM: The Promotion
Just as he was packing up, an email arrived from the Director of IT. Congratulations were in order. Dale was being promoted to Engineering Manager. He would now lead the infrastructure team, own the budget, run the hiring, and attend the leadership meeting every Tuesday.
He would also, the email noted cheerfully, continue in his current on-call rotation "until we can backfill the role."
There was no mention of a raise. There was a mention of "growth opportunities."
Dale read it twice, said "huh," and turned off the light.
Coin earned: Achievement Unlocked: The Manager-Developer
8:30 PM: The Debrief
At dinner, his daughter asked him what he'd done at work today.
Dale thought about it. He thought about the DNS forwarder in the dark, the gift-card scammer, the printer and its imaginary tray, the load-bearing comment, the certificate and the bakery, the storm, the spreadsheet holding up the world, Ohio, and the semicolon he hadn't typed.
"Nothing much," he said. "Nothing broke."
And that, in the end, is the thing about IT work. When you do it well, it's invisible. Nobody notices the outage that didn't happen, the breach that was stopped at the inbox, the data that didn't get deleted. There are no parades for the people who keep the lights on. There's rarely even a thank-you. The only people who truly understand what it took are the ones who've stood the same watch.
Monday, 8:05 AM: The Coin
On Monday morning there was something on Dale's keyboard: a heavy, worn bronze coin, and a sticky note in handwriting he recognized. It belonged to Ray from the network team, an Army veteran who'd seen the 3 AM alert in the team channel, and who had heard about Ohio.
Dale knew what it was. In the service, Ray had told him once, a challenge coin isn't a trophy. Someone presses it into your hand, and it says, without needing to say it, I saw what you did. I know what it cost.
The sticky note said: "For valor. Nobody else will ever know. I do. - Ray"
Dale put it next to the rubber duck. It's still there.
Why We Make These Coins
Dale is made up. Ray is made up. The coin on the keyboard is the part that's true, or at least the part that should be. Almost nobody in IT gets one.
IT Challenge Coins exist because people like Dale deserve one too. Every coin in this shop marks a battle that someone, somewhere, fought at 3 AM in their socks. They're funny, because if you didn't laugh at this job you'd cry. But underneath the joke, every one of them says the same thing: we see you.
So find the Dale on your team. You know who it is. Give them a coin.