Few things are as scary as data loss due to encryption failure within the information security realm. So frequently, we plan for it, test it, use it, and eventually, bitrot sets in, and it fails. This is why your RAID needs constantly rotating disks, especially when a checksum failure or S.M.A.R.T. error pops up.
No one could be as happy as an engineer with a RAID 15 (1+5) environment where every service is duplicated, any work processes can fail, and your site has procedural queries for everything. Even better, you have a tokenization solution that guarantees you never store credit card information in the clear. The nice thing about tokenization is that it is a rather ingenious use of encryption and hashing; the not-lovely thing about it is that it has a key vault for all the little secrets. Unfortunately, the vendor didn’t know how to mirror the servers properly, and they were coming out of sync. This is a problem when you’re dealing with time-based key-value stores that weren’t matching the key assigned to them. You can see where this is going; my phone rings.
It’s the “Under Pressure” opening ringtone, which means it’s my mentor and something went wrong. She emails me to schedule anything and texts if it’s routine, which means if she’s calling from her desk from the caller I.D., there are suits surrounding her looking for a solution, and she’s calling me. I answer.
“Hey, how’s it going?” “Another beautiful day in paradise; how’s the farm?” “Doing great, the horses are happy. We’ll have hay to throw again soon.” “Sounds like a great time; what can I do for you?”
We constantly banter like this at the beginning of calls; it drives anyone listening crazy, and we have our little games.
“So, we’ve got a tokenization system, as you know, for our card processing,” she reminds me to benefit those around her. “Yeah, the system I spec’ed out last year,” I reply “Yeah, it’s been performing great,” she hints “Until…” I give in “This morning” Ah fuck. “I’m already on my way, coming from Uptown.”
I pack up my laptop and signal to my waitress that I’m running out, and the approximate amount is on the table. I make it a generous tip; they give me power, caffeine, and company. Plus, it’s basically my office away from home.
Continued..