I managed to reach management in IT out in Ohio... and was bored silly - and then my guys started telling me I was "out of touch," because I hadn't been "hands on," in the datacenter in months - which wasn't wrong. 

I also like crisis mitigation. The most exciting thing I've done in IT in the last 10 years - a CIO I know of a healthcare called me up, got hit with ransomware - all their NUCs at their branches were locked down... needed me to go and hit all their NorCal branch locations, take machines offline, reimage them if possible - quarantine ones that were too locked up and send them back to corporate. 7 days, 12 branches, ending up in Sacramento, room, board and transportation included, 8 hours minimum at $250/hr even if I did 3 hours. I ended up spending about 12 hours typically - came home with a $15k check for 7 days, and got to visit family in Sacramento on the final evening - and did a bang up job bringing them back online. 

If I could do that consistently - 4 -6 jobs like that a year - I'd open a consultancy. But I really like crisis management. I feel the most engaged with that kind of pressure and deadline to bring a ship that is dead in the water back up to speed. 

And as you pointed out - with clustering and failover and imaging - those skills are also underappreciated now. 

Reminds me of an episode back in the mid 90s. Before ransomware really took off and became a daily event.

I forget the name of it, but it was a poorly crafted email to entice you do to stuff. Even back then normally we would 'eh, garbage' and dismiss it. However, at the time i worked with several people from the far east. In other plants we had some from the middle east. So, that it was poorly worded didn't really 'register' with them.  it started out with something like "love you"

Was sitting in the computer room and my exchange server load went thru the ceiling. We got flooded with 100k emails suddenly ( thankfully i was 'one of those people' and monitored my stuff, even back then ).  saw one of the mails, "oh crap" I yanked our network line to the rest of the world ( actually i broke it, i literally yanked the wire apart on the wan port. ) and literally ran across the office to the president's office and unplugged his machine while he was about to press enter on the message ( he was Japanese "it seemed odd, but i know him, and he was from India, so i donno" was his words ).  Ran over to the next executives office, same thing "yank". ( he was Chinese )  " ill be right back" and off i ran thru the office warning everyone. "corporate just got infected, dont touch your email"  

Was sort of funny, they both just rolled back and blinked. They trusted me, and figured it was really bad if did something that extreme ( which is out of character for me )..  turned out one of the people in another plant ( was also an Indian ) clicked, and opened the flood gates.   

Got my exchange server cleaned out in an hour, and updated some filters to kill any more.. But the rest of the company took days.   Put us back online so we could get to the mainframe up in Wisconsin and continue working, but turned off the exchange server until the smoke had cleared just to avoid the flood.

..and no, not slamming any of their heritages, but it does make 'translation' harder at times. Just a fact of life.

Sun May 03 2026 05:04:43 UTC from ParanoidDelusions

I managed to reach management in IT out in Ohio... and was bored silly - and then my guys started telling me I was "out of touch," because I hadn't been "hands on," in the datacenter in months - which wasn't wrong. 

I also like crisis mitigation. The most exciting thing I've done in IT in the last 10 years - a CIO I know of a healthcare called me up, got hit with ransomware - all their NUCs at their branches were locked down... needed me to go and hit all their NorCal branch locations, take machines offline, reimage them if possible - quarantine ones that were too locked up and send them back to corporate. 7 days, 12 branches, ending up in Sacramento, room, board and transportation included, 8 hours minimum at $250/hr even if I did 3 hours. I ended up spending about 12 hours typically - came home with a $15k check for 7 days, and got to visit family in Sacramento on the final evening - and did a bang up job bringing them back online. 

If I could do that consistently - 4 -6 jobs like that a year - I'd open a consultancy. But I really like crisis management. I feel the most engaged with that kind of pressure and deadline to bring a ship that is dead in the water back up to speed. 

 

I managed to reach management in IT out in Ohio... and was bored silly - and then my guys started telling me I was "out of touch," because I hadn't been "hands on," in the datacenter in months - which wasn't wrong. 

I hear that.  People say there's a point in technology careers where you supposedly have to decide to abandon the craft and become a manager if you want to advance anymore.

Depends on the company, really.  In 2014 I moved from production support up to "engineering" and then a few years later "architecture" and last year to an infosec role, and I've done well despite that very first transition being my exit from being a manager/supervisor.  The "lead engineer" path proves that there is a third path outside of support ticket hell and management hell.

Was sitting in the computer room and my exchange server load went thru the ceiling.

Dude, you know I have to say this.

"Serves you right for running Exchange"

:)

I didn't have a lot of say so at the time for that, tho i was working to at least migrate our file servers to Linux. Sort of stuck due to having to link to the rest of the company even if we were sort of an outcast. No pun intended ( we were a casting company ).

Also, this was mid 90s.... Not a lot of viable enterprise options back then.  Speaking of, my predecessor named the exchange server "Enterprise". He was a "Next Generation" fanatic, took me a bit to get things named something more coherent: Like "east_dock_print_server" instead of "worf'. but Enterprise = Enterprise email.. sort of ironic so i left it )

But to be fair, when we got hit, it wouldn't matter much what we ran.. 

Sun May 03 2026 22:24:29 UTC from IGnatius T Foobar

 

Was sitting in the computer room and my exchange server load went thru the ceiling.

Dude, you know I have to say this.

"Serves you right for running Exchange"

:)

 

The place in Ohio ran Exchange. I got so good at it, I could create a fake, offline domain that matched our production domain, and bring the Exchange server back online there and do things that were not practical on the actual production Exchange server. That was hard - took me weeks to figure out all the things you have to get right for that to work - but once I did - I literally had the keys to the Exchange kingdom that an admin should have actually always had. 

I probably could have done something with that at the time - but there were other things going on in my life with my wife and kid, too... so I went a different direction. :) 

My production servers were pretty bullet proof - other than executives using them as file servers - and I was enough of a dick Admin to get that in check pretty quickly. 

Wed May 06 2026 09:09:39 EDT from IGnatius T Foobar

Just like the rest of you, I became way too familiar with ESEUTIL, running it whenever Exchange blew itself up (which was extremely often). Real PTSD there, some of the most awful experiences.