Tech-aura effect via cellphone call
Posted by N11Ordo@reddit | talesfromtechsupport | View on Reddit | 29 comments
A tale from before the pandemic when i was doing a lot more server maintenance.
Due to a botched update I got the fun(1) task to do midnight maintenance on one of the applications we are supporting.
Midnight hour strikes and I ssh into the machine, stop the application, untangle the bad update parts and do the needful. Pretty straight forward job, just remove a few folders and the application will recreate them correctly on startup. Time spent on this is maybe 5 minutes tops.
Now to restart the application and verify my work as good. I enter in the command to start up the application and switch to my browser to verify that it starts.
Nothing. Application URL only shows the hated nginx landing page.
Ok, something must be wrong. Let’s not panic, I’ll just restart the application. Maybe it couldn’t recreate the folders.
I restart the application again. No deal. Ok, time to do a deep dive in the logs.
I scour the logs, tailing in one tab and manually searching the same log in another tab. After scanning the log i can definitely guarantee that there is nothing out of the ordinary in them.
By this time my brain is thinking in terms most commonly transcribed in comics books by symbols most commonly found below the F-keys and the letters on a standard keyboard.
*ticktock ticktock time’s ticking*
I update the application browser page again in the vain hope it’ll work. No dice, still just the nginx landing page. After staring at the nginx page for almost 10 minutes I weigh my options:
a) Double down and bullhead my way through, hoping something i do will fix it
>bad idea, might frack it over worse
b) Leave it as is and grab SrTech first thing in the morning.
>worse idea, will generate a lot of incident reports about application being down before i get in to work tomorrow
c) Bite the bullet and call SrTech for help, waking him up in the middle of the night when i know he has a long day ahead of him tomorrow
>…oh blast. This is the best choice, isn’t it?
A few more minutes pass as i hope against hope i won’t have to call SrTech. No good, gotta do it.
I call up SrTech and the conversation goes something like this:
SrTech: *mumbled sleepy greeting*
Me: Hi SrTech, It’s me. Sorry for waking you up at this hour but i think i have messed up the application restart
SrTech: OK, walk me through what you have done
Me: I shut down the application, did the needful and then when i started up the application it won’t start. It just goes to the nginx landing page even if i update…
Here i updated my browser to emphasise my words, even though i knew SrTech couldn’t see what i did.
It. Bloody. Worked.
Application is up and running as smoothly as if nothing ever happened
Me: …um nevermind. It works now. Sorry for waking you. I’ll let you get back to sleep.
SrTech: * mumbled sleepy goodbye *
(1) Fun in the Dwarf Fortress sense of the word. Remember: Losing is Fun.
SeanBZA@reddit
Did not do the CTRL F5 refresh, you were getting a cached page, and thus till the cache timed out you do not get an updated page, which will have the nocache flag set on it with a running application, unlike the nginx default page, which has the flag not set by default, so as to avoid slamming the server during startup.
dickcheney600@reddit
I've actually made this joke with a few bosses and with a few IT techs, asking them to "first try it yourself and see if your mere presence makes it magically work again"
__wildwing__@reddit
I’m the opposite! Not IT, merely user. I’m at my desk (no computer) and my coworker is at her desk, working on data entry. She’s tried the standard procedure five times and it won’t work. Leans back in her chair, asks me if I can leave the room for a few.
“Computer problems?” I ask
“Yup…”
“No problem, I’ll be in the hall. Holler when you’re done.”
I got through the doorway and BAM everything worked fine.
This was not an infrequent occurrence.
N11Ordo@reddit (OP)
Classic case of extreme bogon emissions.
https://en.wikipedia.org/wiki/Quantum_bogodynamics
__wildwing__@reddit
And I’m now cackling in the middle of my company car park! That’s great!
Equivalent-Salary357@reddit
I suspect we have some DNA in common. Perhaps we share a distant relative. LOL, maybe not that distant...
AshleyJSheridan@reddit
It's always the cache.
_9a_@reddit
At least it didn't progress to being !!fun!!
WackoMcGoose@reddit
points at my flair
Been there done that. And I'm only tech support for my family, yet I still want to commit "cancels Print: No target to print to" on a weekly basis (mom insists on printing bank statements and manually balancing a checkbook "to have a paper trail they can't change")...
rob-entre@reddit
It wasn’t a bank but i believe in 2022 we received a phone call/email from a state agency stating our registration was expired and would require a lengthy process and hefty reinstatement fees to bring us back into compliance. We logged onto the site and sure enough, none of our renewal info was there. Just completing the process a couple of months earlier, we provided a copy of the paper trail, they said thank you and all was done.
They did “not” have a server fault that required restoration from backup, and the last couple of months backup were “not” corrupted…
It doesn’t have to be printed paper, but keep a “paper trail” for anything you’re not willing to lose.
Black_Handkerchief@reddit
You know, I don't blame her on wanting copies. If something DOES go wrong and you need help fixing it, having the paper copies is irrefutable proof you can offer to get them to act and (if necessary) escalate the matter. Without, they might just go 'our system shows nothing wrong' and start to treat you like a crazy problem customer who doesn't understand technology.
Most of the work in customer-facing support positions is to weed out the idiots and solve basic issues. Complicated niche 'glitch in the matrix' stuff are alien to those employees, and due to the nature of their role, they may well be prejudiced towards assuming it is a PEBKAC type problem rather than a BANK problem... and once they get that in their head it can be impossible to get stuff fixed because all senior personnel go 'well, first line support said it was obviously a customer issue' which makes them less rigorous and lean towards just verifying the basic facts that first line support personnel also saw but don't properly demonstrate the problem.
I get that banks have a lot of safety systems and all that jazz, but as a customer it is best to have your own proof and administrative stuff so you aren't even reliant on the company having their shit in order.
(Also, as a bonus to all this, paper records don't get deleted from the web interface after X amount of months/years. Which can also be useful when you want to dig into stuff for the sake of proving stuff if you somehow get audited a decade later and they go 'you cheated on your taxes in 2012'...)
Beginning_Method_442@reddit
You know a good IT tech…. They scare things into working!! (Very old IT truism)
Dumbname25644@reddit
The amount of times I was offered a job to "just stand next to this bloody printer, Because the damn thing works when you are here". I really hate intermittent problems especially when I never see the actual problem.
DragonAteMyHomework@reddit
My kids are convinced that computers are scared of me.
bikesandlego@reddit
I have years of FLL teams convinced that the robots are scared of me. I actively encourage this feeling, of course. 😂
Stryker_One@reddit
FLL? FIRST LEGO League?
bikesandlego@reddit
Yes
NotYourNanny@reddit
I tell people it's because I'm so smart that when I walk by, they get smarter. Works over the phone, too.
rickbb80@reddit
People used to tell me I just needed to lay hands on it to make it work.
sillyconfused@reddit
My husband is like that. If I ever have a tech problem, I just call him to come into the room, and magically, it works!
TinyNiceWolf@reddit
Sometimes the software gods require a sacrifice. In this case, the restful sleep of a senior tech proved to be an acceptable sacrifice.
bikesandlego@reddit
And it's a lot simpler to provide to the gods than trying to find a virgin & a volcano.
meitemark@reddit
Virgin: Just find anyone that plays CS 1.6 or brag about playing Fortnite. Those are always virgins.
Volcano: I have heard weird things about american "taco-bell" and that those can provide a sort of volcano.
nymalous@reddit
Dwarf Fortress...
salttotart@reddit
!!FUN!!
It's red, bloody, and on fire.
WackoMcGoose@reddit
This somehow sounds better than their official name, "grawlix". Also referred to as "I'm the only person I know with a less colorful vocabulary than Beetle Bailey..."
hadriangates@reddit
It only started working because you called SrTech. If you hadnt, it would have been down all night.
Overall-Tailor8949@reddit
My last place there were several computers and a couple of printers that would, once a month or so, randomly go apesh1t. Outlook wouldn't open, or the AVID newsroom software wouldn't run stupid stuff. We would walk into the room, sit at the desk glaring at the machine and sure enough, the damned thing would work PERFECTLY. Unless there were other fires to put out, I'd usually spend a little time "faking it" by clearing temp files, caches etc... Then tell the user that it's working again.
imnotk8@reddit
Yep. it always starts working immediately when the tech (in this case, sleepy snr tech) enters the room.