I'm on tonight, you know my logs don't lie and I'm starting to feel I'm right.

Posted by speddie23@reddit | talesfromtechsupport | View on Reddit | 64 comments

This is a classic story of work avoidance by blaming IT.

Way back I used to work the night shift at a hospital that ran 24/7. Helpdesk was 24/7, but all other IT operations were mainly done during business hours. Important to this story is that the helpdesk was a different physical location to the helpdesk.

As helpdesk resources were kept minimal after hours, even if we were onsite, anything after hours that requires actually looking at something at the user's work locations was a ticket to be looked at during business hours.

There was one user who only worked nights. They would somewhat frequently call with their PC being offline. When they called, their PC was always actually offline (not pingable, they can't access network resouces). The call would usually go with can you check cables, is it only affecting you?, have you restarted?, OK we'll get someone to take a look at it in person tomorrow.

Strangely enough, when someone went to go look at it the next day, the PC was always back online, working perfectly.

As no other fault could be found, it was assumed to be an intermittent issue with the PC or network cable. The PC was swapped out and network cable changed.

However, we would still get the occasional "My PC isn't working again" calls from this user.

Suspecting next there might be an issue with the network, rather than the PC, a ticket is logged to the networks team. Luckily, the network team used syslog, so they had a good record of logs from the network switches.

What they could see, what at the time the calls were logged by this user, a few minutes after their shift began, the network logs showed the network port the computer was attached to as going down. A few minutes before their shift ended, the network logs showed the network port the computer was attached to as coming up. There was no other related log entries between these 2 entries, and they always occurred at the same time of day, only when this user was on shift.

It almost seemed as if this person was unplugging their network cable when they started their shift, and plugging it back in when they went home, but we couldn't really outright accuse the use of this without proof.

To eliminate a switchport issue, the networks team replaced the patch cable, moved to a different switchport and even went so far as to change to using a different port between the patch panel and wallport. So now literally everything has been replaced between the switchport and the computer, including the actual computer itself.

Another IT team was performing some other unrelated works at that hospital that night after hours, and this user was about due for another unexpected PC offline issue.

I mentioned to that team what was going on and asked although it's not something that we usually do, if this user calls up again, could they quickly go over and see what cause was. I do mention "It almost seems as if they are unplugging the network cable for the duration of their shift", but in a way that isn't directly accusing them of that.

As predicted, the user calls, and I do my usual log a ticket, and then contact the team onsite. They go to that location and sure enough, the network cable is unplugged, but still resting on the port so it appears plugged in. They covertly take a photo. The next morning, the usual onsite team takes another photo, this time it shows the cable fully plugged in. Now, we are sure the user is intentionally disconnecting the network, very likely to avoid doing work due to "IT issues".

Performance management / discipline isn't IT's role, but we do let their manager know, and forward all the proof we have. They did mention that user "seemed to be having a lot of IT issues lately".

A few days later, we get a user termination request, and I'm sure you can guess who it was for.

[-]

ThunderDwn@reddit

I've had one like that.

Fortunately, I was working at a TV station at the time, and it was trivial to place a surveillance camera where the PC in question could be seen.

The stupid bastard completely ignored the "You are being monitored and recorded by CCTV" sign - and pulled the ethernet plug before dozing off for a nap.

Right on camera.

Footage was attached to the helpdesk ticket and forwarded to said individual's manager.

His tenure was abruptly terminated with extreme prejudice

[-]

Kodiak01@reddit

I've only ever intentionally disconnected one time, nearly two decades ago.

We had one customer that would come to us because many other places wouldn't deal with him anymore. He would come in 5-10 minutes before our (much later than average) closing time and start asking a million unrelated questions and price requests. This was a customer where we had permission to come right out and say, "We're closing, time to go."

So it's 8:50pm and I see his truck pull into the lot. Internally groaning, I decide I want nothing to do with his BS tonight. I slip into the server room and flip the power switches on the routers. He walks in and starts his routine, only for me to spin my screen around to show that the Series Of Tubes was down and I couldn't do anything for him at the moment (also lying and saying it's been down for hours).

He stared at me blankly for a moment, then left. On my way to punch out, I turn the routers back on and head home.

[-]

SteamingTheCat@reddit

But why tho? Was he just a bored old man using the hired help for companionship?

[-]

Strazdas1@reddit

I have a friend who goes to supermarkets before closing time and gets the kicks out of being the last customer. He especially likes when they have to reopen a cash register just for him. People are weird.

[-]

WackoMcGoose@reddit

He especially likes when they have to reopen a cash register just for him.

That's a big bag of hell no. Once the registers at my store are closed out in the system, they cannot be reopened until the opening manager flips the "prepare to start business" switch in the system.

In fact, I tell customers that "the registers are tied to the same circuit as the main lighting, when the lights dim, the registers shut off, and if a receipt hasn't finished printing before it happens, you're going home empty handed!" Usually gets them to haul ascii and GTFO from my department on time...

[-]

Strazdas1@reddit

They like to close registers early and prepare to leave because theres usually no customers showing up 15 minutes before closing time. But he is still within the official open hours so they are required to reopen the register and serve him.

[-]

KelemvorSparkyfox@reddit

He was inhumed with an axe?

ITrCool@reddit

I’ve had this occur before:

unplugged Ethernet cable, blamed computer/Internet issues
unplugged power to monitor, blamed computer issues
one that really made me chuckle: kept clicking reboot when I’d try to remote in to “prove” the computer had issues and claimed it just kept rebooting on them.

Event Viewer system log “event ID 1074” entries said otherwise.

OS system logs can tell difference between reset button and power loss? or was he booting OS and rebooting inside it?

Yes they can. They indicate which user initiated shutdown. They also indicate if the system shutdown in an unclean way. System log identifies this.

Theres no "user" if you hit reset button on the case. A properly set up one even physically disconnects power thus doing proper "interrupt".

Not on a typical business PC or laptop. You’re thinking of a home PC.

a typical business PC has a reset button. At least every instance i saw had one. You are right it does not exist on laptops.

Automatic_Rock_2685@reddit

Zero of our Lenovo OR older HPs have reset buttons.

All our old HPs had reset buttons.

Every Dell I have ever worked with or serviced only has a single button on the top or front of the chassis, for power. OptiPlex series especially.

Every Lenovo tower, same thing.

Interesting, our Dells have reset buttons.

Man-In-His-30s@reddit

No I work with SFF optiplexes ( 3050, 3060, 3070, 3080, 3090, 3000, 7020 ) and all of them are single button machines and I’m in the UK

mafiaknight@reddit

The helpdesk is at a different physical location to itself? We have some quantum disentanglement going on? Timy wimy, spacy wacy stuff?

EdStarwind2021@reddit

The whole sort of federal mish-mash.

blockCoder2021@reddit

r/UnexpectedHitchhikers

speddie23@reddit (OP)

It's a quantum helpdesk in superposition.

option-9@reddit

Which level of support the helpdesk offers is fundamentally probabilistic in nature and cannot be determined until measured via phone call. (Dave is overqualified and he might be on break.)

Stryker_One@reddit

Different temporal location. Bitch please, you're just in another timezone!

AStrandedSailor@reddit

As long as its, timy wimy, spacy wacy not wibbly-wobbly, timey-wimey stuff you're ok

keesbeemsterkaas@reddit

We all love spaghetti

JustSomeGuy_56@reddit

I was doing a warehouse automation project. We set up a pilot project in one corner for training and to shake out any bugs. The workers figured out that as soon as our system went live about half of them would lose their jobs. One morning we came it to find a stack of problem tickets that all said “computer broke”.

We went out to the floor and saw the row of work stations where the staff would log in new products as they arrived. Every one of the keyboards had been forcefully embedded into their monitor screen.

sadmac356@reddit

How in the…I mean, I guess "computer broke" is technically an accurate description but…jamming the keyboard into the monitor?

AdSignificant2935@reddit

Twas nothin but a scratch

eggman1995@reddit

What hsppened to the workers? I guess since they were going to be fired, they didnt care that they broke the monitors?

InfiniteTree@reddit

All fired with cause with no benefits instead of getting redundancies, geniuses.

Jesus. I get being mad finding out you're testing you replacement. But atleast you found out ahead of time and can look elsewhere while still being employed and then get redundancy when the time comes.

wtfomg01@reddit

You replied to a different guy, did you notice?

Not at all. It was written so matter of fact, that i failed to notice that there was no "op" mark next to their name.

There's no other outcome to that scenario.

Firing an entire department with no idea who actually did the act seems like a good way to get rid of the problem and generate that many problems in lawsuits.

SeanBZA@reddit

Not really, you fire the lot, because at least some did the act, and the others were complicit in the action, so all are dismissable immediately. plus likely they had CCTV of the ones doing the act, and the rest all watching as well.

we dont know the situation, could have been one disgruntled emploee doing it and others finding the result arriving in the morning.

Night shift, they would all have been working there that shift, and also would have at some time have needed to do that training, so likely also saw the act, and who did it. Also I have never seen any warehouse in the last 20 years that did not have at least 2 cameras per aisle, and one in each room, and others covering entrances, exits, and the corridors to the change rooms and toilets, along with eating and rest areas. so they knew who went in there, who went afterwards, and what was done. So fire the lot, because of failing to act to not allow damage to company property, and also fire the instigator, instead of charging them for criminal destruction of property, as that is going to be cheaper to do instead of paying the legal costs to do the charge, with little probability of getting the money back. Out $5k in monitors and keyboards, likely already depreciated to near zero, and another $20k in court costs, while the lawyer simply goes and shows the video to the employment tribunal or the union higher ups, and tells them this is the reason, at around $1k total, and keeps any monies they would have had to pay out in redundancy payments etc.

LupercaniusAB@reddit

Well done, them.

GamerDad72@reddit

I had a similar series of problems happen at a nursing home we service. We introduced WYSE terminals at the various nurse stations so the staff could put their charting into the new EMR software. Over the next couple months I would get tickets saying that the terminal at one particular nurse station could not be used so the staff couldn't do their charting. I would go onsite the next day and find the keyboard mostly unplugged, monitor cable mostly unplugged, power cable mostly unplugged, etc. All of them just removed far enough to break connection but still look connected. Got bad enough that the Nurse manager called to complain about the device. My boss actually looked up all the tickets for the device, put the ticket numbers and dates into a spreadsheet with what the problem was and how I resolved it and sent it to the manager. Needless to say we didn't hear another word from the manager and the problems "mysteriously" stopped happening.

xxvivivild@reddit

Sounds like a classic case of the ol' "my PC magically heals itself overnight" situation... happening more often with this user than with a Netflix show auto-playing the next episode, amirite?

he really was stupid about it. Once he saw stuff getting replaced he should have left the cable out for the morning shift to find it. Then a real issue would be identified and IT would be driving themselves nuts considering how does that cable keeps slipping out of connection every 2 weeks.

If IT saw that next day that cable will have a cable clip on it, screwed into the case, and with a second one further along, with a cover around the port area, so that removing the cable requires tools. Easy enough to make from scrap PC parts, using some pliers, the odd dead power supply, and some cutters to make it.

given that it took months for IT to even check the place in person, i think you are overestimating how much the IT cares.

micaturtle@reddit

My question is, how did the user get away with not being told they would have to switch to "offline procedures" (which are usually a lot more labor intensive and more annoying)?

He did it rarely enough? Also while i could switch to offlline mode for most part right now, the project of getting everything into cloud storage by 2027 we have here would mean that in two years no viable offline options were possible.

Possibly data entry, which kind of needs a working connection to a central database to work, so the computer being offline means the day shift gets left with a whole lot of extra work to do, in addition to all the rest, which often piles up so that the night shift has to enter the overflow.

I had this unintentionally once. Every day i have internet problems in the morning, ethernet cable coming loose. Until one day i stay late doing something that needed to be done "yesterday" and spot the cleaning staff, casually swiping the mop across all the wires and unplugging my Ethernet cable.

deeseearr@reddit

I saw a similar issue many years ago. I was supporting a warehouse which issued all of the workers with a small handheld computer -- This was in the previous century, over ten years before the coming of the Jobsian Fondleslab so that was a good deal

Ok, i have to ask... Jobsian Fondleslab?

Jobsian Fondleslab. The term Fondleslab was first used by theregister.co.uk around 2009 and could refer to a number of different devices so a qualifier had to be added when talking about the one designed by Steve Jobs, or at least by people who were afraid of him.

Jboyes@reddit

Google "Did you mean Jobs In Fondles Lab?" LOL

DaddyBeanDaddyBean@reddit

If one were to build a small gadget with a battery-powered motor spinning a neodymium magnet, and placed said gadget directly on top of the network cable, I'll bet that would cause significant interference. If it was small and quiet and unobtrusive, one could probably get away with it for quite a while.

Does not do anything, the small magnet means small induced voltage, and twisted pair is remarkably good at rejecting common mode voltage, and the isolation transformers also are extremely good at removing that small voltage, which, compared to even 10M signals, is basically a DC offset, and which 100M will totally not even notice, there will be more errors generated by actual switching noise inside the silicon die itself, corrected transparently by the PHY layer without even requiring a packet to be resent. now if you had something like a plama ball with the cable wrapped around the globe, and the plasma ball grounded, so there were regular electrical discharges into the cable itself, those might be noticeable in slow traffic and lots of retries.

(tips hat)

vivi_is_wet4_420@reddit

Sounds like that PC was pulling a disappearing act...probably off having a siesta when it should be on the clock!

Dom_Shady@reddit

Important to this story is that the helpdesk was a different physical location to the helpdesk.

Something went wrong here.