What is something good you made/did that got you in trouble?

[-]

StaticBroom@reddit

We were doing a SQL Server migration that impacted about 100+ smaller applications. All code in our shared code library matched all the code in the prod apps. We were pretty good about that.

The plan was to spend the weekend on Zoom while updating every app with the new SQL connection info, compile, roll out...like good little boys and girls. A couple of us wanted to slowly update code in the weeks leading up, be we got overruled due to other "needs of the business"

Funny thing...all the apps...the folders existed in the same network share. Every one of them. Huh, that's awfully convenient. That Friday morning I put together something quick in C# that would scan each folder for config files, make an immediate copy in the same target folder called .config_BAK, then update every qualifying prod .config file's SQL connection to the new server. Right when we gathered together, as soon as the change window started, I launched the app and had everything updated in production in a couple minutes, also with an original version of the config as a backup just in case.

Just like that, weekend was free and all we had to do was update the configs in our library at our convenience. I created a shareable spreadsheet to mark off which apps in our library were updated and checked back in.

Got yelled at for going outside the rules of rolling out production code.

[-]

JustDandy07@reddit

Well, yeah, you probably should have told HR you were doing this and maybe verified that the data you were pulling was accurate.

This is the type of lone wolf stuff that makes people dislike IT.

[-]

mrtuna@reddit

The amount of people posting in here completetely disregarding Change management is staggering. Then acting like they're doing everyone a favour when they break stuff lol

[-]

iama_bad_person@reddit

maybe verified that the data you were pulling was accurate.

What? Call every user just in case HR sucks at their job? ERPs should be the source of truth for data in most cases.

[-]

XelNika@reddit

OP was aware that the HR ERP data was wrong/incomplete. OP was aware that the AD data was wrong/incomplete. Instead of taking steps to fix the data, he just synced the two data sources.

my boss what pissed because I wasn’t more collaborative

I ran into an executive who was storing important emails in her deleted folder. When I asked if we could empty it (she was having storage space issues) she said, "no, I need those emails." 😳

[-]

[-]

land8844@reddit

I set Outlook on my work laptop to empty the recycle bin upon exit. I don't store shit in there anyway, but it gives me peace of mind that I'm not retarded.

[-]

ZeroSkribe@reddit

No, you are

[-]

devloz1996@reddit

GPO / Storage Sense: Every week, Empty recycle bin after 30 days
Gaslight users into thinking that it has always been this way and you have no idea what they are talking about
Drive usage alerts drop by \~20%

have lost count of the time I've clicked into someones download folder and found 8 copies of the file they have downloaded.

posixUncompliant@reddit

I worked for a place that sent three alerts every 15 minute that process wasn't sleeping. That was the good state.

Two days after I took over all the monitoring services, that was gone. It would've been faster, but approving the change had a built in delay so management could review stuff (read so that certain managers couldn't pull a fast one on each other with change approvals).

I'd written the monitor that report if the process was hung, suspended, sleeping, or otherwise not functioning properly oh, 9 months before. That monitor worked, and had even been triggered once by a real event (one not noticed by the not sleeping monitor).

[-]

mercurygreen@reddit

I didn't CHANGE anything. I fixed some contacts that had been migrated incorrectly from 2003 (seriously).

[-]

Otherwise_Time3371@reddit

Found the senior guy

[-]

Remarkable-Cut-981@reddit

Senior Junior Intermediate Principal

It's all just a title bro

We have 2 flavors of architects, Enterprise Architects and Infosec Architects that work together to address the gaps. Principal Support technician is not a thing, those will not go anywhere beyond associate in our structure.

Associate - lowest level.

Sr Associate - L3 SME tier of knowledge.

Lead Associate - usually work within their silo with product owner to enhance products they manages.

Advisor - IC work with product owner, high impact contributions, knowledge of business impact of tech capabilities enhanced.

Principal - exceptional contributions on the org level across multiple products and silos.

Distinguished - only a handful of folks we got maybe a dozen that have ear of SVPs on strategy and direction.

I am on both architecture review board to represent our products and voice any concerns and also of course all sides of blameless reviews and sending suggestions to architecs based on any relevant SLA related incidents. The worst I deal with is business side developers

[-]

posixUncompliant@reddit

The Principal Support Tech was a title at the most dysfunctional org I've ever had the displeasure of attending a meeting at. I still wonder what their job actually was.

So, I work in research computing (supercomputers). I consider myself a storage guy, and as I am technical, not sales, and not a straight developer, in my world that means I'm support. If I'm working on issue with your org, I expect to need to talk about the storage systems (which are both arrays and servers), the storage network, the compute nodes, and possibly security. How many people are in the room? Will they be the same people when I'm out again in a couple years because you migrated to our competition and now need help coming back to us?

I can, and do, build our appliances, install them, and depending on how busy I am, I take customer calls ranging from part replacements to truly novel issues. I'm the guy who does the new stuff first, the visible stuff, and especially the high stakes stuff. I am responsible also for our internal test and validation architecture. That's been true even before I moved into research. What level am I in your org?

Do you mean business development, or some flavor of programmer on the business side? The first have either been the best people in the org to know, or the worst type of marketing idiot to slime their way into a suit. The second have always been very odd.

[-]

Remarkable-Cut-981@reddit

If you are fisacted on titles you have an ego issue

Also titles vary from company to company, as well as from team to team making them meaningless and worthless

You could be called a junior engineer in one company

And be paid more than someone who is titled a senior engineer in one company

And vice versa

Not in another team, but in another hierarchy.

I've had some of the dumbest titles. The difference between Systems Infrastructure Engineer and Infrastructure Systems Engineer was about 5 years and $40k. Oddly the better paying one was several steps further from the top technical role, but also much further from the basic technical role. Also oddly, both of them were essentially storage positions.

[-]

horus-heresy@reddit

I feel like my title triggered you and that's fine, but they mean a lot regardless of your rambling. level of calls you attend and agency you have over work you choose to do yourself (individual contributor\staff track with advisor\principal\distinguished titles) over tasks you get assigned by management (associate\sr associate\lead). Even in tiny companies with 700 employees and 5-10 tech folks on staff titles can distinguish who you are working and dealing with. No need to get so worked up. CTO at startup might be dumbfuck with less knowledge than our sr associate, in that regard yeah titles don't matter, but L7 at amazon, google, microsoft, cap one will be very comparable with clear expectations of you and your understanding of what to expect from the role

[-]

Remarkable-Cut-981@reddit

I'm not insecure bro

I just gave my opinion

But you take it for what it is

[-]

Creative-Dust5701@reddit

On this concept for testing you want to insert deliberate errors so that you know that the test script catches and alerts on errors. Of course you also want to trap error codes on exit so you know the script exited normally or abnormally

[-]

technos@reddit

I kind of understand this though.. During a break in my IT life, I worked for a transportation company, and I used to run about a dozen automated reports every night. On a good night half of them produced nothing.

And then the reporting system broke for a week. I didn't know, I was still getting my empty emails. My first clue was my great-grand boss calling up from three states away to ask why I was fucking him in the ass holding things up with Mexico and if it was intentional or incompetent.

It didn't turn into a big thing for me (not my fault, after all) but IT got positively reamed. One of my friends over there said she'd been screamed at by fourteen people that could fire her in single day.

Still, to catch things in the future, all my reports were changed. Now they'd do whatever they were designed to do and then they'd tell me if someone with my driver's license and license plate had tendered a container of depleted uranium tank rounds for the US Army.

That particular query hit all of our databases. Driver, truck, chassis, container, hazmat and sensitive.

Now I had emails that always had at least one line of data so long as the reporting system worked.

A year later I got emails without it. Oh, there were phone calls, where everyone swore shit was good, but my manual reports were different, so I kept talking to people.

And that's how the second outage during my time there was caught.

[-]

mercurygreen@reddit

And that's a DELIBERATE test against data.

[-]

This is the level of genius one associates with end users. At least usually.

[-]

To note: some VS Code malware specifically targeted vulnerability researchers.

[-]

malikto44@reddit

Don't laugh. A NTP pool.

At a previous job, I had a pool of a few machines to ensure the UNIX side was good for NTP, even if the Internet connections went down. The pool had a CDMA/GPS card to ensure that the clock sync was correct.

All went okay until the upstream NTP server for the Windows side failed, causing AD to not authenticate and requiring the Windows machines to all need reboots. The UNIX side (Solaris, Linux, etc.) all worked without issue.

Next day, Windows side was up, and management started demanding A RCA... from the UNIX admins. Yes, I had to give a root cause analysis of why the UNIX side stayed up while the Windows side didn't, and present it.

Management wasn't convinced. so they hired an outside auditing team to go through every UNIX workstation, server, network box, and hit everything with full credentialed scans. The manager doing this kept saying that if a mastercrafted OS like Windows went down, then why did a "patchwork quilt of hacks and kludges" (indicating Solaris and Linux) remained up.

After a huge amount of money spent, the auditors came up with pretty much the same thing I had -- the UNIX machines had their clocks synced up, while the Windows machines fetched time from outside the company.

[-]

lurker_lurks@reddit

mastercrafted OS like Windows

O_o

You say so Boss. Lol.

[-]

malikto44@reddit

I should have had a "/s" in there.

[-]

lurker_lurks@reddit

Same. I was replying to your ex-manager.

[-]

Patching servers that had been running Server 2016 RTM code for 4 years.. "wait you did what? we want up-time, not patches, what if you broke something"

Suggesting and then proving that the guest Wifi network was actually just straight on the corp network, and not a secure separate network as believed. "no its a separate secure network, you're dumb" then why can I ping our domain controllers and file servers from the guest wifi network?

Suggesting that (running a custom self written (in house) RADIUS server that ran on ONE server, when the guy who wrote it was fired 8 years ago, and there was no way to move it to another server, for wifi authenication) was a bad idea (and suggested a fix), and then proven correct when that server crashed. "man, you should have told us this earlier so we could fix it. you should have came up with a solution sooner. you maybe arent qualified for this role if you cant keep our systems up" also them "get approval for everything before doing it, and it better not cost any money, including your time"

[-]

Character-Guest-2804@reddit

To be fair, I recently discovered our "separate secure network" for guest wifi was able to ping the main network. Turned out to be a default rule in WatchGuard firewalls. All other traffic was dropped.

[-]

traydee09@reddit

I kept it simple, but wifi "guests" are also getting DHCP from an "internal" server. and none of our "internal" traffic goes through a firewall, so theres nothing to block the inter-vlan traffic. So it wasnt just a simple oversight.

[-]

Any_Particular_Day@reddit

Working T2 support for a medical transcription company, some 3000 transcriptionists working at home typing up recorded medical reports. Anyone who’s been around transcription knows they live with their word expanders. So instead of typing “35 year-old female” into reports a million times, they’d enter “35yf” and it would fill the rest in. Some of the transcrptionists who’ve been at this a while would have tens of thousands of entries.

Well, developers made a change and everyone on a large account (several hundred transcriptionists) got a new word expander, with zero entries, and the old software couldn’t read the old,word expanded file. Developers said it was impossible to convert, so everyone would have to start from the beginning again. But it’s just a text file at heart, so I got a copy of someone’s old word expander file and dug in. I’m not much of a programmer, and I had no real tools, but I applied the old mainframe processes I’d use, but in Word VBA. It COBOL - Read old file line by line, reformat, save to new file. Tested it with this one person and she was ecstatic that she could work. I added a few niceties to the Word doc that ran the script and gave to my team. Also sent a copy to the dev team, cc’d my boss, with the explanation what it did and how it worked.

I was a freshly hired security intern and while I was getting my bearings I was reading over documentation and reports I found on our shared drive.

I found the internal vulnerability scan reports for each quarter and they were shockingly clean and small for such a large internal environment. I compared them to the Visio diagrams of our internal network and they didn’t add up.

I had discovered that our outsourced IT company wasn’t doing the quarterly vulnerability scans of our internal environment like they were supposed to be doing in our contract. They were just scanning the same /24 every quarter while in reality they were supposed to be scanning around 50 /24s. This was great for them because in their contract they only had to fix or patch systems that showed up in the vulnerability scan.

I asked our team lead and he had no idea why that was, he escalated up to our director, who asked the outsourcing team, who escalated it up to a bunch of other people. It ended up getting to Internal Audit who threw a fit because they had basically been led to believe we had had clean internal vulnerability scans for years at this point.

Long story short, when they fixed the scan scope and re ran it the network was full of massive vulnerabilities. Like, ms08-067 on hundreds of systems level of bad. (This was ~2011).

My director asked for a meeting with me and told me not to go “digging” anymore because I just caused a “massive headache” for him and his boss. Then he removed my access to the shared drive and told me i was only allowed to do software approval tickets until he told me otherwise. He ended up getting canned a couple months later when internal audit found out he had backdated a few firewall configs to make it look like they had been in place prior to an audit.

My director asked for a meeting with me and told me not to go “digging” anymore because I just caused a “massive headache” for him and his boss.

i hope you reported this to the CEO. they might not like that their director is concealing critical information from them.

[-]

ArthurStevensNZ@reddit

Oh, easy.

A client ran out of licences for a piece of software, there was no senior person to sign off the issuing of a new one and the client was desperate so I used my own initiative to request an extension, I judged that it was better to keep the client happy.

When the bosses were back I got such a bad reprimand that I resigned on the spot, to the general astonishment of everybody (I was part of a team of senior engineers that were delivering the biggest IT project ever seen in my geographical region at that point).

I slammed the door in my way out: we often had these problems because the bosses were disorganised and our reputation was suffering.

A few hours later the boss called to apologise, I accepted the apology and a few weeks later I got a pay rise.

It transpired that my big boss' bosses were not impressed with the situation and he got a bollocking as bad or worse than the one I got. He didn't resign.

I resigned again a few months later for unrelated personal reasons, it felt good. 😁

[-]

retrogamer-999@reddit

I used to work in Voice. Ym team of 6 managed about 300 PBX's ranging from 100 users all the way up to 2000.

One particular PBX was using STUN and the stun servers fell over and stopped responding. The service provider took 2 days to get the issue resolved.

The only solution was to change the stun servers. Smaller customers it was ok but when I done the 2000 seat one and got them back online within an hour of finding out what the issue was. Immediate staff on the ground where happy singing my praises as it was a particularly busy day in the call center. But the directors at both my company and at the customers have a right telling off.

I didn't follow change control.

Since then, No change control = no changes, no ticket = no support.

[-]

40kmoose@reddit

Stepped into an IT director position for a company with no formal IT and an MSP doing on call service and server maintenance. First day on site, I did not have my own personal login so I was told to use the shared meeting room account to surf the web and fill out some forms. I had way to much time those first few days as I was waiting on the MSP to make my user and admin accounts. While on the public, low level, shared account, I figured I would do some digging and see where the company was at from a security and organizational standpoint. I started looking through all the unsecured share drives to see what was available. Mind you I was already signed and paperworked so I was allowed to have access to everything.

I was completely shocked to see a .text file labeled "Admin Account"

In plain text was the user name and password. Of course I immediately used it to sign into windows.

What did this account have accessnto you may ask ?

AD, domain, shares, VM, the works. I signed into AD, made my own accounts and then continued to poke around. I asked around and apparently some VERY non IT employees were using the account to download programs add new workstations to the domain etc. It was blowing my mind coming from a much more formal IT environment.

So I changed the password and figured "Let's see what breaks" Apparently the entire domain. This was the domain trust account with FULL domain god rights.

The next day I came into work and everyone was complaining about no being able to log in and access folders from other sites etc.

This prompted several talks with CFO, CEO, MSP calling me a wild cowboy etc etc. I had choice words on the matter. I lasted a year before I switched to a different company for twice the pay and 3x less headache.

[-]

CantankerousBusBoy@reddit

Well this story is insane, but you don't look great either.
You should not have tried logging in with the account, and resetting this password without consulting with anyone else is no bueno.

Some of IT leadership didn't like it and somewhat scolded me because we were working on a much more capable system but likely wasn't going to be fully implemented for another year.

I suggested a similar thing for a customer and offered to implement the thing while working at an MSP. Customer's own helpdesk guy was instantly like "no-no-no, we'll buy a professional software that will do all that and more". 5 years later as I left my position, they still had no software in place and it was one of the employee's most wanted item.

[-]

stempoweredu@reddit

God I hate this in industry. I have run into far too many employers who let perfect be the enemy of good, and rather than letting their team write a quick app that does 90% of the functionality, they spin their wheels for years looking for this unicorn product that offers 105% of the features they want for 40% of the cost such a product would demand, and years later, we still don't have that feature our developers could have easily implemented.

[-]

da_apz@reddit

Smart and talented people quitting in frustration is the anthem of the field, unfortunately. Quite often I've noticed how people flat out reject simple solutions to problems and like you said, start dreaming up this monstrosity of a solution with 3 kitchen sinks integrated and since there's no one to implement that, we're just left with a daily frustrating problem for which there's no solution.

shortfinal@reddit

One of my first "real jobs" was working for Geeksquad at Bestbuy. We had to wear stupid clip-on ties and setup new computers for customers that the sales guys managed to push all of this ridiculous shit on.

Part of that was a checklist of going through to remove a bunch of the stock bloatware that came with standard HP/Dell machines of the day (do they still?) then installing Norton (ughh) and Office, plus all of the incremental windows updates.

It was considerably more pointy-clicky in 2004-2005.

Anyway, so my smartass makes a CD with a batch script that automates a lot of this away. I make half a dozen copies of the CD and give it to my coworkers to use too. Now we're all moving onto other things instead of babysitting these new installs for 2 hours at a time. Troubleshooting PCs in the back, etc.

Direct managers, the kings of little fiefdoms they are, worried about how busy their little bees are, became livid with me. Told me I was using "unapproved software" on customer computers. This wasn't the process, etc. Confiscated all the CDs, etc.

Two months later? Corporate has this "new process" for doing installs that involves dropping a CD into the machine and clicking this shiny batch process that does all the things. It was a bit shinier than my solution for sure.

I worked in Geek Squad when they had multiple CDs for everything. Some guys got tired of using multiple CDs so they created a "SuperDisc". It was just both of the CDs combined and could be used for everything.

Not sure how long it was used for but knew it was way before me. One of those remote Indian asshats reported it and we had to get all new CDs along with a reprimand.

I left about 2 months later but we burned the SuperDisc with the new CDs and only used them on stuff we manually fixed.

[-]

AntelopeUpset6427@reddit

Shadow IT ftw

[-]

blackletum@reddit

truly, no good deed goes unpunished

[-]

raging_radish@reddit

I had a system on our intranet where staff could check out hardware, usually networking gear and scanners. If whatever was borrowed hadn't been returned by a designated date, the staff in question would receive an automated reminder email. In the footer I had written: "This message was brought to you by the $CompanyName Nag-O-Matic^TM."

They made me take it out :(

[-]

kawaiikuronekochan@reddit

RIP Nag-O-Matic, cute name!

[-]

RumpleForeskin990@reddit

Wrote a script to process terminations in PowerShell. Was at an MSP. They didn't like that I was learning to code. Didn't like that I was doing things too efficiently and not racking up billable hours. At the same time they'd complain about human errors on terminations. Eventually I got sick of arguing in circles and just used my script myself and dressed my tickets up to look like they were done manually. Ie screenshots of account configs before I run the script. Time entries that sounded manual. Then I spent 20 minutes playing games or watching TV. I left 6 months later because I felt theyd say anything to avoid using scripts and automation. Got tired of the same disingenuous conversation. Felt like my shop was very set in its ways and slow to change. Which in tech is asinine imo. Got 60% more money to go from jr sysadmin to senior sysadmin.

[-]

punkwalrus@reddit

I think I considered it more of a joke that my company was clearly violating security and HIPAA by having someone from India having access to private medical data. I knew he was not a US citizen, because he flew in from India for the developer meetings, and kept trying to cover it up badly. On paper he was working from the US state of Georgia and was a US citizen who passed a background check.

He flew into one or our conferences, and I asked him how his trip was.

"Great!"

"Did US customs hassle you?"

"No, it was fine-- Uh... I mean, I didn't go through customs! I went through the TSA!"

"Those Georgia Bulldogs aren't doing so well this year. You think they'll get the Stanley cup?" [The bulldogs are football, the Stanley Cup is hockey]

"Yes, we can always hope!"

And so on. During a video call, you could see out his window it was dark outside when it should have been 2pm in Georgia. Frequently his timestamps were 8.5 hours ahead of the east coast of the US, and that's a specific peccadillo to India time zone, that half hour bonus. Just so obvious he was an Indian, from India, who lived and worked in India on stuff he should have never had access to.

I got in BIG trouble one day when I was told to STOP talking about him or addressing him in person in meetings. That's when I knew that my company was not only aware, but trying to cover it up. Two years after I left, major security breach with a military medical organization HMMMMMMMMM....

[-]

Indiana school for the death?

[-]

Monsterlime@reddit

Imperial Star Destroyer 😜

[-]

digitalnoise@reddit

ISD = Independent School District (typically in this context).

I'd agree 99% with what you said - but while the way the manager handled it was terrible, there is some understandable reason for him to be hesitant and nervous about a newbie writing scripts like that. He should have talked with him about what can/should be automated and been receptive to the idea, but questioning it is not a bad thing.

The last thing a manger needs is for a newbie to write a script than handles 95% of occurrences of some task well, and screws up the edge cases, and then start trumpeting about how they can automate the whole department to the point where upper management hears about this still unproven script, and develops an expectation that it will fully 100% automate the department.

Boss should have said "that's great, let's keep this quiet while we test it thoroughly and figure out the best way to use it" and then gone from there.

[-]

ciabattabing16@reddit

I like the fantasy you live in where you have a technical manager. If this land of make believe exists then I'd agree.

[-]

PrincipleExciting457@reddit

I don’t usually tell people when I automate something. Then I can just cruise. I understand being new and wanting to share a success though.

[-]

rollingviolation@reddit

or find a team that encourages it.

The more powershell scripts and bash stuff my team learns, the better off we are. Automation allows us to do more cool stuff. Fuck me if I'm going to upgrade the RAM on 500 virtual desktops by hand, and if someone on my team is doing it by hand, they can go back to the help desk.

see the senior IT pros recognize that you implement the script and just don't tell anyone LOL

[-]

Geminii27@reddit

Well of course. The boss could sit back, run the script, and keep getting the same budget for fewer salary costs now. Or they could save it up for a rainy day and claim a massive budget savings, get a bonus, and springboard to a promotion.

You being there would interfere with their narrative of them being the clever one or financial whiz.

[-]

randomman87@reddit

You needed to present this to your bosses boss, or even higher. Still might have happened but yeah definitely not to your boss. New to the workforce naivety?

[-]

Proper_Cranberry_795@reddit

Automated yourself out of a job eh. Sorry to hear. It’s Better to automate your job and not tell anyone.

[-]

Idk if it counts but in my early IT days. I created a Windows update script to update all machines and keep them current. .inside the script it also said reboot when needed.

Pushed it out and rebooted about 1000 computers at once....on a monday....at 11 AM...

Work-study at some IT training company. I think it was Novell training. We used Windows for Workgroups 3.1 I think. I identified something that used DOS mode and improved it somehow. I can’t remember the details.

They ended my work-study. :(

You’d think a school would appreciate someone improving an IT process.

[-]

Tb1969@reddit

I updated a program that I wrote 15 years ago that did merging of various individual reports into tailored multi-reports for specific users to then be emailed out to those users.

I worked hard to make the process faster and smaller resulting report file size with new error checking to get the end of the day report employees off the clock earlier by ten minutes ending their day that much quicker. Well, the lead User for end of day reports who usually runs it, got territorial since they wanted to be THE report person. They deliberately caused problems on the upgrade implementation. The user intentionally tried to make me look bad for no other reason than to make themselves look better in comparison to their superiors.

I had layers of upgrades to the code and added automation to implement to shave time off time off the process. That user melted down on the first layer I was applying. Long story short, I locked in the first phase of change, froze any further changes, and walked away.

I went above and beyond the call of my job, working hard on something that would have benefitted that user more than anyone else and the user retaliated.

At 4:20p EST every weekday, I have a quiet alarm go off on my phone to remind me there is a self-centered User working an extra ~8 minutes per day at that time because the User is an asshat that hurt themselves while trying to hurt my reputation.

Over a year that 8 minutes totals nearly 34 hours that the user will be doing extra work. Well, deserved extra work.

[-]

visibleunderwater_-1@reddit

This sound VERY familiar. Are you in some "regulated" space? I do ISSEC at an 800-171 DoD contractor company, member of DiB, a handful of us have to keep a clearance, etc. I'm the same way, I don't GAF who you are...the DOJ isn't FA anymore, and when CMMC finally comes down it will be worse. I've told various Directors, VPs, etc that "we can't do that" for a variety of reasons. I will report anyone after the 15-day window as outlined in our contractual and regulatory requirements. It's either them or me, once I've found something.

Oh, in some sh*t environments, fixing things, providing solutions, etc. gets one in trouble, e.g. some examples from one such fscked up environment I contracted in for a bit:

I suggest exactly what needed to be done to find and isolate the issue, I get reamed for it and almost terminated.
I'm given a problem in production to fix, I find the issue, document the issue and how to fix it, trace the origins of the issue - all the way back to who uploaded the problem code, also well note that it looks like code that may not have been deployed to only one host, and all such (potential) hosts should likewise be checked and fixed as appropriate. I get royally reamed for so much as looking at code written by another group or saying it has a "bug" (when it absolutely did), suggesting how to fix it, suggesting more than one host may be implemented, etc.

There were lots more examples from that royally fscked up work environment, but those are at least two that quickly jump to mind.

[-]

spin81@reddit

FYI you can say fuck in this sub

[-]

HoustonBOFH@reddit

In this industry, you often have to!

[-]

miscdebris1123@reddit

Some people don't like to curse.

I don't think you had any idea what those accounts were, lol.

At the end of the day you did a massive restructure of a critical business tool with no approval or foresight by accessing confidential data without permission. From the sounds of it you didn't even do a trial run before just running it.

Tried to automate onboarding with a Microsoft form. Needed to ask for the desired 365 password and just hitting enter on a question with the word “password” set off all kinds of alarms.

"I found a random free PDF to word document converter online to save us time and my boss got so pissed at me for some reason. Sometimes it feels like high school".

That's what OP sounds like to me. Not your infra, not your risk or liability to take.

[-]

g3n3@reddit

That is right. Just immaturity. I would also say the leadership isn’t impressing enough upon juniors about security.

[-]

AntelopeUpset6427@reddit

IT is the security, even if you don't work on risk assessment

[-]

MekanicalPirate@reddit

A Windows 2008 R2 server was failing processing Group Policy and wasn't getting all the proper GPOs applied. Fixed the GPO processing issue then found out that the server had been failing processing for so long that our policy posture had changed so much since since it started that once the server finally successfully processed all current-day (at that time) GPOs that something conflicted with the hosted application and brought it offline.

Of course, was talked to about bringing the application offline with no mention of fixing the Group Policy processing issue.

[-]

I designed, sourced, and built a SAN. This was back before storage was well understood. I made two mistakes. First, I listened to our high priced consultant to use soft zoning instead of hard zones. It worked fine after I put it back. Second I built an secondary SAN using disk that StorageTek said we shouldn't use for block level storage. It worked fine, but the sales guy threw a hissy fit to my bosses boss. I left shortly thereafter.

[-]

wrt-wtf-@reddit

That's a top 10 rule in govt: Never make your boss seem inadequate by doing something they wouldn't or couldn't themselves.

[-]

mingepop@reddit

It’s crazy hearing the government wants to tax its citizens more as they can’t afford public services. Then you hear about stories like these and wonder if government jobs could be more efficient

[-]

hoeskioeh@reddit

Unsure, maybe it was really me who f-ed up?

This Intel microcode x129(?) bug/fix for frying their CPUs turned up as a topic a while ago.
Someone asked me for a list of endpoint/CPU/BIOS version.
Sure. No problem.

At that point in time all I had at hand was the filter "13th and 14th generation". I swear to my favourite drink!

List came up in the hundreds.

Some weeks and meetings later I somehow ended up in the meeting about finally informing the users about some major disruption, mandatory restart. After quite some time in, after listening to arguments on how best to phrase it in a way that users do not power off during BIOS update I remarked, that the numbers where inflated. 500+ was too high, since by now a list of apecific CPUs was out.
So I regenerated the list. With the new filter.
29 endpoints, 10 of which already had the fix installed (plus one VM).

...

Meeting ended with "let's just walk these down individually, no need to scare several thousand people."
The guy who had tested remoting the update for days just hung up.

I could have kept the list current when new infos came out, but I didn't check Intel's press releases regularly. But so could have done whoever was in charge of that project...

[-]

flapanther33781@reddit

There was too much drama in government jobs.

People have no idea.

Take the worst office politics you've seen in the private sector, and multiply that by 5x-10x. Front-line employees backstab each other like they're gunning for VP positions, daily.

Don't even dream about outshining your supervisor unless it's at a task they specifically asked you to do, and under absolutely no circumstances should you ever disagree with them, especially if they were in the service. 95% of the time they won't make a scene, but they'll start a paperwork trail on you and get you flagged as a problematic employee and make your world so horribly annoying that you'll want to quit or leave to another team.

[-]

reddit_username2021@reddit

Our team has been working very hard for years to migrate our primary enterprise-wide application to AWS. Some time later, internal developers developed a system I know very little about and connected it to the application server. Since I take care of the application server and they were literally running DOS on it, I blocked the traffic. It didn't last long because management forced me to unblock it. The management tries hard to silence me, I take the blame from the users for the slow performance of the application server, and nobody wants to do anything about it...

[-]

JudgeCastle@reddit

Similar. We had a bunch of accounts which were generics for our road techs. High turnover position. One got compromised so we wanted to force Pw reset on that OU. Decided to throw some pwsh code together, do it in one swoop vs individually. Talking like 200 accounts.

BigBangFlash@reddit

Back when I was lvl 1 tech support, I got a ticket about a specific engineering software crashing everytime the computer went to sleep. If users didn't save their work and went to lunch, they basically lost their morning's work since they had to reboot to even reopen the software.

By tinkering a bit, I figured out the HP Zserver crap installed for remote access that was rushed when Covid started was interfering with X11, and the app didn't actually crash but had trouble displaying the windows when recuperating from sleep. While I couldn't fix it, I found that if we simply restarted the XServer (or killed the windows, I don't exactly remember), it fixed itself. So I wrote a super simple bash script that I put on the users desktop, double-click it and you get your work back. All users were extremely happy, even though it was only a workaround for the moment.

Well I got yelled at by the sys admin in charge for taking this ticket. While I thought I was doing my job (I got a ticket, answered the ticket and found a work-around), he told me I should have opened another ticket with the internal software team. Even though they had like 5 tickets about that exact issue that had been opened for over a month for which I found a work-around for in about 2-3 hours of tinkering.

At the same job, I wrote a simple EDID file to point the Xserver to since users were all working from home and "we didn't have enough monitors" for the job. The way HP-ZServer works is that it remotes into the actual Xserver, so if we plug 1080p monitors at work, the users get 1080p resolution remotely. If we plug in 4K monitors, as long as the users have 4K monitors at home they can use them at full resolution. It's such a weird software. Anyways, instead of having them buy like 100 monitors, I pushed a very simple EDID file to a few workstations with varying resolutions to show how we could save tens of thousands of dollars. Well, you could have guessed it, the same sys admin yelled at me for not opening a ticket, have THEM review it and implement it. So they bought like a 100 EDID HDMI plugs and a few 4k monitors.

I left right after, that one single guy was seriously unhinged.

[-]

Quiksilver15@reddit

I just stayed late and spun up spiceworks, got it going, sent everyone logins and instructions and said we're doing this instead of outlook, fully expecting to get a reaming but she was SO happy, the service desk was so happy and I was blown away that it scaled to over 4000 tickets, self hosted on some shitty old workstation in the corner of my office with zero performance issues and was flawless. Seriously a great ticketing system and it bridged the gap until we joined up with the new company and transitioned to their ticketing system.

Sometimes a POC that you just ramrod into people's faces will overcome their irritation with you refusing to listen - especially if it costs the company nothing in labor or cash.

[-]

User1539@reddit

I was asked to evaluate AI chat bot solutions. I was assigned this task with the instructions "Check over these two, and if you can think of any other options bring them to the table."

At the time I was working in a place with a lot of sensitive data, and they wanted an AI chat bot to help answer questions about that data. But, in evaluating the options, security told me we couldn't actually send any of our data to the bot.

As a workaround I spun up some vector databases and used the embedding system to create a back-end that would figure out what the question was, go into our database and get all the relevant records, along with instructions on how to answer the question, then anonymize the data, and send that whole mess to the AI ... then, on the record's return, it would re-fill in the anonymous bits.

I almost got fired, because upper management had already chosen a solution, and when they demoed mine, it looked 1,000 times better, and was so obviously a better solution that everyone looked dumb saying 'Well, we already signed a contract ... sooo '

[-]

peekeend@reddit

Whe had a environment for students by students. The freedom it had some teachers where taking examens of this environment wich was fine if you told me.

Now whe had a project to bring pfsense into the high availability, this student who thought he could do it, oke try it.
Now this person pulls a backup from pf01 and deploys it to pf02. now we have two routers with the same ip address. I know that this was happening so it means down time, nice learning moment for this student. This Teacher that did mention to me that he had examens running got mad at me. While i had a perfect learning moment for this student.

[-]

DrDuckling951@reddit

I setup PowerShell to automate some email notifications. Later Network Engineer blocked port 25 on some servers (the server I have PowerShell running). Email stops. We didn’t realized what happened until 2-3 days later.

Not really getting into trouble for the PowerShell but got into trouble for not documenting the process in the main workflow on OneNote/Visio.

[-]

packet_weaver@reddit

Silly network engineer. When blocking ports that were open already, you monitor the usage, track down the users of it and find out if it is legit and then create rules for that specific traffic before blocking the port.

[-]

MortadellaKing@reddit

I fixed a problem with a client's app server on a weekend once, but it caused everyone to have to reboot their machines. Instead of being down Monday morning all they had to do was reboot or log out and log in again. Of course it wasn't taken very well and they sent me a stern email about doing work without informing them beforehand. So next time they had some major network issue on the weekend I just ignored them until Monday. "Sorry Bob, just following your instructions!"

[-]

DiaDollasignPora@reddit

See this is exactly what my company does,

They give high end machines to the devs,

New employees get new latitudes.

The helpdesk gets the oldest jankiest computers, because why would you want the people troubleshooting helping others rather than chase their own tails??

Also, our toilet paper disintegrates once it touches your ass cheeks. We profit 70 mil a year.

[-]

vdragonmpc@reddit

I wrote a script that tallied logins/outs to company equipment. Some folks were claiming late work and project time.

Boss said something about it to the CEO.

Oh boy did that go well.

[-]