A few months into letting non-technical staff use AI coding tools
Posted by allmightybrandon@reddit | sysadmin | View on Reddit | 86 comments
A while ago I posted about our company giving Claude Code to non-technical staff without much of a plan around review, ownership, access, or support.
Original post: https://www.reddit.com/r/sysadmin/comments/1s9oj5z/rolling_out_ai_coding_tools_to_nontechnical_staff/
Figured I'd share where things landed after the initial excitement wore off.
It has not been a disaster. Nobody vibe-coded our warehouse systems into the ground. Most people tried it for a few days, hit the first confusing error, and stopped.
A small group kept using it though. Mostly for practical internal tasks: CSV cleanup, weekly reports, small dashboards, moving data between systems, and replacing bits of spreadsheet-driven process.
Some of it is genuinely useful. Annoyingly useful.
The problem is not dramatic AI failure. It is boring sysadmin stuff.
Scripts running from laptops. Personal API tokens. Scheduled jobs nobody can see. CSV processors that quietly become part of a team's morning routine.
One report script worked fine until the person who wrote it went on holiday and their laptop was off. Apparently that was now an outage.
So now we are trying to put a lightweight path around this:
- shared data means it goes in a repo
- no personal tokens beyond local testing
- scheduled jobs need to run somewhere visible
- every tool needs a business owner
- anything other teams rely on gets some technical review
Nothing revolutionary. Just the rules we already wanted for scripts and internal tools, except now more people can create them faster.
I still do not think "everyone is a developer now" is the right framing. Most people just want the horrible spreadsheet/manual copy-paste thing to go away.
Curious how others are handling this phase. Treating it as shadow IT, or creating a lightweight path before these things become unofficial production systems?
Walbabyesser@reddit
Must be funny to see the decline of used tokens after the first wave đ¤
allmightybrandon@reddit (OP)
It was pretty much exactly that.
Big spike at the start, then most people hit one confusing error message and quietly went back to Excel. The interesting part is the few who kept using it. Those are the ones where you suddenly realise a "quick script" has become part of someoneâs daily process.
BoysenberryDue3637@reddit
We found that these people had no comprehension that they are now responsible for managing these applets - not IT. I called them the Access DB of the 2020's.
Maro1947@reddit
That is so accurate!
Walbabyesser@reddit
Must be funny to see the decline of used tokens after the first wave
Access enters the room
peterk_se@reddit
It's funny, I'm one of those non-IT that have vibed together tools at work to act as middleware between our various systems... Access is doing a phenomenal job đ ... I use SharePoint lists though to store the data
Walbabyesser@reddit
Urhh
peterk_se@reddit
đâ¤ď¸đŤĄ
Walbabyesser@reddit
Will make a suggestion to my superior to buy a flamethrower to handle those kind of situation
peterk_se@reddit
I mean linking external databases, ACE -> List. All part of access, fair game the way I see it!
Gullible-Surround486@reddit
pretty much, except now the Access db can also hit prod a lot faster.
linuxdragons@reddit
And exfiltrate all your data
Neither-Cup564@reddit
The cost of progress.
touchytypist@reddit
Our AI chatbot was announced about 20% of the employees used it the first day, a month later only about 2% use it. It only provides useful answers about 50% of the time.
AtarukA@reddit
Sounds higher than off shore helpdesk tbh.Â
Walbabyesser@reddit
Accurate data is important đ
Makav3lli@reddit
Youâre data can be the only thing it knows and still hallucinate if you donât have good hardware.. which is anywhere from a few hundred thousand to over a million depending on what youâre getting.
Unless of course youâre using a cloud then expect a double digit cloud spend a month on that project lol
Iconically_Lost@reddit
Can't have that. Need to set a minimum token usage into the KPI's /s
Frothyleet@reddit
I have heard of this actually happening at some places and it just boggles my mind
BenadrylCrumplsnatch@reddit
The only time I could ever imagine a KPI like this being reasonable is if we were doing product evaluation and needed to drive usage. I cannot fathom being forced to use tooling like this just for the sake of it.
rubixstudios@reddit
So how much time and money has it cost to review all the slop. đ The AI craze for non-dev is now in the billing phase. Where reverse engineering is the billing factor.
Lopoetve@reddit
We did the same for a bunch of internal stuff and the number one use?
"This is a miserable 30 minute copy/paste/edit job in a spreadsheet. Now I call up a claude skill, tell it a set of numbers from my side, and it does the rest for me by reaching out to public APIs to calculate their side."
bageloid@reddit
It's automating tedium away.
johnsmithdoe15@reddit
all these folks gleefully training their replacement lol
Lopoetve@reddit
Exactly. And since somehow that always ended up in the laps of senior folk - they're quite happy with "I tell it this and 10 minutes later I have it all in a standardized sheet to submit".
Ssakaa@reddit
All the while feeding your company's internal finances, etc., out to a third party that's totally never demonstrated a crass disregard for legal ownership/usage rights related to data.
VintageSin@reddit
Ideally you install the model locally. If your enterprise is using a public model... That's bad. The issue is small business and the in between won't use enterprise solutions.
Ssakaa@reddit
Ah... yes. Because enterprise definitely isn't buying into SaaS eqivalent AI solutions. They've definitely not been at the forefront of "offload that to someone else, we'll figure out how to pay for it later" for decades, through all the cloud side of things, etc...
VintageSin@reddit
Would this be PaaS not SaaS. SaaS can have local applications. It's when platforms obfuscate what it's doing where this is a problem. If it's able to be in your infrastructure and not on someone else's then that's what you want. But yes, I agree there are probably quite a few enterprises who will farm it out.
Lopoetve@reddit
I mean - pricing of our products is public information; it's pulling operational costs and details from public APIs and web pages to build overall ops modelling and break-even points on cloud software ops. Not much to leak there :p.
Ssakaa@reddit
Because surely none of your vendors would increase prices because they found out you have room in the budget, right? And competitors totally couldn't use a clear picture of everything you're spending on to know what you're doing before you announce it... and... etc.
Lopoetve@reddit
Not how it works - I AM one of the vendors. The price lists for what we consume are public too. Our competitors price sheets are also public. And it's not me spending the money - it's cost of setup and ops for things people purchase and run.
Hell of a lot easier having Claude calculate based on today's AWS or Azure or GCE or Dell or Cisco pricing sheets than it is for me to pull it up into their calculators, type it all in, copy data down, copy my internal ops estimates in, etc etc etc. What used to take me 45 minutes now takes 10. For others it was over an hour - same 10 minutes. I'm trying to cut that to 5, but our internal system (which ALSO feeds on various public pricing APIs but knows how many commands we send and the current equations on how) can't be easily scraped by AI unless I literally give it chrome. So we don't do that.
siegevjorn@reddit
So, more workload on admin/IT? Did you get a raise? Did your team hire more?
C-redditKarma@reddit
What youâve described is is a growing set of shadow it operations and itâs working for you now because itâs still early. None of this is scalable.
The report not working because someone was on PTO is a micro example of this.
Wait until your vibe coders are committing API tokens to source control.
Your blast radius is narrow than expected for now but if you continue to prop up these shadow IT workloads your Helpdesk is going to have an impossible job and youâll be making security/compliance concessions without possibly even knowing.
toddtimes@reddit
Everything you describe is so easily mitigated that it just comes across to me as hand wringing from someone whoâs too stuck in their ways and unwilling to grow as technology changes. This is a huge opportunity for massive boosts in productivity and all it needs is a bit of structure, which can be coded into the AI tools being used so the people donât even need to be responsible for making sure it happen.
Complete-Cricket-351@reddit
Hmmm an account that hides their posts doesn't inspire confidence - can't tell if a person or a bot. I'm on both sides - I'm IT and I vibe code. I think CRK's post was quite balanced.
toddtimes@reddit
People on Reddit are weird. Too often I interact in one place and they decide to go and leave strange responses on every post Iâve made in the last week. So I just turn off the visible history.
Complete-Cricket-351@reddit
Okay it doesn't sound like a bot so I'll kick inÂ
Short answer is both things can be true at the same timeÂ
And there is a middle pathÂ
So I have developed my own automations for job hunting generating blog and linked in posts a chatbot rag setup trained on specialist domains and a rehumanizer feature which applies editorial tools and removes AI smell from AI generated copyÂ
Use Claude co work blah blah... Old dog new tricksÂ
I have also rolled out AI governance in a couple of places and what we did for the halfway house was let people have sandpits now this was in regulated industries but most industries have some level of regulation like you're not going to contract out of privacy law whatever you doÂ
C-redditKarma@reddit
At no point did I suggest any of this was unsolve-able or unmitigate-able.
I am not anti-ai.
I am anti âshadow ITâ. Anti-chaos.
Pro-process.
Pro-system. Technological or otherwise.
You appear to be making a lot of assumptions.
toddtimes@reddit
I donât know how else to read âNone of this is scalable.â
Nothing in your OP suggests what you just wrote here. But glad to here what I read wasnât what you actually think.Â
C-redditKarma@reddit
âScripts running from laptops. Personal API tokens. Scheduled jobs nobody can see. CSV processors that quietly become part of a team's morning routine.â
This is shadow IT being propped up by a corporate culture that lacks process or governance being aided by a tool called AI.
Those scripts ran from laptops, scheduled jobs that one person owns: itâs not scalable.
This isnât an AI problem but AI is certainly allowing the velocity of this shadow IT to grow rapidly.
And left unchecked it will create a lot of tech debt.
I stand by my original post. Iâm not sure how this is so confusing or why you believe this is a personal attack on âAI.â
Have a good day.
itishowitisanditbad@reddit
Wait until there are 20 secret hidden apps you have no idea about and key people start leaving and those apps are 'critically necessary' and you're spending huge amounts of time repairing it.
They're taking on tech debt and just sweeping it out of eyesight and thinking its all good because they're not having to pay the tech debt bill yet.
acquiesce88@reddit
I can see it spiraling out of control as these new processes become business critical. But if not properly documented, they will become a nightmare to manage and support.
allmightybrandon@reddit (OP)
I don't exactly disagree, that's definitely the risk if thereâs no process around it.
My hope is that as people get used to the workflow, we get better at spotting which tools are becoming important and pulling those into proper infra early.
iam_ag@reddit
This is the thought I keep coming back to with regard to AI at least until we hit AGI [if]. We as humans are entirely capable of cobbling together a system or process too complex to realistically maintain, but now abstract it further with AI.
xThomas@reddit
How do you manage IT when everyone can create scripts by asking their âagentâ? It seems like itâd be impossible to track everything.
IWantsToBelieve@reddit
Do you just let regular staff access posh to execute this stuff on endpoints? How the heck do you secure the final product? What removes key person dependency risk for everything built? Repos+run servers or similar?
Kazukii@reddit
The laptop holiday problem is going to become the new classic story. Lightweight governance is the only path that makes sense. Banning things just pushes it underground. The people who actually want to solve their spreadsheet hell will find a way regardless. Giving them a safe path before it becomes critical is smart. Most of these scripts probably won't survive the vacation test anyway.
VintageSin@reddit
Sounds like more complex shadow it. It'll take a few years for those cracks to be more visible.
Neither-Cup564@reddit
Reminds me of when Microsoft Access was bundled with Office and suddenly someoneâs random DB is now critical production and they left 4 years ago and no one knows how it works.
AtarukA@reddit
Thankfully our access users pull data directly from the database... Hold up...
chesser45@reddit
Reading this as a positive post and hoping thatâs how itâs framed. Getting really exhausted by the circlejerk of AI bad, business people trying to be creative bad, trying to facilitate other people innovating bad.
Like I get it, AI is a tool and you can do dumb stuff with it but fuck our purpose is to empower the organization and also do our best to build and support systems that are secure. But itâs not our company, if leadership says jump, either jump or leave.
Personally I love seeing people be creative and then helping to turn bobs side project into something that can be safely used by his team or the org. Iâm creative but I have definite limits to my creativity. Let someone else do all the hard work and Iâll work to turn it into something that follows the rules.
SchemaAndShell@reddit
This thread delivered a refreshingly unexpected tale of positivity.
CraftyPancake@reddit
How on earth do you secure it in terms of app locker/app control. Everyone running all these unsigned programs
jks513@reddit
Reminds me so much of the everyone using MS Access doing their own thing to this really needs to be in a real, centralized SQL Server era.
Neither-Cup564@reddit
Haha I just commented the same. Those were fun times.
Aggressive-Fix241@reddit
>
>
I think the framing matters. "Shadow IT" sounds like something to eliminate. "Proof of concept that needs a home" sounds like something to adopt. Same scripts, different path.
nemor3@reddit
The laptop-as-server problem is older than AI tools. What's new is the rate at which it happens now.
The pattern you're describing, things that run on a timer with no visible owner, is exactly where incidents come from. Not because someone did something wrong, but because there's no single place that tracks "this thing is supposed to be running, and someone should know if it stops."
Scripts are one version of it. Scheduled jobs another. SSL certs, domain renewals, API keys with expiry dates - same failure mode. Something worked fine, nobody was watching, it expired or stopped, and the first sign was someone downstream asking where the output went.
The lightweight governance path you're building is right. The hardest sell is usually making people register the thing before it becomes critical, not after.
choss-board@reddit
FWIW this is going to sound reasonable and then nuts, but this problem is pushing us towards:
Plus all the usual DevSecOps stuff is on steroids now.
InsideAge3465@reddit
Same impression, I'm cautiously optimistic.Â
My worry was mostly with credentials and with those few users you can't deny write access to due to hierarchy; most other people can't do real damage and it does speed up some very tedious jobs.
The two major things we did which mitigated the risks were: 1. buying an automation tool like n8n - the automations are just XML files which can straight up be generated by LLMs (often built-in) and the entire thing has decent security safeguards 2. system prompt that directs the LLM to provide a solution importing the credentials through the password manager and steers it towards safety. Users who are not technical will go with whatever the machine suggests, as they don't know which steps are "extra"
TheCaptain53@reddit
Sounds like a whole lot of not your fucking problem!
Pervect66@reddit
The question is though did they vibecode a standalone script/app without using actual company data? If not, your data was technically leaked.... Which is a whole other issue.
Frothyleet@reddit
No leakage, he's saying they deployed Claude via an enterprise subscription (i.e. with the "we won't train on your data" agreement).
xSchizogenie@reddit
But you donât know which data was used by whom.
Frothyleet@reddit
I may not be understanding you correctly, but if we're talking about governance over existing company data sources, that's an issue that would/could occur irrespective of the use of vibecoding tools (i.e. if there aren't controls around access to data sources, that's a problem regardless of whether someone vibe codes or hand codes tools that touch them)
xSchizogenie@reddit
Exactly. Iâm sure, if in EU, no one of those made the EU AI Act certification.
Pervect66@reddit
Show me company leadership that gets this...
xSchizogenie@reddit
Well show them the fines that come with data breach and they should get it. Hit them where it hurts and a dog will bark.
gscjj@reddit
Yeah alot of AI concerns are overblown, a little bit of governance and itâs no different than managing any other type of tech.
dllhell79@reddit
Governance right now is the problem IMO. The tools around AI governance are few and far between right now. Vendors are beginning to catch up some, but there's still few generally affordable tools available.
bageloid@reddit
I mean, there are bunch of tools, they have just been all acquired multiple times in the last 6 months.
Training_Yak_4655@reddit
Couldn't you vibe code one?
BaconEatingChamp@reddit
My concerns are the costs of everything going up because of them. The memory, storage, energy.
MBILC@reddit
So you let people run local agents? Hope you have those secured down and monitoring.....
Frothyleet@reddit
That doesn't sound like what he's talking about. It sounds like he's saying someone used Claude Code to build a script (python or whatever) and run it as a scheduled job on their workstation. None of the parties involved would have the expertise to know why that's not a good way to deploy a production tool.
Pervect66@reddit
The smart ones will automate processes and not tell anyone. They can do their job in 10 hours instead of 40, so get to relax all day, and still come up with better (more consistent) work than their colleagues.....
Int-Merc805@reddit
Shhh, the first rule about automation club is you don't talk about Automation Club. The second rule is scheduled to run at 5:01 so I look like an overachiever...
beerpolice@reddit
canât you write the post on your own? All these ai posts written by ai. âCurious ifâ âcurious howâ jfc. Nobody has a brain anymore!
zero0n3@reddit
What? This post was unlikely to be made from AI. Maybe passed thru and revised, but has no major flags of a âlow effortâ AI generated content post.
Frankly Iâd rather read AI posts when itâs done by someone who knows the tool well. Almost always get better (efficiency and density) info out of the AI pass output
MBILC@reddit
This does get annoying, so long as the content is true, factual and useful, who cares? People didn't complain when people use Grammerly or other tools to help them write...
beerpolice@reddit
Nice backticks on your post too.
Fantastic-Shirt6037@reddit
Every other threat has people calling out ai âproduct researchâ posts for always ending with âcurious how others are handling thisâ
Fantastic-Shirt6037@reddit
Haha had me in the first half, not gonna lie.
The end was the best part, âcurious how others are handling this phaseâ đ
notospez@reddit
So you're actively writing and deploying steering files for Claude telling it all this, right?
kdanovsky@reddit
Have you ended up building a platform for the tools that become critical, or are you moving them into existing infra?
We recently open-sourced a tool for this exact problem. Would love to hear whether this fits what you're running into: https://github.com/compartmentdev/compartment
Commercial_Steak_657@reddit
Honestly, this sounds less like an AI problem and more like classic shadow IT. The tool changed, but the risk didn't. The real issue is finding out a business critical process depends on someones laptop or personal token months later.