Anyone else feel like Linux courses don’t translate well to real production issues?
Posted by meissloth@reddit | sysadmin | View on Reddit | 44 comments
I work in DevOps and my linux is good enough until something breaks.
then I realize I don’t actually understand things properly.
I can follow docs and run commands, but troubleshooting (services, perms, networking, logs, containers) is where I get stuck or slow.
I’ve tried the usual stuff but it doesn’t really translate when you’re dealing with real issues.
Maybe I’m learning it wrong, but “just learn Linux” hasn’t helped much lol
Looking for something practical that actually helps with real-world debugging.
What worked for you?
ErikTheEngineer@reddit
TBH the difficult thing in a devops environment isn't the day to day system administration...it's troubleshooting the Rube Goldberg machine of CICD, GitHub actions, running 20,000 build steps on 11 nested machines to perform a task. Developers look at this and see a brilliant orchestrated well oiled machine...traditional admins focused on simplicity and uptime see it as something with too many moving parts and things to go wrong. And the annoying thing is when you show them what's failing, their attitude is that you couldn't possibly understand their brilliant code and go clean my build for me. I think that's where a lot of people get tripped up...what's an example of something you don't get?
If you're coming from the Windows world, the biggest thing to understand is that "the system" isn't one big monolithic stack that gets updated once a month. It's blown apart into 37,484 pieces, all of which can be interchanged for Yet Another piece that does the same function. Every distro has a slightly different spin on things, and if you work in an environment with "clever" developers you'll often see their "clever" preferences and work on display making things more nonstandard. This carries over into the whole cloud native practice...so fundamentals are important because if you learn one toolset and stack, no one will hire you because it isn't "their" stack which is just a bunch of random stuff pulled from that list so someone can do resume-driven development.
vogelke@reddit
Can show you an unhappy server and ask for fixes.
Basic skills needed by an admin.
Basic steps for troubleshooting a network issue
Nice evaluation-only description of general troubleshooting. I'd be willing to buy this as a book.
58-page sample chapter of Kyle Rankin book on standard troubleshooting.
696-page Peter Harrison book. It's a little old (2005) but it has a ton of troubleshooting stuff.
SadServers_com@reddit
:-)
Anonymity_Is_Good@reddit
The learning curve might not be right for everyone, but the free tier on SadServers is arguably worth at least a look.
SadServers_com@reddit
hopefully the paid ones are worth it as well ;-)
jlrueda@reddit
That is the difference between degree and experience and applies to everything in life.
chronop@reddit
do you have a lab environment? that was the biggest thing for me, once you set up a homelab and start spinning up services of your own and breaking them / fixing them, you tend to have a leg up with troubleshooting because you've already seen half the stuff already.
Break2FixIT@reddit
This is how I expelled in troubleshooting!
CowardyLurker@reddit
Seems like you’re paying attention, right on track. Good on you.
Experience, it’s valuable stuff.
unix_heretic@reddit
This, by definition, is not "good enough". You've learned tools without understanding how they fit together: you read docs, run commands, and that is the limit of your interest.
If you're serious about learning, you need to start from fundamentals. Kernel basics, directory conventions, filesystem permissions, process management (e.g. init systems,
/proc), disk partitioning (both static partitions and LVM), networking config (especially how iptables ties in).If you need something practical, study the RHCSA and take the exam. It's a practical, rather than quiz-style setup.
Burgergold@reddit
Ive worked as a unix sysadmin for the last 23y
AIX for 12y, linux (mainly rhel) and some windows since then. I've learned more that way that with course. Course is basic knowledge, real task is when you learn
ThimMerrilyn@reddit
You’ll never learn how anything truly works until you break it and have to fix it. cleaning ip clusterfucks are how you level up.
mithoron@reddit
Pulling the trigger on my Linux gaming laptop is helping. Real world, weird shit problems you probably care about. Not esoteric word problems.
_SleezyPMartini_@reddit
run BSD instead!
rejectionhotlin3@reddit
FreeBSD, zfs, bhyve and jails is life.
Anonymity_Is_Good@reddit
FreeBSD, at least in the early days, used rc scripts and a basic inittab to do most of the things. Tracing through how the system starts up and going through the run levels was a good introduction to what needs to happen on UNIX/Linux.
Frothyleet@reddit
It's too insecure, they had that unpatched vulnerability for like 25 years!
(jk just referencing that first anthropic mythos announcement)
coukou76@reddit
Your linux is beginner. Keep grinding
ShadowSlayer1441@reddit
r/homelab
SkywardSyntax@reddit
seriously - homelab issues translate pretty well to SMB prod issues
Just yesterday - ran into a scenario of a failing drive on a remote installation. Ran into a similar issue using a cheap nvme that failed in one of my proxmox server lmfao
cdoublejj@reddit
F'nA, nothing like forgetting all passwords on a FUBAR vmwares ESXi cluster after 2 power outages and weaseling your way in to the domain controller to reset your password and reboot the vSphere controller VM and then realize one of the RAID arrays has a dead drive but, none of the drives are numbered to know which one it is.
ought make it in board game form https://imgur.com/a/jm9NTk1
lurkandprosper@reddit
This is the main gap that trips people up in devops, because knowing commands is not the same as knowing why the system is angry. I would look for a course covering:
systemd
logs
permissions
networking
processes
disk
Pretty much not just cd and ls for the hundredth time.
The devops path on boot dev should be tailored for your needs becasue it has coverage for linux, python, docker, aws, and ci cd
cdoublejj@reddit
car owners manuals used to tell you have to adjust the valves in the engine and now they say not to drink battery acid and call the dealer.
i remember when Microslop has read KB web pages with real documentation. it seems the whole industry has gotten dumber, like how stuff works is a secret but, maybe i'm just butt hurt.
sobrique@reddit
I am pretty sure the same can be said of Windows though. Doing the basics isn't at all the same as understand how it all works.
On both platforms that's a lot more than "basic training" can ever give.
lurkandprosper@reddit
For your use case I would care less about whether the course says Linux and more about whether it forces troubleshooting. Services failing to start, ports not listening, users and groups being wrong, containers behaving differently than the host, log files filling disks, that kind of thing. KodeKloud has a lot of DevOps lab style material from what I have read, so I would compare that kind of hands on format against Boot dev
byrontheconqueror@reddit
Ditto to all of that. Seems like there is a gap. I only seem to find the super beginner where they go over ls and cd or on the opposite end where they walk you through recompiling the kernel.
sobrique@reddit
Of course not. But the same is true of Windows for a lot of the same reasons.
And if you go on a training course to learn python, C, HTML, web design or whatever it still doesn't really give you more than the basics.
There's a reason why degrees are 3-4 years.
A software engineer who's "done a course" is a junior for much the same reason.
Sysadmin in particular is a confluence of "knowing the basics" but with a knack of analysing and problem solving and finished with a layer of broad experience and deep knowing of the underlying tech concepts. Also elements of business analysis and integration.
25 years in and I am still learning. I know enough to get started with most problems, and some I have seen before ... But plenty are new too.
sdrawkcabineter@reddit
It's like they're not able to teach a system of operations, but some smaller, common kernel...
FreeBSD is an operating system designed to be learned.
ProperEye8285@reddit
I'm going to go old school and suggest some O'Reilly Books:
https://www.amazon.com/stores/OReillyMedia/page/A2798F2E-D247-4F52-A80C-D695D9792A74
The "Fish" book on Learning BASH was a requirement! Maybe pick that one up first, go through the lessons and see if it helps you. If so, move on to the Linux Cookbook or Understanding the Kernel.
TerrorsOfTheDark@reddit
I learned linux by building my own distribution by compiling everything from scratch. The entire ecosystem makes a lot more sense when you become very familiar with every design choice from bios to kernel jumps all the way up to a graphical desktop.
Swimsuit-Area@reddit
As much as I hate to admit it, STIG testing did the most for my Linux knowledge. I would change settings, it would BF my server and I’d have to fix it.
highcaterpillarr@reddit
yeah the intro linux stuff gets useless fast once production starts doing production things. you want something that makes you debug broken services, weird permissions, failed networking, bad mounts, logs that lie to your face, the whole mess. otherwise it is just youtube telling you what grep does again. i bet your empoyer will cover whichever one you take
itishowitisanditbad@reddit
Well thats devops in a nutshell.
entaille@reddit
this is pretty universally true for any formal education vs real world experience. I'd suggest tapping into AI tools as much as possible - not just for troubleshooting a problem, but for creating educational lab testing scenarios and learning purposes. there's a big difference between using it to figure something out for you vs using it to explain things to you in technical detail and working through the issues to learn from them.
Dirty_Techie@reddit
Definitely a homelab environment, it's helped me to deploy and maintain Linux solutions in a enterprise environment.
I mainly dabble in Proxmox, debian to help me get the experience and understanding but I still couldn't tell you if rm -rf / as sudo would fix your permission issue /s
spellstealyoslowfall@reddit
It can be
TurbulentAmoebaa@reddit
this reminds me of learning the basement of a building you already work in every day. you can ignore it for a while, but when the pipes burst everyone suddenly expects you to know where every valve is. look for a course that teaches the shape of the system, not just a bag of commands
loupgarou21@reddit
I'll admit that it's been a looooooong time since I went through linux classes, but taking a linux class is not that different from taking an active directory class. The class is going to teach you enough so you know how to poke around, but not how to fix all of the problems or best practices.
Use a search engine, or AI, or whatever, research the problems you're running into, fix them, rinse, repeat. After a few years, you'll forget what you didn't know and tell other people to just learn linux.
Top-Perspective-4069@reddit
This will seem really basic but do you understand the fundamental concepts of troubleshooting? In general, troubleshooting anything at all follows a pretty similar process.
Step 1 is scoping a problem - what is it doing that it shouldn't be or what is it not doing that it should be? Step 2 is data collection, almost always in the form of logs. Step 3 is acting on what you got from step 2.
Is your problem that you don't know how to find logs, don't know how to read logs, or something else entirely? Where does what you do actually break down?
smolgremlinn@reddit
The linux foundation courses are more formal from what I have seen. I think the comment recommending the devops path on boot dev is worth looking at
Frothyleet@reddit
Do you feel better equipped to troubleshoot other platforms like Windows? In my experience, the fundamental skills for troubleshooting translate perfectly fine between environments, but obviously having more experience and familiarity expedite the process.
If that's the gap, it's hard to suggest anything else besides spending time with it.
Nonaveragemonkey@reddit
Best education you'll ever get in linux is real world building, breaking and fixing.
digitaltransmutation@reddit
Instead of asking "how do I fix this", ask "What info can I gather to fully understand this problem." This is the difference between yolo-pasting snippets from the internet and actually getting an RCA.
MrNiceBalls@reddit
What you described is the opposite of good enough, but that's just different backgrounds talking.
Just do an RHCSA or at least go through its prerequisites and then continue from there.