Managing 200 Linux machines with no automation – AWX or alternatives?
Posted by sRonk96@reddit | linuxadmin | View on Reddit | 75 comments
Hi everyone,
I’m about to start a new job where I’ll be responsible for around 200 Linux machines… with basically zero automation in place.
In my previous experience, I’ve always relied on AWX (Ansible Tower), so that’s what I’m most comfortable with. However, before I jump in and standardize everything around it, I’d really like to hear some opinions from the community.
Do you think AWX is still a solid choice in this scenario?
Would you recommend any alternatives or complementary tools?
Any advice, experiences, or suggestions are more than welcome. Thanks in advance!
Highpanurg@reddit
Just git repo with Ansible playbook and simple ci/cd. That will be enough.
TimekillerTK@reddit
Take a look at the nix package manager and NixOS for configuration management: https://nixos.org/
There's a steep learning curve, if you're not familiar with defining machines with what is essentially a functional programming language, but the payoff is absolutely bonkers.
Our devs manage the configuration of their Linux/macOS workstations (along with managing their development environments with nix) and we have several machines in prod on NixOS. We've never been happier.
eman0821@reddit
Not meant for production environments. Almost no company uses NixOS especially with the lack of support for most applications nor its made for enterprise environments. RHEL is the defacto standard in enterprise IT.
TimekillerTK@reddit
You can also use the nix package manager on RHEL, and gain access to a large repository of packaged linux software that RHEL repositories do not have and benefit from both.
jw_ken@reddit
I would recommend command-line Ansible, with Semaphore or Rundeck in front of it.
Semaphore is more Ansible-centric, Rundeck is a more generic runbook automation product. Both support RBAC, web hooks / API for other integrations, secret storage, visual interface for executing playbooks, and task scheduling / automation.
We use Rundeck at our current org as an Ansible / scripting job runner, and it works well. It has some neat tricks up its sleeve that AWX didn't have, like cascading job options. (Imagine an interactive job survey for expanding LUNs. User picks the environment in option #1, then option 2 auto-populates with choices based on what was selected in option 1, etc).
Rundeck let us wrap a nice candy coating around many of our scripts and playbooks, making them easy for an ops team to run self-service- while seasoned engineers could still run stuff directly if they wanted.
bikernaut@reddit
+1 for rundeck. We manage thousands of vms with it. Lots of delegated self serve jobs too. It’s the only job runner I have found that lets you assign permissions based on the ansible groups you assign to machines.
Ad hoc commands are killer too. I rarely ssh to machines now just do everything through rundeck
nitroman89@reddit
I used Rundeck a few years ago and it worked with a lot of nuances. Semaphore is a lot better on integrating with the playbooks especially using a Git repository.
See-9@reddit
Tower blows. Run ansible from cli or pipelines. Use it for specific runs (ci/cd state change), use salt or something if you want long running drift/poll based config.
sRonk96@reddit (OP)
Not for me, but if some colleague unfamiliar with the terminal should use it, the graphical interface helps
shyouko@reddit
I'm not letting anyone could can't handle simple CLI handle the servers, tbh
vaulden42@reddit
Yeah, but many of us don't have a say in the matter.
Big-Minimum6368@reddit
It's a bunch of Linux boxes, if they are uncomfortable with a CLI they probably need to rethink their life's choices.
nerdyviking88@reddit
helps, but also hand holds.
Depends on your use cases, obviously, but people don't learn until they have to.
eman0821@reddit
Just build an Ansible server and setup a Git repo and be done with it. No need to over complicate everything.
s1lv3rbug@reddit
Use git + ansible to manage playbooks.
sRonk96@reddit (OP)
I do the same with awx
denisgukov@reddit
AWX no longer being developed. The last update was in 2024. It's very strange to start a new project on an outdated platform.
Zehicle@reddit
Can you give some more background? Bare metal or VM? How often do you want to update? Is there a performance requirement or speed to reset need? Single vendor or multiple? Any specialized networking? What's the workload?
sRonk96@reddit (OP)
Mixed, some vm in cloud high vmware, no physical machine. Different sales but I hope with my arrival to put standards (since most of the machines are to be redone), I don't have all the other information at the moment, I'll start working next week
Zehicle@reddit
If everything is VMware, their management tools are a good bet. BUT, very few want to double down on VMware and I couldn't recommend that. I've heard good things about Morpheus (now owned by HPE) for your use cases.
hlamark@reddit
The Foreman/Katello stack together with Ansible is a good choice. If you’re looking for a more stable, quality-assured solution with enterprise-grade support, consider downstream offerings like Red Hat Satellite 6 (RHEL-only) or ATIX orcharhino (supports multiple Linux distributions).
itsgottabered@reddit
AWX is dying. I'd go with semaphore.
sRonk96@reddit (OP)
Why is dying? I have never used semaphore, it’s hard?
Eulerious@reddit
Look at the GitHub repo:
no updates for almost 2 years is... Brave for software that is so critical for your infrastructure
you can bet your ass that AWX won't get better for users with this refactoring (if they ever finish)
itsgottabered@reddit
RH are pushing people towards AAP. I don't think semaphore is harder than anything else out there, and compares well from a feature perspective.
nerdyviking88@reddit
Redhat has refactored how they are releasing AWX now, as they revamp into a more modular development. As such, AWX is no longer serving as a true upstream to tower.
They're also, iirc, no longer releasing any packages or such for AWX, asking you to build from source.
cgherman@reddit
For this number of servers Salt (https://saltproject.io/) is a very good solution. You can store the state files (playbooks) in git and master pull latest changes from there
It is very fast compared to Ansible
ryebread157@reddit
Use Semaphore or Ascender, AWX not updated in over a year and its future is uncertain.
glotzerhotze@reddit
Take a look at https://uyuni-project.org
nitroman89@reddit
I use it and it's great! Especially coming from Satellite or Oracle Linux Manager. You factor in using salt state configs and it covers most of the use cases. I still use Ansible with Semaphore for some situations.
sRonk96@reddit (OP)
I have the vomit of uyunj, horrible
glotzerhotze@reddit
Nice. Thanks for elaborating what your issues are! This will help a lot of other people, I‘m really sure.
cemo1304@reddit
If it's a new job and you already have Ansible experience, then definitely go with that. You can do the same thing with ansible/puppet/chef/salt, just in slightly different ways. Try to make your life easier by using a familiar tool.
sRonk96@reddit (OP)
I'll go to awx then, thanks!
Vuiz@reddit
The issue with AWX right now is its lack of support. Redhat may or may not get it back on track. Make sure you are aware of the pros & cons of it.
For ~200 machines we are not talking about a huge number of staff. You could also look into Semaphore instead. Me and my own Team ran it for our own servers of ~70 or so, worked pretty well.
eraser215@reddit
AWX was never supported. Now that it is subsumed into the far bigger, more complex solution that is ansible automation platform, building it as a standalone component probably isn't a priority for RH. I don't think AWX had community contributions.
nitroman89@reddit
I use a combination of Uyuni Project for patching and state configs then Ansible with Semaphore UI for my adhoc configurations. I'm managing about 110 Linux servers, mostly Ubuntu and about 6 Oracle Linux for the DB dipshits.
Idlafriff0@reddit
Ansible is too slow, so I would use Pyinfra. Here is a speed comparison between Ansible, Pyinfra, and Fabric.
https://docs.pyinfra.com/en/3.x/performance.html
If you're interested, I recommend giving it a try. You might also want to take a look at this document.
https://marp.kalvad.com/fosdem_2026
SuperQue@reddit
If you want performance, why go with Python when there's MGMT.
Idlafriff0@reddit
No, it’s not just about speed. I also like how clean and concise the code is in Python. It feels great to be free from the YAML hell of Ansible.
srekkas@reddit
Add AWX schedule and forget. Who care it took minute or two :)
sRonk96@reddit (OP)
I agree
drunkenjunkconstruct@reddit
awx rbac was useful when i had multiple teams touching the same playbooks tbh
Shake69@reddit
NL?
seanhead@reddit
For long running machines/vms I'd probably do this with puppet.
Pretend-Weird26@reddit
Run the Community (CLI) version of Ansible. I have ever used Tower, no one could afford, it but have heard nothing good. Learning yaml is a valuable skill. It is easy to go from CLI to GUI/packaged tools but is hard to go the other way. it's like bash coding or Vim; you can make a good career at cheap companies having experience doing it the hard way.
If that is off base AWX is solidly in your wheelhouse. That many servers, the wrong choice will haunt you for years. Experiment in a dev env. The worst is troubleshooting your tools during an "event".
BloodyIron@reddit
AWX is the community (not paid) version of Ansible Tower, btw.
Pretend-Weird26@reddit
K. Can't keep up with the names.
Off my lawn you young whipper snappers
BloodyIron@reddit
Apparently I might not be fully up to speed either!
grumpysysadmin@reddit
It’s not tower anymore either. It’s Ansible Automation Platfoem (AAP). Unless that’s changed too…
BloodyIron@reddit
Hmmmm I might need to catch up on some things then...
sRonk96@reddit (OP)
But it's not that with awx you don't write yaml code, in the end I connect awx to a github repo and I write the yaml codes there what changes?
Pretend-Weird26@reddit
Yes. Last few places I have worked have been deeply suspicious of GitHub. Current place has air gapped environments. Doing it the hard way fills my day. Pays well, but yeah.
BloodyIron@reddit
Well could be completely achievable with self-hosted GitLab btw :)
oloryn@reddit
Or a self-hosted Gitea (will run on a smaller server).
BloodyIron@reddit
Sure!
BloodyIron@reddit
IMO if you have it work against a code repo, set up self-hosted GitLab so you maintain total control.
pnutjam@reddit
Tower is really good at afew things that are necessary for a good enterprise install. If you don't need this it's overkill.
1. vaulting passwords
managing access levels (ie. write a play others can run)
storing logs of plays that run
scheduling playbooks
You can do this stuff with the CLI, but it's more labor intensive.
Pretend-Weird26@reddit
I would have used the word fiddly, but labor intensive is fair. Would be great for compliance audit. Guess it also would depend on the industry sector
Loud_Posseidon@reddit
I'd go with salt/puppet/chef/cfengine (personally prefer cfengine due to how easy it is to maintain - 1 package, deploy - worked OOB with no additional setup needed, and how easy it is on resources, though at the cost of having fairly complex DSL).
Ran central cfengine hub (enterprise, for support reasons) for 4k VMs on 2 CPU/4GB of RAM, plus it scaled linearly, as it serves as distribution point and evaluation is done on the endpoints. So more endpoints meant only shutting down hub VM, adding CPUs, booting it up and continuing.
There was a git pipeline around cfengine repo, so it was easy finding out who and when did what change under what change request for what reasons. This has helped us a ton.
The changes were applied across all the servers within minutes (literally 5-6 minutes), no need to wait for next ansible run and by its mode of operation (autonomous agent), cfengine captured/managed/configured even machines that came online after a while etc. Something you don't get by design with ansible.
HeadlessChild@reddit
We use CFEngine at my $ORG for ~4K hosts and it still holds up. It has a relatively steep learning curve but when you are familiar with it is quite simple, in q good sense.
Acrobatic_Method_320@reddit
Use semaphore it is opensource,
jimsu@reddit
I run CLI myself.. works great . But I setup semaphoreui for others to kick off jobs without needing access/credentials, and also to automate things.
I liked awx before it went full kubernetes (call me old).. and we looked at tower.. but as soon as we saw the annual and it was only for 100 nodes.. funk dat!
Kahless_2K@reddit
I manage 7000 machines with Foreman. It's a significant lift to set up, but it's pretty great.
swissarmychainsaw@reddit
Go with what you are good at. Use claude to help write scripts.
Going from nothing to automation of any kind will be a huge win.
BloodyIron@reddit
I would say it depends on what distro you're dealing with. If you're dealing with RHEL that's fully licensed, I'm pretty sure you should be able to spin up a RH Satellite system with the included licensing to manage them. If that is the case, I would HIGHLY recommend you do that.
I personally have used RH Satellite to manage literally thousands of systems (this was already set up before I walked into the environment so alts like AWX/Terraform/Salt/etc weren't feasible at the time) and it was actually really great for that job! But... it's not exactly "easy", just very good at managing lots of RH related systems.
If it's Ubuntu, maybe look at Landscape options.
I'm not averse to AWX/Ansible Tower, and that might be the "right" answer based on what distros you use. But throwing ideas out there for your consideration.
gforke@reddit
I'm using foreman+ansible for the linux machines at the small company I work for, most stuff is still just login via ssh and run commands tough....
6stringt3ch@reddit
+1 for Foreman/Katello with Ansible
Xionous_@reddit
I use Kestra, you can run ansible playbooks with it
roiki11@reddit
Automation platform is really good if you have the usecases for it. But you can also work with semaphore which works about as well. Or gitlab with its runners can build an entire gitops system with ansible and no external tools. With automation git is kinda mandatory to keep your work organized and as a source of truth for the automation. Whatever eventually runs it.
But you really should look into foreman. Makes the whole fleet management experience way easier and works nicely with ansible.
Though is prefer to use automation platform for the actual automation part. Foreman(satellite for me) is for the package delivery, updates, reboots and such. But you can also integrate dns and dhcp into foreman.
And if you deal with certificates and secrets then vault(or openbao) is a good tool to integrate a bunch of secrets and access related tasks into one tool.
And standardize into one distro. Makes management a thousand times easier.
sRonk96@reddit (OP)
Thanks for the advice! I have to study semaphore and foreman
mdub881@reddit
if AWX/Ansible Tower is overkill for your use then look into Ansible Semaphore.
ollybee@reddit
Unsexy and underated is making good use of custom repositories and custom packages. Both rpm and deb allow you to deploy scripts that run on install or upgrade and optionally overwrite config file to stop drift. You can easily do graduated roll outs as well. I still use Ansible but only for things that cant be achieved with package updates.
ITViking@reddit
Rundeck
equinoxstar@reddit
consider puppet/chef iac style of keeping things consistent