containers on prem?
Posted by crankysysadmin@reddit | sysadmin | View on Reddit | 19 comments
Anyone running containers on prem? Our workload is not large enough to try to run k8s but too large to just run docker on a server. There in between space that is on prem is not ideal
RandomThrowAways0@reddit
Docker Swarm or Hashicorp Nomad are good for smaller setups. There are other, simpler, "lightweight" k8s distributions as well like k3s and k0s, even canonical has Microk8s. Kubernetes can be as hard or as simple as you make it. If your staff like using GUIs, look into Portainer for simplified management.
ashimbo@reddit
We're currently a 100% windows shop, and I haven't found any good use-case for containers in our environment, though I'm always hopeful.
roiki11@reddit
This really depends on your specific application but k8s isn't really that to set up. There's not really workload "small enough" that you couldn't run it on k8s.
And the next step up from docker is k8s. Unless you want to try swarm or nomad or such.
Ssakaa@reddit
From my homelab experience with it, cannot recommend swarm, simply because it's incredibly misleading in how much it looks like standard docker compose but then behaves just differently enough to break random assumptions you'll make.
BlueHatBrit@reddit
I disagree with this, swarm is totally fine within it's wheel-house. In a business setting you'd expect someone to spend some time upskilling and training on it. I'd never want anything in my business to be running on something that we're just exclusively making assumptions about.
I'd agree that it's documentation isn't fantastic, and the fact it's so close to docker and docker compose does create confusion. But that confusion is undone with some experimentation and reading.
Kuipyr@reddit
Just keep in mind you don't want to be the only one in your shop capable of managing a Kubernetes environment or you will have a bad time.
man__i__love__frogs@reddit
Why not docker container instances in Azure or AWS?
codatory@reddit
I'm a huge fan of podman quadlets for simple container deployments. Systemd integration, auto updates, almost no setup... If i need multi server, I'm probably running a kubernetes of some sort (probably k0s).
Hotshot55@reddit
Yes, loads of them.
crankysysadmin@reddit (OP)
well what is your setup?
Hotshot55@reddit
All sorts, docker, kubernetes, k3s, podman. Different tools for different requirements.
Sufficient_Yak2025@reddit
Why do you think you need large workloads to run k8s?
imnotonreddit2025@reddit
Docker Swarm might be the sweet spot for you.
AxisNL@reddit
A lot of companies do kubernetes even in really small setups, the concept is the same. But I know a lot of companies that use docker in swarm mode, whether or not with portainer for easy management. Or the really poor man’s solution is manually create some servers or VMs and manually decide which containers run on which machines with docker-compose or local portainer.
crankysysadmin@reddit (OP)
running docker on a bunch of servers is what doesn't scale or go so well
AxisNL@reddit
Well, the logical solution would be kubernetes or docker in swarm mode, but I have a client where we had about a hundred VM’s all running docker with a local docker compose service, deployed using puppet, across a VMware vsphere cluster with 7 or 8 nodes. Perhaps it wasn’t the best setup, but it had its advantages and disadvantages, and VMware DRS did a pretty good job of balancing load ;)
eruffini@reddit
That really depends on your environment needs.
Throwing up K8S for the sake of K8S without architecting your application(s) to use K8S other than "containers" is the wrong approach.
Sometimes it's as simple as deploying Docker on a bunch of servers and centrally managing them.
Sometimes it's a full-blown K8S stack with multiple pods, CNI, etc.
Both of these can scale, they just do it differently, and for different reasons.
Pristine_Curve@reddit
Can't really guide you unless we know what specific part is falling short.
"We have a bunch of stuff to manage, some of it is containers. Managing it is mostly about config management and setup consistency, and not programmatic flexibility."
Will produce different guidance vs:
"We are constantly creating/destroying containers and are looking to automate the process more with an eye towards application driven provisioning for workloads etc..."
Ssakaa@reddit
For "just a few", where what you need is a container but what you have are VMs, ansible does a nice job if standing up a container or three in a vm it built a couple minutes before.
If you actually want the benefits of containers, bite the bullet, stand up a small few k8s nodes, get people started down that road.