How do you guys provide your developers with Rebooting ability on their Ubuntus?

Posted by Good_Panic1986@reddit | linuxadmin | View on Reddit | 15 comments

Our users ubuntu machines have either been configured on MAAS in the server room or are on VM in the vSphere. From time to time they need their ubuntu machines get rebooted due to so many dangling dockers eating their CPU and they have to submit a ticket so we do it for them from the server side.

I wanted to see how other teams are handling this and how we can provide our users the reboot availability on their own?

[-]

deacon91@reddit

Configure Ubuntu w/ AAP/AWX user and service key
Create AAP/AWX job that sends sudo reboot command
profit

[-]

reddit-MT@reddit

"Reboot to fix" is a Windows mentality. Under UNIX-like systems, there's usually a more elegant solution.

[-]

SuperQue@reddit

ubuntu machines get rebooted due to so many dangling dockers

Ahh, XY Problems.

[-]

UsedToLikeThisStuff@reddit

Yeah, why not write a script that kills off all the containers and restarts the dockerd, and give sudo rights to just run that?

[-]

SuperQue@reddit

Kubernetes has entered the chat

[-]

Resource_account@reddit

If they’re not that many containers on the host I would even suggest a simple podman quadlet setup. Create free .container files, generate the equivalent unit files and set them to run at the user level. Let systemd handle the rest

[-]

whetu@reddit

From time to time they need their ubuntu machines get rebooted due to so many dangling dockers eating their CPU

Make sure their users are in the docker group and train them with how to use docker properly?

And add a cronjob that routinely runs some cleanup tasks? Like this:

docker container prune -f --filter "until=24h"
docker image prune -a -f --filter "until=24h"

[-]

lightmatter501@reddit

Granting docker group perms = granting root perms. If they already have root I’m guessing they wouldn’t need help rebooting.

I agree on no blind reboots, just write a script which force stops all running containers and prunes the environment and use sudo to let them run it.

[-]

whetu@reddit

Granting docker group perms = granting root perms. If they already have root I’m guessing they wouldn’t need help rebooting.

Yeah, unfortunately docker's security position is fucking atrocious. I recently invested too much time ansibling it to be selinux-on, rootless, and subject to firewalld without its bullshit iptables bypass. I shudder to think how much worse it would be with ufw and apparmor under Ubuntu. Really, I would much rather have our dev team grow up and switch to podman.

just write a script which force stops all running containers

One-liner :)

docker stop $(docker ps -q)

[-]

IridescentKoala@reddit

Why do they need to reboot? Just launch another one.

[-]

michaelpaoli@reddit

sudo, ssh with forced command, many possibilities. Can also build into it whatever logging or controls you want, e.g. make them supply a reason, require some additional person to approve it ... whatever. But yeah, if it's that regular, may make lots of sense to automate it.

[-]