NFSv4 - Admin permission issues
Posted by OneInchPunchMan@reddit | linuxadmin | View on Reddit | 22 comments
Hey r/linuxadmin , I have a weird one.
I have a NAS and a Server where the NAS serves /mnt/storage via NFSv4 to the Server.
There is also a user gitea:gitea (5203:5203) on both the NAS and Server admin is part of the gitea group.
The dir structure is:
/mnt/storage/ (775 admin:admin)
/mnt/storage/a.txt (775 gitea:gitea)
/mnt/storage/gitea/ (775 gitea:gitea + setgid)
My problem is that both admins can rw the a.txt file fine (appear to be in group gitea), however they cannot make now files in gitea/ dir (appear to be in "others").
How and why is that and am I missing some key concept here?
OneInchPunchMan@reddit (OP)
I seem to have nailed it down as per: GitHub Issue and Documentation
I was banging my head for far too long on this. I guess LDAP it is then...
trapexit@reddit
Or manually sync perms as you need or use the host's users/groups list within the container.
OneInchPunchMan@reddit (OP)
Is there a better way of doing this somehow other than manually going on both machines and adding the groups and users. Also from what I understand now, there is the issue that mergerfs (not it's fault) won't pick up the group changes since it is a "expensive operation"? Is LDAP (OpenLDAP) my only option because it seems a bit daunting to set it up :/
trapexit@reddit
Copying files is the simplest way. You could use any number of config management platforms (like ansible) to do it for you but if you don't expect to change perms much manual is easier.
Without you saying what version of mergerfs you are using I can't comment. Newer versions can have their gid cache purged at runtime and naturally a restart would do the same. Future versions remove the concept all together.
kai_ekael@reddit
Checking, do the UIDs and GIDs match on the NAS and Server? If so, how?
Next, what is exact error message when an admin user tries to create a file in /mnt/storage/gitea. Note the timestamp, get the error from the command and check local syslog and NAS log as well at the same timestamp.
OneInchPunchMan@reddit (OP)
They do match. "admin" user and group uid/gid 1000, "gitea" user and group uid/gid 5203. Both present and same on NAS and Server.
The error is: touch: cannot touch 'c.txt': Permission denied The error is not present in journalctl tho. Ill check the timestamps in the morning
kai_ekael@reddit
What on the NAS manages accounts and memberships, ie how does the NAS know admin is in gitea group?
Test an admin user (or make a new one) by making its primary group gitea and trying the same command. Example to be clear:
Note the gid versus groups, gid is the primary group.
OneInchPunchMan@reddit (OP)
I added a new user "someuser" and the output is the same as yours. I then set the primary group of said user to "gitea" and not it can write in gitea/. I also checked it with the "admin" user, setting it's primary group to "gitea", also works. (I rebooted the machine just to make sure the changes are applied everytime). It sort of appears as if touch does not recognize supplementary groups.. What still is weird to me is that "admin" with supp group "gitea" can edit a.txt (644, gitea:gitea). Like nano sees the groups but touch doesn't.
kai_ekael@reddit
You're putting it on the wrong side. The NAS is responsible for security, not the server. The NAS needs a method to know supplementary group membership.
OneInchPunchMan@reddit (OP)
Here is the explanation about it. It appears it is a problem of mergerfs being passed to the VM. Thanks though
trapexit@reddit
The docs you link to are about containers and cross OS identity management (which is what the same thing). This isn't a particular problem with mergerfs but with any identity concerns across systems. I mention it in docs because it is extremely common for people who have no experience with cross system identity management or with containers to miss this fact.
OneInchPunchMan@reddit (OP)
I kind of understand the premise now and I'll dig a it deeper into mergerfs to try and understand it under the hood. I quite like mergerfs so I'm trying to find the least painful option to solve this cross system id sync
dodexahedron@reddit
Good thoights.
But might be a red herring. Since it is NFS4, it should be using names anyway and mapping them on the server side, unless OP forced use of IDs.
But your thoughts gave me an idea that would address your concern by making it guaranteed to be bame-based and would also add some security to it:
OP could instead just not use auth_sys for NFS and instead use kerberos. If the server side supports it, that is. NFS4 itself supports kerberos implicitly. But, if the configuration for what OP is using doesn't expose a way to use kerberos auth, that would be a show-stopper.
kai_ekael@reddit
Red Herring: depends on the NAS implementation and setup. Worth testing to prove so the problem is known.
No mention of using NFSv4 ACLs at all. Far too many grey spots in the NFSv4 RFCs as well.
Kerberos can be a real dream or a giant mess, just depends on what does which. Any Microsoft involved, run away.
dodexahedron@reddit
Agreed.
And yeah, MS + NFS (at least until VERY recently, and even then still to an extent) can only lead to sorrow.
kai_ekael@reddit
Don't forget (ever!) MS bastardization of Kerberos.
OneInchPunchMan@reddit (OP)
God please not kerberos and ACLs hahaha. This is just my "simple" homelab
ihatepowershell@reddit
If the permissions on the /mnt/storage/gitea/ directory are as above(775 gitea:gitea+setgid) then the admins would not have permissions to write(create new files) in that directory.
You could do one of the following: A) add the members of the admin group to the gitea group B) relax the permissions for other(may be a security issue) C) Add an ACL to the directory(NFSv4 ACLs are a little tricky) D) Have the directory owned by either admin:gitea or gitea:admin
I think option D is simplest depending on your environment.
You may want to check the group membership, e.g. getent group admin getent group gitea
The issue that seems unexplained is why the admin user can write to a.txt with those permissions.
OneInchPunchMan@reddit (OP)
Admin is part of the gitea group and that's the confusing part. Like if admin interacts with a.txt it is a part of the group gitea, but when entering gitea/ it is part of the "others". Seems really dubious. Might go with D) since A) is obviously not working :/
dodexahedron@reddit
Did you check the journal for selinux/apparmor/[whatever MAC you use] denied actions when you try to create the files?
wezelboy@reddit
The admin group doesn't mean that its automatically root. You need to use sudo.
OneInchPunchMan@reddit (OP)
Could this have something to do with virtiofs, where the Host is Passing the directory to the NAS VM?