[OpenAFS] best practice for a service to access a user AFS token?
and why ruid instead of euid?
Fri, 18 Nov 2016 00:59:23 -0500
This is a cryptographically signed message in MIME format.
This is a multi-part message in MIME format.
Content-Type: text/plain; charset=utf-8
Thanks for your note.
On 11/17/2016 2:27 PM, Todd Tannenbaum wrote:
> Hi OpenAFS gurus, I am in desperate need of your advice!
> We are adding OpenAFS support to HTCondor (http://htcondor.org).
> I read in the docs that the Cache Manager identifies token by either th=
> user's UNIX UID or by a process authentication group (PAG). In the
> non-PAG case, I expected the cache manager to identify the token by
> _effective_ uid. But my testing implies that the token is identified b=
> the _real_ uid. Is there a way to change this behavior?
Your description is close but not quite correct. Tokens are always
tracked by a Process Authentication Group. The distinction is whether
the PAG identifier is a unique identifier independent of the user id or
is the user id. It is an explicit security property of a PAG that one
user (including root) cannot switch a process into a PAG. To change
this behavior would break the security guarantees that PAGs provide.
> My situation is hopefully not new or unique : I have a long-running
> service (HTCondor in this case) running as root that is implemented via=
> a cooperating set of processes, and this service needs to impersonate
> different users in order read/write to the filesystem on their behalf.
The behavior of HTCondor is similar to that of IBM's Platform Load
Sharing Facility (LSF) which distributes jobs across an HPC environment
where the jobs have the need to authenticate to and access resources
over a network. LSF supports the ability to forward Kerberos v5 ticket
granting tickets with the jobs, renew them on the individual nodes. LSF
has the ability to create a PAG but does not have any specific knowledge
of how to obtain afs (or AuriStorFS) tokens. Those tokens are obtained
by executing a shell script that can be configured with the job.
When jobs will require weeks or months to complete it is critical from a
security perspective to distribute Kerberos tickets that have a
short-lifetime but can be renewed for the length of the job provided
that the represented principal has not been disabled.
> Ideally I'd like this service to be as ignorant of AFS is possible. It=
> currently performs impersonation by changing out the effective UID,
> which of course works fine for the local filesystem.
> Meanwhile, I have another service running (a home-brewed "credential
> manager") on the machine that keeps AFS tokens refreshed for all users
> of the service by doing an "aklog" as each user. Both HTCondor and the=
> credential manager are not running in a PAG, as my understanding is a
> PAG can only hold one user token per cell.
It is correct that a cache manager can only associate one token (or
identity) per cell in each PAG. If a PAG stored more than one identity
per cell there would be an identity selection problem.
There is nothing that prevents a user-land application from storing more
than one token per cell and setting the desired token into the cache
manager prior to the file system requests that must use it are issued.
> The problem is the HTCondor service cannot access AFS on behalf of the
> user being impersonated by simply switching effective UID; it apparentl=
> needs to switch over the real UID as well. Thanks to the saved UID and=
> the setresuid() syscall, it is trivial to change HTCondor to switch bot=
> the read and effective UID over to the user being impersonated, and
> still switch back to root afterwards. Doing so would likely make
> everything work great with AFS. But the problem is changing the
> real-uid is a potential security hole, as now HTCondor service processe=
> could receive signals (i.e. SIGKILL!) from the unprivileged users it is=
> simply trying to impersonate. This is why OpenAFS's reliance on using=
> the real UID for identifying tokens seems very broken to me; seems like=
> it should be euid (or specifically on Linux clients, the file-system
> uid) like every other similar system...
The security hole that you want to avoid introducing into HTCondor is
exactly the security hole that would be introduced into every
application if PAGs followed the effective uid.
> Any thoughts/advice?
> One idea is I could have all the HTCondor service processes run in thei=
> own PAG and perform some sort of very lightweight "aklog" whenever
> impersonation is required. But I need this "aklog" to be very
> fast/lightweight; I don't want to be blocked on network communication t=
> some KDC or even file I/O if I can help it. The service would have
> access to a KRB5 credential cache for the user that has an AFS service
> principal; given that, is there some lightweight "aklog"-like in-proces=
> library call I can use (that avoids I/O)? Something the HTCondor
> service could dlopen()/dlsym() (so I don't introduce a pile of
> dependencies) ?
There are four steps that need to be performed:
1. obtain and/or renew Kerberos v5 tickets
2. obtain the afs (or AuriStorFS token)
3. create the PAG for the process
4. set the token for the PAG
Step 3 needs to be performed once per process and does not involve the
Step 4 needs to be performed each time the token is replaced due to
renewal. It does not require network.
Steps 1 and 2 do require network but can be performed in the background
after the initial acquisition.
> Another idea, as I only care about supporting Linux, would be to
> leverage Linux kernel keyrings. I am thinking perhaps my credential
> manager could link the "afs_pag: _pag" key to the user keyring, and the=
> the HTCondor service could link this key into its session keyring when
> impersonating. Does anyone think that would work? Or is there more to=
> swapping PAGs in and out (i.e. besides the key on the keyring, pagsh
> seems to do some magic with groups as "/bin/id" shows some magic groups=
> when in a PAG...) ? Is the keyring-based idea crazy talk or a good ide=
> to pursue if Linux is my only target?
The groups are not required on Linux. The cache manager uses keyrings
internally to track the PAG.
The behavior you are seeking with user-land manipulation of keyrings to
store afs tokens is the one that David Howell's kafs client will provide
when it is complete.
> I've seen the great lengths (i.e. immense amount of code, security
> side-steps of creating their own krb4 tickets) that Samba has done to
> support AFS; I am hoping there is an easier way.
Samba had a different problem than HTCondor. An SMB 1.1 server used to
authenticate a client by the client sending a username and password over
the wire to the server which could then be used with kinit (or
equivalent library calls) to obtain a Kerberos TGT. In the 90s this was
more often than not Kerberos v4.
In SMB 1.2 NTLM authentication was introduced and later GSS-Kerberos v5
authentication. Both avoided the transmission of the user's password.
Therefore, Samba had to obtain a Kerberos v5 ticket granting ticket or
an AFS token some other way. This typically made use of an
impersonation service to acquire a TGT or Token after asserting it had
authenticated the user identity. Many single sign-on web authentication
services utilize a similar model.
> Your suggestions greatly appreciated.
The best approach in my opinion is to follow the LSF model.
Content-Type: text/x-vcard; charset=utf-8;
adr:Suite 6B;;255 West 94Th Street;New York;New York;10025-6985;United St=
title:Founder and CEO
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature