[OpenAFS] Testing the Linux built in Kernel kafs client
Jonathan Billings
jsbillin@umich.edu
Wed, 18 Jul 2018 14:42:22 -0400
--00000000000060543c05714a6c8f
Content-Type: text/plain; charset="UTF-8"
Hello,
I've been building a test environment to run the AFS client that's part of
the Linux kernel. This has been mostly my side project that hasn't got a
lot of testing, but I feel like it'd be great if others could give it a
test. I've been able to run a full Fedora desktop with my $HOME in /afs,
although I have run into some problems that could use some extra testing
(I'm not certain if it's my cell or a bug in the client). Please disregard
this email if you're not interested in the in-kernel Linux client.
Thanks go to David Howells for doing all the work on the linux kernel and
userspace code! I'm just a sysadmin, he's been very patient with my
efforts to package up this software.
I want to start by saying that this post is in no way a slight against the
OpenAFS developers, particularly those working hard on getting the OpenAFS
Client working on Linux systems.
As of the 4.18 prereleases, all the code needed to run a client is in the
upstream kernel. I'm a Fedora Linux user, so I built packages with the
kAFS features for Fedora 28. I just took the rawhide kernel, enabled the
following features and built the package for f28:
CONFIG_AF_RXRPC_IPV6=y
CONFIG_AF_RXRPC_DEBUG=y
CONFIG_AF_RXRPC=m
CONFIG_AFS_DEBUG=y
CONFIG_AFS_FSCACHE=y
CONFIG_AFS_FS=m
Along with the kernel, I built packages for kafs-utils[1] (so you can run
'afs vos ...' commands) and kafs-client, which sets up the /afs mountpoint
and a service to get you tokens for systemd. I've also included a
kafs-aklog package that works with the in-kernel RXRPC module.[2] The COPR
repo also includes an updated keyutils package which supports AFSDB
records.[3]
On a fedora 28 system, you can get these packages by running 'dnf copr
enable jsbillings/kafs' and then 'dnf upgrade' to get the new kernel and
keyutils. You can install kafs-aklog, kafs-client, and kafs-utils from
there too.
You'll need to have Kerberos set up for your realm. I use pam_krb5 in my
tests since it's super simple. Since there's no working pam module for
getting AFS tokens with kafs, I have a really hackish PAM solution that
just has this at the end of /etc/pam.d/password-auth:
session optional pam_exec.so seteuid quiet /usr/bin/aklog umich.edu
(obviously, replace umich.edu with your cell)
So far, I'm impressed with the client. Once I built a kernel with all the
correct features, it is quite fast. (note: don't turn on
CONFIG_AF_RXRPC_INJECT_LOSS. :)
One of the neat things is the way the mounts appear:
$ df -t afs
Filesystem 1K-blocks Used Available Use%
Mounted on
none 1 1 0 100%
/afs
#umich.edu:root.cell 5000 17 4983 1%
/afs/umich.edu
#umich.edu:user 1000 38 962 4%
/afs/umich.edu/user
#umich.edu:user.j 20000 13496 6504 68%
/afs/umich.edu/user/j
%umich.edu:user.jsbillin 10485760 8231929 2253831 79%
/afs/umich.edu/user/j/s/jsbillin
You can see my home directory has every separate volume in the path
mounted. They all show up with the correct percentage used in df. Also,
you can directly mount volumes in paths outside of /afs. I've got some of
our software volumes mounted directly in /usr/caen so I can test some of
our engineering software, and it just is like any other kind of network
mount.
AFS tokens are stored in the keyring, which you can see with this:
$ keyctl show @s
Keyring
855773135 --alswrv 263726 1000 keyring: _ses
393094001 --alswrv 263726 65534 \_ keyring: _uid.263726
320980772 --als-rv 0 0 \_ rxrpc: afs@umich.edu
The kafs-utils are still a work in progress, but I love that it's all
python3 code. There's no 'fs' commands yet, but I can see this as a
potential client for a non-administrative AFS client. There's more
documentation the kafs utils page about the progress with those commands.
It's not quite ready for prime time, but I am hoping that once we've worked
out all the kinks, we can start convincing the distro makers to enable the
kafs module in default kernels, bringing AFS clients to major distros
out-of-the-box. Keep an eye on my fedora repo[4] for any new packages I
might build. Please let me know if anything looks wrong (not an OpenAFS
developer) or if I've made any mistakes in the above information.
1. https://www.infradead.org/~dhowells/kafs/kafs_utils.html
2. https://wiki.openafs.org/devel/LinuxKAFSNotes/
3. http://people.redhat.com/~dhowells/keyutils/
4. https://copr.fedorainfracloud.org/coprs/jsbillings/kafs/
--
Jonathan Billings <jsbillin@umich.edu>
--00000000000060543c05714a6c8f
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div>Hello,</div><div><br></div><div>I've been buildin=
g a test environment to run the AFS client that's part of the Linux ker=
nel.=C2=A0 This has been mostly my side project that hasn't got a lot o=
f testing, but I feel like it'd be great if others could give it a test=
.=C2=A0 I've been able to run a full Fedora desktop with my $HOME in /a=
fs, although I have run into some problems that could use some extra testin=
g (I'm not certain if it's my cell or a bug in the client).=C2=A0 P=
lease disregard this email if you're not interested in the in-kernel Li=
nux client.<br></div><div><br></div><div>Thanks go to David Howells for doi=
ng all the work on the linux kernel and userspace code!=C2=A0 I'm just =
a sysadmin, he's been very patient with my efforts to package up this s=
oftware. <br></div><div><br></div><div>I want to start by saying that this =
post is in no way a slight against the OpenAFS developers, particularly tho=
se working hard on getting the OpenAFS Client working on Linux systems.=C2=
=A0 <br></div><div><br></div><div>As of the 4.18 prereleases, all the code =
needed to run a client is in the upstream kernel.=C2=A0 I'm a Fedora Li=
nux user, so I built packages with the kAFS features for Fedora 28.=C2=A0 I=
just took the rawhide kernel, enabled the following features and built the=
package for f28:</div><div><pre><code>CONFIG_AF_RXRPC_IPV6=3Dy
CONFIG_AF_RXRPC_DEBUG=3Dy
CONFIG_AF_RXRPC=3Dm
CONFIG_AFS_DEBUG=3Dy
CONFIG_AFS_FSCACHE=3Dy
CONFIG_AFS_FS=3Dm</code></pre></div><div>Along with the kernel, I built pac=
kages for kafs-utils[1] (so you can run 'afs vos ...' commands) and=
kafs-client, which sets up the /afs mountpoint and a service to get you to=
kens for systemd.=C2=A0 I've also included a kafs-aklog package that wo=
rks with the in-kernel RXRPC module.[2]=C2=A0 The COPR repo also includes a=
n updated keyutils package which supports AFSDB records.[3]<br></div><div><=
br></div><div>On a fedora 28 system, you can get these packages by running =
'dnf copr enable jsbillings/kafs' and then 'dnf upgrade' to=
get the new kernel and keyutils.=C2=A0 You can install kafs-aklog, kafs-cl=
ient, and kafs-utils from there too.</div><div><br></div><div>You'll ne=
ed to have Kerberos set up for your realm.=C2=A0 I use pam_krb5 in my tests=
since it's super simple.=C2=A0 Since there's no working pam module=
for getting AFS tokens with kafs, I have a really hackish PAM solution tha=
t just has this at the end of /etc/pam.d/password-auth:</div><div><br></div=
><div>session=C2=A0=C2=A0=C2=A0=C2=A0 optional=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 pam_exec.so seteuid quiet /usr/bin/aklog <a href=3D"http://umich.edu">u=
mich.edu</a><br></div><div><br></div><div>(obviously, replace <a href=3D"ht=
tp://umich.edu">umich.edu</a> with your cell)</div><div><br></div><div>So f=
ar, I'm impressed with the client.=C2=A0 Once I built a kernel with all=
the correct features, it is quite fast.=C2=A0 (note: don't turn on CON=
FIG_AF_RXRPC_INJECT_LOSS.=C2=A0 :)<br></div><div><br></div><div>One of the =
neat things is the way the mounts appear:</div><div><br></div><div><font si=
ze=3D"1"><span style=3D"font-family:monospace,monospace">$ df -t afs<br>Fil=
esystem=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1K-blocks=C2=A0=C2=A0=C2=A0 Used Availabl=
e Use% Mounted on<br>none=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 1=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =
0 100% /afs<br>#umich.edu:root.cell=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 5000=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 17=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0 4983=C2=A0=C2=A0 1% /afs/<a href=3D"http://umich.e=
du">umich.edu</a><br>#umich.edu:user=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1000=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 38=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 962=C2=A0=C2=
=A0 4% /afs/<a href=3D"http://umich.edu/user">umich.edu/user</a><br>#umich.=
edu:user.j=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0 20000=C2=A0=C2=A0 13496=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 65=
04=C2=A0 68% /afs/<a href=3D"http://umich.edu/user/j">umich.edu/user/j</a><=
br>%umich.edu:user.jsbillin=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 10485760 8231929=C2=A0=C2=
=A0 2253831=C2=A0 79% /afs/<a href=3D"http://umich.edu/user/j/s/jsbillin">u=
mich.edu/user/j/s/jsbillin</a><br></span></font></div><div><br></div><div>Y=
ou can see my home directory has every separate volume in the path mounted.=
=C2=A0 They all show up with the correct percentage used in df.=C2=A0 Also,=
you can directly mount volumes in paths outside of /afs.=C2=A0 I've go=
t some of our software volumes mounted directly in /usr/caen so I can test =
some of our engineering software, and it just is like any other kind of net=
work mount.<br></div><div><br></div><div>AFS tokens are stored in the keyri=
ng, which you can see with this:</div><div><br></div><div><font size=3D"1">=
<span style=3D"font-family:monospace,monospace">$ keyctl show @s<br>Keyring=
<br>=C2=A0855773135 --alswrv=C2=A0 263726=C2=A0 1000=C2=A0 keyring: _ses<br=
>=C2=A0393094001 --alswrv=C2=A0 263726 65534=C2=A0=C2=A0 \_ keyring: _uid.2=
63726<br>=C2=A0320980772 --als-rv=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=
=A0=C2=A0=C2=A0 0=C2=A0=C2=A0 \_ rxrpc: <a href=3D"mailto:afs@umich.edu">af=
s@umich.edu</a></span></font><br></div><div><br></div><div>The kafs-utils a=
re still a work in progress, but I love that it's all python3 code.=C2=
=A0 There's no 'fs' commands yet, but I can see this as a poten=
tial client for a non-administrative AFS client.=C2=A0 There's more doc=
umentation the kafs utils page about the progress with those commands.</div=
><div><br></div><div>It's not quite ready for prime time, but I am hopi=
ng that once we've worked out all the kinks, we can start convincing th=
e distro makers to enable the kafs module in default kernels, bringing AFS =
clients to major distros out-of-the-box.=C2=A0 Keep an eye on my fedora rep=
o[4] for any new packages I might build. Please let me know if anything loo=
ks wrong (not an OpenAFS developer) or if I've made any mistakes in the=
above information.<br></div><div><br></div><div><br></div><div>1. <a href=
=3D"https://www.infradead.org/~dhowells/kafs/kafs_utils.html">https://www.i=
nfradead.org/~dhowells/kafs/kafs_utils.html</a></div><div>2. <a href=3D"htt=
ps://wiki.openafs.org/devel/LinuxKAFSNotes/">https://wiki.openafs.org/devel=
/LinuxKAFSNotes/</a><br></div><div>3. <a href=3D"http://people.redhat.com/~=
dhowells/keyutils/">http://people.redhat.com/~dhowells/keyutils/</a></div><=
div>4. <a href=3D"https://copr.fedorainfracloud.org/coprs/jsbillings/kafs/"=
>https://copr.fedorainfracloud.org/coprs/jsbillings/kafs/</a><br></div><div=
><br>-- <br><div class=3D"gmail_signature">Jonathan Billings <<a href=3D=
"mailto:jsbillin@umich.edu" target=3D"_blank">jsbillin@umich.edu</a>><br=
><br></div>
</div></div>
--00000000000060543c05714a6c8f--