[OpenAFS] containers / AFS / Ubuntu - stopped working

[Charles (Chas) Williams]

I will submit something later today for it and hopefully we can get it
approved.

While mounting /proc and /sys as read only does provide more isolation,
it obviously doesn't provide complete isolation.  I don't think anyone
would be surprised by that though.  However, OpenAFS should really be
made aware of containers with respect to tokens.  Tokens really
shouldn't be shared across containers if that is your isolation
boundary.  I know it makes sense in certain use cases though.
                          
On Mon, 2015-11-30 at 17:21 +0000, Neil Davies wrote:
> Chas
> 
> Didn’t get the time to get around to it yesterday, but did today. 
> 
> I can confirm that changing the appropriate line in src/sys/glue.c to open /proc/fs/openafs/afs_ioctl RDONLY works.
> 
> I used the current ubuntu sources (1.6.7 + PATCHES), built a new version with the patch and installed it into a container which, prior to the install, failed with the "aklog: a pioctl failed while obtaining tokens for cell ….” and post the install I was able to get tokens and access the AFS.
> 
> Could I ask that the patch gets pushed into the next release? (eventually it will make it my ubuntu/containers) - in the mean time I’ll change my container build approach to make the appropriate patch during the build process….
> 
> Thanks for the steer - it was precisely what was needed.
> 
> Cheers
> 
> Neil
> 
> On 29 Nov 2015, at 10:20, Neil Davies <semanticphilosopher@gmail.com> wrote:
> 
> > Chas
> > 
> > This sounds like a plan!
> > 
> > I've got a few things to do first thing today, but I'll try and get round to putting up an appropriate test system and trying this later today.
> > 
> > Neil
> > 
> > On 28 Nov 2015, at 22:44, Charles (Chas) Williams <3chas3@gmail.com> wrote:
> > 
> >> Strangely, I don't see a reason for this file to opened read/write by
> >> the OpenAFS utilities.  We only use ioctl() and I believe that only
> >> needs O_RDONLY.  Change src/sys/glue.c to be O_RDONLY instead of O_RDWR
> >> when it opens PROC_SYSCALL_FNAME.
> >> 
> >> I don't happen to have a test system right now, or I would check it
> >> myself.
> >> 
> >> On Sat, 2015-11-28 at 21:19 +0000, Neil Davies wrote:
> >>> I can confirm that this sis the problem
> >>> 
> >>> There was a change in docker 1.2.1 (a CVE related fix) that now forces /proc/fs to be mounted read-only
> >>> 
> >>> use of the --privileged  argument to docker run does allow openafs to run ok, but only at the cost of loosing
> >>> all of the container isolation!
> >>> 
> >>> I spent some time trying to work out how to _just_ permit read-write access to the appropriate portion of 
> >>> the /proc/fs filestore, but not cracked it. 
> >>> 
> >>> It is potentially possible to mount the host's /proc/fs/openafs under a different name (with read-write access)
> >>> within the container - but that would imply a change to the openafs building process....
> >>> 
> >>> Obviously I could modify the docker sources, submit a patch etc.. 
> >>> 
> >>> Any suggestions? I'm just wondering if there is any other bits of functionality that the docker folks might have 
> >>> broken this way - looking to see if there we, as a community, are not alone here.
> >>> 
> >>> Neil
> >>> 
> >>> On 27 Nov 2015, at 19:06, Charles (Chas) Williams <3chas3@gmail.com> wrote:
> >>> 
> >>>> On Nov 27, 2015, at 13:42 , Neil Davies wrote:
> >>>>> After this upgrade I am no longer able, in the container, able to push tokens into the kernel - it gives a pioctl.
> >>>> 
> >>>> Is there any chance you can run an strace on this?
> >>>> 
> >>>> I believe that /proc was changed to read-only at some point for docker
> >>>> containers.  OpenAFS tries to open /proc/fs/openafs/afs_ioctl read/write
> >>>> in order to handle pioctl's.
> >>>> 
> >>>> 
> >>> 
> >> 
> >> 
> > 
>