[OpenAFS] containers / AFS / Ubuntu - stopped working

Neil Davies SemanticPhilosopher@gmail.com
Thu, 3 Dec 2015 09:24:27 +0000


Thanks Chas

On the isolation policy I agree - different threat scenarios have =
different solutions. Our current use of AFS In this sense is really a =
access isolation issue (got access to one container doesn=92t get you =
access to other AFS tokens) and cost reduction (running multiple =
containers in a cloud VM).

My beef is that people =93selling=94 containers are not articulating the =
hazards appropriately - causes strife in large organisations (with which =
we work) where the upper echelons are sold a vision (usually associated =
with cost reduction) and then are shocked when a negative risk analysis =
occurs. Its all about human dynamics more than anything else.

Neil

On 2 Dec 2015, at 15:32, Charles (Chas) Williams <3chas3@gmail.com> =
wrote:

> I will submit something later today for it and hopefully we can get it
> approved.
>=20
> While mounting /proc and /sys as read only does provide more =
isolation,
> it obviously doesn't provide complete isolation.  I don't think anyone
> would be surprised by that though.  However, OpenAFS should really be
> made aware of containers with respect to tokens.  Tokens really
> shouldn't be shared across containers if that is your isolation
> boundary.  I know it makes sense in certain use cases though.
>=20
> On Mon, 2015-11-30 at 17:21 +0000, Neil Davies wrote:
>> Chas
>>=20
>> Didn=92t get the time to get around to it yesterday, but did today.=20=

>>=20
>> I can confirm that changing the appropriate line in src/sys/glue.c to =
open /proc/fs/openafs/afs_ioctl RDONLY works.
>>=20
>> I used the current ubuntu sources (1.6.7 + PATCHES), built a new =
version with the patch and installed it into a container which, prior to =
the install, failed with the "aklog: a pioctl failed while obtaining =
tokens for cell =85.=94 and post the install I was able to get tokens =
and access the AFS.
>>=20
>> Could I ask that the patch gets pushed into the next release? =
(eventually it will make it my ubuntu/containers) - in the mean time =
I=92ll change my container build approach to make the appropriate patch =
during the build process=85.
>>=20
>> Thanks for the steer - it was precisely what was needed.
>>=20
>> Cheers
>>=20
>> Neil
>>=20
>> On 29 Nov 2015, at 10:20, Neil Davies <semanticphilosopher@gmail.com> =
wrote:
>>=20
>>> Chas
>>>=20
>>> This sounds like a plan!
>>>=20
>>> I've got a few things to do first thing today, but I'll try and get =
round to putting up an appropriate test system and trying this later =
today.
>>>=20
>>> Neil
>>>=20
>>> On 28 Nov 2015, at 22:44, Charles (Chas) Williams <3chas3@gmail.com> =
wrote:
>>>=20
>>>> Strangely, I don't see a reason for this file to opened read/write =
by
>>>> the OpenAFS utilities.  We only use ioctl() and I believe that only
>>>> needs O_RDONLY.  Change src/sys/glue.c to be O_RDONLY instead of =
O_RDWR
>>>> when it opens PROC_SYSCALL_FNAME.
>>>>=20
>>>> I don't happen to have a test system right now, or I would check it
>>>> myself.
>>>>=20
>>>> On Sat, 2015-11-28 at 21:19 +0000, Neil Davies wrote:
>>>>> I can confirm that this sis the problem
>>>>>=20
>>>>> There was a change in docker 1.2.1 (a CVE related fix) that now =
forces /proc/fs to be mounted read-only
>>>>>=20
>>>>> use of the --privileged  argument to docker run does allow openafs =
to run ok, but only at the cost of loosing
>>>>> all of the container isolation!
>>>>>=20
>>>>> I spent some time trying to work out how to _just_ permit =
read-write access to the appropriate portion of=20
>>>>> the /proc/fs filestore, but not cracked it.=20
>>>>>=20
>>>>> It is potentially possible to mount the host's /proc/fs/openafs =
under a different name (with read-write access)
>>>>> within the container - but that would imply a change to the =
openafs building process....
>>>>>=20
>>>>> Obviously I could modify the docker sources, submit a patch etc..=20=

>>>>>=20
>>>>> Any suggestions? I'm just wondering if there is any other bits of =
functionality that the docker folks might have=20
>>>>> broken this way - looking to see if there we, as a community, are =
not alone here.
>>>>>=20
>>>>> Neil
>>>>>=20
>>>>> On 27 Nov 2015, at 19:06, Charles (Chas) Williams =
<3chas3@gmail.com> wrote:
>>>>>=20
>>>>>> On Nov 27, 2015, at 13:42 , Neil Davies wrote:
>>>>>>> After this upgrade I am no longer able, in the container, able =
to push tokens into the kernel - it gives a pioctl.
>>>>>>=20
>>>>>> Is there any chance you can run an strace on this?
>>>>>>=20
>>>>>> I believe that /proc was changed to read-only at some point for =
docker
>>>>>> containers.  OpenAFS tries to open /proc/fs/openafs/afs_ioctl =
read/write
>>>>>> in order to handle pioctl's.
>>>>>>=20
>>>>>>=20
>>>>>=20
>>>>=20
>>>>=20
>>>=20
>>=20
>=20
>=20