[OpenAFS] Re: Package Management in AFS
Phillip Moore
w.phillip.moore@gmail.com
Tue, 21 Dec 2010 13:37:05 -0500
--0016362842ea5e3b740497efec8b
Content-Type: text/plain; charset=ISO-8859-1
As the architect of OpenEFS, I guess I should pipe up and say something...
First of all, there's no single solution to this problem, and trying to
install *everything* in AFS is possible, but extremeley challenging for some
products. This is in fact what we accomplished in the 1990s at Morgan
Stanley in the AFS-based distributed environment I helped design and
engineer. We went to the ultimate extreme, and even managed the operating
system out of AFS by implementing dataless AFS clients. This worked, but
proved to be very challenging to maintain.
At the other end of the spectrum, trying to maintain everything local to
every node scales far worse, and results in environments with a very large
degree of otherwise unnecessary heterogeniety. Now, to put this context,
I'm refering to Enterprise environments with 10000+ client nodes distributed
in multiple data centers on multiple continents. At medium to small scale
(and we would have to debate just what those definitions mean) local
installations can be managed fairly well. Classic case of YMMV.
My goal has been to design an infrastructure layer that makes both options
feasible at large to extreme scale. EFS evolved from the Morgan Stanley VMS
system, and the immediate predecessor was implemented using NFSv3 as the
backend filesystem, and we focused on open source software and 3rd party
vendor products as our content deliverables. We have NOT attempted to
integrate the OS with EFS, as I personally think the return on investment is
relatively small, and it requires a very significant amount of custom
engineering to accomplish.
The OpenEFS implementation currently only supports NFSv3 out of the box, but
I have done most of the work necessary to use AFS. However, that work is
currently on hold, pending, well.. someone else being interested in it.
My current employers are very unlikely to invest in an AFS infrastructure,
and hopefully I'll find out for sure in early 2011 (I am not very
optimistic).
We have done a lot of work to automate integrating open source products with
OpenEFS, and anything that is built using the GNU autoconf suite or
CPAN-style perl packages (Makefile.PL, Build.PL) can be integrated fairly
easily. We have a lot of pre-compiled content that be downloaded and used
immediately, for example, and we would love to build an open source
community around the product.
EFS is fundamentally designed to support software deployment and change
control in a SINGLE authentication domain, and the goal for OpenAFS is to
support distribution to a collection of AFS cells that are all part of the
same kerberos realm. This is based on the basic model we used to deploy
AFS at Morgan Stanley, where we had a single Kerberos realm but 50+ AFS
cells in data centers around the world.
The main challenge to using EFS is that it supports a very different
deployment paradigm, since it completely decouples software deployment from
the clients, but this is precisely why the predecessors to OpenEFS have
proven to be successful.
If anyone is interested in the product, let us know....
On Mon, Dec 20, 2010 at 1:37 PM, Andrew Deason <adeason@sinenomine.net>wrote:
> On Mon, 20 Dec 2010 18:46:32 +0100
> Dirk Heinrichs <dirk.heinrichs@altum.de> wrote:
>
> > I'm currently thinking about a good way to deploy software packages in
> > (eventually replicated) AFS volumes. One possible way I can think of
> > is to use (x)stow, but that would imply a lot of manual work
> > (download, unpack, compile, install to rw volume, xstow, vos release).
> >
> > Does anyone know of a simpler (more automated) solution, maybe
> > something like Gentoo portage or Nix?
>
> You may be interested in OpenEFS <http://www.openefs.org/>, where I
> believe AFS support is being worked on. While you can perhaps get
> something to work that just combines something stow-like with
> pkgsrc/portage/etc... depending on how 'production' this setup is,
> you're going to encounter some problems sooner or later with
> dependencies and release engineering, etc, that systems like EFS or VMS
> try to make easier.
>
> --
> Andrew Deason
> adeason@sinenomine.net
>
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info
>
--0016362842ea5e3b740497efec8b
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
<br>As the architect of OpenEFS, I guess I should pipe up and say something=
...<div><br></div><div>First of all, there's no single solution to this=
problem, and trying to install *everything* in AFS is possible, but extrem=
eley challenging for some products. =A0This is in fact what we accomplished=
in the 1990s at Morgan Stanley in the AFS-based distributed environment I =
helped design and engineer. =A0We went to the ultimate extreme, and even ma=
naged the operating system out of AFS by implementing dataless AFS clients.=
=A0This worked, but proved to be very challenging to maintain.</div>
<div><br></div><div>At the other end of the spectrum, trying to maintain ev=
erything local to every node scales far worse, and results in environments =
with a very large degree of otherwise unnecessary heterogeniety. =A0Now, to=
put this context, I'm refering to Enterprise environments with 10000+ =
client nodes distributed in multiple data centers on multiple continents. =
=A0 At medium to small scale (and we would have to debate just what those d=
efinitions mean) local installations can be managed fairly well. =A0 Classi=
c case of YMMV.</div>
<div><br></div><div>My goal has been to design an infrastructure layer that=
makes both options feasible at large to extreme scale. =A0EFS evolved from=
the Morgan Stanley VMS system, and the immediate predecessor was implement=
ed using NFSv3 as the backend filesystem, and we focused on open source sof=
tware and 3rd party vendor products as our content deliverables. =A0We have=
NOT attempted to integrate the OS with EFS, as I personally think the retu=
rn on investment is relatively small, and it requires a very significant am=
ount of custom engineering to accomplish.</div>
<div><br></div><div>The OpenEFS implementation currently only supports NFSv=
3 out of the box, but I have done most of the work necessary to use AFS. =
=A0However, that work is currently on hold, pending, well.. =A0 someone els=
e being interested in it. =A0My current employers are very unlikely to inve=
st in an AFS infrastructure, and hopefully I'll find out for sure in ea=
rly 2011 (I am not very optimistic).</div>
<div><br></div><div>We have done a lot of work to automate integrating open=
source products with OpenEFS, and anything that is built using the GNU aut=
oconf suite or CPAN-style perl packages (Makefile.PL, Build.PL) can be inte=
grated fairly easily. =A0 We have a lot of pre-compiled content that be dow=
nloaded and used immediately, for example, and we would love to build an op=
en source community around the product.</div>
<div><br></div><div>EFS is fundamentally designed to support software deplo=
yment and change control in a SINGLE authentication domain, and the goal fo=
r OpenAFS is to support distribution to a collection of AFS cells that are =
all part of the same kerberos realm. =A0 This is based on the basic model w=
e used to deploy AFS at Morgan Stanley, where we had a single Kerberos real=
m but 50+ AFS cells in data centers around the world.</div>
<div><br></div><div>The main challenge to using EFS is that it supports a v=
ery different deployment paradigm, since it completely decouples software d=
eployment from the clients, but this is precisely why the predecessors to O=
penEFS have proven to be successful.=A0</div>
<div><br></div><div>If anyone is interested in the product, let us know....=
</div><div><br></div><div><div class=3D"gmail_quote">On Mon, Dec 20, 2010 a=
t 1:37 PM, Andrew Deason <span dir=3D"ltr"><<a href=3D"mailto:adeason@si=
nenomine.net">adeason@sinenomine.net</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex;"><div class=3D"im">On Mon, 20 Dec 2010 18:46=
:32 +0100<br>
Dirk Heinrichs <<a href=3D"mailto:dirk.heinrichs@altum.de">dirk.heinrich=
s@altum.de</a>> wrote:<br>
<br>
> I'm currently thinking about a good way to deploy software package=
s in<br>
> (eventually replicated) AFS volumes. One possible way I can think of<b=
r>
> is to use (x)stow, but that would imply a lot of manual work<br>
> (download, unpack, compile, install to rw volume, xstow, vos release).=
<br>
><br>
> Does anyone know of a simpler (more automated) solution, maybe<br>
> something like Gentoo portage or Nix?<br>
<br>
</div>You may be interested in OpenEFS <<a href=3D"http://www.openefs.or=
g/" target=3D"_blank">http://www.openefs.org/</a>>, where I<br>
believe AFS support is being worked on. While you can perhaps get<br>
something to work that just combines something stow-like with<br>
pkgsrc/portage/etc... depending on how 'production' this setup is,<=
br>
you're going to encounter some problems sooner or later with<br>
dependencies and release engineering, etc, that systems like EFS or VMS<br>
try to make easier.<br>
<font color=3D"#888888"><br>
--<br>
Andrew Deason<br>
<a href=3D"mailto:adeason@sinenomine.net">adeason@sinenomine.net</a><br>
</font><div><div></div><div class=3D"h5"><br>
_______________________________________________<br>
OpenAFS-info mailing list<br>
<a href=3D"mailto:OpenAFS-info@openafs.org">OpenAFS-info@openafs.org</a><br=
>
<a href=3D"https://lists.openafs.org/mailman/listinfo/openafs-info" target=
=3D"_blank">https://lists.openafs.org/mailman/listinfo/openafs-info</a><br>
</div></div></blockquote></div><br></div>
--0016362842ea5e3b740497efec8b--