[OpenAFS-devel] Patches for Openafs compression support Part II
Sven Oehme
oehmes@de.ibm.com
Wed, 12 Jan 2005 20:16:11 +0100
This is a multipart message in MIME format.
--=_alternative 006998E1C1256F87_=
Content-Type: text/plain; charset="US-ASCII"
not right now ...
my "in head" vision is :
1. step implement file compression this for the move , release, dump stuff
..
2. step look at byte level replication like rsync on volume level
3. step use this build prereqs by other parts of openafs (backup, etc ...)
4. step create a more generic backup interface.
5. step use the rsync like transfer mechanism also to fetch/update
client's cache files instead of transferring whole files between client
and server.
......and a lot more :-)
but this will all be discussed, when we finished step 1 and have time to
work on step 2....
but in general, ideas are welcome :-)
Sven
-------------------------------------------------------------------------------------------------------------------------
Dept. A141, TG/SSG EMEA AIS Strategy and Architecture
Development Leader Stonehenge
IBM intranet ---> http://w3.ais.mainz.de.ibm.com/stonehenge/
internet ---> http://www-5.ibm.com/services/de/storage/stonehenge.html
Phone (+49)-6131-84-3151
Fax (+49)-6131-84-6708
Mobil (+49)-171-970-6664
E-Mail : oehmes@de.ibm.com
Matthew Andrews <matt@slackers.net>
Sent by: openafs-devel-admin@openafs.org
12/01/2005 19:49
To
openafs-devel@openafs.org
cc
Subject
Re: [OpenAFS-devel] Patches for Openafs compression support Part II
Have you looked at all at how dificult to gat the afs backup
utils(backup, butc) to take advantage of this?
Sven Oehme wrote:
>
> Hi ,
>
> here is an update to the compression support and some answers to
> questions :-)
>
> Purpose:
> - patch for versions of openafs 1.2.13 and 1.3.77 to support gzip and
> bz2 compression at "vos dump", "vos release", "vos restore" and "vos
> move".
>
> we still haven't prepared the changes for WinNT (will come with next
> update
> - haven't done any changes to libadmin (AFS API) yet.
> - these patches are full patches, not upgrade. Has to be applied on the
> original version (of src/volser directory).
>
> Installation:
> cd openafs-1.2.13
> (where the src folder and configure script is in openafs-1.2.13)
> patch -p1 <../openafs-1.2.13z.patch
> [recompile the source]
>
> Modified binaries are: "vos" and "volserver"
> The command "vos" has the new flag "-z [1..9]" and "-bz2 [1..9]", and
> volserver has the support with its modified interface.
> The compression support is directly over the RPC call layer.
> The modified binaries must work fully with the unpatched openafs
> versions.
>
> Implementation: At some places [dumpstuff.c, vos.c] rx_Read and
> rx_Write has been replaced to the new wrapping functions which support
> compression. See "ziorx.[c,h]".
>
>
>
> Sven
>
>
-------------------------------------------------------------------------------------------------------------------------
> Dept. A141, TG/SSG EMEA AIS Strategy and Architecture
> Development Leader Stonehenge
> IBM intranet ---> http://w3.ais.mainz.de.ibm.com/stonehenge/
> internet ---> http://www-5.ibm.com/services/de/its/filestore.html
> Phone (+49)-6131-84-3151
> Fax (+49)-6131-84-6708
> Mobil (+49)-171-970-6664
> E-Mail : oehmes@de.ibm.com
>
>
> *Mitch Collinsworth <mitch@ccmr.cornell.edu>*
> Sent by: openafs-devel-admin@openafs.org
>
> 04/01/2005 19:31
>
>
> To
> Sven Oehme/Germany/IBM@IBMDE
> cc
> openafs-devel@openafs.org
> Subject
> Re: [OpenAFS-devel] Patches for Openafs compression
support
>
>
>
>
>
>
>
>
>
>
> On Wed, 22 Dec 2004, Sven Oehme wrote:
>
> > we have started a new project to enhance the data transfer (first on
> > volume level) to slow link connected sites.
> > we implemented the gzip algorithm and will add bzip2 compression
later.
> >
> > i have attached two patches for Openafs 1.2.13 and 1.3.77 to add gzip
> > compression to vos dump , release and restore
>
> Question: How is it decided when to compress and when to not? Does
> the invoker have to request it, is it done by default, etc?
>
> -Mitch
> _______________________________________________
> OpenAFS-devel mailing list
> OpenAFS-devel@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-devel
>
_______________________________________________
OpenAFS-devel mailing list
OpenAFS-devel@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-devel
--=_alternative 006998E1C1256F87_=
Content-Type: text/html; charset="US-ASCII"
<br><font size=2 face="sans-serif">not right now ...</font>
<br>
<br><font size=2 face="sans-serif">my "in head" vision is :</font>
<br>
<br><font size=2 face="sans-serif">1. step implement file compression this
for the move , release, dump stuff ..</font>
<br><font size=2 face="sans-serif">2. step look at byte level replication
like rsync on volume level</font>
<br><font size=2 face="sans-serif">3. step use this build prereqs by other
parts of openafs (backup, etc ...)</font>
<br><font size=2 face="sans-serif">4. step create a more generic backup
interface.</font>
<br><font size=2 face="sans-serif">5. step use the rsync like transfer
mechanism also to fetch/update client's cache files instead of transferring
whole files between client and server.</font>
<br>
<br><font size=2 face="sans-serif">......and a lot more :-)</font>
<br>
<br><font size=2 face="sans-serif">but this will all be discussed, when
we finished step 1 and have time to work on step 2....</font>
<br>
<br><font size=2 face="sans-serif">but in general, ideas are welcome :-)</font>
<br>
<br><font size=2 face="sans-serif">Sven</font>
<br><font size=2 face="sans-serif"><br>
-------------------------------------------------------------------------------------------------------------------------<br>
Dept. A141, TG/SSG EMEA AIS Strategy and Architecture<br>
Development Leader Stonehenge <br>
IBM intranet ---> http://w3.ais.mainz.de.ibm.com/stonehenge/<br>
internet ---> http://www-5.ibm.com/services/de/storage/stonehenge.html<br>
Phone (+49)-6131-84-3151<br>
Fax (+49)-6131-84-6708<br>
Mobil (+49)-171-970-6664<br>
E-Mail : oehmes@de.ibm.com</font>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td width=40%><font size=1 face="sans-serif"><b>Matthew Andrews <matt@slackers.net></b>
</font>
<br><font size=1 face="sans-serif">Sent by: openafs-devel-admin@openafs.org</font>
<p><font size=1 face="sans-serif">12/01/2005 19:49</font>
<td width=59%>
<table width=100%>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">To</font></div>
<td valign=top><font size=1 face="sans-serif">openafs-devel@openafs.org</font>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">cc</font></div>
<td valign=top>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">Subject</font></div>
<td valign=top><font size=1 face="sans-serif">Re: [OpenAFS-devel] Patches
for Openafs compression support Part II</font></table>
<br>
<table>
<tr valign=top>
<td>
<td></table>
<br></table>
<br>
<br>
<br><font size=2><tt>Have you looked at all at how dificult to gat the
afs backup <br>
utils(backup, butc) to take advantage of this?<br>
<br>
Sven Oehme wrote:<br>
<br>
><br>
> Hi ,<br>
><br>
> here is an update to the compression support and some answers to <br>
> questions :-)<br>
><br>
> Purpose:<br>
> - patch for versions of openafs 1.2.13 and 1.3.77 to support gzip
and <br>
> bz2 compression at "vos dump", "vos release",
"vos restore" and "vos <br>
> move".<br>
><br>
> we still haven't prepared the changes for WinNT (will come with next
<br>
> update<br>
> - haven't done any changes to libadmin (AFS API) yet.<br>
> - these patches are full patches, not upgrade. Has to be applied on
the<br>
> original version (of src/volser directory).<br>
><br>
> Installation:<br>
> cd openafs-1.2.13<br>
> (where the src folder and configure script is in openafs-1.2.13)<br>
> patch -p1 <../openafs-1.2.13z.patch<br>
> [recompile the source]<br>
><br>
> Modified binaries are: "vos" and "volserver"<br>
> The command "vos" has the new flag "-z [1..9]"
and "-bz2 [1..9]", and <br>
> volserver has the support with its modified interface.<br>
> The compression support is directly over the RPC call layer.<br>
> The modified binaries must work fully with the unpatched openafs <br>
> versions.<br>
><br>
> Implementation: At some places [dumpstuff.c, vos.c] rx_Read and <br>
> rx_Write has been replaced to the new wrapping functions which support
<br>
> compression. See "ziorx.[c,h]".<br>
><br>
><br>
><br>
> Sven<br>
><br>
> -------------------------------------------------------------------------------------------------------------------------<br>
> Dept. A141, TG/SSG EMEA AIS Strategy and Architecture<br>
> Development Leader Stonehenge<br>
> IBM intranet ---> http://w3.ais.mainz.de.ibm.com/stonehenge/<br>
> internet ---> http://www-5.ibm.com/services/de/its/filestore.html<br>
> Phone (+49)-6131-84-3151<br>
> Fax (+49)-6131-84-6708<br>
> Mobil (+49)-171-970-6664<br>
> E-Mail : oehmes@de.ibm.com<br>
><br>
><br>
> *Mitch Collinsworth <mitch@ccmr.cornell.edu>*<br>
> Sent by: openafs-devel-admin@openafs.org<br>
><br>
> 04/01/2005 19:31<br>
><br>
> <br>
> To<br>
> Sven
Oehme/Germany/IBM@IBMDE<br>
> cc<br>
> openafs-devel@openafs.org<br>
> Subject<br>
> Re:
[OpenAFS-devel] Patches for Openafs compression support<br>
><br>
><br>
><br>
> <br>
><br>
><br>
><br>
><br>
><br>
><br>
> On Wed, 22 Dec 2004, Sven Oehme wrote:<br>
><br>
> > we have started a new project to enhance the data transfer (first
on<br>
> > volume level) to slow link connected sites.<br>
> > we implemented the gzip algorithm and will add bzip2 compression
later.<br>
> ><br>
> > i have attached two patches for Openafs 1.2.13 and 1.3.77 to
add gzip<br>
> > compression to vos dump , release and restore<br>
><br>
> Question: How is it decided when to compress and when to not?
Does<br>
> the invoker have to request it, is it done by default, etc?<br>
><br>
> -Mitch<br>
> _______________________________________________<br>
> OpenAFS-devel mailing list<br>
> OpenAFS-devel@openafs.org<br>
> https://lists.openafs.org/mailman/listinfo/openafs-devel<br>
><br>
<br>
_______________________________________________<br>
OpenAFS-devel mailing list<br>
OpenAFS-devel@openafs.org<br>
https://lists.openafs.org/mailman/listinfo/openafs-devel<br>
</tt></font>
<br>
--=_alternative 006998E1C1256F87_=--