[OpenAFS-devel] Regarding GSoC 2010 Collaborative Caching Project
Derrick Brashear
shadow@dementia.org
Wed, 14 Apr 2010 10:45:18 -0400
--Apple-Mail-5-891343853
Content-Type: text/plain;
charset=us-ascii;
format=flowed;
delsp=yes
Content-Transfer-Encoding: 7bit
Steve,
You should like it. The idea was already implemented at Michigan once
(ifs, the intermediate fileserver)
Derrick
On Apr 14, 2010, at 10:36 AM, Steve Simmons <scs@umich.edu> wrote:
> On Apr 13, 2010, at 10:12 AM, shruti jain wrote:
>> The project aims at developing a system which would use
>> collaborative caching techniques to improve the read accesses in
>> OpenAFS. This project is based on two observations.
>>
>> Firstly, in a cluster environment, a large number of clients need
>> same datasets to work on i.e. the data on which client nodes need
>> to execute is same for many other nodes on the network. Currently,
>> each client contacts the server individually to fetch the data.
>> This increase load on the server unnecessarily. If the size of the
>> file is very large then the problem would be highly magnified.
>>
>> Second observation is that the local bandwidth are mostly fast and
>> runs into Gbps. In a cluster, many clients would share the same
>> geography and thus have fast interconnects between them. The server
>> might be connected through a slow network link. In this situation,
>> accessing data from another client would be much faster than
>> accessing data from server itself.
>>
> Just one guy's opinion, but I like this. The more I think about it,
> the more I like it.
>
> One long-term benefit of this is in clusters. We've seen a number of
> instances where folks have attempted to use AFS as if it were a
> clustered file system, keeping various lockfiles, etc in the fs. It
> beat the hell out of our AFS servers when it happened. If a fs could
> delegate responsibility for such lockfiles to a local node, it's
> make that sort of thing feasible and fast.
>
> Steve
--Apple-Mail-5-891343853
Content-Type: text/html;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
<html><body =
bgcolor=3D"#FFFFFF"><div>Steve, </div><div><br></div><div>You =
should like it. The idea was already implemented at Michigan once (ifs, =
the intermediate =
fileserver)<br><br>Derrick<div><br></div></div><div><br>On Apr 14, 2010, =
at 10:36 AM, Steve Simmons <<a =
href=3D"mailto:scs@umich.edu">scs@umich.edu</a>> =
wrote:<br><br></div><div></div><blockquote type=3D"cite"><div><div><div>On=
Apr 13, 2010, at 10:12 AM, shruti jain wrote:</div><blockquote =
type=3D"cite"><span class=3D"Apple-style-span" style=3D"border-collapse: =
separate; font-family: Courier; font-size: medium; font-style: normal; =
font-variant: normal; font-weight: normal; letter-spacing: normal; =
line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; =
white-space: normal; widows: 2; word-spacing: 0px; =
-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
auto; -webkit-text-stroke-width: 0px; "><p>The project aims at =
developing a system which would use collaborative caching techniques to =
improve the read accesses in OpenAFS. This project is based on two =
observations.</p><p>Firstly, in a cluster environment, a large number of =
clients need same datasets to work on i.e. the data on which client =
nodes need to execute is same for many other nodes on the network. =
Currently, each client contacts the server individually to fetch the =
data. This increase load on the server unnecessarily. If the size of the =
file is very large then the problem would be highly =
magnified.</p><p>Second observation is that the local bandwidth are =
mostly fast and runs into Gbps. In a cluster, many clients would share =
the same geography and thus have fast interconnects between them. The =
server might be connected through a slow network link. In this =
situation, accessing data from another client would be much faster than =
accessing data from server itself.</p></span></blockquote></div>Just one =
guy's opinion, but I like this. The more I think about it, the more I =
like it.<div><br></div><div>One long-term benefit of this is in =
clusters. We've seen a number of instances where folks have attempted to =
use AFS as if it were a clustered file system, keeping various =
lockfiles, etc in the fs. It beat the hell out of our AFS servers when =
it happened. If a fs could delegate responsibility for such lockfiles to =
a local node, it's make that sort of thing feasible and =
fast.</div><div><div><br></div><div>Steve</div></div></div></blockquote></=
body></html>=
--Apple-Mail-5-891343853--