[OpenAFS] poor out of cache behavior on writing

[Dexter "Kim" Kimball]

>I wonder if something as simple as "track the total bytes written to a
>particular file while that file is currently open, once it exceeds x% of
>the cache, bypass" would perform reasonably?

Is the write behavior necessarily this dependent on cache size alone?  I'm
speculating that cache state would be relevant.

If I've got a large empty cache or mostly unused cache, do I get the same
write performance as with an about-to-purge-I'm-full cache?

Or might there be a dependency on the number/percentage of dirty chunks ...
those have got to be flushed back to the fileserver before they can be
reclaimed, so the worst case might be a large cache with lots of dirty
chunks from lots of small files.

I've only been "bothered" by the write performance limitations once or
twice -- when continuously and rapidly writing/closing/opening an output
file from an "indexer" into AFS filespace.  Indexing software was
proprietary, couldn't get vendor to change it .... so finally moved the
output file outside of AFS -- automatically moved it back when the indexing
was complete.

It took a while to figure out what the bottleneck was, though, and it would
by all means be great to have a reasonable bypass algorithm.  Or even the
ability to tag a file/directory/volume/partition as "don't cache."   Which
again presupposes the willingness to identify the file/directory/...


----- Original Message -----
From: "Neulinger, Nathan" <nneul@umr.edu>
To: <openafs-info@openafs.org>
Sent: Monday, February 17, 2003 4:24 PM
Subject: RE: [OpenAFS] poor out of cache behavior on writing


I wonder if something as simple as "track the total bytes written to a
particular file while that file is currently open, once it exceeds x% of
the cache, bypass" would perform reasonably?

It wouldn't help the small cases.

Another possible scenario would be assume bypass until the file has been
read once. That would cause all initial creates to bypass, but later
appends/edits would return to normal speed.

------------------------------------------------------------
Nathan Neulinger                       EMail:  nneul@umr.edu
University of Missouri - Rolla         Phone: (573) 341-4841
Computing Services                       Fax: (573) 341-4216


> -----Original Message-----
> From: Derrick J Brashear [mailto:shadow@dementia.org]
> Sent: Monday, February 17, 2003 5:14 PM
> To: openafs-info@openafs.org
> Subject: Re: [OpenAFS] poor out of cache behavior on writing
>
>
> On Mon, 17 Feb 2003, Paul Blackburn wrote:
>
> > You probably will get better data transfer from ftp-client
> => ftp-server
> > than a distributed filesystem.
>
> This, at least, is a protocol. scp is a hack, which is why
> I'm reluctant
> to use it.
>
> > When it comes to "uploading" large datafiles from machine
> to machine,
> > I now prefer using SSH's scp which shows me a progress bar and ETA.
> >
> > For most all other tasks, I like AFS which gives me other
> capabilities
> > unheard of in NFS.
>
> I wonder if we could find a sucker^H^H^H^H^H^Hgrad student to
> figure out
> ("guess") when to bypass the cache and when to use it, and
> incorporate it
> (at least as an option) in the clients.
>
> I know CITI has done cache-bypass stuff, but it used the
> volume name (I
> think) as a hint.
>
>
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info
>
_______________________________________________
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info