[OpenAFS] Windows cache problem revisited...

[Rodney M Dyer]

Rob, Derrick, Jeffrey,

I'm not sure what the right way to respond is here...so I'll just start in.

Jeffrey, thank you for the polite lesson in mailing list social 
etiquette.  It may be deserved, maybe not.  I did say ahead of time that I 
was "sorry for the rant".  Sometimes I use this list to vent my 
frustrations.  I wasn't directing my frustration toward you at all, and I'm 
sorry if you may have interpreted it that way.  I do not in the least want 
to imply that the AFS/OpenAFS developers haven't done a good job in keeping 
the software alive, especially for Windows users.  I am thoroughly pleased 
with the community at large, that this list exists, and that continued 
development is being done.  I do however like to make problems known, and 
sometimes I do this with a manner that probably isn't in some peoples best 
interests.  Many of the best rants and heated debates take place in mailing 
lists.  Just look at the Linux kernel mailing list for example.  To think 
that I might be annoying someone here, how outrageous!  ;)

Moving on...

Rob, Jeffrey, your responses were good in that they are some of the deeper 
'technical' information I was looking for.  However, part of the problem is 
that I'm not overjoyed about these so called 'performance improvements' 
made by the "memory mapped chunks" because there don't seem to be any.

Yes, reducing the cache has made my life better, if you can call it 
that.  It is now set to 48 Meg instead of 256 Meg.  The problem now is I 
don't have enough room to really cache much.  I'm now loading down the 
network more than I should.  Part of my frustration is seeing that others 
"can" set their caches to 1 Gig or greater.   See...

http://www.acm.uiuc.edu/wiki/space/Setting+up+your+computer+as+an+AFS+client

So how does the cache implementation differ under the Unix/Linux 
environment.  Does the 'nix environment use a memory mapped file?  If a 
'nix machine has only 256 Meg of real memory, then what do you think a good 
cache size would be?  What do you guys on the mailing list set your caches 
to on 'nix, and Windows?  If the cache implementation is different under 
'nix/Win then why?  Does 'nix slow down when transfering large files?

My problem isn't always related to copying large files.  Many times I will 
walk into our student labs and logon with a test account just to do a quick 
test.  I find that many machines have entered the 'slow' condition and 
logons take considerably longer than normal as well as application slow 
downs.  I find this especially curious because we restart the AFS client 
service every night at 4:00am and delete the cache file.  You would think 
that our lab machines could take 5 or 10 different people logging on and 
off to do work during the day without entering the 'slow' state.  If I 
reset the cache down to 48 Meg for those machines it should 'fix' the 
problem, but then the network will probably be loaded more, which makes the 
cache kind'a moot.  Granted, we are now getting new machines with a Gig of 
memory so this problem may not show up as often.  So maybe I should just 
shut up and throw hardware at the problem right?

In the end, I don't see the efficiency gained by the algorithm used to 
cache AFS data on Windows.  I'd either like this changed in the future, or 
given an option to not use any memory mapping.  Windows itself does it's 
own file caching for recently used files.  Why would AFS need to do it on 
top of that?

Sorry again Jeffrey,

(Thanks for the moral support Mike!)

Rodney

At 02:06 PM 12/4/2003, Jeffrey Altman wrote:
>Rodney:
>
>You mis-read some of what I said.  The file cache is memory mapped not in 
>one huge chunk (if that was true allocating a 256MB cache would result in 
>an instantaneous 256MB memory allocation) but in N chucks of the specified 
>chunk size.  I think the default values are 20MB and 32KB chunks.  Hence, 
>the allocation of up to 640 handles.
>
>I haven't looked at all of the specifics but ... when you write data into 
>the cache a chunk is allocated and mapped to the disk file.  This chunk 
>remains in memory for some period of time to avoid the need to re-read the 
>data from the cache into memory.  When you are copying a huge file the 
>process is sequential and hence all of the data is processed once and then 
>ignored.  Hence for some period of time the entire backing store ends up 
>in memory.  Eventually those cache pages will time out due to lack of use 
>and be flushed.
>There are many things lacking in the cache management. Off the top of my 
>head these are the two big ones:
>
>   1. there is no distinction between the disk cache size and the in
>      memory cache size
>   2. application as part of the open mode for a file have the ability
>      to provide hints regarding the access usage: sequential, random
>      access, etc.  these hints are not seen by the AFS cache and
>      therefore cannot be used to optimize the memory usage
>
><flame on>
>On a personal note, I must say that I am getting a bit tired of your 
>rants.  They are not in the least bit constructive.  Insulting the work of 
>the people who wrote and designed the original code does not help.
>You sound like a spoiled child.  "I want ...", "I want ...", "Why can't I 
>...", "Who's idea was it to ...". <flame off>
>
>We are all aware that the design of OpenAFS for Windows is not appropriate 
>for the Windows 2000/XP/2003 environments with multiple (perhaps 
>simultaneous) users per machine with roaming profiles and integrated 
>Kerberos authentication.  The work that must be done is substantial and 
>for the most part should not be done piece meal.
>Instead, a significant re-design of the architecture must be performed to 
>better integrate AFS into Windows and its local security architecture.
>
>I certainly understand your frustration.  OpenAFS on Windows has certainly 
>been the hand me down sibling of the Unix counterpart.  Rants will not 
>change this.  The only thing that will is a commitment of resources and 
>money.  A really rough estimate of the work to be done falls somewhere 
>between $750,000 and $1 million range.  Clearly none of the individual 
>organizations which use OpenAFS on Windows are can afford to put up that 
>kind of money.  Perhaps smaller chucks collected from a number of 
>organizations would allow an appropriate team of developers to be put 
>together either to perform the work or at least to design an architecture 
>and manage a team of volunteers to do the work.
>
>I would ask that anyone who would be interested in contributing either 
>money or a substantial number of developer hours over an extended period 
>of time to contact me directly.
>
>Jeffrey Altman
>
>
>
>
>Rodney M Dyer wrote:
>
>>All,
>>
>>I've been thinking a bit more about the problem I'm having with OpenAFS 
>>on Windows 2k/XP.  When transfering large files to and from AFS the 
>>Windows OS performance can slow down considerably, all most to the point 
>>of non-usability.  To alleviate this problem I've been told not to use a 
>>large AFS cache because the client service opens a real memory buffer 
>>that is the same size as the cache file.  On closer examination I find 
>>this to be ridiculous and intolerable.  The point of the AFS cache is to 
>>eliminate multiple network reads for files that haven't changed on the 
>>network by grabbing them off the local disk cache.  We should be able to, 
>>and most people do, use very large caches.  I'm familiar with people 
>>using gig sized caches these days.
>>Granted, depending on what your are doing, there will be a point of 
>>diminishing returns with larger and larger cache sizes.  But that is for 
>>me to decide!
>>
>>I want to know why the AFS client service uses a real memory buffer in 
>>the first place?  I want to know why you can't turn this 'feature' 
>>off?  Who decided that the in-memory buffer would be a good idea?  Can we 
>>get this changed?
>>
>>Thanks ahead for any feedback (sorry for the rant),
>>
>>Rodney
>>
>>Rodney M. Dyer
>>Windows Systems Programmer
>>Mosaic Computing Group
>>William States Lee College of Engineering
>>University of North Carolina at Charlotte
>>Email: rmdyer@uncc.edu
>>Web: http://www.coe.uncc.edu/~rmdyer
>>Phone (704)687-3518
>>Help Desk Line (704)687-3150
>>FAX (704)687-2352
>>Office  267 Smith Building
>>
>>_______________________________________________
>>OpenAFS-info mailing list
>>OpenAFS-info@openafs.org
>>https://lists.openafs.org/mailman/listinfo/openafs-info
>