[OpenAFS] Performance

[Nathan Ward]

I'll do some tests with this today.

Yes of course it will be slower, but it is unbearably slower. I'm not 
sure about local disk stats, but perhaps a better comparison:
It takes ~4 times as long on AFS over NFS, last I checked.

emoy@apple.com wrote:

> Sorry for any confusion, but it was a 1 GB file I used.  So 
> performance  was not up to the theoretical maximum, but I was not on a 
> quiet  network, and the server was on a different subnet.  That is why 
> one has  to compare the performance with NFS directly, and not against 
> the  theoretical maximum (the server in the test was doing both NFS 
> and AFS).
>
> Other than setting the cache size to 2000000 blocks, I used 
> "-chucksize  18" to get the 256 KB chucksize.  NFS write time was 
> about 4:20  compared to 3:30 for 256 KB chucksize and 4:50 for the 
> default 64 KB  chucksize.  System CPU time for NFS was about the same 
> as AFS (around  20 seconds).
>
> As for compiling on AFS versus local disk, of course AFS would be  
> slower.  The files have to be read from the network at least once, 
> and  new/modified files written back.  However, compiling on NFS, for  
> example, will generally be poorer, though perhaps not strikingly so,  
> depending on how much your kernel caches NFS.  And as more users 
> access  files simultaneously, the performance of NFS will degrade 
> faster than  AFS.
> ------------------------------------------------------------------------ 
> -- 
> Edward Moy
> Apple Computer, Inc.
> emoy@apple.com
>
> (This message is from me as a reader of this list, and not a statement
> from Apple.)
>
> On Thursday, February 27, 2003, at 01:24  PM, Nathan Ward wrote:
>
>> Yes, playing with tuning the cache parameters is something thats on 
>> my  run queue.
>> What were your afs client settings you had when you did the 2gb file  
>> in 3.5 min?
>>
>> My calculations say that 2gb in 3.5 minutes is ~10mb/s. A 100Mbit  
>> network can do ~12.5MB/s of packets, so allow for headers and 
>> control  data and i'd say your are limited at this point by your 
>> network.
>>
>> You say it beats NFS, whats the CPU usages for 2GB over OpenAFS vs  
>> NFS? Context switches?
>>
>> My developers complain that code compiles are much much slower in 
>> AFS  than on the local disk, but actual file performance when they 
>> aren't  compiling is not-so-bad. I imagine this is because of the CPU 
>> that AFS  takes to get the files, so CPU usage is important here.
>>
>> emoy@apple.com wrote:
>>
>>> As I understand it, the cache helps reads primarily, though if you  
>>> are  doing random updates of a file, those also get cached, until 
>>> you  do the  finally close, which then writes all to the server.  
>>> That  assumes the  file fits in the cache.  If not, then a write can 
>>> force  the cache to  partially empty, so the latest write can be 
>>> cached.
>>>
>>> But there seems to be some pathological behavior, because doing a   
>>> sequential write should ideally not be affected that much by size 
>>> of   the cache.  But in a test I did with a 1 GB file, with regular  
>>> network  traffic, it took almost 5 minute to write with with a 2 GB  
>>> cache (about  10% slower than NFS in this case), but took almost 11  
>>> minutes with a  0.5 GB cache.  The system CPU time rose from 20  
>>> seconds for the 2 GB  cache to 517 seconds with the 0.5 GB cache.
>>>
>>> As for being slower than NFS in this test case, with such a large   
>>> cache, tuning the cache parameters does help.  Setting the 
>>> chucksize  to  256KB dropped the 2 GB cache speed to about 3.5 
>>> minutes, beating  NFS.
>>>
>>> On Thursday, February 27, 2003, at 11:51  AM, Nathan Ward wrote:
>>>
>>>> Well I'd expect that it goes slower as your cache size is exceeded  
>>>> as  it then needs to start getting that data to the server. Or is  
>>>> the  cache for read operations only?
>>>>
>>>> I notice that there are around about the same number of 
>>>> packets/sec  as  context switches/sec on my client machines. I 
>>>> wonder if switches   between userland and kernel could be to 
>>>> blame... ? Who sends  packets  in OpenAFS, the userspace daemon or 
>>>> the kernel?
>>>>
>>>> Nathan
>>>>
>>>> emoy@apple.com wrote:
>>>>
>>>>> Could the slowness you see with your dd write test be related to  
>>>>> the   cache exhaustion issue that I raised recently, when writing 
>>>>> a  file   larger than your cache size.  Your test writes a 1 GB 
>>>>> file,  so if  your  cache is smaller than this, you will see poor  
>>>>> performance once  your  cache size is exceeded.
>>>>>
>>>>> On Thursday, February 27, 2003, at 11:15  AM, Nathan Ward wrote:
>>>>>
>>>>>> I see pretty bad performance to tell you the truth.
>>>>>> I can read and write ~60mb/s directly to my raid array, but 
>>>>>> when   using  OpenAFS (locally or remotely) to the same array, I 
>>>>>> get  around   6-10MB/s, I have seen up to 25MB/s over a peice of  
>>>>>> 1000Mbps fibre.   Client and Server are both dual P3-1ghz with  
>>>>>> 1024mb ram. I notice  the  context switches on the server at 
>>>>>> this  time jump to ~10000/s,  and on  the client ~40000/s. I 
>>>>>> imagine  this is the source of my  slowdown, but  I havn't had a 
>>>>>> chance to  look into it.
>>>>>>
>>>>>> I'd be interested if anyone else has the same level of context    
>>>>>> switches going on.
>>>>>>
>>>>>> This is while doing a large sequential write operation (dd    
>>>>>> if=/dev/zero of=/afs/alb-nz.esphion.com/public/dd.out bs=256k    
>>>>>> count=4096).
>>>>>>
>>>>>> Michael Robokoff wrote:
>>>>>>
>>>>>>> Does anyone have any Open AFS performance information they can   
>>>>>>> share  with me. I plan on doing a couple benchmarks and I 
>>>>>>> would   like to have  some idea of what to expect.
>>>>>>
>>
>
>
>