[OpenAFS] OpenAFS speed
Nathan Ward
nward@esphion.com
Wed, 25 Jun 2003 09:00:31 +1200
On Wed, 25 Jun 2003 08:53:47 +1200, Nathan Ward <nward@esphion.com> wrote:
> On Tue, 24 Jun 2003 16:25:50 -0400 (EDT), Derrick J Brashear
> <shadow@dementia.org> wrote:
>
>> On Wed, 25 Jun 2003, Nathan Ward wrote:
>>
>>> Several times I have mentioned this and gotten no useful response that
>>> I
>>> can remember.
>>>
>>> I am running OpenAFS on linux machines. Take a look at the context
>>> switches
>>> on the client and the server....
>>>
>>> (vmstat 1, look at the "cs" column)
>>>
>>> NFS solves this problem by having a fully kernel server and client.
>>
>> Our client is fully kernel, and yet it's the client that people seem to
>> indicate is the big problem.
>
> One moment please...
>
> AFS server on serv-1, client is serv-2. Gigabit ethernet between them...
>
> nward@serv-2:/$ vmstat 1
> procs memory swap io system
> cpu
> r b w swpd free buff cache si so bi bo in cs us
> sy id
> 0 0 0 576 349032 37444 389620 0 0 1 2 18 11 0
> 0 6
> 0 0 0 576 282320 37512 456188 0 0 0 0 268 2707 0
> 24 76
> 0 0 0 576 282320 37512 456188 0 0 0 0 876 14909 0
> 43 57
> 0 0 0 576 282320 37512 456188 0 0 0 0 881 14884 0
> 40 60
> 0 0 0 576 282320 37512 456188 0 0 0 0 865 14917 1
> 44 55
> 0 0 0 576 282320 37512 456188 0 0 0 0 846 15008 0
> 39 61
> 1 0 0 576 254496 37596 483928 0 0 0 128 713 12035 0
> 44 56
> 0 0 2 576 238196 37612 500208 0 0 0 0 687 13162 0
> 43 57
> 0 0 2 576 238196 37612 500208 0 0 0 8 821 14754 0
> 43 57
> 0 0 1 576 238196 37612 500208 0 0 0 0 854 14737 0
> 39 61
> 0 0 1 576 238196 37612 500208 0 0 0 8 846 14696 0
> 45 55
> 0 0 0 576 238196 37612 500208 0 0 0 0 777 13821 0
> 34 65
> 0 0 0 576 238264 37612 500208 0 0 0 0 238 2982 0
> 6 93
> 0 0 0 576 238264 37612 500208 0 0 0 0 103 116 0
> 0 100
> 0 0 0 576 238264 37612 500208 0 0 0 0 103 114 0
> 0 100
>
> serv-1:~# vmstat 1
> procs -----------memory---------- ---swap-- -----io---- --system-- ----
> cpu-- --
> r b swpd free buff cache si so bi bo in cs us sy
> id wa
> 0 0 4452 234396 84868 308172 0 0 1 14 10 6 0 1
> 99 0
> 0 0 4452 234388 84868 308172 0 0 0 0 120 45 0 0
> 100 0
> 0 0 4452 234388 84868 308172 0 0 0 0 125 56 0 0
> 100 0
> 0 0 4452 299904 84884 242636 0 0 0 40 114 54 0 3
> 97 0
> 1 0 4452 293592 84892 248844 0 0 0 0 2193 1944 40 12
> 49 0
> 1 0 4452 287040 84900 255388 0 0 0 0 2287 1994 45 12
> 44 0
> 1 0 4452 280476 84904 261948 0 0 0 0 2266 1838 47 12
> 42 0
> 1 0 4452 273932 84916 268476 0 0 0 0 2207 1891 41 12
> 47 0
> 1 0 4452 268280 84920 274124 0 0 0 260 2050 1724 37 14
> 50 0
> 1 0 4452 263988 84924 278412 0 0 0 0 1494 1231 28 7
> 64 0
> 1 0 4452 257436 84932 284956 0 0 0 0 2233 2050 47 9
> 44 0
> 2 0 4452 250884 84940 291500 0 0 0 0 2260 1823 48 10
> 42 0
> 1 0 4452 244320 84944 298060 0 0 0 0 2251 1922 43 14
> 43 0
> 1 0 4452 238168 84952 304204 0 0 0 168 2132 1857 41 10
> 49 0
> 0 0 4452 234196 84956 308172 0 0 0 0 1361 1150 25 8
> 66 0
> 0 0 4452 234196 84956 308172 0 0 0 0 127 60 0 0
> 100 0
>
> nward@serv-2:/$ dd if=/dev/zero of=/afs/alb-nz/public/blah bs=256k
> count=256 256+0 records in
> 256+0 records out
>
With native memcache on on the client machine... (-stat 2000 -memcache -
chunksize 14 -daemons 3 -volumes 50)
It's a LOT faster, but there are not less context switches.. on a busy
machine this could be a problem..
nward@serv-2:/$ vmstat 1
procs memory swap io system
cpu
r b w swpd free buff cache si so bi bo in cs us sy
id
1 0 0 400 189604 39152 496496 0 0 1 2 18 11 0 0
6
0 0 0 400 189604 39152 496496 0 0 0 128 137 138 0 0
100
0 0 1 400 221744 39152 464292 0 0 0 0 873 21399 0 37
63
2 0 0 400 205148 39152 480888 0 0 0 0 2378 56966 0 59
40
0 0 2 400 189536 39152 496496 0 0 0 0 2435 56344 0 59
41
0 0 0 400 189604 39152 496496 0 0 0 0 747 14851 0 10
90
0 0 0 400 189604 39152 496496 0 0 0 125 127 114 0 0
100
--
Nathan Ward
Esphion Ltd.