[OpenAFS] Re: [OpenAFS] [Q] Funny Traffic ...

Mon, 31 Mar 2003 12:11:27 -0800

Hi,

About the 256k(18) chunksize, do you have a "-dcache <size>" set also?=20
According to the afsd man page, when you're using a memcache you override=
 the=20
cacheinfo file if you specify both dcache and chunksize. In this case, th=
e=20
size of the cache is dcache*chunksize, which if you're using a large dcac=
he=20
might be why your kernel won't load with the large chunksize value. You m=
ay=20
have inadvertantly specified a cache larger than available memory.

To get a 50MB memory cache with chunksize set to 18 for example, you'd wa=
nt to=20
specify a dcache size of 200. I was recently setting up an embedded syste=
m=20
using memcache (it's not good to put the cache on flash memory...) and=20
realized I was doing precisely this.

Regards,

Lester Barrows
Asani Solutions, LLC
Code IC Systems Group
NASA Ames Research Center
Voice: 650-604-2639

On Sunday 30 March 2003 08:09 pm, S.J.Chun wrote:
> Thanks for your help and sorry for late reply. I've configured with ver=
y
> small size of memory cache, 50MB of memory cache with default chunk
> size.(This is because with chunksize of 256K(18) the kernel cannot load=
 afs
> client so I did not tried on this more) The peformance seems improved v=
ery
> much. But I cannot be sure the client uses its traffic(100Mb) fully. An=
d
> the server's load is below 20~30%.
>
> Another strange thing: why I cannot kill the process if the process is =
in
> the DW state(I think it is wait until the afs client can give required =
data
> for it. With the current configuration(2 Pentium III with 1Gb of memory=
,
> the maximum client(the real client, not afs client) is 100 concurrent
> clients. More than this causes queuing and the service seems almost
> stopped.
>
> Thanks again for your help and sorry for my poor english.
>
>   ----- Original Message -----
>   From: "Dexter \"Kim\" Kimball" <dhk@ccre.com>
>   To: Sungjin Chun <chunsj@workspace.buchun.thrunet.com>
>   Cc: openafs-info@openafs.org
>   Sent: Thu, 27 Mar 2003 12:21:55 -0700
>   Subject: Re: [OpenAFS] [Q] Funny Traffic ...
>
>   Hello again,
>
>   You'll want to look at the value "cacheBlocksInUse" and the value
>   "cacheBlocksTotal."
>
>   Set the refresh time for afsmonitor to 3 seconds or so.
>
>   When the cache appears to be almost full (cacheBlocksInUse approaches
>   cacheBlocksTotal), the number of dcache items in use
>   will begin to drop -- meaning that the cache manager is beginning to =
free
> up some cache space.
>
>   It's at this point I experienced the "hesitation," with very large ca=
ches
>   and very small data files (not quite the same setup as yours).
>
>   You're also interested in "cacheCurrDirtyChunks."  Before dirty chunk=
s
> are garbage collected, they have to be pushed back to the fileserver.  =
It
> would take a lot of dirty chunks to affect garbage-collection time (the=
y're
> ignored when cache space is reclaimed, unless there are so many that th=
ey
> _must_ be freed up), but it shouldn't be overlooked.
>
>   If you don't see this behavior (cacheBlocksInUse approaches
>   cacheBlocksTotal, and then reduction in cacheBlocksInUse) let me know=
=2E
>   There are other things that will trigger cache purges -- like running=
 out
> of Vfiles -- that will cause a cache purge to take place even though yo=
u've
> got lots of cache (disk) space left.
>
>   WRT to chunk size, a client was looking for performance improvements =
and
>   experimented with 1 MB chunk size.  In their environment (AFS on a Cr=
ay,
>   CDDI to every user's desktop, several hundred megabyte data files) th=
ey
> saw no improvement over the 64K default. Then again, they had a lot of
> hardware horsepower at the time, had the Cray set up to run multiple
> instances of the OS, and ran AFS server software for each instance of t=
he
> OS.
>
>   According to the IBM AFS Administration Guide for AFS 3.6, setting
>   the -chunksize argument to 0 or setting it to anything greater than 3=
0
> will cause afsd to set default chunksize.
>
>   In other words, valid values are 1-30, unless you want to reset to th=
e
>   default of 64K.
>
>   Chunk size is set as 2 to the power of -chunksize.  That is, if you
> specify "-chunksize 2" you'll get a chunksize of 4 bytes, and "-chunksi=
ze
> 16" will get you a 64 K chunk size, which is the default.
>
>   2E30 or 1.073741824e9  (a gigabyte) is the largest chunk size permitt=
ed
> by IBM/Transarc AFS 3.6.
>
>   So "-chunksize 30" should give a one gigabyte chunk size.
>
>   And "-chunksize 21" will give you a 2MB chunk size.
>
>   "-chunksize 20" will give you a 1 MB chunk size
>
>   and your chunksize value, to get the 256K chunks, must be 18, right?
>
>   Probably more than you wanted/needed to know.
>
>   Let me know what works in your circumstances, please.  I run into thi=
s
>   question frequently, and am way out of date on results.
>
>   It's been several years since I fooled around with any of this, and
> current hardware speeds may invalidate what I found then.
>
>
>   Kim
>
>   ---------------------------------------------------
>   Dexter "Kim" Kimball
>   CCRE, Inc.
>   dhk@ccre.com
>   P970.207.1525
>   C970.215.6359
>   F970.207.1474
>   ---------------------------------------------------
>
>
>   ----- Original Message -----
>   From: "Sungjin Chun" <chunsj@workspace.buchun.thrunet.com>
>   To: "Dexter Kim Kimball" <dhk@ccre.com>
>   Cc: <openafs-info@openafs.org>
>   Sent: Wednesday, March 26, 2003 7:11 AM
>   Subject: Re: [OpenAFS] [Q] Funny Traffic ...
>
>   > Thanks for your help, here comes another question :-)
>   >
>   > How can I know that the "hesitation" occurs using afsmonitor, what
>   > value should I consult? And current chunksize is 256K, if bugger on=
e
>   > shows
>
>   better
>
>   > performance, what is the limit ofr chunksize?
>   >
>   > Thanks in advance.
>   >
>   > On Tue, Mar 25, 2003 at 08:31:44AM -0700, Dexter Kim Kimball wrote:
>   > > Use afsmonitor to look at your cache behavior.
>   > >
>   > > You may find that the "hesitations" occur when the cache is purgi=
ng.
>   > >
>   > > If so, reduce cache size and repeat benchmarking until the
>   > > hesitations
>
>   go
>
>   > > away.
>   > >
>   > > With the large files you describe, I'd also look at changing the
>   > > default chunksize (look at arguments to "afsd") from 64K to somet=
hing
>   > > much
>
>   larger.
>
>   > > Kim
>   > > ---------------------------------------------------
>   > > Dexter "Kim" Kimball
>   > > CCRE, Inc.
>   > > dhk@ccre.com
>   > >
>   > >
>   > > ----- Original Message -----
>   > > From: "S.J.Chun" <chunsj@embian.com>
>   > > To: <openafs-info@openafs.org>
>   > > Sent: Tuesday, March 25, 2003 8:18 AM
>   > > Subject: [OpenAFS] [Q] Funny Traffic ...
>   > >
>   > > > Hi,
>   > > >
>   > > > We are using one OpenAFS server and one OpenAFS client, through=
 the
>
>   client
>
>   > > we are serving files(
>   > >
>   > > > rather big sized files, normally 600~800MB) using apache http
>   > > > server.
>
>   Now,
>
>   > > we have BIG performance
>   > >
>   > > > problem. We've chosen AFS for its high scalability but the
>   > > > performance
>
>   is
>
>   > > very bad. We are using
>   > >
>   > > > 100M dedicated line and expected the download speed as up to 80=
0K
>   > > > or
>   > >
>   > > 1,000K through our cable
>   > >
>   > > > or ADSL line, but only got 40 ~ 80K. What is the best configura=
tion
>
>   for
>
>   > > our purpose?
>   > >
>   > > > And if I try to get file using ftp, the data transfer is rather
>   > > > funny;
>
>   it
>
>   > > gets 256K of data, then waits some
>   > >
>   > > > times(several seconds) then again gets another 256K of data. Wh=
y
>   > > > the
>
>   data
>
>   > > transfer is not continuous?
>   > >
>   > > > Misconfiguration?
>   > > >
>   > > > The server has 2 TB of storage and the client has 9GB of cache.=
 And
>   > > > OS
>
>   is
>
>   > > Linux.
>   > >
>   > > > Thanks in advance.
>   > > > _______________________________________________
>   > > > OpenAFS-info mailing list
>   > > > OpenAFS-info@openafs.org
>   > > > https://lists.openafs.org/mailman/listinfo/openafs-info
>   > >
>   > > ----- Original Message -----
>   > > From: "S.J.Chun" <chunsj@embian.com>
>   > > To: <openafs-info@openafs.org>
>   > > Sent: Tuesday, March 25, 2003 8:18 AM
>   > > Subject: [OpenAFS] [Q] Funny Traffic ...
>   > >
>   > > > Hi,
>   > > >
>   > > > We are using one OpenAFS server and one OpenAFS client, through=
 the
>
>   client
>
>   > > we are serving files(
>   > >
>   > > > rather big sized files, normally 600~800MB) using apache http
>   > > > server.
>
>   Now,
>
>   > > we have BIG performance
>   > >
>   > > > problem. We've chosen AFS for its high scalability but the
>   > > > performance
>
>   is
>
>   > > very bad. We are using
>   > >
>   > > > 100M dedicated line and expected the download speed as up to 80=
0K
>   > > > or
>   > >
>   > > 1,000K through our cable
>   > >
>   > > > or ADSL line, but only got 40 ~ 80K. What is the best configura=
tion
>
>   for
>
>   > > our purpose?
>   > >
>   > > > And if I try to get file using ftp, the data transfer is rather
>   > > > funny;
>
>   it
>
>   > > gets 256K of data, then waits some
>   > >
>   > > > times(several seconds) then again gets another 256K of data. Wh=
y
>   > > > the
>
>   data
>
>   > > transfer is not continuous?
>   > >
>   > > > Misconfiguration?
>   > > >
>   > > > The server has 2 TB of storage and the client has 9GB of cache.=
 And
>   > > > OS
>
>   is
>
>   > > Linux.
>   > >
>   > > > Thanks in advance.
>   > > > _______________________________________________
>   > > > OpenAFS-info mailing list
>   > > > OpenAFS-info@openafs.org
>   > > > https://lists.openafs.org/mailman/listinfo/openafs-info
>
>   _______________________________________________
>   OpenAFS-info mailing list
>   OpenAFS-info@openafs.org
>   https://lists.openafs.org/mailman/listinfo/openafs-info
>
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info