[OpenAFS] Re: [OpenAFS] [Q] Funny Traffic ...
Dexter "Kim" Kimball
Dexter \"Kim\" Kimball" <dhk@ccre.com
Sat, 5 Apr 2003 20:15:41 -0700
I'm quite pleased that someone else has set memcache size such that it
consumes all of available memory and wants more.
My HP box appeared to be quite dead, though I was able to access it (slowly)
via telnet or rlogin or something.
Took a minute to figure it out. It was definitely amusing.
Kim
----- Original Message -----
From: "Lester Barrows" <barrows@email.arc.nasa.gov>
To: <openafs-info@openafs.org>
Sent: Monday, March 31, 2003 1:11 PM
Subject: Re: [OpenAFS] Re: [OpenAFS] [Q] Funny Traffic ...
Hi,
About the 256k(18) chunksize, do you have a "-dcache <size>" set also?
According to the afsd man page, when you're using a memcache you override
the
cacheinfo file if you specify both dcache and chunksize. In this case, the
size of the cache is dcache*chunksize, which if you're using a large dcache
might be why your kernel won't load with the large chunksize value. You may
have inadvertantly specified a cache larger than available memory.
To get a 50MB memory cache with chunksize set to 18 for example, you'd want
to
specify a dcache size of 200. I was recently setting up an embedded system
using memcache (it's not good to put the cache on flash memory...) and
realized I was doing precisely this.
Regards,
Lester Barrows
Asani Solutions, LLC
Code IC Systems Group
NASA Ames Research Center
Voice: 650-604-2639
On Sunday 30 March 2003 08:09 pm, S.J.Chun wrote:
> Thanks for your help and sorry for late reply. I've configured with very
> small size of memory cache, 50MB of memory cache with default chunk
> size.(This is because with chunksize of 256K(18) the kernel cannot load
afs
> client so I did not tried on this more) The peformance seems improved very
> much. But I cannot be sure the client uses its traffic(100Mb) fully. And
> the server's load is below 20~30%.
>
> Another strange thing: why I cannot kill the process if the process is in
> the DW state(I think it is wait until the afs client can give required
data
> for it. With the current configuration(2 Pentium III with 1Gb of memory,
> the maximum client(the real client, not afs client) is 100 concurrent
> clients. More than this causes queuing and the service seems almost
> stopped.
>
> Thanks again for your help and sorry for my poor english.
>
> ----- Original Message -----
> From: "Dexter \"Kim\" Kimball" <dhk@ccre.com>
> To: Sungjin Chun <chunsj@workspace.buchun.thrunet.com>
> Cc: openafs-info@openafs.org
> Sent: Thu, 27 Mar 2003 12:21:55 -0700
> Subject: Re: [OpenAFS] [Q] Funny Traffic ...
>
> Hello again,
>
> You'll want to look at the value "cacheBlocksInUse" and the value
> "cacheBlocksTotal."
>
> Set the refresh time for afsmonitor to 3 seconds or so.
>
> When the cache appears to be almost full (cacheBlocksInUse approaches
> cacheBlocksTotal), the number of dcache items in use
> will begin to drop -- meaning that the cache manager is beginning to
free
> up some cache space.
>
> It's at this point I experienced the "hesitation," with very large
caches
> and very small data files (not quite the same setup as yours).
>
> You're also interested in "cacheCurrDirtyChunks." Before dirty chunks
> are garbage collected, they have to be pushed back to the fileserver. It
> would take a lot of dirty chunks to affect garbage-collection time
(they're
> ignored when cache space is reclaimed, unless there are so many that they
> _must_ be freed up), but it shouldn't be overlooked.
>
> If you don't see this behavior (cacheBlocksInUse approaches
> cacheBlocksTotal, and then reduction in cacheBlocksInUse) let me know.
> There are other things that will trigger cache purges -- like running
out
> of Vfiles -- that will cause a cache purge to take place even though
you've
> got lots of cache (disk) space left.
>
> WRT to chunk size, a client was looking for performance improvements and
> experimented with 1 MB chunk size. In their environment (AFS on a Cray,
> CDDI to every user's desktop, several hundred megabyte data files) they
> saw no improvement over the 64K default. Then again, they had a lot of
> hardware horsepower at the time, had the Cray set up to run multiple
> instances of the OS, and ran AFS server software for each instance of the
> OS.
>
> According to the IBM AFS Administration Guide for AFS 3.6, setting
> the -chunksize argument to 0 or setting it to anything greater than 30
> will cause afsd to set default chunksize.
>
> In other words, valid values are 1-30, unless you want to reset to the
> default of 64K.
>
> Chunk size is set as 2 to the power of -chunksize. That is, if you
> specify "-chunksize 2" you'll get a chunksize of 4 bytes, and "-chunksize
> 16" will get you a 64 K chunk size, which is the default.
>
> 2E30 or 1.073741824e9 (a gigabyte) is the largest chunk size permitted
> by IBM/Transarc AFS 3.6.
>
> So "-chunksize 30" should give a one gigabyte chunk size.
>
> And "-chunksize 21" will give you a 2MB chunk size.
>
> "-chunksize 20" will give you a 1 MB chunk size
>
> and your chunksize value, to get the 256K chunks, must be 18, right?
>
> Probably more than you wanted/needed to know.
>
> Let me know what works in your circumstances, please. I run into this
> question frequently, and am way out of date on results.
>
> It's been several years since I fooled around with any of this, and
> current hardware speeds may invalidate what I found then.
>
>
> Kim
>
> ---------------------------------------------------
> Dexter "Kim" Kimball
> CCRE, Inc.
> dhk@ccre.com
> P970.207.1525
> C970.215.6359
> F970.207.1474
> ---------------------------------------------------
>
>
> ----- Original Message -----
> From: "Sungjin Chun" <chunsj@workspace.buchun.thrunet.com>
> To: "Dexter Kim Kimball" <dhk@ccre.com>
> Cc: <openafs-info@openafs.org>
> Sent: Wednesday, March 26, 2003 7:11 AM
> Subject: Re: [OpenAFS] [Q] Funny Traffic ...
>
> > Thanks for your help, here comes another question :-)
> >
> > How can I know that the "hesitation" occurs using afsmonitor, what
> > value should I consult? And current chunksize is 256K, if bugger one
> > shows
>
> better
>
> > performance, what is the limit ofr chunksize?
> >
> > Thanks in advance.
> >
> > On Tue, Mar 25, 2003 at 08:31:44AM -0700, Dexter Kim Kimball wrote:
> > > Use afsmonitor to look at your cache behavior.
> > >
> > > You may find that the "hesitations" occur when the cache is purging.
> > >
> > > If so, reduce cache size and repeat benchmarking until the
> > > hesitations
>
> go
>
> > > away.
> > >
> > > With the large files you describe, I'd also look at changing the
> > > default chunksize (look at arguments to "afsd") from 64K to
something
> > > much
>
> larger.
>
> > > Kim
> > > ---------------------------------------------------
> > > Dexter "Kim" Kimball
> > > CCRE, Inc.
> > > dhk@ccre.com
> > >
> > >
> > > ----- Original Message -----
> > > From: "S.J.Chun" <chunsj@embian.com>
> > > To: <openafs-info@openafs.org>
> > > Sent: Tuesday, March 25, 2003 8:18 AM
> > > Subject: [OpenAFS] [Q] Funny Traffic ...
> > >
> > > > Hi,
> > > >
> > > > We are using one OpenAFS server and one OpenAFS client, through
the
>
> client
>
> > > we are serving files(
> > >
> > > > rather big sized files, normally 600~800MB) using apache http
> > > > server.
>
> Now,
>
> > > we have BIG performance
> > >
> > > > problem. We've chosen AFS for its high scalability but the
> > > > performance
>
> is
>
> > > very bad. We are using
> > >
> > > > 100M dedicated line and expected the download speed as up to 800K
> > > > or
> > >
> > > 1,000K through our cable
> > >
> > > > or ADSL line, but only got 40 ~ 80K. What is the best
configuration
>
> for
>
> > > our purpose?
> > >
> > > > And if I try to get file using ftp, the data transfer is rather
> > > > funny;
>
> it
>
> > > gets 256K of data, then waits some
> > >
> > > > times(several seconds) then again gets another 256K of data. Why
> > > > the
>
> data
>
> > > transfer is not continuous?
> > >
> > > > Misconfiguration?
> > > >
> > > > The server has 2 TB of storage and the client has 9GB of cache.
And
> > > > OS
>
> is
>
> > > Linux.
> > >
> > > > Thanks in advance.
> > > > _______________________________________________
> > > > OpenAFS-info mailing list
> > > > OpenAFS-info@openafs.org
> > > > https://lists.openafs.org/mailman/listinfo/openafs-info
> > >
> > > ----- Original Message -----
> > > From: "S.J.Chun" <chunsj@embian.com>
> > > To: <openafs-info@openafs.org>
> > > Sent: Tuesday, March 25, 2003 8:18 AM
> > > Subject: [OpenAFS] [Q] Funny Traffic ...
> > >
> > > > Hi,
> > > >
> > > > We are using one OpenAFS server and one OpenAFS client, through
the
>
> client
>
> > > we are serving files(
> > >
> > > > rather big sized files, normally 600~800MB) using apache http
> > > > server.
>
> Now,
>
> > > we have BIG performance
> > >
> > > > problem. We've chosen AFS for its high scalability but the
> > > > performance
>
> is
>
> > > very bad. We are using
> > >
> > > > 100M dedicated line and expected the download speed as up to 800K
> > > > or
> > >
> > > 1,000K through our cable
> > >
> > > > or ADSL line, but only got 40 ~ 80K. What is the best
configuration
>
> for
>
> > > our purpose?
> > >
> > > > And if I try to get file using ftp, the data transfer is rather
> > > > funny;
>
> it
>
> > > gets 256K of data, then waits some
> > >
> > > > times(several seconds) then again gets another 256K of data. Why
> > > > the
>
> data
>
> > > transfer is not continuous?
> > >
> > > > Misconfiguration?
> > > >
> > > > The server has 2 TB of storage and the client has 9GB of cache.
And
> > > > OS
>
> is
>
> > > Linux.
> > >
> > > > Thanks in advance.
> > > > _______________________________________________
> > > > OpenAFS-info mailing list
> > > > OpenAFS-info@openafs.org
> > > > https://lists.openafs.org/mailman/listinfo/openafs-info
>
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info
>
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info
_______________________________________________
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info