[OpenAFS] Cache Consistency problem
Paul Blackburn
mpb@est.ibm.com
Tue, 17 Dec 2002 09:44:41 +0000
This is a multi-part message in MIME format.
--------------070700050307010708060200
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit
Kağan Coşkun wrote:
> Hi All,
>
> We are having some cache consistency problems. Sometimes the afs
> client on our web server does not represent the actual files on the
> some volumes. That is when someone uploads a new file to his directory
> the file is not noticed on the client side. The volume is a read/write
> volume. And the problem can be overcomed by creating a file on the
> volume using the client and also flush commands work too.
>
> And another point actual cache size is bigger then the defined cache
> size in /usr/vice/etc/cacheinfo. It is 100MB.
>
> [root@cs kaan]# du /usr/vice/cache/ -h
> 32k /usr/vice/cache/D0
> 25M /usr/vice/cache/D1
> 54M /usr/vice/cache/D2
> 53M /usr/vice/cache/D3
> 44M /usr/vice/cache/D4
> 175M /usr/vice/cache
>
>
> Our client is openafs-client-1.2.3-rh7.2.2.
>
> Any ideas to track the problem?
>
> Kaan
>
> _________________________________________________________________
> Protect your PC - get McAfee.com VirusScan Online
> http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963
>
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info
Hello Kağan,
The method that AFS uses to maintain cache consistency is called "Callback".
"Callback" is a promise from an AFS fileserver to an AFS client that
if the server-side copy of a file is updated then the callback will be
"broken".
The way that a client notices a broken Callback is when the file is closed
and then the same file is read again.
The AFS Cache Manager (afsd) will then "see" the Callback is broken
and re-fetch the file from the file server (instead of from the local
cache).
Therefore, it is possible for a server-side copy of a file to be different
from a client-side copy if the client-side has not re-read the changed file.
Normal behaviour on the client-side is for files to be opened, read and
closed
so that when they are again opened and read the new server copy will be
seen.
Regarding the size of the AFS cache: it is recommended that the value
for cache size defined in the cacheinfo file is no larger than about 84%
of the size of the actual cache size. So, in your 100mb cacheinfo size
example, the optimum size for the cache would be 125mb. However,
you can safely have less than 84% and your figures of 80/175 give 45%.
On the subject of disk cache, it is a useful to make a separate local
filesystem (typically "/usr/vice/cache"). Then the optimum size in
cacheinfo will be 84% the size of this filesystem.
Example:
$ cat /usr/vice/etc/cacheinfo
/afs:/usr/vice/cache:55969
$ df /usr/vice/cache
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/hda8 65846 50742 11704 82% /usr/vice/cache
$ echo "$(compute 100*55969/65846)% of $(compute 65846*k/m) mb"
84.99% of 64.30 mb
$ # Utilization of cache:
$ echo "$(compute 100*50742/55969)%"
90.66%
The advantages of having a separate filesystem for /usr/vice/cache/ are:
a) Only the AFS Cache Manager will use the disk space
in /usr/vice/cache/ this eliminates the problem of not
having enough space in the /usr/ filesystem if some other
process floods /usr/.
b) You can measure the utilization of the cache as a percentage
of disk blocks used versus cacheinfo defined size. (also, use "df")
c) You can specify a different filesystem type for /usr/vice/cache/
than is used for /usr/.
Earlier discussions here on the openafs-info mailing list
suggest it is better to use ext2 for the AFS disk cache.
You may choose to use ext3 for /usr/.
FWIW, I have found no problems using ext3 for /usr/vice/cache.
The other point to make is that you appear to be using AFS 1.2.3
and you may like to consider upgrading to the current 1.2.7 version
(I understand 1.2.8 is being worked on now).
--
cheers
paul http://acm.org/~mpb
--------------070700050307010708060200
Content-Type: text/plain;
name="compute"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="compute"
#!/bin/sh -
#
# $Header:
# $Locker:
# ---------------------------------------------------------------------------
#
# NAME compute
# AUTHOR Paul Blackburn, AIXSSC IBM UK http://acm.org/~mpb
# DATE Thu Jul 04 15:06:24 BST 1991
# PURPOSE compute using 'bc' with predefined values of:
# k(1024), b(512), m(k*k), p(4*m), and scale=2
# BENEFIT
# Why is "compute" useful. It is very handy for command-line
# calculations. For example: how many megabytes is 3200
# 512-byte blocks? answer: compute 3200*b/m -> 1.56
#
# ---------------------------------------------------------------------------
CMD=`basename ${0}`
usage() {
cat <<eeooff >&2
Usage: ${CMD} "calculation"
compute "calculation" using 'bc' with predefined values of:
k(1024), b(512), m(k*k), p(4*m), and scale=2
for example, how many (512 byte) blocks in 20 (4*m) physical partions?
$ compute "20*p/b"
163840.00
NB: "calculation" should be in double-quotes to avoid shell evaluation (of *).
(exception: where "calculation" is not matched as a filename by the shell)
eeooff
}
error() {
echo "${CMD} error: ${1}" >&2
}
if [ -z "${1}" ]; then
usage
exit 1
fi
echo "k=1024;b=512;m=k*k;p=4*m;scale=2;" "$*" | bc
--------------070700050307010708060200--