[OpenAFS] Re: question about error on i686 arch

[Andrew Deason]

On Thu, 2 Aug 2018 18:03:45 -0400
Gary Gatling <gsgatlin@ncsu.edu> wrote:

> Anyone ever see this error before?
> 
> /var/lib/dkms/openafs/1.8.1/build/src/libafs/MODLOAD-4.17.9-100.fc27.i686-MP/afs_dcache.c:2819!
> [libafs]

If I use memcache on a similar system, I see the same thing. I also see
the following messages in dmesg, and I assume you'll find similar
messages if you look around:

Aug 03 17:19:14 fed-1 kernel: Starting AFS cache scan...
Aug 03 17:19:14 fed-1 kernel: vmap allocation for size 69632 failed: use vmalloc=<size> to increase size
Aug 03 17:19:14 fed-1 kernel: afsd: vmalloc: allocation failure: 65536 bytes, mode:0x14000c0(GFP_KERNEL), nodemask=(null)
[...]
Aug 03 17:19:24 fed-1 kernel: afs_osi_Alloc: Can't vmalloc 65536 bytes.
Aug 03 17:19:24 fed-1 kernel: afsd:  memCache allocation failure at 98688 KB.
Aug 03 17:19:24 fed-1 kernel: afsd: memory cache too large for available memory.
Aug 03 17:19:24 fed-1 kernel: afsd: AFS files cannot be accessed.

If you search for that "use vmalloc=<size> to increase size" message,
you'll find some pages that explain a little what's going on, and how to
work around it (it's not specific to openafs). But basically, you don't
have enough memory for a memcache of that size; we ran out of memory
while allocating the memcache entries.

Your system may have more physical ram than the given memcache size
(e.g. your cache may be 100M, but your machine may have 1G of ram), but
Linux restricts how much of this flavor of memory we can use (vmalloc'd
memory). On amd64, these limits are much higher, so you're less likely
to see it there. To work around this, you can raise the vmalloc limit by
passing vmalloc=<size> on the kernel command line when you boot.

I haven't looked too deep in the details, but I think it's somewhat
difficult for us to avoid this issue entirely, given the way that
memcache works on Linux. We currently of course handle the error
terribly and we'll work on not panicing in this situation, but we may
not be able to avoid the error itself.

So to avoid the error, either use vmalloc=<size> as mentioned above, or
lower your cache size, or ideally, stop using memcache. memcache is used
way less than disk cache and is probably way less stable. If you really
feel you need to avoid hitting the disk, you can mount tmpfs on
/usr/vice/cache. iirc that's less memory-efficient, but it tends to work
better.

-- 
Andrew Deason
adeason@sinenomine.net
(please CC me directly on replies; I don't read the list)