[OpenAFS-devel] current cvs kernel module crash on 2.6.13-rc6
Martin MOKREJŠ
mmokrejs@ribosome.natur.cuni.cz
Wed, 17 Aug 2005 13:35:37 +0200
Hi,
I'm testing current cvs version on linux-2.6.13-rc6 machine with 40GB cache
partition, default "afsd -nosettime" (but with patches from Niklas and Jim) and
# cat /usr/vice/etc/cacheinfo
/afs:/usr/vice/cache:50000
#
I tried to copy 17GB large file from local xfs partition to the ext2 based /vicepa.
Cache is also ext2, btw. The machine has serious problems with interactivity,
the mouse movements stop once in a second or so, probably as a result of the
high context switching? How can I improve the performance?
# fs getcacheparms
AFS using 44060 of the cache's available 50000 1K byte blocks.
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 261G 210G 51G 81% /
udev 1.5G 216K 1.5G 1% /dev
/dev/sda3 12G 2.8G 8.7G 25% /usr/portage
/dev/sdb1 37G 44M 35G 1% /usr/vice/cache
/dev/sdb2 37G 3.0G 32G 9% /vicepa
none 1.5G 0 1.5G 0% /dev/shm
AFS 261G 210G 51G 81% /afs
/dev/hda 670M 670M 0 100% /mnt/cdrom
#
Anyway, so here's what was going on when copying the 17GB file.
# vmstat 1
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 1 0 2135700 8276 680648 0 0 499 25 373 236 2 2 93 3
1 0 0 2082504 8352 732800 0 0 28288 21 484 462 1 54 0 45
1 0 0 2041948 8384 772112 0 0 4736 0 502 22645 4 92 0 4
1 0 0 1939648 8416 872464 0 0 33280 0 923 23195 1 87 0 12
1 0 0 1837100 8448 972816 0 0 33280 0 905 23292 4 86 0 10
1 0 0 1734552 8480 1073168 0 0 33280 0 914 23345 4 85 0 11
0 1 0 1632872 8516 1172520 0 0 33284 24856 972 22643 3 87 0 11
1 0 0 1529208 8552 1273872 0 0 33280 37340 1020 24338 3 86 0 10
1 0 0 1426784 8584 1374224 0 0 33280 33192 990 22727 3 87 0 11
1 0 0 1324236 8616 1474576 0 0 33280 37348 1010 23562 2 88 0 10
0 1 0 1226896 8648 1569700 0 0 33284 24892 908 19647 3 85 0 11
1 0 0 1164524 8652 1631048 0 0 29568 12452 538 4215 1 82 0 17
1 0 0 1118644 8684 1675280 0 0 3712 34200 740 23079 5 94 0 1
1 0 0 1016220 8716 1775632 0 0 33280 53884 1039 23539 3 86 0 11
1 1 0 913176 8748 1875984 0 0 33280 52188 1021 23543 3 86 0 11
1 0 0 864444 8748 1924368 0 0 24320 1000 665 419 0 50 0 50
0 1 0 820920 8776 1966884 0 0 8964 12436 462 16368 2 93 0 5
2 1 0 770800 8788 2015048 0 0 21120 20728 488 7154 5 81 0 14
1 0 0 708180 8820 2076688 0 0 12160 33160 682 23184 4 92 0 4
1 0 0 605384 8852 2177040 0 0 33280 33812 1166 23540 3 87 0 11
...
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 0 85160 7340 2669712 0 0 16672 785 759 1160 21 50 0 29
3 0 124 85284 7368 2668900 0 0 16644 33160 1093 23053 3 93 0 4
0 1 192 85408 7336 2669040 0 0 26496 8292 518 765 20 57 0 23
3 1 196 85316 7340 2669176 0 0 6784 29584 1219 22213 3 95 0 2
1 0 196 85564 7312 2669468 0 0 32256 200 816 975 3 64 0 33
6 0 196 85564 7344 2669364 0 0 1024 33160 773 22118 4 95 0 0
1 0 196 85284 7308 2670060 0 0 25304 4 473 834 21 53 0 26
4 0 196 85160 7328 2669984 0 0 8064 33160 1369 22442 3 94 0 3
2 0 196 85440 7296 2669644 0 0 32768 0 611 1113 6 69 0 25
3 1 196 85904 7304 2669276 0 0 772 33172 1365 22705 9 90 0 0
1 0 196 92724 7300 2662068 0 0 33024 0 675 1054 3 64 0 33
4 0 196 85284 7328 2669592 0 0 24 24872 1073 22393 4 96 0 0
5 0 196 85160 7300 2669924 0 0 31104 8292 612 978 5 63 0 32
4 0 196 85532 7328 2669528 0 0 2416 29025 1325 22574 5 92 0 3
1 1 196 85532 7304 2669328 0 0 13460 4148 693 732 38 34 0 28
2 1 196 83828 7324 2669924 0 0 20236 59703 1112 21674 3 90 0 7
2 1 196 85688 7304 2669516 0 0 18432 8 541 2049 2 52 0 46
2 0 196 85472 7332 2669592 0 0 14848 29048 926 22828 2 94 0 4
2 1 196 86960 7324 2668268 0 0 33280 12444 539 772 1 66 0 33
1 0 196 85100 7352 2669812 0 0 8 25412 721 22495 3 97 0 0
I cancelled the cp(1) process and as I'm nasty user, I tried:
# umount /afs
Segmentation fault
#
inode freed while on LRU------------[ cut here ]------------
kernel BUG at /scratch/openafs/src/libafs/MODLOAD-2.6.13-rc6-SP/rx_kcommon.c:131!
invalid operand: 0000 [#1]
PREEMPT DEBUG_PAGEALLOC
Modules linked in: libafs radeon drm reiserfs snd_rtctimer snd_seq_virmidi snd_seq_midi snd_rawmidi snd_intel8x0 snd_ac97_codec snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_pcm snd_timer snd_page_alloc snd_mixer_oss snd uhci_hcd ohci_hcd ehci_hcd intel_agp agpgart
CPU: 0
EIP: 0060:[<fac97c8e>] Tainted: P VLI
EFLAGS: 00010282 (2.6.13-rc6)
EIP is at osi_Panic+0x28/0x36 [libafs]
eax: 0000001b ebx: ce6bcdf0 ecx: 00000000 edx: 00000001
esi: ce6bcf14 edi: ce6bcdf0 ebp: dbd71e9c esp: dbd71e88
ds: 007b es: 007b ss: 0068
Process umount (pid: 13099, threadinfo=dbd70000 task=e6ae1b10)
Stack: facbcce0 faca4d88 00000000 ce6bcdf0 ce6bcdf0 dbd71ea8 faca4dac ce6bcdf0
dbd71ebc c01740db cdb67df0 ce6bcdf0 cad6fdf8 dbd71ecc c017521a ce6bcdf0
ce6bcdf0 dbd71ed4 c0175266 dbd71ee0 c01752be dbd70000 dbd71ef0 faca6dab
Call Trace:
[<c0103dd4>] show_stack+0x7a/0x90
[<c0103f59>] show_registers+0x156/0x1ce
[<c0104167>] die+0xf4/0x17e
[<c0104272>] do_trap+0x81/0xb8
[<c010454c>] do_invalid_op+0xa3/0xad
[<c0103a33>] error_code+0x4f/0x54
[<faca4dac>] afs_clear_inode+0x24/0x3e [libafs]
[<c01740db>] clear_inode+0xc7/0xc9
[<c017521a>] generic_forget_inode+0x145/0x17b
[<c0175266>] generic_drop_inode+0x16/0x18
[<c01752be>] iput+0x56/0x69
[<faca6dab>] afs_dentry_iput+0x7a/0x98 [libafs]
[<c0171d9a>] dput+0x168/0x267
[<c01617a2>] generic_shutdown_super+0x39/0x150
[<c0162258>] kill_anon_super+0xc/0x4b
[<c016165e>] deactivate_super+0x65/0x85
[<c0176aef>] __mntput+0x28/0x33
[<c0168847>] path_release_on_umount+0x28/0x2b
[<c01770b0>] sys_umount+0x37/0x76
[<c0177108>] sys_oldumount+0x19/0x1b
[<c0102f2f>] sysenter_past_esp+0x54/0x75
Code: ff 5d c3 55 89 e5 53 bb 10 c2 cb fa 83 ec 10 85 c0 0f 44 c3 8b 5d 08 89 4c 24 08 89 5c 24 0c 89 54 24 04 89 04 24 e8 e7 7d 48 c5 <0f> 0b 83 00 68 95 cb fa 83 c4 10 5b 5d c3 55 83 fa 01 89 e5 57