[OpenAFS] Possible RO volume corruption, AFS 1.4.1 on Solaris 8

Thu, 03 Aug 2006 20:07:20 -0400

This is a cryptographically signed message in MIME format.

--------------ms040607000308080706070305
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

This is fixed in 1.4.2-beta1 and above.

Jeffrey Altman

Kevin Hildebrand wrote:
> 
> Ok, here's some more info- I added some debugging code to the fileserver
> at each of the calls to VTakeOffline that didn't already have it.  The
> volume is being taken offline because rx_WritevAlloc is failing inside
> FetchData_RXStyle.  Relevant logs below:
> 
> Thu Aug  3 19:15:36 2006 [102] FindClient: authenticating connection:
> authClass=0
> Thu Aug  3 19:15:36 2006 [102] WhoAreYou success on 128.8.111.219:7001
> Thu Aug  3 19:15:36 2006 [102] InitCallBackState3 success on
> 128.8.111.219:7001
> Thu Aug  3 19:15:36 2006 [102] SAFS_FetchStatus,  Fid = 1970897351.1.1,
> Host 128
> .8.111.219:7001, Id 32766
> Thu Aug  3 19:15:36 2006 [102] SAFS_FetchStatus returns 0
> Thu Aug  3 19:15:36 2006 [102] SAFS_FetchStatus,  Fid =
> 1970897351.1348.1070438,
>  Host 128.8.111.206:7001, Id 32766
> Thu Aug  3 19:15:36 2006 [102] SAFS_FetchStatus returns 0
> Thu Aug  3 19:15:37 2006 [102] SAFS_FetchStatus,  Fid =
> 1970897351.1348.1070438,
>  Host 128.8.111.206:7001, Id 32766
> Thu Aug  3 19:15:37 2006 [102] SAFS_FetchStatus returns 0
> Thu Aug  3 19:15:37 2006 [102] SRXAFS_FetchData, Fid =
> 1970897351.1348.1070438
> Thu Aug  3 19:15:37 2006 [102] SRXAFS_FetchData, Fid =
> 1970897351.1348.1070438,
> Host 128.8.111.206:7001, Id 32766
> Thu Aug  3 19:15:37 2006 [102] FetchData_RXStyle: Pos 0, Len 524288
> Thu Aug  3 19:15:37 2006 [102] FetchData_RXStyle: file size 1854506
> Thu Aug  3 19:15:37 2006 [102] FetchData_RXStyle failed - rx_WritevAlloc
> returned <= 0
> Thu Aug  3 19:15:37 2006 [102] VOffline: Volume 1970897351
> (s.common.readonly) i
> s now offline
> Thu Aug  3 19:15:37 2006 [102]
> Thu Aug  3 19:15:37 2006 [102] SRXAFS_FetchData returns 5
> 
> Kevin
> 
> On Thu, 3 Aug 2006, Kevin Hildebrand wrote:
> 
>>
>> Hello, we've been having problems recently with one of our volumes
>> having most or all of its RO replications go offline at approximately
>> the same time. The RW volume has remained stable, so it's only the ROs
>> that we're having problems with.
>>
>> This volume is released on an hourly basis, and normally has 3 RO
>> replications.  What's been happening, is that some point in between
>> replications, the volume is taken offline-
>>
>> FileLog:
>> Thu Aug  3 12:46:42 2006 VAttachVolume: volume salvage flag is ON for
>> /vicepc//V1970897351.vol; volume needs salvage
>>
>> VolserLog:
>> Thu Aug  3 12:46:42 2006 VAttachVolume: volume salvage flag is ON for
>> /vicepc/V1970897351.vol; volume needs salvage
>>
>> There is no other relevant entry in the logs as to WHY the volume is
>> being taken offline.  I'll be adding some debug code to the fileserver
>> shortly to see if I can nail down where this is occurring, if no one
>> else has any leads.
>>
>> Here's the volume info-
>>
>> # /usr/afs/bin/volinfo -volumeid 1970897351
>> Inode 219522: Good magic 78a1b2c5 and version 1
>> Inode 219523: Good magic 99776655 and version 1
>> Inode 219524: Good magic 88664433 and version 1
>> Volume header for volume 1970897351 (s.common.readonly)
>> stamp.magic = 78a1b2c5, stamp.version = 1
>> inUse = 0, inService = 1, blessed = 1, needsSalvaged = 1, dontSalvage = 0
>> type = 1 (readonly), uniquifier = 1070251, needsCallback = 0,
>> destroyMe = 0
>> id = 1970897351, parentId = 1970897350, cloneId = 1970897351, backupId
>> = 1970897352, restoredFromId = 0
>> maxquota = 200000, minquota = 0, maxfiles = 0, filecount = 1022,
>> diskused = 125611
>> creationDate = 1154622174 (2006/08/03.12:22:54), copyDate = 1154622174
>> (2006/08/03.12:22:54)
>> backupDate = 1154577821 (2006/08/03.00:03:41), expirationDate = 0
>> (1969/12/31.19:00:00)
>> accessDate = 0 (1969/12/31.19:00:00), updateDate = 1154622150
>> (2006/08/03.12:22:30)
>> owner = 0, accountNumber = 0
>> dayUse = 36575; week = (0, 0, 0, 0, 0, 0, 0), dayUseDate = 1154540160
>> (2006/08/02.13:36:00)
>>
>> Thanks,
>>
>> Kevin Hildebrand
>> University of Maryland, College Park
>> Project Glue
>> _______________________________________________
>> OpenAFS-info mailing list
>> OpenAFS-info@openafs.org
>> https://lists.openafs.org/mailman/listinfo/openafs-info
>>
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info

--------------ms040607000308080706070305
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJeTCC
AxcwggKAoAMCAQICEBW00lKwoWJXt8wbmTl1M0kwDQYJKoZIhvcNAQEEBQAwYjELMAkGA1UE
BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT
I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA2MDUyNzIyMDMzMloX
DTA3MDUyNzIyMDMzMlowczEPMA0GA1UEBBMGQWx0bWFuMRUwEwYDVQQqEwxKZWZmcmV5IEVy
aWMxHDAaBgNVBAMTE0plZmZyZXkgRXJpYyBBbHRtYW4xKzApBgkqhkiG9w0BCQEWHGphbHRt
YW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB
AQC19SD7DncCP/+wfQlLzAAcxf1nJ/7UQgh4o/nxzvuY55XwHdLQjqWuFUnM5vecfyZKwq0o
fGCucDfcQbSIrkhHD9z4TZ8vDaYWVY9Foz8Rp8G0PNdbRFoFtfJbaeVBX5hG3aQXIc/T1b9U
8uN3kLyqXAFIGWKO8DJVGTKKtOiPVOp1U+9CwujyYmUSKF+suutKABhhK1ZGHsTnFczLZ2g0
ma0H7PiFJ2kLfOf///07E1fbr4IRb+cd87kpWLcjtEZ0rbBr9HlOy9dkeEii/qFoo1ahfKCD
A9bNErMiOXA3dudaNNzXlN/70slq5fboBXbepamJGrsnXYcCsS9+LtCTAgMBAAGjOTA3MCcG
A1UdEQQgMB6BHGphbHRtYW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wDAYDVR0TAQH/BAIwADAN
BgkqhkiG9w0BAQQFAAOBgQDBzWhkrW+ol3iyT1rV8ZBQB0+z/6dFH3djQfNf7jDXNoXx4Vbo
pA7BAR4ihAPibv7j7ZaxmyMxWiDACRGS934uvUS0K6L6q14hTWMostJgFsAEDArrmbrES03v
L3EVETiGFqTB2sLp5MLc6+z+72pLXRuDPL3lO2GOQuBbILswRzCCAxcwggKAoAMCAQICEBW0
0lKwoWJXt8wbmTl1M0kwDQYJKoZIhvcNAQEEBQAwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoT
HFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25h
bCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA2MDUyNzIyMDMzMloXDTA3MDUyNzIyMDMzMlow
czEPMA0GA1UEBBMGQWx0bWFuMRUwEwYDVQQqEwxKZWZmcmV5IEVyaWMxHDAaBgNVBAMTE0pl
ZmZyZXkgRXJpYyBBbHRtYW4xKzApBgkqhkiG9w0BCQEWHGphbHRtYW5Ac2VjdXJlLWVuZHBv
aW50cy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC19SD7DncCP/+wfQlL
zAAcxf1nJ/7UQgh4o/nxzvuY55XwHdLQjqWuFUnM5vecfyZKwq0ofGCucDfcQbSIrkhHD9z4
TZ8vDaYWVY9Foz8Rp8G0PNdbRFoFtfJbaeVBX5hG3aQXIc/T1b9U8uN3kLyqXAFIGWKO8DJV
GTKKtOiPVOp1U+9CwujyYmUSKF+suutKABhhK1ZGHsTnFczLZ2g0ma0H7PiFJ2kLfOf///07
E1fbr4IRb+cd87kpWLcjtEZ0rbBr9HlOy9dkeEii/qFoo1ahfKCDA9bNErMiOXA3dudaNNzX
lN/70slq5fboBXbepamJGrsnXYcCsS9+LtCTAgMBAAGjOTA3MCcGA1UdEQQgMB6BHGphbHRt
YW5Ac2VjdXJlLWVuZHBvaW50cy5jb20wDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQQFAAOB
gQDBzWhkrW+ol3iyT1rV8ZBQB0+z/6dFH3djQfNf7jDXNoXx4VbopA7BAR4ihAPibv7j7Zax
myMxWiDACRGS934uvUS0K6L6q14hTWMostJgFsAEDArrmbrES03vL3EVETiGFqTB2sLp5MLc
6+z+72pLXRuDPL3lO2GOQuBbILswRzCCAz8wggKooAMCAQICAQ0wDQYJKoZIhvcNAQEFBQAw
gdExCzAJBgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUg
VG93bjEaMBgGA1UEChMRVGhhd3RlIENvbnN1bHRpbmcxKDAmBgNVBAsTH0NlcnRpZmljYXRp
b24gU2VydmljZXMgRGl2aXNpb24xJDAiBgNVBAMTG1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFp
bCBDQTErMCkGCSqGSIb3DQEJARYccGVyc29uYWwtZnJlZW1haWxAdGhhd3RlLmNvbTAeFw0w
MzA3MTcwMDAwMDBaFw0xMzA3MTYyMzU5NTlaMGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxU
aGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwg
RnJlZW1haWwgSXNzdWluZyBDQTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAxKY8VXNV
+065yplaHmjAdQRwnd/p/6Me7L3N9VvyGna9fww6YfK/Uc4B1OVQCjDXAmNaLIkVcI7dyfAr
hVqqP3FWy688Cwfn8R+RNiQqE88r1fOCdz0Dviv+uxg+B79AgAJk16emu59l0cUqVIUPSAR/
p7bRPGEEQB5kGXJgt/sCAwEAAaOBlDCBkTASBgNVHRMBAf8ECDAGAQH/AgEAMEMGA1UdHwQ8
MDowOKA2oDSGMmh0dHA6Ly9jcmwudGhhd3RlLmNvbS9UaGF3dGVQZXJzb25hbEZyZWVtYWls
Q0EuY3JsMAsGA1UdDwQEAwIBBjApBgNVHREEIjAgpB4wHDEaMBgGA1UEAxMRUHJpdmF0ZUxh
YmVsMi0xMzgwDQYJKoZIhvcNAQEFBQADgYEASIzRUIPqCy7MDaNmrGcPf6+svsIXoUOWlJ1/
TCG4+DYfqi2fNi/A9BxQIJNwPP2t4WFiw9k6GX6EsZkbAMUaC4J0niVQlGLH2ydxVyWN3amc
OY6MIE9lX5Xa9/eH1sYITq726jTlEBpbNU1341YheILcIRk13iSx0x1G/11fZU8xggNkMIID
YAIBATB2MGIxCzAJBgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5
KSBMdGQuMSwwKgYDVQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQQIQ
FbTSUrChYle3zBuZOXUzSTAJBgUrDgMCGgUAoIIBwzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcN
AQcBMBwGCSqGSIb3DQEJBTEPFw0wNjA4MDQwMDA3MjBaMCMGCSqGSIb3DQEJBDEWBBRM4wry
vvDxV+cy732QP3274HlNlTBSBgkqhkiG9w0BCQ8xRTBDMAoGCCqGSIb3DQMHMA4GCCqGSIb3
DQMCAgIAgDANBggqhkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG9w0DAgIBKDCBhQYJKwYB
BAGCNxAEMXgwdjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcg
KFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3Vpbmcg
Q0ECEBW00lKwoWJXt8wbmTl1M0kwgYcGCyqGSIb3DQEJEAILMXigdjBiMQswCQYDVQQGEwJa
QTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhh
d3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEBW00lKwoWJXt8wbmTl1M0kwDQYJ
KoZIhvcNAQEBBQAEggEAEwcVjlH1amjYVPCvAylkE3DtpZsjMh0Y1s0UAKUKQ4JzFH4+L8Ez
UTIax73x9gv2M1uW9UHvh6FNVjxobiARE03nyJ5p+5d2+eL+YG0psWkjOIwKFfa4pyIgQ/O3
ZGz61txxyL5nFGnloLxeQb1SLf74IknzA7WsTFgFNlaZrRggNnT7DQnkUV61qI/yJpkbYzso
tie+RRUfIctCyiVKD+JUZG5Io0N0H2hOW3AujiHXyA2NG5Eb/tWUtC4J3za/gh1AWBIjt5cU
NKOwz6laQiETK8R98AwEDevCzcxZKz3L3zRNBXMpyqhODcICe3W/8L22VvLVhOs3o+YamhXh
MQAAAAAAAA==
--------------ms040607000308080706070305--