[OpenAFS-devel] Re: [OpenAFS release-team] Re: volume time stamp analysis

Wed, 03 Dec 2014 13:34:46 -0500

This is a cryptographically signed message in MIME format.

--------------ms030000050604030908030102
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

[CC'ing openafs-devel]

One of the areas of concern that I have is the use of the vos -clone
option and the impact it has on the resulting volume timestamps.

I will reply in more depth later but I am unable to do so now.

Jeffrey Altman

On 12/3/2014 12:17 PM, Jeffrey Hutzelman wrote:
> On Wed, 2014-12-03 at 06:43 -0500, Jeffrey Altman wrote:
>=20
>>> The updateDate is set to the same as the copy date when a volume is
>>> newly created.  It is propagated during cloning, recloning, and
>>> forwarding.  However, as with the creation date, recent versions of '=
vos
>>> restore' allow the user to select between setting a new date, using t=
he
>>> one in the dump, or preserving a target's existing date.  The default=

>>> for 'vos restore' is to use the time in the dump, but any other calle=
rs
>>> of UV_RestoreVolume will default to setting a new time.  This last is=

>>> actually a bug, introduced in 2004 when the flags to 'vos restore' we=
re
>>> added.  Prior to that time, the only available behavior was to use th=
e
>>> updateDate contained in the dump.
>>
>> The bug introduced to UV_RestoreVolume in 2004 will be triggered when
>> volumes are restored via butc or when the Norbert perl binding is used=
=2E
>=20
> Yup.  And potentially by other backup tools as well, if they use
> UV_RestoreVolume and don't do anything else to set the dates (which is
> unlikely, especially for updateDate, because the ability to set it via
> an RPC was introduced at the same time as the vos options described
> above, and the bug itself.
>=20
> We really should get this bug fixed, now that we know about it.
>=20
>=20
>>> Additionally, the updateDate is modified whenever the modify timestam=
p
>>> on a vnode changes.  However, it is _not_ modified when volume metada=
ta
>>> changes.
>>
>> I consider it a bug that the updateDate is not modified when volume
>> metadata that is reflected in a dump is modified.  I believe that
>> OpenAFS should fix this.
>=20
> It's not that straightforward, and I think it's a topic for discussion
> on openafs-devel.  An argument could certainly be made that anything
> that touches a volume, whether it appears in a dump or not, should bump=

> the update stamp.  However, that's more than a bug fix; it's a semantic=

> change.  The updateDate currently describes the _contents_ of the
> volume; you want to change it to describe the volume itself.  That coul=
d
> be important, especially since it may be used in a variety of situation=
s
> for creating and processing incremental dumps, as an indicator of which=

> vnodes need to be processed.
>=20
>=20
>>> For a backup volume, the backupDate (if set) is clearly safe, and is
>>> also the newest possible safe time, because it explicitly records the=

>>> snapshot date.  However, it may be unset (zero).  In such a case, the=

>>> best we can do is to either use the backupDate or treat it like any
>>> other clone, which means using the creationDate (see below).  The onl=
y
>>> difference in handling between backup volumes and other clones are th=
at
>>> a backup volume is always the result of a cloning and that its
>>> backupDate is set.
>>
>> The result of 11433+11468 is that the backupDate is always used for
>> backup volumes.  To handle the case where backupDate might be unset, w=
e
>> could check for V_backupDate(vp) =3D=3D 0 and use creationDate in that=

>> situation.
>=20
> Agreed.  I think that check would even be a reasonable addition to
> 11468.
>=20
>=20
>>> The gotcha case has to do with volume which are themselves the produc=
t
>>> of a restore done using 'vos restore' or UV_RestoreVolume().  In such=

>>> volumes, the creationDate has usually been reset to the time of the
>>> restore (always, with older AFS), and with versions since 2004 or so,=

>>> the updateDate may also have been reset.  This means that the normall=
y
>>> "correct" timestamps may in fact be dangerously new.
>>>
>>> I think we can disregard the case where the updateDate is incorrect,
>>> especially if we fix the bug which causes UV_RestoreVolume() to reset=
 it
>>> by default.  Admins who explicitly do something else should know what=

>>> they are getting themselves into.  This means that we can basically
>>> assume that it is OK to use updateDate for RW volumes.
>>
>> Sites that use the OpenAFS backup tooling might have triggered this bu=
g
>> and that is a problem.
>=20
> It is, but in practice it's probably not.  A volume restored as RW can
> be modified simply by modifying it, at which point any information abou=
t
> the updateDate of the volume it came from is destroyed anyway.  For the=

> situation to be problematic, the following sequence has to occur:
>=20
> (1) dump from volume A
> (2) volume A changes
> (3) restore the dump from (1) to volume B
> (4) dump from volume B
> ... arbitrarily long delay ...
> (5) incremental dump from volume A
> (6) apply the dump from (4) to volume B
> (7) incremental dump from volume B.
>=20
> The restore in (3) will set the update date on volume B to something
> that is newer than the change in (2).  Then the dump in (4) will contai=
n
> an incorrect to time.  However, this only matters if the change in (2)
> is eventually going to make its way into volume B.  That means you have=

> to do further incremental transfers from A to B after volume B is
> online.
>=20
> Further, note that an actual change to volume B anytime between (3) and=

> (4) would have the same effect as the bug and would also mean B is no
> longer an appropriate target for incremental transfer from A.  Thus, fo=
r
> the problem to actually matter, you have to not plan on making any
> changes to the RW volume B until after doing any more incremental
> restores from A.
>=20
> And of course, the problem we're discussing is the to time in dump (4),=

> and how it affects the selection of the from time in (7).  So if you
> don't start doing backups of B until all the incremental transfers are
> done, it doesn't matter.
>=20
> Finally, let's not even get started on what happens if you begin doing
> dumps of clones of B, all of which bear timestamps with no relation to
> anything that went on in A, because A is not their parent.
>=20
>=20
> In short, a newly restored RW volume is a new volume, not a clone of th=
e
> original, and must be treated as such.  That means there are going to b=
e
> issues no matter what we do.  However, operationally, I don't think it'=
s
> a problem, because in most cases such a volume either won't be backed u=
p
> or won't receive additional incremental transfers once the initial
> series of restores is done, and especially not from dumps made after th=
e
> new volume is created.
>=20
> IOW, this doesn't affect temporary restores that aren't backed up, and
> it doesn't affect permanent restores to replace a lost volume (since th=
e
> original no longer exists).
>=20
>=20
>>> Unfortunately, we cannot so easily disregard the issue with
>>> creationDate, since that is reset by default during a restore and alw=
ays
>>> has been.  I'm not entirely sure what to do here -- creationDate is
>>> certainly better than copyDate, but it also might be wrong.
>>
>> The behavior of "vos restore" and UV_RestoreVolume is might have anoth=
er
>> set of bugs since the default behavior of setting the creationDate
>> should take into account whether the dump is from a RW, a RO, or a BK
>> and whether or not it is being restored as a RW or a clone.
>=20
>=20
>=20
>> When performing a restore to a RO (vos restore -readonly) it is not sa=
fe
>> to set the creationDate to the restore timestamp or perhaps even to ke=
ep
>> the existing volume's creationDate.  Instead creationDate should be se=
t
>> to one of the dump's timestamps.
>=20
> Perhaps, but AFS has more or less always done this, so we ought to cope=

> with it as best we can, and we ought to think about it carefully before=

> we change it.  I'll comment on the specifics below, but perhaps this is=

> a conversation we should continue on openafs-devel.
>=20
>=20
>>   If the dump was taken from a BK then
>> the backupDate will be non-zero.  If so, using that value for the
>> creationDate seems like a good choice.
>=20
> This seems like a good idea, but only if the volume type in the dump is=

> actually BKVOL.  Otherwise, the backupDate doesn't actually have
> anything to do with the contents of the dump.  Unfortunately, this is a=

> bit hard to get at, because RestoreVolume throws it away (the type of
> the restored volume is set when it is created; it cannot be changed).
>=20
>>    If backupDate is zero, then
>> perhaps the newer of creationDate and updateDate since if creationDate=

>> is newer, that would indicate that the dump was created from a clone.
>> The problem with using creationDate is that it could be the result of =
a
>> bug and therefore be too new.
>=20
> Well, if the dump was done from an RW, then clearly you should just use=

> the updateDate.  This has the same problems as using backupDate from a
> BK volume, in that it is quite difficult for UV_RestoreVolume to find
> out what volume type was in the dump.  Doing that requires new
> interfaces into dumpstuff.c and a new RPC (probably the right thing is
> for a volserver transaction to record the parentId, cloneId, and type
> contained in the last volume header restored as part of that
> transaction, and provide an RPC to retrieve that).
>=20
> If the dump was from an RO, all bets are off, because the creationDate
> and updateDate of that volume may both be wrong, depending on how it wa=
s
> restored, and the copyDate and backupDate may be too old to be useful.
>=20
> I think the least-bad thing is for restores to restore the creationDate=

> and updateDate from the dump.  If the dump was from a volume where thos=
e
> dates are wrong, due to a bug or admin intervention, then they'll be
> wrong.  But at least they won't be any more wrong than they were when
> the dump was made.
>=20
> -- Jeff
>=20

--------------ms030000050604030908030102
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIINITCC
BkIwggUqoAMCAQICEDirAC//rpa3Vv85Wvtd5xswDQYJKoZIhvcNAQEFBQAwgcoxCzAJBgNV
BAYTAlVTMRcwFQYDVQQKEw5WZXJpU2lnbiwgSW5jLjEfMB0GA1UECxMWVmVyaVNpZ24gVHJ1
c3QgTmV0d29yazE6MDgGA1UECxMxKGMpIDE5OTkgVmVyaVNpZ24sIEluYy4gLSBGb3IgYXV0
aG9yaXplZCB1c2Ugb25seTFFMEMGA1UEAxM8VmVyaVNpZ24gQ2xhc3MgMSBQdWJsaWMgUHJp
bWFyeSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eSAtIEczMB4XDTExMDkwMTAwMDAwMFoXDTIx
MDgzMTIzNTk1OVowgaYxCzAJBgNVBAYTAlVTMR0wGwYDVQQKExRTeW1hbnRlYyBDb3Jwb3Jh
dGlvbjEfMB0GA1UECxMWU3ltYW50ZWMgVHJ1c3QgTmV0d29yazEeMBwGA1UECxMVUGVyc29u
YSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5TeW1hbnRlYyBDbGFzcyAxIEluZGl2aWR1YWwg
U3Vic2NyaWJlciBDQSAtIEc0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxuwn
/R1j9DsdisHTHMjIgoa2uEqGkqqBXHLKMA0vnkEiVzAhJZCao/SsKsaIF4ZhchN2LuwDyyeb
jyCAN+DkitpVplAP/LlcI2mJQqG6H6/vDvmkyQrx+DeyxtmSSq5937hEH5u6P4wG/tgjT0hR
I2pghKjuJy9g35byGiqMPI8AzE/L+iCOvDX24fCatgXz/B0/xhR7DtryBeTTgwKmxWlwtKnk
VunbHVz0pjbia7UeKi3cvrvuOgSwMAitX2hsxr0GloiE5+apZC28ODC7iCbDZ2ZmtLR3+cCh
xw5y72bi5bnK4POFdzWY3tQcsP5mceI4y258T0BV65fZqBge7QIDAQABo4ICRDCCAkAwOAYI
KwYBBQUHAQEELDAqMCgGCCsGAQUFBzABhhxodHRwOi8vcGtpLW9jc3AudmVyaXNpZ24uY29t
MBIGA1UdEwEB/wQIMAYBAf8CAQAwbAYDVR0gBGUwYzBhBgtghkgBhvhFAQcXATBSMCYGCCsG
AQUFBwIBFhpodHRwOi8vd3d3LnN5bWF1dGguY29tL2NwczAoBggrBgEFBQcCAjAcGhpodHRw
Oi8vd3d3LnN5bWF1dGguY29tL3JwYTA0BgNVHR8ELTArMCmgJ6AlhiNodHRwOi8vY3JsLnZl
cmlzaWduLmNvbS9wY2ExLWczLmNybDAOBgNVHQ8BAf8EBAMCAQYwKQYDVR0RBCIwIKQeMBwx
GjAYBgNVBAMTEVZlcmlTaWduTVBLSS0yLTk3MB0GA1UdDgQWBBSt+cOTci21uShh5KTXYNXE
Cl4aATCB8QYDVR0jBIHpMIHmoYHQpIHNMIHKMQswCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVy
aVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOjA4BgNVBAsT
MShjKSAxOTk5IFZlcmlTaWduLCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxRTBD
BgNVBAMTPFZlcmlTaWduIENsYXNzIDEgUHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBB
dXRob3JpdHkgLSBHM4IRAItbdVaEVIULAM+vOEjOsaQwDQYJKoZIhvcNAQEFBQADggEBANaP
wdqbiPKzbE0fWC+6AVFddMFG6MO4e5/WQPHv/zK6iWvADjRDn6SZ5qTwXUgzYoWFYf4jiCKM
YJsrnGVJlMSiOCRIpVylUEto6WIip5PomSJuPVu7EEIOH0x1RzRWCY/4vYw881y70pZwVHBi
Te/REL6dSCxe7IZrB4LwPeElJygs4BZ2HrP95WKW0oo9Xyuu+1zCE7dlY8s0dkOf1oeZq26t
lcEAP0Yngf813iMOQ9wUXzL5yinvwlIw9ZnduYH4OiUgjYJo8rkhhXRmBOGGORYy8i3WKqjJ
3tkAAk/jGCDFpYFWtpXe04Kt+HslvmR8LqC6cCz4+XXidE0HbYQwggbXMIIFv6ADAgECAhA5
oFEXaG88XscBgkTPSsu4MA0GCSqGSIb3DQEBBQUAMIGmMQswCQYDVQQGEwJVUzEdMBsGA1UE
ChMUU3ltYW50ZWMgQ29ycG9yYXRpb24xHzAdBgNVBAsTFlN5bWFudGVjIFRydXN0IE5ldHdv
cmsxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3MDUGA1UEAxMuU3ltYW50ZWMg
Q2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBHNDAeFw0xMzEyMjMwMDAwMDBa
Fw0xNTAxMTYyMzU5NTlaMIHOMS4wLAYDVQQDDCVQZXJzb25hIE5vdCBWYWxpZGF0ZWQgLSAx
MzU4Mjc1NTk5Njg2MSswKQYJKoZIhvcNAQkBFhxqYWx0bWFuQHNlY3VyZS1lbmRwb2ludHMu
Y29tMQ8wDQYDVQQLDAZTL01JTUUxHjAcBgNVBAsMFVBlcnNvbmEgTm90IFZhbGlkYXRlZDEf
MB0GA1UECwwWU3ltYW50ZWMgVHJ1c3QgTmV0d29yazEdMBsGA1UECgwUU3ltYW50ZWMgQ29y
cG9yYXRpb24wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCtcVgrUA+Zl527P0yx
xfkiLZzymUZLAwif9amX6c79OHd17CN5Hg3TIRXi0exjq/z/K2eftbSfj3XatgllPtlmCktq
i4daJTVLifx5G7qbcYjQgKpex/8FA3gfvJJNgMef5OwOTE1HqMJsp5CAquFjw/ReiXQqduou
1nHhUhX1AXBpaQmYDOQZwrTD41yT7qm22N67vV0viWG9x/1RdFqqtIOIyKR+ojD3IN0wufES
gy7hsWgmghh4jkrclmMIXuo+AAtmJHzwzF4hCrSqdwiRXU4aghTjsmehtMT1nFfzDPylaclO
p7IY4xeQm/q1cbU3uEqxhy06sYeVbh6zfTDBAgMBAAGjggLVMIIC0TAMBgNVHRMBAf8EAjAA
MA4GA1UdDwEB/wQEAwIFoDAgBgNVHSUBAf8EFjAUBggrBgEFBQcDBAYIKwYBBQUHAwIwHQYD
VR0OBBYEFLNBzIZb/xB6K+oeDwfBVLaB3qbUMCcGA1UdEQQgMB6BHGphbHRtYW5Ac2VjdXJl
LWVuZHBvaW50cy5jb20wHwYDVR0jBBgwFoAUrfnDk3IttbkoYeSk12DVxApeGgEwggErBggr
BgEFBQcBAQSCAR0wggEZMIIBFQYIKwYBBQUHMAKGggEHbGRhcDovL2RpcmVjdG9yeS52ZXJp
c2lnbi5jb20vQ04lMjAlM0QlMjBTeW1hbnRlYyUyMENsYXNzJTIwMSUyMEluZGl2aWR1YWwl
MjBTdWJzY3JpYmVyJTIwQ0ElMjAtJTIwRzQlMkMlMjBPVSUyMCUzRCUyMFBlcnNvbmElMjBO
b3QlMjBWYWxpZGF0ZWQlMkMlMjBPVSUyMCUzRCUyMFN5bWFudGVjJTIwVHJ1c3QlMjBOZXR3
b3JrJTJDJTIwTyUyMCUzRCUyMFN5bWFudGVjJTIwQ29ycG9yYXRpb24lMkMlMjBDJTIwJTNE
JTIwVVM/Y0FDZXJ0aWZpY2F0ZTtiaW5hcnkwXQYDVR0fBFYwVDBSoFCgToZMaHR0cDovL3Br
aS1jcmwuc3ltYXV0aC5jb20vY2FfNTYxYzEwMzY5MGM5N2E2OTI0N2EwZWYwNzFhYzgxYWYv
TGF0ZXN0Q1JMLmNybDBsBgNVHSAEZTBjMGEGC2CGSAGG+EUBBxcBMFIwJgYIKwYBBQUHAgEW
Gmh0dHA6Ly93d3cuc3ltYXV0aC5jb20vY3BzMCgGCCsGAQUFBwICMBwaGmh0dHA6Ly93d3cu
c3ltYXV0aC5jb20vcnBhMCoGCmCGSAGG+EUBEAMEHDAaBhFghkgBhvhFARABAgIEAYazFxYF
MTA5MjIwDQYJKoZIhvcNAQEFBQADggEBAACPkJV5NIxzjKc+WveaoM8Uc86wX0yLBm1A33z4
rLVXTWPi5kMIJ6kfE+dFWcMdyyOgZ2VcxIwneZ50LcITFv1VOfRkrX32vVChQqs8XGqerIo/
K3epyFEg01qHq/4byolXW6UOvmZb3oHhtHDGS94Vv6Fu6wV7irAdoM18cqzQsxU0nZDMnY5k
0pKJHLTrsC/uKuoWGz8xLLyeayi37ZsXsbGdazqzVMIoLvFTMjaFuoCetEbiFQZvnuHKwdbV
YqyCY28Cl8DVRHrInZrz84xqFiGZNSfFRWOougT47VRDA8SVy6pOtDaOmkxcYXlh5Ezo29FB
OiA0+tF8qgMmq3QxggRSMIIETgIBATCBuzCBpjELMAkGA1UEBhMCVVMxHTAbBgNVBAoTFFN5
bWFudGVjIENvcnBvcmF0aW9uMR8wHQYDVQQLExZTeW1hbnRlYyBUcnVzdCBOZXR3b3JrMR4w
HAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxNzA1BgNVBAMTLlN5bWFudGVjIENsYXNz
IDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVyIENBIC0gRzQCEDmgURdobzxexwGCRM9Ky7gwCQYF
Kw4DAhoFAKCCAmswGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcN
MTQxMjAzMTgzNDQ2WjAjBgkqhkiG9w0BCQQxFgQUI4m4nXJiR+H9mdF58rMIaM+j2CcwbAYJ
KoZIhvcNAQkPMV8wXTALBglghkgBZQMEASowCwYJYIZIAWUDBAECMAoGCCqGSIb3DQMHMA4G
CCqGSIb3DQMCAgIAgDANBggqhkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG9w0DAgIBKDCB
zAYJKwYBBAGCNxAEMYG+MIG7MIGmMQswCQYDVQQGEwJVUzEdMBsGA1UEChMUU3ltYW50ZWMg
Q29ycG9yYXRpb24xHzAdBgNVBAsTFlN5bWFudGVjIFRydXN0IE5ldHdvcmsxHjAcBgNVBAsT
FVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3MDUGA1UEAxMuU3ltYW50ZWMgQ2xhc3MgMSBJbmRp
dmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBHNAIQOaBRF2hvPF7HAYJEz0rLuDCBzgYLKoZIhvcN
AQkQAgsxgb6ggbswgaYxCzAJBgNVBAYTAlVTMR0wGwYDVQQKExRTeW1hbnRlYyBDb3Jwb3Jh
dGlvbjEfMB0GA1UECxMWU3ltYW50ZWMgVHJ1c3QgTmV0d29yazEeMBwGA1UECxMVUGVyc29u
YSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5TeW1hbnRlYyBDbGFzcyAxIEluZGl2aWR1YWwg
U3Vic2NyaWJlciBDQSAtIEc0AhA5oFEXaG88XscBgkTPSsu4MA0GCSqGSIb3DQEBAQUABIIB
AAGbUgVrVbKx1G1MZHS3ANUe48Eca+jvXcTwFSzlPe4Fl4YnyevdphN47/YV5sKknhF8DHNY
W4+Kpc8shJjZHwtaqYjAiSCymDQo88irBUSJVPhEPtDRfekPMfMIG8i4jTTJqHwpQuQQ2gst
KN6Jgb+qDqRa8tlJZmnxLWo4t7PsloqNVVUUWT4UcJXh8RQcjr/yVkhOr7+4IZLk2IVvPYTY
8f01hTwLil1CLIG3Cex1aiMjxv+35VHpyOaLPjD+j1ppgF3iZv69QD2t47H7bk0I/xmUJpCg
mViW9MvVPg3rX83fl1hXImXW+W6t22XarprCxIuAPPbalPnFg47OmpcAAAAAAAA=
--------------ms030000050604030908030102--