[OpenAFS-devel] Request to create a Dump Tag registry
Jeffrey E Altman
openafs-devel@openafs.org" <openafs-devel@openafs.org
Wed, 12 Oct 2022 23:40:09 -0400
This is a cryptographically signed message in MIME format.
--------------ms070405000404050506010705
Content-Type: multipart/alternative;
boundary="------------74Zi0JE9HnrPq0zpCxyga8gk"
--------------74Zi0JE9HnrPq0zpCxyga8gk
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
AuriStor has been investigating extending the Dump Stream to support the
functionality proposed in April 2011 as RPC Refresh (Gerrit 4573) as
well as additional data and metadata types implemented by AuriStorFS.
To avoid unnecessary collisions and to maximize interoperability I
propose the creation of a Dump Tag Registry.
The Dump Format is poorly documented and many tags and sub-tags have
been squatted upon over the years without creation of a registry. The
original Dump Format did not provide any mechanism for backward
compatibility. The structure of the data following a Tag or Sub-Tag
could not be determined independent of any particular implementation.
For example, when OpenAFS introduced the D_VNODE 'h' Sub-Tag to
represent the lengths of vnodes larger than 2GB any dump stream that
included the D_VNODE 'h' sub-tag could not be parsed by an earlier
version of IBM AFS or OpenAFS. It can be argued that in this particular
case the ability to understand 64-bit lengths should be considered
critical and therefore parsing the dump stream should fail. The same
case cannot be made for the D_VOLUMEHEADER 'V' (volume update counter)
sub-tag that was introduced in October 2005. An inability to parse and
restore this counter should not prevent a backup dump stream from being
used to recover the contents of a Volume.
In March 2005 Peter Somogyi proposed a solution submitted as OpenAFS RT
17947 which requested that OpenAFS implement a mechanism for storing
Vnode Data in compressed form. The data compression implementation was
never adopted by OpenAFS but in September 2009 OpenAFS adopted a
framework which would permit tags and sub-tags to be introduced in the
future without breaking compatibility with earlier versions of OpenAFS
that implemented the framework. OpenAFS 1.4.12 was the first production
release to implement the framework.
The text of the proposed framework was forwarded to afs3-standardization
in October 2007.
https://lists.openafs.org/pipermail/afs3-standardization/2007-October/001053.html
OpenAFS adopted the TLV standard for future tags by merging
https://gerrit.openafs.org/#/c/191/4
The framework extends the pre-existing Dump Format as follows:
1. Valid Dump or "D" Header Tag values range from 0x1 to 0x15
1. The four tags in use before the adoption of the framework are
declared "Legacy"
1. 0x01 D_DUMPHEADER
2. 0x02 D_VOLUMEHEADER
3. 0x03 D_VNODE
4. 0x04 D_DUMPEND
2. The remaining tags are designated Tag-Length-Value (TLV) tags
3. Each D Header tag has its own private namespace of sub-tags
2. Valid Sub-Tags values range from 0x16 to 0x7d
1. All D_VOLUMEHEADER and D_VNODE Sub-tags in use before the
adoption of the framework are declared "Legacy"
2. Non-legacy sub-tags between 0x16 and 0x60 are designated TLV
sub-tags.
3. Non-legacy sub-tags between 0x61 and 0x7a are designated
Standard sub-tags where the sub-tag is followed by a fixed
32-bit integer value in network-byte-order.
4. Non-legacy sub-tags between 0x7b and 0x7d are designated
Dataless sub-tags which are not followed by any data.
3. Tag 0x0 is reserved and currently invalid
4. Tag0x7e is dataless and has the special meaning that the Next Tag or
Sub-tag is CRITICAL. Failure to recognize the meaning of the Tag or
Sub-tag must result in a fatal failure.
5. Tag 0x7f is reserved.
TLV tags are followed by a variable length "Length" field which is then
followed by the specified number of octets. The variable length
"Length" is parsed as follows:
1. If the first octet is less than or equal to 0x7f, then the Value
that follows is that number of octets.
2. If the first octet is 0x80, then the Value that follows is of
indefinite length. The value data that follows must be
self-describing with regards to length. In other words, a parser
that understands the data format must be able to identify the end of
the value from the contents of the data. If an indefinite length
tag or sub-tag is unrecognized, it is a fatal error regardless of
whether or not the CRITICAL tag preceded it.
3. If the first octet 'L' is between 0x81 and 0x88, the length is value
of the next (L & 0x0f) octets combined in most significant byte
(MSB) order. 0x81 indicates one byte, 0x82 two bytes, 0x88 eight bytes.
4. If the first octet is greater than 0x88, it is an invalid length
which is a fatal error.
The OpenAFS implementation has the following deficiencies:
1. It does not handle parsing of unrecognized D Tags.
2. The MIN_TLV_TAG sub-tag must be 0x16 not 0x5.
3. Tag values up to and including D_MAX (20) must not be used as sub-tags.
4. The Legacy tag table is incomplete.
Initial Registry Contents:
I propose that all of the following tags and sub-tags be added to the
registry. Sub-tags designated as deprecated are known to have been
stored in backups and therefore might be present in a restored dump
stream at some point in the future. Deprecated sub-tags should be
avoided but might be reallocated if all other sub-tags are in use.
I propose that new tags be allocated on a first come, first served basis
with a documentation requirement. Documentation must be provided that is
sufficient to permit a dump stream to be restored if non-critical
unrecognized tags are included in the stream. A stream containing
unrecognized critical tags will fail to restore but sufficient
documentation must be present for a site to evaluate the risk associated
with the criticality.
When designing new tags for registration developers should consider use
by the entire ecosystem. For example, in addressing the Year 2038
problem new tags to transmit 64-bit 100ns timestamps for vnodes and
volumes should be reusable by many implementations. The same should be
true for 64-bit data versions, 96-bit vnode ids, 64-bit volume ids,
64-bit user ids, longer volume names, 64-bit volume quotas, and 64-bit
volume space usage. The Dump Header will require new tags for 64-bit
100ns timestamps, 64-bit volume ids, and longer volume names,
D Header Tags:
0x01 D_DUMPHEADER IBM Legacy
0x02 D_VOLUMEHEADER IBM Legacy
0x03 D_VNODE IBM Legacy
0x04 D_DUMPEND IBM Legacy
D_DUMPHEADER Sub-Tags
21 0x15 64-bit Volume ID AuriStor TLV
22 0x16 100ns Dump Times AuriStor TLV
'n' 0x6e Volume Name IBM Legacy: C-String
't' 0x74 32-bit Dump Times IBM Legacy
'v' 0x76 32-bit Volume ID IBM Legacy
D_VOLUMEHEADER Sub-Tags
21 0x15 64-bit Volume IDs AuriStor TLV
22 0x16 Volume Maximum ACL AuriStor TLV
23 0x17 Volume Sec Levels AuriStor TLV
24 0x18 64-bit Max Quota AuriStor TLV
25 0x19 64-bit Disk Usage AuriStor TLV
26 0x1a 100ns Timestamps AuriStor TLV
27 0x1b 64-bit Features AuriStor TLV
28 0x1c 64-bit Owner AuriStor TLV
29 0x1d 64-bit Min Quota AuriStor TLV
30 0x1e 64-bit File Count AuriStor TLV
'A' 0x41 32-bit Last Access Date OSD Legacy
'B' 0x42 32-bit Backup Date IBM Legacy
'C' 0x43 32-bit Creation Date IBM Legacy
'D' 0x44 32-bit Day Use Date IBM Legacy
'E' 0x45 32-bit Expiration Date IBM Legacy
'F' 0x46 OSD Policy OSD Legacy: 32-bit [deprecated - replaced by 'P']
'M' 0x4d MOTD IBM Legacy: C-String
'O' 0x4f Offline Message IBM Legacy: C-String
'P' 0x50 OSD Policy OSD Legacy: 32-bit [deprecated - replaced by 'y']
'U' 0x55 32-bit Update Date IBM Legacy
'V' 0x56 32-bit Update Counter OpenAFS Legacy
'W' 0x57 Week Use Statistics IBM Legacy: structure
'Z' 0x5a Day Use Statistics IBM Legacy
'a' 0x61 Account Number IBM Standard [deprecated]
'b' 0x62 Blessed Flag IBM Legacy: 8-bit
'c' 0x63 32-bit Clone Id IBM Standard
'd' 0x64 32-bit Disk Usage IBM Standard
'f' 0x66 32-bit File Count IBM Standard
'i' 0x69 32-bit Volume Id IBM Standard
'm' 0x6d 32-bit Volume Min Quota IBM Standard
'n' 0x6e Volume Name IBM Legacy: C-String
'o' 0x6f 32-bit Volume Owner IBM Standard
'p' 0x70 32-bit Parent Volume Id IBM Standard
'q' 0x71 32-bit Volume Max Quota IBM Standard
'r' 0x72 OSD Max number of files OSD Standard
's' 0x73 In-Service Flag IBM Legacy: 8-bit
't' 0x74 Volume Type IBM Legacy: 8-bit
'u' 0x75 32-bit Next Uniquifier IBM Standard
'v' 0x76 32-bit Stamp Version IBM Standard [deprecated]
'y' 0x79 OSD Policy OSD Standard
D_VNODE Sub-Tags
21 0x15 File ACL AuriStor TLV [deprecated]
22 0x16 100ns Timestamps AuriStor TLV
23 0x17 64-bit A, O, G IDs AuriStor TLV
24 0x18 96-bit Vnode# Parent# AuriStor TLV
25 0x19 64-bit Data Version AuriStor TLV
26 0x1a 64-bit Ext ACL AuriStor TLV
'A' 0x41 32-bit ACL IBM Legacy: C-String
'L' 0x4c OSD vnode length (hi,lo) OSD TLV
'O' 0x4f OSD metadata string OSD TLV
'P' 0x50 OSD dir policy index OSD Legacy: 32-bit [deprecated - replaced by 'd']
'a' 0x61 32-bit Author ID IBM Standard
'b' 0x62 Unix Mode Bits IBM Legacy: 16-bit
'd' 0x64 OSD dir policy index OSD Standard
'f' 0x66 File Stream - Small IBM Legacy: structured
'g' 0x67 32-bit Group ID IBM Standard
'h' 0x68 File Stream - Large OpenAFS Legacy: structured
'l' 0x6c Link Count IBM Legacy: 16-bit
'm' 0x6d 32-bit UNIX Modify Time IBM Standard
'o' 0x6f 32-bit Owner ID IBM Standard
'p' 0x70 32-bit Parent Vnode # IBM Standard
's' 0x73 32-bit Server Modify Time IBM Standard
't' 0x74 Vnode Type IBM Legacy: 8-bit
'u' 0x75 32-bit Last Access Time OSD Standard [deprecated]
'v' 0x76 32-bit Data Version IBM Standard
'x' 0x78 OSD File Online Flag OSD Standard
'y' 0x79 OSD vnode length OSD Legacy: structure [deprecated - replaced by 'L']
'z' 0x7a OSD metadata string OSD Legacy: C-String [deprecated - replaced by 'O']
Guidance for Use of New Tags
1. If a new sub-tag is allocated to support a larger value, for example
D_VOLUMEHEADER 0x15 "64-bit IDs" which can replace 0x69 "32-bit
Volume Id", 0x70 "32-bit Parent Id", and 0x63 "Clone Id", then the
new tag should only be generated if the data values cannot be
represented by the older tag. If the new tag is used, it should be
preceded by the CRITICAL tag.
2. If a new sub-tag is allocated to augment existing data, then both
sub-tags should be generated. For example, the D_VOLUMEHEADER 0x16
"100ns timestamps" provides 100ns granularity for accessDate,
updateDate, creationDate, backupDate and expirationDate. This
sub-tag should be generated in addition to the 32-bit second
granularity equivalents. The new sub-tag should not be preceded by
the CRITICAL tag. This permits the dump stream to be restored both
to systems that understand 100ns timestamps and those that do not.
Please note that I did not include the RT 17947 Compression D Tags:
D_COMPRESS_TAG (0x5) and D_FLAGS_TAG (0x6) in the list of registered
tags. There are two reasons for this. First, to the best of my
knowledge the Compression Proposal was never put into production.
Second, I believe the allocation of D_COMPRESS_TAG and D_FLAGS_TAG out
of the Dump Header namespace was a mistake. If anyone revisits the
Compression proposal I believe that the details of the compression
should be sent either via D_DUMPHEADER sub-tags or D_VOLUMEHEADER
sub-tags; and a new D_VNODE sub-tag should be used to store the
Compressed File Stream. The choice of D_DUMPHEADER vs D_VOLUMEHEADER
depends on whether or not all of the components of a "merged dump"
stream must share the same compression options. If yes, then
D_DUMPHEADER is the preferred location. If not, then D_VOLUMEHEADER
should be used.
For those that are not familiar with a "merged dump", incremental dump
streams can be merged into a full dump stream by:
1. merging the D_DUMPHEADER "from".."to" time range into the
D_DUMPHEADER at the start of the stream
2. remove the D_DUMPEND header from the end of the first stream
3. remove the D_DUMPHEADER from the stream to be appended
4. concatenate the two streams producing a stream with multiple
D_VOLUMEHEADER blocks each of which are followed by one or more
D_VNODE blocks.
A merged stream can be used to restore a volume in a single RPC.
If anyone is familiar with other Tags and Sub-tags that are in use,
please let me know.
I will provide additional details of the AuriStor sub-tags to the
Registrar once a Registry has been published.
Sincerely,
Jeffrey Altman
--------------74Zi0JE9HnrPq0zpCxyga8gk
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<p><font face="Gill Sans MT">AuriStor has been investigating
extending the Dump Stream to support the functionality proposed
in April 2011 as RPC Refresh (Gerrit 4573) as well as additional
data and metadata types implemented by AuriStorFS. To avoid
unnecessary collisions and to maximize interoperability I
propose the creation of a Dump Tag Registry.<br>
</font></p>
<p><font face="Gill Sans MT">The Dump Format is poorly documented
and many tags and sub-tags have been squatted upon over the
years without creation of a registry. The original Dump Format
did not provide any mechanism for backward compatibility. The
structure of the data following a Tag or Sub-Tag could not be
determined independent of any particular implementation. For
example, when OpenAFS introduced the D_VNODE 'h' Sub-Tag to
represent the lengths of vnodes larger than 2GB any dump stream
that included the D_VNODE 'h' sub-tag could not be parsed by an
earlier version of IBM AFS or OpenAFS. It can be argued that in
this particular case the ability to understand 64-bit lengths
should be considered critical and therefore parsing the dump
stream should fail. The same case cannot be made for the
D_VOLUMEHEADER 'V' (volume update counter) sub-tag that was
introduced in October 2005. An inability to parse and restore
this counter should not prevent a backup dump stream from being
used to recover the contents of a Volume.<br>
</font></p>
<p>In March 2005 Peter Somogyi proposed a solution submitted as
OpenAFS RT 17947 which requested that OpenAFS implement a
mechanism for storing Vnode Data in compressed form. The data
compression implementation was never adopted by OpenAFS but in
September 2009 OpenAFS adopted a framework which would permit tags
and sub-tags to be introduced in the future without breaking
compatibility with earlier versions of OpenAFS that implemented
the framework. OpenAFS 1.4.12 was the first production release to
implement the framework.</p>
<p>The text of the proposed framework was forwarded to
afs3-standardization in October 2007. <br>
</p>
<p><font face="Gill Sans MT">
<a class="moz-txt-link-freetext"
href="https://lists.openafs.org/pipermail/afs3-standardization/2007-October/001053.html">https://lists.openafs.org/pipermail/afs3-standardization/2007-October/001053.html</a></font></p>
<p><font face="Gill Sans MT">OpenAFS adopted the TLV standard for
future tags by merging <br>
</font></p>
<p><font face="Gill Sans MT"> <a class="moz-txt-link-freetext"
href="https://gerrit.openafs.org/#/c/191/4">https://gerrit.openafs.org/#/c/191/4</a></font></p>
<p><font face="Gill Sans MT">The framework extends the pre-existing
Dump Format as follows:</font></p>
<ol>
<li><font face="Gill Sans MT">Valid Dump or "D" Header Tag values
range from 0x1 to 0x15</font></li>
<ol>
<li><font face="Gill Sans MT">The four tags in use before the
adoption of the framework are declared "Legacy"</font></li>
<ol>
<li><font face="Gill Sans MT">0x01 D_DUMPHEADER</font></li>
<li><font face="Gill Sans MT">0x02 D_VOLUMEHEADER</font></li>
<li><font face="Gill Sans MT">0x03 D_VNODE</font></li>
<li><font face="Gill Sans MT">0x04 D_DUMPEND</font></li>
</ol>
<li><font face="Gill Sans MT">The remaining tags are designated
Tag-Length-Value (TLV) tags</font></li>
<li><font face="Gill Sans MT">Each D Header tag has its own
private namespace of sub-tags</font></li>
</ol>
<li><font face="Gill Sans MT">Valid Sub-Tags values range from
0x16 to 0x7d</font></li>
<ol>
<li><font face="Gill Sans MT">All D_VOLUMEHEADER and D_VNODE
Sub-tags in use before the adoption of the framework are
declared "Legacy"</font></li>
<li><font face="Gill Sans MT">Non-legacy sub-tags between 0x16
and 0x60 are designated TLV sub-tags.</font></li>
<li><font face="Gill Sans MT">Non-legacy sub-tags between 0x61
and 0x7a are designated Standard sub-tags where the sub-tag
is followed by a fixed 32-bit integer value in
network-byte-order.</font></li>
<li><font face="Gill Sans MT">Non-legacy sub-tags between 0x7b
and 0x7d are designated Dataless sub-tags which are not
followed by any data.</font></li>
</ol>
<li><font face="Gill Sans MT">Tag 0x0 is reserved and currently
invalid</font></li>
<li><font face="Gill Sans MT">Tag</font><font face="Gill Sans MT">
0x7e is dataless and has the special meaning that the Next Tag
or Sub-tag is CRITICAL. Failure to recognize the meaning of
the Tag or Sub-tag must result in a fatal failure. </font></li>
<li><font face="Gill Sans MT">Tag 0x7f is reserved.</font></li>
</ol>
<p>TLV tags are followed by a variable length "Length" field which
is then followed by the specified number of octets. The variable
length "Length" is parsed as follows:<br>
</p>
<ol>
<li>If the first octet is less than or equal to 0x7f, then the
Value that follows is that number of octets.</li>
<li>If the first octet is 0x80, then the Value that follows is of
indefinite length. The value data that follows must be
self-describing with regards to length. In other words, a
parser that understands the data format must be able to identify
the end of the value from the contents of the data. If an
indefinite length tag or sub-tag is unrecognized, it is a fatal
error regardless of whether or not the CRITICAL tag preceded it.</li>
<li>If the first octet 'L' is between 0x81 and 0x88, the length is
value of the next (L & 0x0f) octets combined in most
significant byte (MSB) order. 0x81 indicates one byte, 0x82 two
bytes, 0x88 eight bytes.</li>
<li>If the first octet is greater than 0x88, it is an invalid
length which is a fatal error.<br>
</li>
</ol>
<p><font face="Gill Sans MT">The OpenAFS implementation has the
following deficiencies:</font></p>
<ol>
<li><font face="Gill Sans MT">It does not handle parsing of
unrecognized D Tags.</font></li>
<li><font face="Gill Sans MT">The MIN_TLV_TAG sub-tag must be 0x16
not 0x5.</font></li>
<li><font face="Gill Sans MT">Tag values up to and including D_MAX
(20) must not be used as sub-tags.</font></li>
<li><font face="Gill Sans MT">The Legacy tag table is incomplete.</font></li>
</ol>
<p><font face="Gill Sans MT"><br>
</font></p>
<p><font face="Gill Sans MT">Initial Registry Contents:</font></p>
<p><font face="Gill Sans MT">I propose that all of the following
tags and sub-tags be added to the registry. Sub-tags
designated as deprecated are known to have been stored in
backups and therefore might be present in a restored dump stream
at some point in the future. Deprecated sub-tags should be
avoided but might be reallocated if all other sub-tags are in
use.<br>
</font></p>
<font face="Gill Sans MT">I propose that new tags be allocated on a
first come, first served basis with a documentation requirement.
Documentation must be provided that is sufficient to permit a dump
stream to be restored if non-critical unrecognized tags are
included in the stream. A stream containing unrecognized critical
tags will fail to restore but sufficient documentation must be
present for a site to evaluate the risk associated with the
criticality.<br>
</font>
<p><font face="Gill Sans MT">When designing new tags for
registration developers should consider use by the entire
ecosystem. For example, in addressing the Year 2038 problem new
tags to transmit 64-bit 100ns timestamps for vnodes and volumes
should be reusable by many implementations. The same should be
true for 64-bit data versions, 96-bit vnode ids, 64-bit volume
ids, 64-bit user ids, longer volume names, 64-bit volume quotas,
and 64-bit volume space usage. The Dump Header will require new
tags for 64-bit 100ns timestamps, 64-bit volume ids, and longer
volume names, <br>
</font></p>
<p>D Header Tags:</p>
<pre> 0x01 D_DUMPHEADER IBM Legacy
0x02 D_VOLUMEHEADER IBM Legacy
0x03 D_VNODE IBM Legacy
0x04 D_DUMPEND IBM Legacy</pre>
<p>D_DUMPHEADER Sub-Tags<br>
</p>
<pre> 21 0x15 64-bit Volume ID AuriStor TLV
22 0x16 100ns Dump Times AuriStor TLV
'n' 0x6e Volume Name IBM Legacy: C-String
't' 0x74 32-bit Dump Times IBM Legacy
'v' 0x76 32-bit Volume ID IBM Legacy
</pre>
<p>D_VOLUMEHEADER Sub-Tags</p>
<pre> 21 0x15 64-bit Volume IDs AuriStor TLV
22 0x16 Volume Maximum ACL AuriStor TLV
23 0x17 Volume Sec Levels AuriStor TLV
24 0x18 64-bit Max Quota AuriStor TLV
25 0x19 64-bit Disk Usage AuriStor TLV
26 0x1a 100ns Timestamps AuriStor TLV
27 0x1b 64-bit Features AuriStor TLV
28 0x1c 64-bit Owner AuriStor TLV
29 0x1d 64-bit Min Quota AuriStor TLV
30 0x1e 64-bit File Count AuriStor TLV
'A' 0x41 32-bit Last Access Date OSD Legacy
'B' 0x42 32-bit Backup Date IBM Legacy
'C' 0x43 32-bit Creation Date IBM Legacy
'D' 0x44 32-bit Day Use Date IBM Legacy
'E' 0x45 32-bit Expiration Date IBM Legacy
'F' 0x46 OSD Policy OSD Legacy: 32-bit [deprecated - replaced by 'P']
'M' 0x4d MOTD IBM Legacy: C-String
'O' 0x4f Offline Message IBM Legacy: C-String
'P' 0x50 OSD Policy OSD Legacy: 32-bit [deprecated - replaced by 'y']
'U' 0x55 32-bit Update Date IBM Legacy
'V' 0x56 32-bit Update Counter OpenAFS Legacy
'W' 0x57 Week Use Statistics IBM Legacy: structure
'Z' 0x5a Day Use Statistics IBM Legacy
'a' 0x61 Account Number IBM Standard [deprecated]
'b' 0x62 Blessed Flag IBM Legacy: 8-bit
'c' 0x63 32-bit Clone Id IBM Standard
'd' 0x64 32-bit Disk Usage IBM Standard
'f' 0x66 32-bit File Count IBM Standard
'i' 0x69 32-bit Volume Id IBM Standard
'm' 0x6d 32-bit Volume Min Quota IBM Standard
'n' 0x6e Volume Name IBM Legacy: C-String
'o' 0x6f 32-bit Volume Owner IBM Standard
'p' 0x70 32-bit Parent Volume Id IBM Standard
'q' 0x71 32-bit Volume Max Quota IBM Standard
'r' 0x72 OSD Max number of files OSD Standard
's' 0x73 In-Service Flag IBM Legacy: 8-bit
't' 0x74 Volume Type IBM Legacy: 8-bit
'u' 0x75 32-bit Next Uniquifier IBM Standard
'v' 0x76 32-bit Stamp Version IBM Standard [deprecated]
'y' 0x79 OSD Policy OSD Standard
</pre>
<p>D_VNODE Sub-Tags</p>
<pre> 21 0x15 File ACL AuriStor TLV [deprecated]
22 0x16 100ns Timestamps AuriStor TLV
23 0x17 64-bit A, O, G IDs AuriStor TLV
24 0x18 96-bit Vnode# Parent# AuriStor TLV
25 0x19 64-bit Data Version AuriStor TLV
26 0x1a 64-bit Ext ACL AuriStor TLV
'A' 0x41 32-bit ACL IBM Legacy: C-String
'L' 0x4c OSD vnode length (hi,lo) OSD TLV
'O' 0x4f OSD metadata string OSD TLV
'P' 0x50 OSD dir policy index OSD Legacy: 32-bit [deprecated - replaced by 'd']
'a' 0x61 32-bit Author ID IBM Standard
'b' 0x62 Unix Mode Bits IBM Legacy: 16-bit
'd' 0x64 OSD dir policy index OSD Standard
'f' 0x66 File Stream - Small IBM Legacy: structured
'g' 0x67 32-bit Group ID IBM Standard
'h' 0x68 File Stream - Large OpenAFS Legacy: structured
'l' 0x6c Link Count IBM Legacy: 16-bit
'm' 0x6d 32-bit UNIX Modify Time IBM Standard
'o' 0x6f 32-bit Owner ID IBM Standard
'p' 0x70 32-bit Parent Vnode # IBM Standard
's' 0x73 32-bit Server Modify Time IBM Standard
't' 0x74 Vnode Type IBM Legacy: 8-bit
'u' 0x75 32-bit Last Access Time OSD Standard [deprecated]
'v' 0x76 32-bit Data Version IBM Standard
'x' 0x78 OSD File Online Flag OSD Standard
'y' 0x79 OSD vnode length OSD Legacy: structure [deprecated - replaced by 'L']
'z' 0x7a OSD metadata string OSD Legacy: C-String [deprecated - replaced by 'O']
</pre>
<p>Guidance for Use of New Tags</p>
<ol>
<li>If a new sub-tag is allocated to support a larger value, for
example D_VOLUMEHEADER 0x15 "64-bit IDs" which can replace 0x69
"32-bit Volume Id", 0x70 "32-bit Parent Id", and 0x63 "Clone
Id", then the new tag should only be generated if the data
values cannot be represented by the older tag. If the new tag
is used, it should be preceded by the CRITICAL tag.</li>
<li>If a new sub-tag is allocated to augment existing data, then
both sub-tags should be generated. For example, the
D_VOLUMEHEADER 0x16 "100ns timestamps" provides 100ns
granularity for accessDate, updateDate, creationDate, backupDate
and expirationDate. This sub-tag should be generated in
addition to the 32-bit second granularity equivalents. The new
sub-tag should not be preceded by the CRITICAL tag. This
permits the dump stream to be restored both to systems that
understand 100ns timestamps and those that do not.</li>
</ol>
<p>Please note that I did not include the RT 17947 Compression D
Tags: D_COMPRESS_TAG (0x5) and D_FLAGS_TAG (0x6) in the list of
registered tags. There are two reasons for this. First, to the
best of my knowledge the Compression Proposal was never put into
production. Second, I believe the allocation of D_COMPRESS_TAG
and D_FLAGS_TAG out of the Dump Header namespace was a mistake.
If anyone revisits the Compression proposal I believe that the
details of the compression should be sent either via D_DUMPHEADER
sub-tags or D_VOLUMEHEADER sub-tags; and a new D_VNODE sub-tag
should be used to store the Compressed File Stream. The choice of
D_DUMPHEADER vs D_VOLUMEHEADER depends on whether or not all of
the components of a "merged dump" stream must share the same
compression options. If yes, then D_DUMPHEADER is the preferred
location. If not, then D_VOLUMEHEADER should be used.</p>
<p>For those that are not familiar with a "merged dump", incremental
dump streams can be merged into a full dump stream by:</p>
<ol>
<li>merging the D_DUMPHEADER "from".."to" time range into the
D_DUMPHEADER at the start of the stream</li>
<li>remove the D_DUMPEND header from the end of the first stream</li>
<li>remove the D_DUMPHEADER from the stream to be appended</li>
<li>concatenate the two streams producing a stream with multiple
D_VOLUMEHEADER blocks each of which are followed by one or more
D_VNODE blocks.</li>
</ol>
<p>A merged stream can be used to restore a volume in a single RPC.<br>
</p>
<p>If anyone is familiar with other Tags and Sub-tags that are in
use, please let me know.</p>
<p>I will provide additional details of the AuriStor sub-tags to the
Registrar once a Registry has been published.</p>
<p>Sincerely,</p>
<p>Jeffrey Altman</p>
<p><br>
</p>
</body>
</html>
--------------74Zi0JE9HnrPq0zpCxyga8gk--
--------------ms070405000404050506010705
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature
MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCC
DHEwggXSMIIEuqADAgECAhBAAYJpmi/rPn/F0fJyDlzMMA0GCSqGSIb3DQEBCwUAMDoxCzAJ
BgNVBAYTAlVTMRIwEAYDVQQKEwlJZGVuVHJ1c3QxFzAVBgNVBAMTDlRydXN0SUQgQ0EgQTEz
MB4XDTIyMDgwNDE2MDQ0OFoXDTI1MTAzMTE2MDM0OFowcDEvMC0GCgmSJomT8ixkAQETH0Ew
MTQxMEQwMDAwMDE4MjY5OUEyRkQyMDAwMjMzQ0QxGTAXBgNVBAMTEEplZmZyZXkgRSBBbHRt
YW4xFTATBgNVBAoTDEF1cmlTdG9yIEluYzELMAkGA1UEBhMCVVMwggEiMA0GCSqGSIb3DQEB
AQUAA4IBDwAwggEKAoIBAQCkC7PKBBZnQqDKPtZPMLAy77zo2DPvwtGnd1hNjPvbXrpGxUb3
xHZRtv179LHKAOcsY2jIctzieMxf82OMyhpBziMPsFAG/ukihBMFj3/xEeZVso3K27pSAyyN
fO/wJ0rX7G+ges22Dd7goZul8rPaTJBIxbZDuaykJMGpNq4PQ8VPcnYZx+6b+nJwJJoJ46kI
EEfNh3UKvB/vM0qtxS690iAdgmQIhTl+qfXq4IxWB6b+3NeQxgR6KLU4P7v88/tvJTpxIKkg
9xj89ruzeThyRFd2DSe3vfdnq9+g4qJSHRXyTft6W3Lkp7UWTM4kMqOcc4VSRdufVKBQNXjG
IcnhAgMBAAGjggKcMIICmDAOBgNVHQ8BAf8EBAMCBPAwgYQGCCsGAQUFBwEBBHgwdjAwBggr
BgEFBQcwAYYkaHR0cDovL2NvbW1lcmNpYWwub2NzcC5pZGVudHJ1c3QuY29tMEIGCCsGAQUF
BzAChjZodHRwOi8vdmFsaWRhdGlvbi5pZGVudHJ1c3QuY29tL2NlcnRzL3RydXN0aWRjYWEx
My5wN2MwHwYDVR0jBBgwFoAULbfeG1l+KpguzeHUG+PFEBJe6RQwCQYDVR0TBAIwADCCASsG
A1UdIASCASIwggEeMIIBGgYLYIZIAYb5LwAGAgEwggEJMEoGCCsGAQUFBwIBFj5odHRwczov
L3NlY3VyZS5pZGVudHJ1c3QuY29tL2NlcnRpZmljYXRlcy9wb2xpY3kvdHMvaW5kZXguaHRt
bDCBugYIKwYBBQUHAgIwga0MgapUaGlzIFRydXN0SUQgQ2VydGlmaWNhdGUgaGFzIGJlZW4g
aXNzdWVkIGluIGFjY29yZGFuY2Ugd2l0aCBJZGVuVHJ1c3QncyBUcnVzdElEIENlcnRpZmlj
YXRlIFBvbGljeSBmb3VuZCBhdCBodHRwczovL3NlY3VyZS5pZGVudHJ1c3QuY29tL2NlcnRp
ZmljYXRlcy9wb2xpY3kvdHMvaW5kZXguaHRtbDBFBgNVHR8EPjA8MDqgOKA2hjRodHRwOi8v
dmFsaWRhdGlvbi5pZGVudHJ1c3QuY29tL2NybC90cnVzdGlkY2FhMTMuY3JsMB8GA1UdEQQY
MBaBFGphbHRtYW5AYXVyaXN0b3IuY29tMB0GA1UdDgQWBBQB+nzqgljLocLTsiUn2yWqEc2s
gjAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUHAwQwDQYJKoZIhvcNAQELBQADggEBAJwV
eycprp8Ox1npiTyfwc5QaVaqtoe8Dcg2JXZc0h4DmYGW2rRLHp8YL43snEV93rPJVk6B2v4c
WLeQfaMrnyNeEuvHx/2CT44cdLtaEk5zyqo3GYJYlLcRVz6EcSGHv1qPXgDT0xB/25etwGYq
utYF4Chkxu4KzIpq90eDMw5ajkexw+8ARQz4N5+d6NRbmMCovd7wTGi8th/BZvz8hgKUiUJo
Qle4wDxrdXdnIhCP7g87InXKefWgZBF4VX21t2+hkc04qrhIJlHrocPG9mRSnnk2WpsY0MXt
a8ivbVKtfpY7uSNDZSKTDi1izEFH5oeQdYRkgIGb319a7FjslV8wggaXMIIEf6ADAgECAhBA
AXA7OrqBjMk8rp4OuNQSMA0GCSqGSIb3DQEBCwUAMEoxCzAJBgNVBAYTAlVTMRIwEAYDVQQK
EwlJZGVuVHJ1c3QxJzAlBgNVBAMTHklkZW5UcnVzdCBDb21tZXJjaWFsIFJvb3QgQ0EgMTAe
Fw0yMDAyMTIyMTA3NDlaFw0zMDAyMTIyMTA3NDlaMDoxCzAJBgNVBAYTAlVTMRIwEAYDVQQK
EwlJZGVuVHJ1c3QxFzAVBgNVBAMTDlRydXN0SUQgQ0EgQTEzMIIBIjANBgkqhkiG9w0BAQEF
AAOCAQ8AMIIBCgKCAQEAu6sUO01SDD99PM+QdZkNxKxJNt0NgQE+Zt6ixaNP0JKSjTd+SG5L
wqxBWjnOgI/3dlwgtSNeN77AgSs+rA4bK4GJ75cUZZANUXRKw/et8pf9Qn6iqgB63OdHxBN/
15KbM3HR+PyiHXQoUVIevCKW8nnlWnnZabT1FejOhRRKVUg5HACGOTfnCOONrlxlg+m1Vjgn
o1uNqNuLM/jkD1z6phNZ/G9IfZGI0ppHX5AA/bViWceX248VmefNhSR14ADZJtlAAWOi2un0
3bqrBPHA9nDyXxI8rgWLfUP5rDy8jx2hEItg95+ORF5wfkGUq787HBjspE86CcaduLka/Bk2
VwIDAQABo4IChzCCAoMwEgYDVR0TAQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAYYwgYkG
CCsGAQUFBwEBBH0wezAwBggrBgEFBQcwAYYkaHR0cDovL2NvbW1lcmNpYWwub2NzcC5pZGVu
dHJ1c3QuY29tMEcGCCsGAQUFBzAChjtodHRwOi8vdmFsaWRhdGlvbi5pZGVudHJ1c3QuY29t
L3Jvb3RzL2NvbW1lcmNpYWxyb290Y2ExLnA3YzAfBgNVHSMEGDAWgBTtRBnA0/AGi+6ke75C
5yZUyI42djCCASQGA1UdIASCARswggEXMIIBEwYEVR0gADCCAQkwSgYIKwYBBQUHAgEWPmh0
dHBzOi8vc2VjdXJlLmlkZW50cnVzdC5jb20vY2VydGlmaWNhdGVzL3BvbGljeS90cy9pbmRl
eC5odG1sMIG6BggrBgEFBQcCAjCBrQyBqlRoaXMgVHJ1c3RJRCBDZXJ0aWZpY2F0ZSBoYXMg
YmVlbiBpc3N1ZWQgaW4gYWNjb3JkYW5jZSB3aXRoIElkZW5UcnVzdCdzIFRydXN0SUQgQ2Vy
dGlmaWNhdGUgUG9saWN5IGZvdW5kIGF0IGh0dHBzOi8vc2VjdXJlLmlkZW50cnVzdC5jb20v
Y2VydGlmaWNhdGVzL3BvbGljeS90cy9pbmRleC5odG1sMEoGA1UdHwRDMEEwP6A9oDuGOWh0
dHA6Ly92YWxpZGF0aW9uLmlkZW50cnVzdC5jb20vY3JsL2NvbW1lcmNpYWxyb290Y2ExLmNy
bDAdBgNVHQ4EFgQULbfeG1l+KpguzeHUG+PFEBJe6RQwHQYDVR0lBBYwFAYIKwYBBQUHAwIG
CCsGAQUFBwMEMA0GCSqGSIb3DQEBCwUAA4ICAQB/7BKcygLX6Nl4a03cDHt7TLdPxCzFvDF2
bkVYCFTRX47UfeomF1gBPFDee3H/IPlLRmuTPoNt0qjdpfQzmDWN95jUXLdLPRToNxyaoB5s
0hOhcV6H08u3FHACBif55i0DTDzVSaBv0AZ9h1XeuGx4Fih1Vm3Xxz24GBqqVudvPRLyMJ7u
6hvBqTIKJ53uCs3dyQLZT9DXnp+kJv8y7ZSAY+QVrI/dysT8avtn8d7k7azNBkfnbRq+0e88
QoBnel6u+fpwbd5NLRHywXeH+phbzULCa+bLPRMqJaW2lbhvSWrMHRDy3/d8HvgnLCBFK2s4
Spns4YCN4xVcbqlGWzgolHCKUH39vpcsDo1ymZFrJ8QR6ihIn8FmJ5oKwAnnd/G6ADXFC9bu
db9+532phSAXOZrrecIQn+vtP366PC+aClAPsIIDJDsotS5z4X2JUFsNIuEgXGqhiKE7SuZb
rFG9sdcLprSlJN7TsRDc0W2b9nqwD+rj/5MN0C+eKwha+8ydv0+qzTyxPP90KRgaegGowC4d
UsZyTk2n4Z3MuAHX5nAZL/Vh/SyDj/ajorV44yqZBzQ3ChKhXbfUSwe2xMmygA2Z5DRwMRJn
p/BscizYdNk2WXJMTnH+wVLN8sLEwEtQR4eTLoFmQvrK2AMBS9kW5sBkMzINt/ZbbcZ3F+eA
MDGCAxQwggMQAgEBME4wOjELMAkGA1UEBhMCVVMxEjAQBgNVBAoTCUlkZW5UcnVzdDEXMBUG
A1UEAxMOVHJ1c3RJRCBDQSBBMTMCEEABgmmaL+s+f8XR8nIOXMwwDQYJYIZIAWUDBAIBBQCg
ggGXMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTIyMTAxMzAz
NDAwOVowLwYJKoZIhvcNAQkEMSIEICtDQLREhJ5Giq3JqbXIe1rmdw15zECLynikayw9Nnzy
MF0GCSsGAQQBgjcQBDFQME4wOjELMAkGA1UEBhMCVVMxEjAQBgNVBAoTCUlkZW5UcnVzdDEX
MBUGA1UEAxMOVHJ1c3RJRCBDQSBBMTMCEEABgmmaL+s+f8XR8nIOXMwwXwYLKoZIhvcNAQkQ
AgsxUKBOMDoxCzAJBgNVBAYTAlVTMRIwEAYDVQQKEwlJZGVuVHJ1c3QxFzAVBgNVBAMTDlRy
dXN0SUQgQ0EgQTEzAhBAAYJpmi/rPn/F0fJyDlzMMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZI
AWUDBAEqMAsGCWCGSAFlAwQBAjAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZI
hvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwDQYJKoZIhvcNAQEBBQAEggEAecrr
yh1sBjoO9JLqasqEm0AXx6XiIjpKt1y02SLhv+LJROYaU5522RsdIAmssSmg9TMTcYfc+Ek6
Z0pbjzgrF1CJIYorza5jSAd1wYWh+ODcHma4LKblOj6xkbH7m1cU25IJ+iuIVdPh5YkzV5rV
3UmsXYrklUSApmN9h9qyJVnuABI6Pr7UH1egFcVvprqHKvoYxcePp4NKYfdcgTACAxb94PJi
6W/paW1gU5FF4tTNK7VxxUi1oDbzeDJs1nHkrOz9s1jCxKDehMJMZ1Rz/MmXADsUJNRWpG7w
xFoJtake9VjVwLT+jWroRRikkh4fmKohO4zo6a3oG7hT7YoaBQAAAAAAAA==
--------------ms070405000404050506010705--