[OpenAFS] How to Upgrade 1.6. to 1.8
Jeffrey Altman
jaltman@auristor.com
Wed, 13 Nov 2024 22:58:18 -0500
--Apple-Mail=_94B08A19-FE25-422C-85A7-4F4F5371EDFD
Content-Type: multipart/alternative;
boundary="Apple-Mail=_6BABEE81-0D1D-45DD-B315-7BA81E24D564"
--Apple-Mail=_6BABEE81-0D1D-45DD-B315-7BA81E24D564
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
> On Nov 10, 2024, at 5:12=E2=80=AFAM, Hirste <norgay@hirschhofer.net> =
wrote:
>=20
> Hi all,
>=20
> I have to update an existing single server installation from 1.6. to =
1.8.
>=20
> A brief overview of the situation:
>=20
> Existing Machine: Ubuntu Xenial, AFS Version 1.6.23 (through PPA) =
=3DafsA
>=20
> Target Machine: Ubuntu Noble with AFS Version 1.8.10 (from Distro =
Repo) afsB
>=20
> The Data ar provided via a handful of vdisks, each beeing a single AFS =
volume, summing up to +3 TB.
>=20
> Clients: Ubuntu Focal, AFS Client Version 1.8.10 (through PPA, ugly =
Kerberos hack to allow weak crypto)
>=20
> This is an inherited setup, I would describe my AFS abilities and =
knowledge as weak / not to good.
>=20
I would describe the stated goals as follows:
Migrate the single server cell to new hardware
Upgrade from OpenAFS 1.6 to OpenAFS 1.8
Are there unstated goals? For example,
Changing the IP address and hostname assigned to the database services
Mitigating Brute force DES attacks: =
https://www.openafs.org/pages/security/#OPENAFS-SA-2013-003
Switching from afs@<REALM> service principal names to afs/<cell>@<REALM> =
service principal names
Since you describe your AFS expertise as weak I will assume that you are =
unaware that UNIX OpenAFS clients rely upon the IP addresses of the =
database services not changing. Any change to the IP address or =
hostname of the database services will require distribution of new =
configuration files to all clients UNLESS all clients are obtaining the =
service information solely from DNS AFSDB or SRV records.
You mention a weak crypto hack which implies that the cell server has =
either not been upgraded to 1.6.5 (or later) OR at the very least the =
Kerberos service principal was never rekeyed to deploy =
aes256-cts-hmac-sha1-96 service principal key in place of des-cbc-crc. =
If that is the case, I would suggest upgrading to 1.8 and then =
performing the rekeying to a non-DES encryption type.
Switching from afs@<REALM> to afs/<cell>@<REALM> will make the client =
authentication faster because all versions of aklog shipped in the last =
15 years try to acquire a ticket for afs/<cell>@<REALM> before =
afs@<REALM>.
> What I would have done without advise or help:
>=20
> Setup the new server, add vdisks as the origin machine has, rsyncing =
the data, repointing DNS entries
> so that no changes on the clients are needed (some of them beeing road =
warriors on laptops), demoting afsA.
>=20
> This will lead to a significant downtime since rsync will take its =
time. Since there are no changes in Client Configuration
> there could ba an easy way back.
>=20
The upside of this approach is that the clients require no configuration =
changes. If there are no DNS AFSDB or SRV records for the cell and all =
of the clients maintain local configuration with hard coded IP =
addresses, then it is really important that the IP address currently =
assigned to afsA remain an active database server even if it is not a =
fileserver.
There are other downsides of this approach in addition to the downtime. =
It makes assumptions that the vice partition content consists of normal =
file data whereas the AFS vice partitions should really be thought of as =
a proprietary database of object stores where the fileserver and related =
processes (volserver, salvager) which operate as =E2=80=9Croot=E2=80=9D =
are permitted to violate the normal filesystem rules to encode metadata. =
As such, volume level operations should always be performed using =
=E2=80=9Cvos=E2=80=9D initiated operations. For example, =E2=80=9Cmove=E2=
=80=9D, =E2=80=9Caddsite=E2=80=9D, =E2=80=9Cremsite=E2=80=9D, =
=E2=80=9Cremove=E2=80=9D, =E2=80=9Crelease=E2=80=9D, etc.
> Instead I think it would be better to
>=20
> add afsB to the existing cell (beeing only an additional file server)
As Jose mentioned, AFS is designed to be a reliable service which =
permits maintenance to be performed without cell-wide outages. Three =
database instances are required to ensure that there isn=E2=80=99t a =
single point of failure which interrupts services. I would suggest =
that you add two database service instances as the first step. =
Database service instances can in general be relatively small. VMs are =
perfectly fine as would be Intel NUC class servers. Separating the =
database servers from the fileservers is also beneficial. Unless the =
goal is to run the service as a single server.
> Migrating the AFS Volumes in an AFS manner to afsB (does this work =
online?)
Yes. =E2=80=9Cvos move=E2=80=9D, =E2=80=9Cvos addsite=E2=80=9D, =E2=80=9C=
vos remove=E2=80=9D, =E2=80=9Cvos remsite=E2=80=9D, =E2=80=9Cvos =
release=E2=80=9D.
> move the other AFS roles (database server, binary distribution =
machine, system control machine)=20
> from afsA to afsB
see above
> demote afsA
As mentioned the IP addresses and hostname assigned to afsA are the only =
place the existing deployed clients will look to. Therefore, any =
shutdown or removal of services from this address will result in an =
outage.
> I am currently reading and trying to understand =
https://docs.openafs.org/AdminGuide but would appreciate
> help from experienced users.
>=20
Once you have the new servers in place you should tackle the Kerberos =
service principal rekeying. OpenAFS 1.8 requires the use of the =
akeyconvert tool.
> Is it possible to have mixed fileserver versions ( 1.6 vs. 1.8) in a =
Cell? What are caveats?
>=20
The intra cell protocol messages used between cell services in 1.6 and =
1.8 are unchanged. Mixing 1.6 and 1.8 is just fine.
> Is there a better strategy with less risk to migrate the server?
>=20
> Appreciate any help and advise.
>=20
> Hirste
>=20
Good luck.
Jeffrey Altman
AuriStor, Inc.
--Apple-Mail=_6BABEE81-0D1D-45DD-B315-7BA81E24D564
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=utf-8
<html><head><meta http-equiv=3D"content-type" content=3D"text/html; =
charset=3Dutf-8"></head><body style=3D"overflow-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: =
after-white-space;"><br><div><blockquote type=3D"cite"><div>On Nov 10, =
2024, at 5:12=E2=80=AFAM, Hirste <norgay@hirschhofer.net> =
wrote:</div><br class=3D"Apple-interchange-newline"><div>
=20
<meta http-equiv=3D"content-type" content=3D"text/html; =
charset=3DUTF-8">
=20
<div><p>Hi all,</p><p>I have to update an existing single server =
installation from 1.6.
to 1.8.</p><p>A brief overview of the situation:<br>
</p><p>Existing Machine: Ubuntu Xenial, AFS Version 1.6.23 (through =
PPA)
=3DafsA<br>
</p><p>Target Machine: Ubuntu Noble with AFS Version 1.8.10 (from =
Distro
Repo) afsB<br>
</p><p>The Data ar provided via a handful of vdisks, each beeing a
single AFS volume, summing up to +3 TB.<br>
</p><p>Clients: Ubuntu Focal, AFS Client Version 1.8.10 (through =
PPA,
ugly Kerberos hack to allow weak crypto)</p><p>This is an =
inherited setup, I would describe my AFS abilities and
knowledge as weak / not to =
good.</p></div></div></blockquote><div>I would describe the stated goals =
as follows:</div><div><ul class=3D"MailOutline"><li>Migrate the single =
server cell to new hardware</li><li>Upgrade from OpenAFS 1.6 to OpenAFS =
1.8</li></ul><div><br></div><div>Are there unstated goals? For =
example,</div><div><ul class=3D"MailOutline"><li>Changing the IP address =
and hostname assigned to the database services</li><li>Mitigating Brute =
force DES attacks: <a =
href=3D"https://www.openafs.org/pages/security/#OPENAFS-SA-2013-003">https=
://www.openafs.org/pages/security/#OPENAFS-SA-2013-003</a></li><li>Switchi=
ng from afs@<REALM> service principal names to =
afs/<cell>@<REALM> service principal =
names</li></ul><div><br></div></div><div><div style=3D"display: =
block;">Since you describe your AFS expertise as weak I will assume that =
you are unaware that UNIX OpenAFS clients rely upon the IP addresses of =
the database services not changing. Any change to the IP address =
or hostname of the database services will require distribution of new =
configuration files to all clients UNLESS all clients are obtaining the =
service information solely from DNS AFSDB or SRV =
records.</div></div><div style=3D"display: block;"><br></div><div =
style=3D"display: block;">You mention a weak crypto hack which implies =
that the cell server has either not been upgraded to 1.6.5 (or later) OR =
at the very least the Kerberos service principal was never rekeyed to =
deploy aes256-cts-hmac-sha1-96 service principal key in place of =
des-cbc-crc. If that is the case, I would suggest upgrading to =
1.8 and then performing the rekeying to a non-DES encryption =
type.</div><div style=3D"display: block;"><br></div><div style=3D"display:=
block;">Switching from afs@<REALM> to =
afs/<cell>@<REALM> will make the client authentication =
faster because all versions of aklog shipped in the last 15 years try to =
acquire a ticket for afs/<cell>@<REALM> before =
afs@<REALM>.</div></div><blockquote type=3D"cite"><div><div><p>
</p><p>What I would have done without advise or help:<br>
<br>
Setup the new server, add vdisks as the origin machine has,
rsyncing the data, repointing DNS entries<br>
so that no changes on the clients are needed (some of them beeing
road warriors on laptops), demoting afsA.<br>
</p><p>This will lead to a significant downtime since rsync will =
take
its time. Since there are no changes in Client Configuration<br>
there could ba an easy way =
back.<br></p></div></div></blockquote>The upside of this approach is =
that the clients require no configuration changes. If there are no =
DNS AFSDB or SRV records for the cell and all of the clients maintain =
local configuration with hard coded IP addresses, then it is really =
important that the IP address currently assigned to afsA remain an =
active database server even if it is not a =
fileserver.</div><div><br></div><div>There are other downsides of this =
approach in addition to the downtime. It makes assumptions that =
the vice partition content consists of normal file data whereas the AFS =
vice partitions should really be thought of as a proprietary database of =
object stores where the fileserver and related processes (volserver, =
salvager) which operate as =E2=80=9Croot=E2=80=9D are permitted to =
violate the normal filesystem rules to encode metadata. As such, =
volume level operations should always be performed using =E2=80=9Cvos=E2=80=
=9D initiated operations. For example, =E2=80=9Cmove=E2=80=9D, =
=E2=80=9Caddsite=E2=80=9D, =E2=80=9Cremsite=E2=80=9D, =E2=80=9Cremove=E2=80=
=9D, =E2=80=9Crelease=E2=80=9D, etc.</div><div><blockquote =
type=3D"cite"><div><div><p>
</p><p>Instead I think it would be better to <br>
</p>
<ul>
<li>add afsB to the existing cell (beeing only an additional file
server)<br></li></ul></div></div></blockquote>As Jose mentioned, =
AFS is designed to be a reliable service which permits maintenance to be =
performed without cell-wide outages. Three database instances are =
required to ensure that there isn=E2=80=99t a single point of failure =
which interrupts services. I would suggest that you add two =
database service instances as the first step. Database service =
instances can in general be relatively small. VMs are perfectly =
fine as would be Intel NUC class servers. Separating the database =
servers from the fileservers is also beneficial. Unless the goal =
is to run the service as a single server.</div><div><blockquote =
type=3D"cite"><div><div><ul><li>Migrating the AFS Volumes in an AFS =
manner to afsB (does this
work online?)</li></ul></div></div></blockquote>Yes. =
=E2=80=9Cvos move=E2=80=9D, =E2=80=9Cvos addsite=E2=80=9D, =E2=80=9C=
vos remove=E2=80=9D, =E2=80=9Cvos remsite=E2=80=9D, =E2=80=9Cvos =
release=E2=80=9D.<br><blockquote type=3D"cite"><div><div><ul>
<li>move the other AFS roles (database server, binary distribution
machine, system control machine) <br>
from afsA to afsB</li></ul></div></div></blockquote>see =
above<br><blockquote type=3D"cite"><div><div><ul>
<li>demote afsA</li></ul></div></div></blockquote>As mentioned the =
IP addresses and hostname assigned to afsA are the only place the =
existing deployed clients will look to. Therefore, any shutdown or =
removal of services from this address will result in an =
outage.<br><blockquote type=3D"cite"><div><div><ul>
</ul><p>I am currently reading and trying to understand
<a class=3D"moz-txt-link-freetext" =
href=3D"https://docs.openafs.org/AdminGuide">https://docs.openafs.org/Admi=
nGuide</a> but would appreciate<br>
help from experienced users.</p></div></div></blockquote>Once you =
have the new servers in place you should tackle the Kerberos service =
principal rekeying. OpenAFS 1.8 requires the use of the =
akeyconvert tool.<br><blockquote type=3D"cite"><div><div><p>Is it =
possible to have mixed fileserver versions ( 1.6 vs. 1.8)
in a Cell? What are caveats?<br></p></div></div></blockquote>The =
intra cell protocol messages used between cell services in 1.6 and 1.8 =
are unchanged. Mixing 1.6 and 1.8 is just fine.<br><blockquote =
type=3D"cite"><div><div><p>
Is there a better strategy with less risk to migrate the =
server?<br>
</p><p>Appreciate any help and advise.<br>
</p><p>Hirste</p></div></div></blockquote>Good =
luck.</div><div><br></div><div>Jeffrey Altman</div><div>AuriStor, =
Inc.</div><div><br></div></body></html>=
--Apple-Mail=_6BABEE81-0D1D-45DD-B315-7BA81E24D564--
--Apple-Mail=_94B08A19-FE25-422C-85A7-4F4F5371EDFD
Content-Disposition: attachment;
filename=smime.p7s
Content-Type: application/pkcs7-signature;
name=smime.p7s
Content-Transfer-Encoding: base64
MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCCDHEw
ggXSMIIEuqADAgECAhBAAYJpmi/rPn/F0fJyDlzMMA0GCSqGSIb3DQEBCwUAMDoxCzAJBgNVBAYT
AlVTMRIwEAYDVQQKEwlJZGVuVHJ1c3QxFzAVBgNVBAMTDlRydXN0SUQgQ0EgQTEzMB4XDTIyMDgw
NDE2MDQ0OFoXDTI1MTAzMTE2MDM0OFowcDEvMC0GCgmSJomT8ixkAQETH0EwMTQxMEQwMDAwMDE4
MjY5OUEyRkQyMDAwMjMzQ0QxGTAXBgNVBAMTEEplZmZyZXkgRSBBbHRtYW4xFTATBgNVBAoTDEF1
cmlTdG9yIEluYzELMAkGA1UEBhMCVVMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCk
C7PKBBZnQqDKPtZPMLAy77zo2DPvwtGnd1hNjPvbXrpGxUb3xHZRtv179LHKAOcsY2jIctzieMxf
82OMyhpBziMPsFAG/ukihBMFj3/xEeZVso3K27pSAyyNfO/wJ0rX7G+ges22Dd7goZul8rPaTJBI
xbZDuaykJMGpNq4PQ8VPcnYZx+6b+nJwJJoJ46kIEEfNh3UKvB/vM0qtxS690iAdgmQIhTl+qfXq
4IxWB6b+3NeQxgR6KLU4P7v88/tvJTpxIKkg9xj89ruzeThyRFd2DSe3vfdnq9+g4qJSHRXyTft6
W3Lkp7UWTM4kMqOcc4VSRdufVKBQNXjGIcnhAgMBAAGjggKcMIICmDAOBgNVHQ8BAf8EBAMCBPAw
gYQGCCsGAQUFBwEBBHgwdjAwBggrBgEFBQcwAYYkaHR0cDovL2NvbW1lcmNpYWwub2NzcC5pZGVu
dHJ1c3QuY29tMEIGCCsGAQUFBzAChjZodHRwOi8vdmFsaWRhdGlvbi5pZGVudHJ1c3QuY29tL2Nl
cnRzL3RydXN0aWRjYWExMy5wN2MwHwYDVR0jBBgwFoAULbfeG1l+KpguzeHUG+PFEBJe6RQwCQYD
VR0TBAIwADCCASsGA1UdIASCASIwggEeMIIBGgYLYIZIAYb5LwAGAgEwggEJMEoGCCsGAQUFBwIB
Fj5odHRwczovL3NlY3VyZS5pZGVudHJ1c3QuY29tL2NlcnRpZmljYXRlcy9wb2xpY3kvdHMvaW5k
ZXguaHRtbDCBugYIKwYBBQUHAgIwga0MgapUaGlzIFRydXN0SUQgQ2VydGlmaWNhdGUgaGFzIGJl
ZW4gaXNzdWVkIGluIGFjY29yZGFuY2Ugd2l0aCBJZGVuVHJ1c3QncyBUcnVzdElEIENlcnRpZmlj
YXRlIFBvbGljeSBmb3VuZCBhdCBodHRwczovL3NlY3VyZS5pZGVudHJ1c3QuY29tL2NlcnRpZmlj
YXRlcy9wb2xpY3kvdHMvaW5kZXguaHRtbDBFBgNVHR8EPjA8MDqgOKA2hjRodHRwOi8vdmFsaWRh
dGlvbi5pZGVudHJ1c3QuY29tL2NybC90cnVzdGlkY2FhMTMuY3JsMB8GA1UdEQQYMBaBFGphbHRt
YW5AYXVyaXN0b3IuY29tMB0GA1UdDgQWBBQB+nzqgljLocLTsiUn2yWqEc2sgjAdBgNVHSUEFjAU
BggrBgEFBQcDAgYIKwYBBQUHAwQwDQYJKoZIhvcNAQELBQADggEBAJwVeycprp8Ox1npiTyfwc5Q
aVaqtoe8Dcg2JXZc0h4DmYGW2rRLHp8YL43snEV93rPJVk6B2v4cWLeQfaMrnyNeEuvHx/2CT44c
dLtaEk5zyqo3GYJYlLcRVz6EcSGHv1qPXgDT0xB/25etwGYqutYF4Chkxu4KzIpq90eDMw5ajkex
w+8ARQz4N5+d6NRbmMCovd7wTGi8th/BZvz8hgKUiUJoQle4wDxrdXdnIhCP7g87InXKefWgZBF4
VX21t2+hkc04qrhIJlHrocPG9mRSnnk2WpsY0MXta8ivbVKtfpY7uSNDZSKTDi1izEFH5oeQdYRk
gIGb319a7FjslV8wggaXMIIEf6ADAgECAhBAAXA7OrqBjMk8rp4OuNQSMA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRIwEAYDVQQKEwlJZGVuVHJ1c3QxJzAlBgNVBAMTHklkZW5UcnVzdCBD
b21tZXJjaWFsIFJvb3QgQ0EgMTAeFw0yMDAyMTIyMTA3NDlaFw0zMDAyMTIyMTA3NDlaMDoxCzAJ
BgNVBAYTAlVTMRIwEAYDVQQKEwlJZGVuVHJ1c3QxFzAVBgNVBAMTDlRydXN0SUQgQ0EgQTEzMIIB
IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAu6sUO01SDD99PM+QdZkNxKxJNt0NgQE+Zt6i
xaNP0JKSjTd+SG5LwqxBWjnOgI/3dlwgtSNeN77AgSs+rA4bK4GJ75cUZZANUXRKw/et8pf9Qn6i
qgB63OdHxBN/15KbM3HR+PyiHXQoUVIevCKW8nnlWnnZabT1FejOhRRKVUg5HACGOTfnCOONrlxl
g+m1Vjgno1uNqNuLM/jkD1z6phNZ/G9IfZGI0ppHX5AA/bViWceX248VmefNhSR14ADZJtlAAWOi
2un03bqrBPHA9nDyXxI8rgWLfUP5rDy8jx2hEItg95+ORF5wfkGUq787HBjspE86CcaduLka/Bk2
VwIDAQABo4IChzCCAoMwEgYDVR0TAQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAYYwgYkGCCsG
AQUFBwEBBH0wezAwBggrBgEFBQcwAYYkaHR0cDovL2NvbW1lcmNpYWwub2NzcC5pZGVudHJ1c3Qu
Y29tMEcGCCsGAQUFBzAChjtodHRwOi8vdmFsaWRhdGlvbi5pZGVudHJ1c3QuY29tL3Jvb3RzL2Nv
bW1lcmNpYWxyb290Y2ExLnA3YzAfBgNVHSMEGDAWgBTtRBnA0/AGi+6ke75C5yZUyI42djCCASQG
A1UdIASCARswggEXMIIBEwYEVR0gADCCAQkwSgYIKwYBBQUHAgEWPmh0dHBzOi8vc2VjdXJlLmlk
ZW50cnVzdC5jb20vY2VydGlmaWNhdGVzL3BvbGljeS90cy9pbmRleC5odG1sMIG6BggrBgEFBQcC
AjCBrQyBqlRoaXMgVHJ1c3RJRCBDZXJ0aWZpY2F0ZSBoYXMgYmVlbiBpc3N1ZWQgaW4gYWNjb3Jk
YW5jZSB3aXRoIElkZW5UcnVzdCdzIFRydXN0SUQgQ2VydGlmaWNhdGUgUG9saWN5IGZvdW5kIGF0
IGh0dHBzOi8vc2VjdXJlLmlkZW50cnVzdC5jb20vY2VydGlmaWNhdGVzL3BvbGljeS90cy9pbmRl
eC5odG1sMEoGA1UdHwRDMEEwP6A9oDuGOWh0dHA6Ly92YWxpZGF0aW9uLmlkZW50cnVzdC5jb20v
Y3JsL2NvbW1lcmNpYWxyb290Y2ExLmNybDAdBgNVHQ4EFgQULbfeG1l+KpguzeHUG+PFEBJe6RQw
HQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMA0GCSqGSIb3DQEBCwUAA4ICAQB/7BKcygLX
6Nl4a03cDHt7TLdPxCzFvDF2bkVYCFTRX47UfeomF1gBPFDee3H/IPlLRmuTPoNt0qjdpfQzmDWN
95jUXLdLPRToNxyaoB5s0hOhcV6H08u3FHACBif55i0DTDzVSaBv0AZ9h1XeuGx4Fih1Vm3Xxz24
GBqqVudvPRLyMJ7u6hvBqTIKJ53uCs3dyQLZT9DXnp+kJv8y7ZSAY+QVrI/dysT8avtn8d7k7azN
BkfnbRq+0e88QoBnel6u+fpwbd5NLRHywXeH+phbzULCa+bLPRMqJaW2lbhvSWrMHRDy3/d8Hvgn
LCBFK2s4Spns4YCN4xVcbqlGWzgolHCKUH39vpcsDo1ymZFrJ8QR6ihIn8FmJ5oKwAnnd/G6ADXF
C9budb9+532phSAXOZrrecIQn+vtP366PC+aClAPsIIDJDsotS5z4X2JUFsNIuEgXGqhiKE7SuZb
rFG9sdcLprSlJN7TsRDc0W2b9nqwD+rj/5MN0C+eKwha+8ydv0+qzTyxPP90KRgaegGowC4dUsZy
Tk2n4Z3MuAHX5nAZL/Vh/SyDj/ajorV44yqZBzQ3ChKhXbfUSwe2xMmygA2Z5DRwMRJnp/BscizY
dNk2WXJMTnH+wVLN8sLEwEtQR4eTLoFmQvrK2AMBS9kW5sBkMzINt/ZbbcZ3F+eAMDGCAqYwggKi
AgEBME4wOjELMAkGA1UEBhMCVVMxEjAQBgNVBAoTCUlkZW5UcnVzdDEXMBUGA1UEAxMOVHJ1c3RJ
RCBDQSBBMTMCEEABgmmaL+s+f8XR8nIOXMwwDQYJYIZIAWUDBAIBBQCgggEpMBgGCSqGSIb3DQEJ
AzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTI0MTExNDAzNTgxOVowLwYJKoZIhvcNAQkE
MSIEIAGNMvr2QaE1738Ww1RHIzB+IE2krKG21UOXgjPtHJYRMF0GCSsGAQQBgjcQBDFQME4wOjEL
MAkGA1UEBhMCVVMxEjAQBgNVBAoTCUlkZW5UcnVzdDEXMBUGA1UEAxMOVHJ1c3RJRCBDQSBBMTMC
EEABgmmaL+s+f8XR8nIOXMwwXwYLKoZIhvcNAQkQAgsxUKBOMDoxCzAJBgNVBAYTAlVTMRIwEAYD
VQQKEwlJZGVuVHJ1c3QxFzAVBgNVBAMTDlRydXN0SUQgQ0EgQTEzAhBAAYJpmi/rPn/F0fJyDlzM
MA0GCSqGSIb3DQEBCwUABIIBABmGwDU0g1HSobjsEynjCcMvp5vAUNFlygDlmsU/ljZrnKvbEoEh
CGtQsZhBpHoaMgw1YyidCc8U3oZlGsr6bK68PA8x1bMoV4Cygt0FLT6sUv28TGC+lj3x/YMbnnvT
fTRPG8cTH4C2E3GmjKJarE3dXlaWalKpWcCajDtxZo/8MKDjfPA4UuIJA24FysmeOrEBscneEAz6
vTV4fzqC9uxDSyxy/ZeFwWR64NtD9ts0UGS/3Qjs3jMLnCAanKAHLX82JsKXkAeEVsfCjklR0nji
3NbVEVLc8Iyq6ameYZDBj3USCZmC4q9kYWcBFvrpIdXC957+ByiwLcy6Elnhv5UAAAAAAAA=
--Apple-Mail=_94B08A19-FE25-422C-85A7-4F4F5371EDFD--