[OpenAFS] Problem with Off-line volumes...unable to bring On-line

McKee, Shawn smckee@umich.edu
Mon, 24 Jan 2011 08:50:23 -0500


------=_NextPart_000_00A4_01CBBBA3.BCD06B30
Content-Type: multipart/alternative;
	boundary="----=_NextPart_001_00A5_01CBBBA3.BCD06B30"


------=_NextPart_001_00A5_01CBBBA3.BCD06B30
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi Everyone,

=20

I am having a problem with one of my OpenAFS file servers.   About =BD =
of the
volumes are =93Off-line=94 and I am unable to bring them online.  First =
some
system info and then I will list problem details and what I have tried.

=20

The system is running Scientific Linux 5.5/x86_64 (basically CentOS 5.5
64-bit).  The openafs rpms are:

=20

[atums2:~]# rpm -qa | grep openafs

openafs-kpasswd-1.4.12-6.cern

openafs-client-1.4.12-6.cern

kernel-module-openafs-2.6.18-194.3.1.el5-1.4.12-5.cern

openafs-1.4.12-6.cern

kernel-module-openafs-2.6.18-194.8.1.el5-1.4.12-5.cern

openafs-krb5-1.4.12-6.cern

kernel-module-openafs-2.6.18-238.1.1.el5-1.4.12-6.cern

openafs-server-1.4.12-6.cern

=20

The version of =91e2fsprogs=92 is 1.39

=20

The system has an ext3 1TB partition for AFS:

=20

[atums2:~]# df /vicepb

Filesystem           1K-blocks      Used Available Use% Mounted on

/dev/sda1            1007931664 635382472 321349196  67% /vicepb

=20

The system has 931 volumes and only 470 are On-line while 461 are =
Off-line:

=20

[atums2:~]# vos listvol atums2=20

Total number of volumes on server atums2 partition /vicepb: 931

chamber.OLD_eml4a07               536872814 RW    8634169 K Off-line

chamber.OLD_eml4a07.readonly      536872815 RO    8634169 K On-line

chamber.OLD_eml4a09               536872817 RW     702642 K Off-line

chamber.OLD_eml4a09.readonly      536872818 RO     702642 K On-line

=85

=20

Total volumes onLine 470 ; Total volumes offLine 461 ; Total busy 0

=20

I have run =91bos salvage=92 on the partition multiple times.   I have =
restarted
the system.  I have run a force fsck.ext3 check on the underlying =
partition
(no problems found).   Only RW volumes are Off-line.   All RO volumes =
are
On-line.   There are a few RW volumes On-line (8 out of 469) but the =
rest
won=92t come On-line.

=20

Here is a particular volume which is Off-line:

=20

[atums2:~]# vos examine chdata.sn

chdata.sn                         536871656 RW        598 K  Off-line

    atums2.cern.ch /vicepb

    RWrite  536871656 ROnly          0 Backup          0

    MaxQuota   10000000 K

    Creation    Fri May 26 04:02:49 2006

    Copy        Wed Oct 11 12:35:42 2006

    Backup      Sun Jun 11 00:30:10 2006

    Last Access Fri Jan  7 16:38:32 2011

    Last Update Wed Apr  4 15:29:42 2007

    0 accesses in the past day (i.e., vnode references)

=20

    RWrite: 536871656     ROnly: 536871657     RClone: 536871657

    number of sites -> 3

       server atums1.cern.ch partition /vicepi RO Site  -- Old release

       server atums2.cern.ch partition /vicepb RW Site  -- New release

       server atums2.cern.ch partition /vicepb RO Site  -- New release

=20

Try to bring online:

=20

[atums2:~]# vos online -server atums2 -partition /vicepb -id chdata.sn

=20

The FileLog shows:

=20

Sun Jan 23 22:57:03 2011 GetBitmap: addled vnode index in volume =
chdata.sn;
volume needs salvage

Sun Jan 23 22:57:03 2011 VAttachVolume: error getting bitmap for volume
(/vicepb//V0536871656.vol)

=20

Try to Salvage:

=20

[atums2:~]# bos salvage atums2 /vicepb chdata.sn

Starting salvage.

bos: salvage completed

=20

The SalvageLog shows:

=20

[atums2:~]# tail /usr/afs/logs/SalvageLog

@(#) OpenAFS 1.4.12 built  2010-12-13 1928681 19919656

01/23/2011 22:58:19 STARTING AFS SALVAGER 2.4 (/usr/afs/bin/salvager =
/vicepb
536871656)

01/23/2011 22:58:19 2 nVolumesInInodeFile 64

01/23/2011 22:58:19 CHECKING CLONED VOLUME 536871657.

01/23/2011 22:58:19 chdata.sn.readonly (536871657) updated 04/04/2007 =
15:29

01/23/2011 22:58:19 Partially allocated vnode 2 deleted.

=20

Try again:

=20

[atums2:~]# vos online -server atums2 -partition /vicepb -id chdata.sn


FileLog has the same message:

=20

Sun Jan 23 22:59:05 2011 GetBitmap: addled vnode index in volume =
chdata.sn;
volume needs salvage

Sun Jan 23 22:59:05 2011 VAttachVolume: error getting bitmap for volume
(/vicepb//V0536871656.vol)

=20

Salvage attempt again:

=20

[atums2:~]# bos salvage atums2 /vicepb chdata.sn

Starting salvage.

bos: salvage completed

=20

[atums2:~]# tail /usr/afs/logs/SalvageLog

@(#) OpenAFS 1.4.12 built  2010-12-13 1928681 19919656

01/23/2011 23:00:07 STARTING AFS SALVAGER 2.4 (/usr/afs/bin/salvager =
/vicepb
536871656)

01/23/2011 23:00:07 2 nVolumesInInodeFile 64

01/23/2011 23:00:07 CHECKING CLONED VOLUME 536871657.

01/23/2011 23:00:07 chdata.sn.readonly (536871657) updated 04/04/2007 =
15:29

01/23/2011 23:00:07 Partially allocated vnode 2 deleted.

=20

Same result as if the prior salvage didn=92t do anything.    This is =
exactly
what happens on other volumes I have tried to bring online.=20

=20

So how would I fix this?   Any suggestions for how to get the rest of =
these
volumes On-line? =20

=20

Let me know if you need further details.  Thanks,

=20

Shawn

=20

=20


------=_NextPart_001_00A5_01CBBBA3.BCD06B30
Content-Type: text/html;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Diso-8859-1">
<html xmlns:v=3D"urn:schemas-microsoft-com:vml" =
xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:x=3D"urn:schemas-microsoft-com:office:excel" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" =
xmlns=3D"http://www.w3.org/TR/REC-html40"><head><meta name=3DGenerator =
content=3D"Microsoft Word 12 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:11.0pt;
	font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal;
	font-family:"Calibri","sans-serif";
	color:windowtext;}
span.EmailStyle18
	{mso-style-type:personal-reply;
	font-family:"Calibri","sans-serif";
	color:#1F497D;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-size:10.0pt;}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]--></head><body lang=3DEN-US link=3Dblue =
vlink=3Dpurple><div class=3DWordSection1><p class=3DMsoNormal>Hi =
Everyone,<o:p></o:p></p><p class=3DMsoNormal><o:p>&nbsp;</o:p></p><p =
class=3DMsoNormal>I am having a problem with one of my OpenAFS file =
servers.&nbsp;&nbsp; About =BD of the volumes are &#8220;Off-line&#8221; =
and I am unable to bring them online.&nbsp; First some system info and =
then I will list problem details and what I have tried.<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>The system =
is running Scientific Linux 5.5/x86_64 (basically CentOS 5.5 =
64-bit).&nbsp; The openafs rpms are:<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>[atums2:~]# =
rpm -qa | grep openafs<o:p></o:p></p><p =
class=3DMsoNormal>openafs-kpasswd-1.4.12-6.cern<o:p></o:p></p><p =
class=3DMsoNormal>openafs-client-1.4.12-6.cern<o:p></o:p></p><p =
class=3DMsoNormal>kernel-module-openafs-2.6.18-194.3.1.el5-1.4.12-5.cern<=
o:p></o:p></p><p =
class=3DMsoNormal>openafs-1.4.12-6.cern<o:p></o:p></p><p =
class=3DMsoNormal>kernel-module-openafs-2.6.18-194.8.1.el5-1.4.12-5.cern<=
o:p></o:p></p><p =
class=3DMsoNormal>openafs-krb5-1.4.12-6.cern<o:p></o:p></p><p =
class=3DMsoNormal>kernel-module-openafs-2.6.18-238.1.1.el5-1.4.12-6.cern<=
o:p></o:p></p><p =
class=3DMsoNormal>openafs-server-1.4.12-6.cern<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>The version =
of &#8216;e2fsprogs&#8217; is 1.39<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>The system =
has an ext3 1TB partition for AFS:<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>[atums2:~]# =
df /vicepb<o:p></o:p></p><p =
class=3DMsoNormal>Filesystem&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp; 1K-blocks&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Used Available =
Use% Mounted on<o:p></o:p></p><p =
class=3DMsoNormal>/dev/sda1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp; 1007931664 635382472 321349196&nbsp; 67% =
/vicepb<o:p></o:p></p><p class=3DMsoNormal><o:p>&nbsp;</o:p></p><p =
class=3DMsoNormal>The system has 931 volumes and only 470 are On-line =
while 461 are Off-line:<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>[atums2:~]# =
vos listvol atums2 <o:p></o:p></p><p class=3DMsoNormal>Total number of =
volumes on server atums2 partition /vicepb: 931<o:p></o:p></p><p =
class=3DMsoNormal>chamber.OLD_eml4a07&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 536872814 =
RW&nbsp;&nbsp;&nbsp; 8634169 K Off-line<o:p></o:p></p><p =
class=3DMsoNormal>chamber.OLD_eml4a07.readonly&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp; 536872815 RO&nbsp;&nbsp;&nbsp; 8634169 K On-line<o:p></o:p></p><p =
class=3DMsoNormal>chamber.OLD_eml4a09&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 536872817 =
RW&nbsp;&nbsp;&nbsp;&nbsp; 702642 K Off-line<o:p></o:p></p><p =
class=3DMsoNormal>chamber.OLD_eml4a09.readonly&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp; 536872818 RO&nbsp;&nbsp;&nbsp;&nbsp; 702642 K =
On-line<o:p></o:p></p><p class=3DMsoNormal>&#8230;<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>Total =
volumes onLine 470 ; Total volumes offLine 461 ; Total busy =
0<o:p></o:p></p><p class=3DMsoNormal><o:p>&nbsp;</o:p></p><p =
class=3DMsoNormal>I have run &#8216;bos salvage&#8217; on the partition =
multiple times.&nbsp;&nbsp; I have restarted the system.&nbsp; I have =
run a force fsck.ext3 check on the underlying partition (no problems =
found).&nbsp;&nbsp; Only RW volumes are Off-line.&nbsp;&nbsp; All RO =
volumes are On-line.&nbsp;&nbsp; There are a few RW volumes On-line (8 =
out of 469) but the rest won&#8217;t come On-line.<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>Here is a =
particular volume which is Off-line:<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>[atums2:~]# =
vos examine chdata.sn<o:p></o:p></p><p =
class=3DMsoNormal>chdata.sn&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp; 536871656 =
RW&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 598 K&nbsp; =
Off-line<o:p></o:p></p><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp; =
atums2.cern.ch /vicepb<o:p></o:p></p><p =
class=3DMsoNormal>&nbsp;&nbsp;&nbsp; RWrite&nbsp; 536871656 =
ROnly&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 0 =
Backup&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
0<o:p></o:p></p><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp; =
MaxQuota&nbsp;&nbsp; 10000000 K<o:p></o:p></p><p =
class=3DMsoNormal>&nbsp;&nbsp;&nbsp; Creation&nbsp;&nbsp;&nbsp; Fri May =
26 04:02:49 2006<o:p></o:p></p><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp; =
Copy&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Wed Oct 11 12:35:42 =
2006<o:p></o:p></p><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp; =
Backup&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Sun Jun 11 00:30:10 =
2006<o:p></o:p></p><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp; Last Access =
Fri Jan&nbsp; 7 16:38:32 2011<o:p></o:p></p><p =
class=3DMsoNormal>&nbsp;&nbsp;&nbsp; Last Update Wed Apr&nbsp; 4 =
15:29:42 2007<o:p></o:p></p><p class=3DMsoNormal>&nbsp;&nbsp;&nbsp; 0 =
accesses in the past day (i.e., vnode references)<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p =
class=3DMsoNormal>&nbsp;&nbsp;&nbsp; RWrite: =
536871656&nbsp;&nbsp;&nbsp;&nbsp; ROnly: =
536871657&nbsp;&nbsp;&nbsp;&nbsp; RClone: 536871657<o:p></o:p></p><p =
class=3DMsoNormal>&nbsp;&nbsp;&nbsp; number of sites -&gt; =
3<o:p></o:p></p><p =
class=3DMsoNormal>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; server =
atums1.cern.ch partition /vicepi RO Site&nbsp; -- Old =
release<o:p></o:p></p><p =
class=3DMsoNormal>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; server =
atums2.cern.ch partition /vicepb RW Site&nbsp; -- New =
release<o:p></o:p></p><p =
class=3DMsoNormal>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; server =
atums2.cern.ch partition /vicepb RO Site&nbsp; -- New =
release<o:p></o:p></p><p class=3DMsoNormal><o:p>&nbsp;</o:p></p><p =
class=3DMsoNormal>Try to bring online:<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>[atums2:~]# =
vos online -server atums2 -partition /vicepb -id =
chdata.sn<o:p></o:p></p><p class=3DMsoNormal><o:p>&nbsp;</o:p></p><p =
class=3DMsoNormal>The FileLog shows:<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>Sun Jan 23 =
22:57:03 2011 GetBitmap: addled vnode index in volume chdata.sn; volume =
needs salvage<o:p></o:p></p><p class=3DMsoNormal>Sun Jan 23 22:57:03 =
2011 VAttachVolume: error getting bitmap for volume =
(/vicepb//V0536871656.vol)<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>Try to =
Salvage:<o:p></o:p></p><p class=3DMsoNormal><o:p>&nbsp;</o:p></p><p =
class=3DMsoNormal>[atums2:~]# bos salvage atums2 /vicepb =
chdata.sn<o:p></o:p></p><p class=3DMsoNormal>Starting =
salvage.<o:p></o:p></p><p class=3DMsoNormal>bos: salvage =
completed<o:p></o:p></p><p class=3DMsoNormal><o:p>&nbsp;</o:p></p><p =
class=3DMsoNormal>The SalvageLog shows:<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>[atums2:~]# =
tail /usr/afs/logs/SalvageLog<o:p></o:p></p><p class=3DMsoNormal>@(#) =
OpenAFS 1.4.12 built&nbsp; 2010-12-13 1928681 19919656<o:p></o:p></p><p =
class=3DMsoNormal>01/23/2011 22:58:19 STARTING AFS SALVAGER 2.4 =
(/usr/afs/bin/salvager /vicepb 536871656)<o:p></o:p></p><p =
class=3DMsoNormal>01/23/2011 22:58:19 2 nVolumesInInodeFile =
64<o:p></o:p></p><p class=3DMsoNormal>01/23/2011 22:58:19 CHECKING =
CLONED VOLUME 536871657.<o:p></o:p></p><p class=3DMsoNormal>01/23/2011 =
22:58:19 chdata.sn.readonly (536871657) updated 04/04/2007 =
15:29<o:p></o:p></p><p class=3DMsoNormal>01/23/2011 22:58:19 Partially =
allocated vnode 2 deleted.<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>Try =
again:<o:p></o:p></p><p class=3DMsoNormal><o:p>&nbsp;</o:p></p><p =
class=3DMsoNormal>[atums2:~]# vos online -server atums2 -partition =
/vicepb -id chdata.sn<o:p></o:p></p><p class=3DMsoNormal><br>FileLog has =
the same message:<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>Sun Jan 23 =
22:59:05 2011 GetBitmap: addled vnode index in volume chdata.sn; volume =
needs salvage<o:p></o:p></p><p class=3DMsoNormal>Sun Jan 23 22:59:05 =
2011 VAttachVolume: error getting bitmap for volume =
(/vicepb//V0536871656.vol)<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>Salvage =
attempt again:<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>[atums2:~]# =
bos salvage atums2 /vicepb chdata.sn<o:p></o:p></p><p =
class=3DMsoNormal>Starting salvage.<o:p></o:p></p><p =
class=3DMsoNormal>bos: salvage completed<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>[atums2:~]# =
tail /usr/afs/logs/SalvageLog<o:p></o:p></p><p class=3DMsoNormal>@(#) =
OpenAFS 1.4.12 built&nbsp; 2010-12-13 1928681 19919656<o:p></o:p></p><p =
class=3DMsoNormal>01/23/2011 23:00:07 STARTING AFS SALVAGER 2.4 =
(/usr/afs/bin/salvager /vicepb 536871656)<o:p></o:p></p><p =
class=3DMsoNormal>01/23/2011 23:00:07 2 nVolumesInInodeFile =
64<o:p></o:p></p><p class=3DMsoNormal>01/23/2011 23:00:07 CHECKING =
CLONED VOLUME 536871657.<o:p></o:p></p><p class=3DMsoNormal>01/23/2011 =
23:00:07 chdata.sn.readonly (536871657) updated 04/04/2007 =
15:29<o:p></o:p></p><p class=3DMsoNormal>01/23/2011 23:00:07 Partially =
allocated vnode 2 deleted.<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p class=3DMsoNormal>Same result =
as if the prior salvage didn&#8217;t do anything.&nbsp; &nbsp;&nbsp;This =
is exactly what happens on other volumes I have tried to bring online. =
<o:p></o:p></p><p class=3DMsoNormal><o:p>&nbsp;</o:p></p><p =
class=3DMsoNormal>So how would I fix this?&nbsp;&nbsp; Any suggestions =
for how to get the rest of these volumes On-line?&nbsp; =
<o:p></o:p></p><p class=3DMsoNormal><o:p>&nbsp;</o:p></p><p =
class=3DMsoNormal>Let me know if you need further details.&nbsp; =
Thanks,<o:p></o:p></p><p class=3DMsoNormal><o:p>&nbsp;</o:p></p><p =
class=3DMsoNormal>Shawn<o:p></o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p><p =
class=3DMsoNormal><o:p>&nbsp;</o:p></p></div></body></html>
------=_NextPart_001_00A5_01CBBBA3.BCD06B30--

------=_NextPart_000_00A4_01CBBBA3.BCD06B30
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIMIzCCA/gw
ggLgoAMCAQICASkwDQYJKoZIhvcNAQEFBQAwdTETMBEGCgmSJomT8ixkARkWA25ldDESMBAGCgmS
JomT8ixkARkWAkVTMQ4wDAYDVQQKEwVFU25ldDEgMB4GA1UECxMXQ2VydGlmaWNhdGUgQXV0aG9y
aXRpZXMxGDAWBgNVBAMTD0VTbmV0IFJvb3QgQ0EgMTAeFw0wMjEyMDUwODAwMDBaFw0xMzAxMjUw
ODAwMDBaMGkxEzARBgoJkiaJk/IsZAEZFgNvcmcxGDAWBgoJkiaJk/IsZAEZFghET0VHcmlkczEg
MB4GA1UECxMXQ2VydGlmaWNhdGUgQXV0aG9yaXRpZXMxFjAUBgNVBAMTDURPRUdyaWRzIENBIDEw
ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC09dYjYaPbCD5mtbiQb7Ka3y1qAm0ZcqKC
FciWcfe8Kwcuy9tjHuIsLf9ZItdkDW4xy8sua9nJlx3KlwjtumTMtOtg35KZCknUd8KM4VGTSFdL
VG9AbNayef76caVCGM1+jyF0Lq03kauGOPTcNfZe1TZa3e1c9rc8ljV5OSWa/mfsCACyS5zFIWu0
yIDNyJdf+n0hwaPN53wllpJ30taD+JBjQ7h2k4xRWzeaznLOb9OztZVRA/1sVze+iczFh2xwa4Vd
Gy0eIIPw1pfvYwxO36rm0S109qvbsNlaroPRbxerPKakQLpKe034Xcx7gBPqUk/FxoRRWin5EWN3
rz9LAgMBAAGjgZ4wgZswDgYDVR0PAQH/BAQDAgGGMBEGCWCGSAGG+EIBAQQEAwIAhzAdBgNVHQ4E
FgQUyhkdEo5upDhdQtQxDgjb2Y0XDV0wHwYDVR0jBBgwFoAUvF1NSC/4NZRZq1yJSz7RsjoUAeow
DwYDVR0TAQH/BAUwAwEB/zAlBgNVHREEHjAcgRpET0VHcmlkcy1DQS0xQGRvZWdyaWRzLm9yZzAN
BgkqhkiG9w0BAQUFAAOCAQEAZNVrIDLqe39CEOiJt7Q7EpBPhAihMvDTSf/42u0SMbUmChww4mLm
ph5DBghZUVF8Yn59kRZMn1QLOtO1HzLqvAvPITacZVPlJgG2IXzlR636YghZFAycbIUEOJDBHR4v
tQO1KDxgZwvAbtmKIoxvhUCq2xsfFt9kCBBn+JYtQ6O5LsBJq3PmuubeMcc7mbQAfJZ7h/3Qghgk
FIhmE1+LBXPJbkuP8vgfg6h2BKoAf5TFfZECgGZKimfN110tBvfedGZwYYd3/GsJc83B0JN1gny0
gqNVPm392UchXGeBRrHnm2gkhIkr48Oq6EmNGV9/a6XfbplQW/JWbtPVPWkaizCCBAAwggLooAMC
AQICASEwDQYJKoZIhvcNAQEFBQAwdTETMBEGCgmSJomT8ixkARkWA25ldDESMBAGCgmSJomT8ixk
ARkWAkVTMQ4wDAYDVQQKEwVFU25ldDEgMB4GA1UECxMXQ2VydGlmaWNhdGUgQXV0aG9yaXRpZXMx
GDAWBgNVBAMTD0VTbmV0IFJvb3QgQ0EgMTAeFw0wMjEwMDgwNzAwMDBaFw0yMjEwMjYwNzAwMDBa
MHUxEzARBgoJkiaJk/IsZAEZFgNuZXQxEjAQBgoJkiaJk/IsZAEZFgJFUzEOMAwGA1UEChMFRVNu
ZXQxIDAeBgNVBAsTF0NlcnRpZmljYXRlIEF1dGhvcml0aWVzMRgwFgYDVQQDEw9FU25ldCBSb290
IENBIDEwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCyhT19UCvOoOts/yl9KUQLUZ0v
FV5qy1A2hTMGcihHrnlsZK8psr57g6OIQgz6yHxMyHAFpychNXr+GlsN5d0Ju+Rg1IsleNNTqeff
Ij0+W2wjJRyXGAQDcETSp77WfA3KFPQWu+balEB1dU0cejZMNQEOYDTEErmecArA0RBrPgXtyUWt
OkSX/JOYMOaHagYwk5BpiMAnMdahLixlznEKDPshCmbw+POTIjymhCmCQUQx4tIfr749Y/IHvUif
CcujC+AftkDS2+3040rrTBeahiSpQoB8OnTSx7ltoJUtSmdw7dYl5DuknaMHJbCr6RYVvXwyH7bv
0znDCziYuX4FAgMBAAGjgZowgZcwEQYJYIZIAYb4QgEBBAQDAgAHMCEGA1UdEQQaMBiBFkVTbmV0
LVJvb3QtQ0EtMUBlcy5uZXQwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUvF1NSC/4NZRZq1yJ
Sz7RsjoUAeowHwYDVR0jBBgwFoAUvF1NSC/4NZRZq1yJSz7RsjoUAeowDgYDVR0PAQH/BAQDAgGG
MA0GCSqGSIb3DQEBBQUAA4IBAQAOxsv4RoupyosJtnwF70DVOznvOlRs/dlTyAsYwsggjFt7mLhQ
CVU4wesETFB/38NNSPqQvQmfkVCIcVqhSiYs/ln945Kvtuaz0NlXuRU3rAf7q1J3MKmSkWjpOUrU
luV0IwiHypO1uqugSZN5zAJuu27UqEXRPTX7HnxKR2ySfnbY0YMicfkF+7Dn7SLKhzFphI+0EwI6
Aib60k12rxyqLEObPmj8km2ekejAgc07FMyZNX20768yZ84tLB+Hsmde1sUY/AcZ8jegRzP9IHq9
m/3Or6361uu/yegNHmwpW3bpxXbGMT7XB+PcqLoE6sihFRHh+4YVZDh9Gxn+ENFXMIIEHzCCAweg
AwIBAgIDAK3kMA0GCSqGSIb3DQEBBQUAMGkxEzARBgoJkiaJk/IsZAEZFgNvcmcxGDAWBgoJkiaJ
k/IsZAEZFghET0VHcmlkczEgMB4GA1UECxMXQ2VydGlmaWNhdGUgQXV0aG9yaXRpZXMxFjAUBgNV
BAMTDURPRUdyaWRzIENBIDEwHhcNMTAwNTEwMTkzODA1WhcNMTEwNTEwMTkzODA1WjBcMRMwEQYK
CZImiZPyLGQBGRYDb3JnMRgwFgYKCZImiZPyLGQBGRYIZG9lZ3JpZHMxDzANBgNVBAsTBlBlb3Bs
ZTEaMBgGA1UEAxMRU2hhd24gTWNLZWUgODM0NjcwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQC64ipPSwPcHoJMXNcMD/dUglGl7Zv6i4lnRMjgwj4Ln5fkPtUcC+2EU1IFcNUi5Fynbbh+
tHUyGlclPjVZubsxQXuYeAPFhiyu12uqu0x0YOjnU7u7U+8oS0sEdPOy+9HJejGxxWkIrOVA54yN
0I9+fO51/p2NgBJ0zI9qfH0agwGuS+9Sfx8htfWn24kTNqQ1SQZRe8wKfBZOaK0e0ZnmZ5DamagV
1lzo07qI1yFuAYqOwd0worVVwYpA840epRF+jWZsAgQE9VwPc7RWY9ML6NyjgncnEdPXWVH8niLB
0LqCecO0KIpBwYHevj2B4b8zk3IFqAx4Qer3DkCOSUVLAgMBAAGjgdwwgdkwEQYJYIZIAYb4QgEB
BAQDAgXgMA4GA1UdDwEB/wQEAwIE8DA2BgNVHSAELzAtMA0GCyqGSIb3TAMHAQMBMAwGCiqGSIb3
TAUCAgEwDgYMKoZIhvdMBQIDAgEBMD4GA1UdHwQ3MDUwM6AxoC+GLWh0dHA6Ly9jcmwuZG9lZ3Jp
ZHMub3JnLzFjM2YyY2E4LzFjM2YyY2E4LmNybDAbBgNVHREEFDASgRBzbWNrZWVAdW1pY2guZWR1
MB8GA1UdIwQYMBaAFMoZHRKObqQ4XULUMQ4I29mNFw1dMA0GCSqGSIb3DQEBBQUAA4IBAQBlyn/k
dOmigE+bXjwXcAUTqSDRuNDCGzBJ6qbSG6GqpByyXFbdfwq4/XAm+vvvQD0iZh1MGZ/iD5xycD5m
kVtFI5T9C+SfhcE+y6VriJOVVvZY5dB9Dpyzhv98ElZQKH/nb2y11CjZ0ELJSg2GLT4fJlT1OW6m
HOw/fGNMr97dK8Y2UdCCx+uW1RXYhAIzV3zFCoudHMR83kH5ee+Pd6TSpc/xBVQxN/zzNCw0YjST
veX3sfLOs0c/A+knug7YNwg7bInoAboE1rP6cARanWoQif9vCGOMXN9r8F1LEJTfozrlmfZPbNg3
9cLMHtrREyb1ARuZKWrjt9IE1N+GIAWPMYIDtzCCA7MCAQEwcDBpMRMwEQYKCZImiZPyLGQBGRYD
b3JnMRgwFgYKCZImiZPyLGQBGRYIRE9FR3JpZHMxIDAeBgNVBAsTF0NlcnRpZmljYXRlIEF1dGhv
cml0aWVzMRYwFAYDVQQDEw1ET0VHcmlkcyBDQSAxAgMAreQwCQYFKw4DAhoFAKCCAhwwGAYJKoZI
hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTEwMTI0MTM1MDIzWjAjBgkqhkiG
9w0BCQQxFgQUJIOG1oy6tmLv2ybsjD1yWuhFJ0cwfwYJKwYBBAGCNxAEMXIwcDBpMRMwEQYKCZIm
iZPyLGQBGRYDb3JnMRgwFgYKCZImiZPyLGQBGRYIRE9FR3JpZHMxIDAeBgNVBAsTF0NlcnRpZmlj
YXRlIEF1dGhvcml0aWVzMRYwFAYDVQQDEw1ET0VHcmlkcyBDQSAxAgMAreQwgYEGCyqGSIb3DQEJ
EAILMXKgcDBpMRMwEQYKCZImiZPyLGQBGRYDb3JnMRgwFgYKCZImiZPyLGQBGRYIRE9FR3JpZHMx
IDAeBgNVBAsTF0NlcnRpZmljYXRlIEF1dGhvcml0aWVzMRYwFAYDVQQDEw1ET0VHcmlkcyBDQSAx
AgMAreQwgbcGCSqGSIb3DQEJDzGBqTCBpjALBglghkgBZQMEASowCwYJYIZIAWUDBAEWMAoGCCqG
SIb3DQMHMAsGCWCGSAFlAwQBAjAOBggqhkiG9w0DAgICAIAwBwYFKw4DAgcwDQYIKoZIhvcNAwIC
AUAwDQYIKoZIhvcNAwICASgwBwYFKw4DAhowCwYJYIZIAWUDBAIDMAsGCWCGSAFlAwQCAjALBglg
hkgBZQMEAgEwCgYIKoZIhvcNAgUwDQYJKoZIhvcNAQEBBQAEggEAO++60w0MWFexIiteRGMucsvd
KbOmwFF9KrKSLcxQmFzp6JJlTAKZIa7VUyyXjKJMAYR2lpPXNH7kbi9p8Qy3U1sM+VxdXpiQft6f
P2StTUEP1aTo8aWeErNtD5V76POp9R/hmtB3SNXtnVlfwHQOaAFFp+WAM91XdreG1m2jQ9wCLAAq
+O7KkwotCrjLTvc/eDSwBP04De6+aJUqHzswKfkktDsTL4irsmI9nrJwUP2U7aHqPR/Yp1/rxTJ3
medYUqqw5Evr/KSCrXEe6W1RkgjKzOTN/dVUOJ2g1Gi6jR3CNlXZS3YBB+SR/OFRwuIEk2kmnE0i
1hc9XAhcVmtd4QAAAAAAAA==

------=_NextPart_000_00A4_01CBBBA3.BCD06B30--