[OpenAFS] vos dumps to S3 via S3 Storage Gateway?

[Jeffrey Altman]

This is a cryptographically signed message in MIME format.

--------------ms060201090604000608090907
Content-Type: multipart/mixed;
 boundary="------------A3CB5EC7F5FC6034D4271252"

This is a multi-part message in MIME format.
--------------A3CB5EC7F5FC6034D4271252
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 3/3/2017 12:29 PM, Harald Barth wrote:
>=20
> adsmpipe replacement:
>=20
> /afs/hpc2n.umu.se/lap/tsmpipe/x.x/src/
>=20
> Used with some scripts do put vos dumps into TSM archive. This is the
> current backup solution for at least 3 AFS cells I know about.
>=20
> Harald.

There is also LTU's tsmafs which is available on GitHub

  https://github.com/mattiaspantzare/tsmafs

On 3/1/2017 1:52 PM, Dave Botsch wrote:
> Sounds like the most recent TSM patches may or may not be in the
> OpenAFS tree?
>
> Are you aware of any reason that this api is not enabled by default? I
> believe it would be a huge win for OpenAFS to be able to advertise
> native TSM support.

The primary reason is because a third party sdk is required to
use the api and that sdk is licensed for use only to customers
that have a valid license for the commercial product.

Beyond that there were other reasons.  The current TSM support was
merged into the OpenAFS repository prior to the existence of the
Gerrit review system and the buildbot continuous integration system.
It was merged without significant review by third parties.  The code
quality is quite poor as Anders noticed in Aug 2015.

  https://gerrit.openafs.org/#/c/11960/

Since there was no method by which the Gatekeepers could test
the TSM functionality nor guarantee that it didn't alter the behavior
of backups for non-TSM using organizations, the decision was made
to merge the code as a build time option for those organizations
that wanted it.

On 3/3/2017 11:00 AM, David Boyes wrote:
> The IBM-supplied TSM butc support relies on a XBSA (an OpenGroup
> standard) compatibility library that was not updated past version 6.1
> of the TSM client on HP/UX, Solaris SPARC and AIX. Linux and Solaris
> x86 were never supported for the XBSA-based client. A fairly
> substantial amount of work would be needed to bring that support up
> to the current client levels (basically recoding to support the
> native TSM API). There was some discussion about doing that circa
> 2009, unclear if anything happened with that.

The XBSA standard was adopted not only by Tivoli Storage Manager but
also by Veritas NetBackup.  As David Boyes said, IBM abandoned the
XBSA standard and now only supports their own proprietary API (loosely
based on the XBSA model.)  OpenAFS only ever included support for TSM,
not NetBackup.

The Backup Tape Controller (butc) when XBSA is supported permits a
remote XBSA enabled backup system to be used in place of a tape
device or local file system.  Full and incremental volume dumps
are sent to butc and stored in the XBSA service and the object
identifier of the specific backup is stored in the AFS backup database
just as if the dump had been stored to a tape device.

XBSA and the Spectrum Protect SDK are fresh in my mind because AuriStor
recently finished integrating Spectrum Protect support into the AuriStor
File System. AuriStorFS now supports IBM Spectrum Protect and the older
Tivoli Storage Manager releases. This is in addition to our support of
Teradactyl's True Incremental Backup System and BackupAFS.
The XBSA implementation is modular so we can add support for Veritas
NetBackup and object stores in the near future.

Jeffrey Altman
AuriStor, Inc.


--------------A3CB5EC7F5FC6034D4271252
Content-Type: text/x-vcard; charset=utf-8;
 name="jaltman.vcf"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
 filename="jaltman.vcf"

begin:vcard
fn:Jeffrey Altman
n:Altman;Jeffrey
org:AuriStor, Inc.
adr:Suite 6B;;255 West 94Th Street;New York;New York;10025-6985;United St=
ates
email;internet:jaltman@auristor.com
title:Founder and CEO
tel;work:+1-212-769-9018
note;quoted-printable:LinkedIn: https://www.linkedin.com/in/jeffreyaltman=
=3D0D=3D0A=3D
	Skype: jeffrey.e.altman=3D0D=3D0A=3D
=09
url:https://www.auristor.com/
version:2.1
end:vcard


--------------A3CB5EC7F5FC6034D4271252--

--------------ms060201090604000608090907
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCC
DIMwggXqMIIE0qADAgECAhBAAVgjDHzXCy5hFo6GsQuLMA0GCSqGSIb3DQEBCwUAMDoxCzAJ
BgNVBAYTAlVTMRIwEAYDVQQKEwlJZGVuVHJ1c3QxFzAVBgNVBAMTDlRydXN0SUQgQ0EgQTEy
MB4XDTE2MTEwMjAzMTkzMFoXDTE3MTEwMjAzMTkzMFowgYYxLTArBgNVBAsMJFZlcmlmaWVk
IEVtYWlsOiBqYWx0bWFuQGF1cmlzdG9yLmNvbTEjMCEGCSqGSIb3DQEJARYUamFsdG1hbkBh
dXJpc3Rvci5jb20xMDAuBgoJkiaJk/IsZAEBEyA3RjAwMDAwMTAwMDAwMTU4MjMwQzdDQTcw
MDAwMDdCMjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALtO/7veyX1tdA4aDvsO
c0gS/fICCTPlhS34dpRNmZYs2mvWb/xGRCKeflva8uZLFqsCE6ybMgz2IaB9rpowRcGSr/R7
CyY6HEBl3Rw0OkkslV7HQTxgp1fBXbGGoS8gOp6/ML64d5UJsMUM6NM59SzC77v8bi0sYkaT
q31SoiJrCcPiI0F2wZiESf1tguQONl3Emt9fhEokmlLRHAlv1DkRw2XeUOyUwVHIVgF1wnzR
4Ao86aYmym62Z669a6NPla3hqLmYk9w+ydTnzcWj0X6nhvaMxb7fhOUyj06Y5Mxhqye+dJg4
bPj/G+8OoNVY9l5h/qbJaWpSbDog4I0LxaUCAwEAAaOCAp0wggKZMA4GA1UdDwEB/wQEAwIF
oDCBhAYIKwYBBQUHAQEEeDB2MDAGCCsGAQUFBzABhiRodHRwOi8vY29tbWVyY2lhbC5vY3Nw
LmlkZW50cnVzdC5jb20wQgYIKwYBBQUHMAKGNmh0dHA6Ly92YWxpZGF0aW9uLmlkZW50cnVz
dC5jb20vY2VydHMvdHJ1c3RpZGNhYTEyLnA3YzAfBgNVHSMEGDAWgBSkc9rvaTWKdcygGXsI
MvhrieRC7DAJBgNVHRMEAjAAMIIBLAYDVR0gBIIBIzCCAR8wggEbBgtghkgBhvkvAAYLATCC
AQowSgYIKwYBBQUHAgEWPmh0dHBzOi8vc2VjdXJlLmlkZW50cnVzdC5jb20vY2VydGlmaWNh
dGVzL3BvbGljeS90cy9pbmRleC5odG1sMIG7BggrBgEFBQcCAjCBrhqBq1RoaXMgVHJ1c3RJ
RCBDZXJ0aWZpY2F0ZSBoYXMgYmVlbiBpc3N1ZWQgaW4gYWNjb3JkYW5jZSB3aXRoIApJZGVu
VHJ1c3QncyBUcnVzdElEIENlcnRpZmljYXRlIFBvbGljeSBmb3VuZCBhdCBodHRwczovL3Nl
Y3VyZS5pZGVudHJ1c3QuY29tL2NlcnRpZmljYXRlcy9wb2xpY3kvdHMvaW5kZXguaHRtbDBF
BgNVHR8EPjA8MDqgOKA2hjRodHRwOi8vdmFsaWRhdGlvbi5pZGVudHJ1c3QuY29tL2NybC90
cnVzdGlkY2FhMTIuY3JsMB8GA1UdEQQYMBaBFGphbHRtYW5AYXVyaXN0b3IuY29tMB0GA1Ud
DgQWBBT6okll+NyYQTyBKCigoq5jFBzw3DAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUH
AwQwDQYJKoZIhvcNAQELBQADggEBABF9+YNU05P6tFiN8wvh52K+dgiUsQijFqN/tmwxqWHj
3TKxLwl8ZtK0F44FYcfeRXgE59bAxG8tZVRYBD7jY3qYEreri0x2DgJcA0oPfV1NHxM51t6h
ST09jpXEcthB4FJu3xWg8O6pB7Oq9Q9kkTjHfVLLru9B3BxrY+1NKZaSkaTBb0JGw6kCN2MM
rXrwPAdLlAWn5ywPRMqTIlgvhxtIQMv+dxpXhco9O5+ckMIdp1uBMJ82Hslfy3K9KCZvm6Cr
BG8C/fBo5TefVGuxKyOwROvup2gx1/KZWoIO0hyMNGzFPrijNZD5fAlZ11FcAJ9Why+pd5FY
mmJwln/BGPcwggaRMIIEeaADAgECAhEA+d5Wf8lNDHdw+WAbUtoVOzANBgkqhkiG9w0BAQsF
ADBKMQswCQYDVQQGEwJVUzESMBAGA1UEChMJSWRlblRydXN0MScwJQYDVQQDEx5JZGVuVHJ1
c3QgQ29tbWVyY2lhbCBSb290IENBIDEwHhcNMTUwMjE4MjIyNTE5WhcNMjMwMjE4MjIyNTE5
WjA6MQswCQYDVQQGEwJVUzESMBAGA1UEChMJSWRlblRydXN0MRcwFQYDVQQDEw5UcnVzdElE
IENBIEExMjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANGRTTzPCic0kq5L6ZrU
JWt5LE/n6tbPXPhGt2Egv7plJMoEpvVJJDqGqDYymaAsd8Hn9ZMAuKUEFdlx5PgCkfu7jL5z
giMNnAFVD9PyrsuF+poqmlxhlQ06sFY2hbhQkVVQ00KCNgUzKcBUIvjv04w+fhNPkwGW5M7A
e5K5OGFGwOoRck9GG6MUVKvTNkBw2/vNMOd29VGVTtR0tjH5PS5yDXss48Yl1P4hDStO2L4w
TsW2P37QGD27//XGN8K6amWB6F2XOgff/PmlQjQOORT95PmLkwwvma5nj0AS0CVp8kv0K2RH
V7GonllKpFDMT0CkxMQKwoj+tWEWJTiDKSsCAwEAAaOCAoAwggJ8MIGJBggrBgEFBQcBAQR9
MHswMAYIKwYBBQUHMAGGJGh0dHA6Ly9jb21tZXJjaWFsLm9jc3AuaWRlbnRydXN0LmNvbTBH
BggrBgEFBQcwAoY7aHR0cDovL3ZhbGlkYXRpb24uaWRlbnRydXN0LmNvbS9yb290cy9jb21t
ZXJjaWFscm9vdGNhMS5wN2MwHwYDVR0jBBgwFoAU7UQZwNPwBovupHu+QucmVMiONnYwDwYD
VR0TAQH/BAUwAwEB/zCCASAGA1UdIASCARcwggETMIIBDwYEVR0gADCCAQUwggEBBggrBgEF
BQcCAjCB9DBFFj5odHRwczovL3NlY3VyZS5pZGVudHJ1c3QuY29tL2NlcnRpZmljYXRlcy9w
b2xpY3kvdHMvaW5kZXguaHRtbDADAgEBGoGqVGhpcyBUcnVzdElEIENlcnRpZmljYXRlIGhh
cyBiZWVuIGlzc3VlZCBpbiBhY2NvcmRhbmNlIHdpdGggSWRlblRydXN0J3MgVHJ1c3RJRCBD
ZXJ0aWZpY2F0ZSBQb2xpY3kgZm91bmQgYXQgaHR0cHM6Ly9zZWN1cmUuaWRlbnRydXN0LmNv
bS9jZXJ0aWZpY2F0ZXMvcG9saWN5L3RzL2luZGV4Lmh0bWwwSgYDVR0fBEMwQTA/oD2gO4Y5
aHR0cDovL3ZhbGlkYXRpb24uaWRlbnRydXN0LmNvbS9jcmwvY29tbWVyY2lhbHJvb3RjYTEu
Y3JsMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEFBQcDBDAOBgNVHQ8BAf8EBAMCAYYwHQYD
VR0OBBYEFKRz2u9pNYp1zKAZewgy+GuJ5ELsMA0GCSqGSIb3DQEBCwUAA4ICAQAN4YKu0vv0
62MZfg+xMSNUXYKvHwvZIk+6H1pUmivyDI4I6A3wWzxlr83ZJm0oGIF6PBsbgKJ/fhyyIzb+
vAYFJmyI8I/0mGlc+nIQNuV2XY8cypPoVJKgpnzp/7cECXkX8R4NyPtEn8KecbNdGBdEaG4a
7AkZ3ujlJofZqYdHxN29tZPdDlZ8fR36/mAFeCEq0wOtOOc0Eyhs29+9MIZYjyxaPoTS+l8x
LcuYX3RWlirRyH6RPfeAi5kySOEhG1quNHe06QIwpigjyFT6v/vRqoIBr7WpDOSt1VzXPVbS
j1PcWBgkwyGKHlQUOuSbHbHcjOD8w8wHSDbL+L2he8hNN54doy1e1wJHKmnfb0uBAeISoxRb
JnMMWvgAlH5FVrQWlgajeH/6NbYbBSRxALuEOqEQepmJM6qz4oD2sxdq4GMN5adAdYEswkY/
o0bRKyFXTD3mdqeRXce0jYQbWm7oapqSZBccFvUgYOrB78tB6c1bxIgaQKRShtWR1zMM0Jfq
UfD9u8Fg7G5SVO0IG/GcxkSvZeRjhYcbTfqF2eAgprpyzLWmdr0mou3bv1Sq4OuBhmTQCnqx
AXr4yVTRYHkp5lCvRgeJAme1OTVpVPth/O7HJ7VuEP9GOr6kCXCXmjB4P3UJ2oU0NqfoQdcS
SSt9hliALnExTEjii20B2nSDojGCAxQwggMQAgEBME4wOjELMAkGA1UEBhMCVVMxEjAQBgNV
BAoTCUlkZW5UcnVzdDEXMBUGA1UEAxMOVHJ1c3RJRCBDQSBBMTICEEABWCMMfNcLLmEWjoax
C4swDQYJYIZIAWUDBAIBBQCgggGXMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZI
hvcNAQkFMQ8XDTE3MDMwMzE4NTAzN1owLwYJKoZIhvcNAQkEMSIEIIsYe9F269Dm2w2r8I9s
fAgNhSitPIRvp9m7VrpYIwAEMF0GCSsGAQQBgjcQBDFQME4wOjELMAkGA1UEBhMCVVMxEjAQ
BgNVBAoTCUlkZW5UcnVzdDEXMBUGA1UEAxMOVHJ1c3RJRCBDQSBBMTICEEABWCMMfNcLLmEW
joaxC4swXwYLKoZIhvcNAQkQAgsxUKBOMDoxCzAJBgNVBAYTAlVTMRIwEAYDVQQKEwlJZGVu
VHJ1c3QxFzAVBgNVBAMTDlRydXN0SUQgQ0EgQTEyAhBAAVgjDHzXCy5hFo6GsQuLMGwGCSqG
SIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAKBggqhkiG9w0DBzAOBggq
hkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwDQYJ
KoZIhvcNAQEBBQAEggEAOKuKLiiOIp9hYzNQ0iHCvrrMrVVi2IlHpb32snEevfyI0w2Bovh5
wM+RZe1nwbxakCEmA/JJkSjalFjQn0+TGZD3h50XCnKSrff585zfoDTUyc9FKoG2K5DvX1re
Niw+pSYdcEdkiKjBJ+mBYgxPZbmEsQEsDDDyzlIr4vXwclzrMjqy+Lcuv0H/jHGyrF4W5KsP
ukw0d72iatkH5OpK6/S6+TZEGVJrUmD9Xr5bL54HRKAy2ZOpMooJBf6iTrL59nnKDVbQmwNP
xwEjRm+LtGmvFZajMxKEzTq5EfJ/yHRhXebbc09PE3H4n0UoD0pXgo0+A/VPq4V/T5Wi38Rf
5AAAAAAAAA==
--------------ms060201090604000608090907--