[OpenAFS] Access an OpenAFS cell in LAN and WAN with dynamic DNS (DDNS) address

Wed, 31 Aug 2016 10:55:07 -0400

This is a cryptographically signed message in MIME format.

--------------ms030403090802090204090003
Content-Type: multipart/mixed;
 boundary="------------020A75994817685A50B760DD"

This is a multi-part message in MIME format.
--------------020A75994817685A50B760DD
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 8/26/2016 9:43 AM, Karl-Philipp Richter wrote:
>
> Are there any plans to use name resolution in OpenAFS? It's a major
> technology that exists for decades and for a reason. It'd make all our
> lives much easier.

Kalle,

There are two types of servers that the AFS cache manager
communicates with.  The first is the Volume Location servers (port 7003)
whose endpoints are determined by querying the client's CellServDB file.
The CellServDB file format is (approximately):

 >cellname optional-linked-cellname # Description of Cell
 ipv4-addr-of-vlserver    #    DNS-hostname-of-vlserver
 ...
 >nextcell ...
 ....

Those not familiar with the CellServDB format mistakenly
assume that the '#' is a comment.  It is not, its a mandatory field
separator.

The list of server records (ipv4-addr # dns-hostname) is optional and if
the AFS cache manager is started with the -afsdb command-line parameter
then cells which cannot be resolved from the CellServDB file will be
resolved via a combination of

 * DNS SRV records or DNS AFSDB records
 * DNS A records

It is important to note that the behavior of the OpenAFS *nix cache
manager and the Windows cache manager and Arla differ.  For example,
whereas the OpenAFS *nix cache manager only uses the ipv4 addresses from
the CellServDB file, the Windows cache manager always performs DNS A or
CNAME lookups on the DNS-hostname.

Whereas the *nix cache manager, only resolves volume location servers
for a cell at startup, the Windows cache manager tracks the DNS TTL and
applies a configurable TTL to the CellServDB entries.

Part of the reason for the difference in behavior in this regard is
because the *nix kernel module is not the entity that reads the
CellServDB file or performs DNS lookups.  The userland "afsd" process
parses the CellServDB info and pushes the results into the kernel.
Similarly to how an end-user can do so using the "fs newcell" command.

The volume location servers maintain a database, VLDB, which stores the
mapping of volume names and numeric identifiers to locations (file
servers and partitions.)   It also maintains a record of file server
identifiers and the list of ipv4 addresses on which the file server can
be contacted.   This latter list is updated each time the file server
restarts which is why it is necessary for the file server to be
restarted each time its ipv4 address configuration changes.

Public NAT addresses are registered by a file server by including it in
the NetInfo file on the file server.  Private NAT addresses are excluded
from the registration by including them in the NetRestrict file.

Each time the file server registration alters the contents of the
matching VLDB record, a counter for that file server is incremented.
This counter combined with the file server identifier are delivered to
cache managers as the location of a volume.  The Volume Location service
also provides an RPC that permits cache managers to query the set of
ipv4 addresses associated with a given file server identifier and
counter.  It is this mechanism that permits the AFS cache manager to
cache the locations of file servers.

The underlying issues that you are facing are:

1. Each time your public NAT address changes you have to update
   each of the server CellServDB and NetInfo files and restart
   the affected servers.  If one of those servers is a Ubik server
   (VLDB, PRDB, etc.) then it is critical that the CellServDB on each
   of the servers be updated to include the updated address. The cell
   must then be restarted.

   For Ubik it is critical that all of the servers in a quorum see
   the same set of ipv4 addresses for all of the servers in the
   quorum.  It is not safe to rely upon the DNS responses that might
   be altered by a NAT gateway and if DNS-SEC is used the NAT
   could not alter them.

   If the cell uses DNS records SRV or AFSDB to distribute the location
   of the Ubik database servers to clients, then those records must
   be updated.

   File servers contact the Volume Location and Protection servers
   and do so by using the ipv4 addresses in the server CellServDB.
   Servers in OpenAFS never use the DNS hostnames to contact one
   another.

2. Each time your public NAT address for a VL server changes you
   have to do one of the following on each *nix cache manager:

   a. update CellServDB and restart
   b. "fs newcell" the updated VL server info for the cell

Knowing that the IP addresses of your cell are unstable you can automate
these operations using scripts that execute periodically to check the
public NAT address.  However, even if you have such scripts in
place there is a strong likelihood that there will be outages since DNS
resolvers are permitted to cache records for the full TTL of the
response.  Therefore, it is possible that the automated script would
fail to notice the address change for up to a TTL.

If OpenAFS supported IPv6 (see Simon Wilkinson's talk "The Road to IPv6"
from the 2015 Best Practices Workshop)

 http://workshop.openafs.org/afsbpw15/talks/thursday/wilkinson-road-to-af=
s.pdf

you could obtain your own IPv6 tunnel from Hurricane Electric and avoid
the IPv4 NAT nonsense when IPv6 is available to your client systems.  Of
course, OpenAFS doesn't. Alternatives include requiring the use of a VPN
to access the cell and ensuring that only private addresses are
published for cell servers.

As indicated above, DNS can be used by clients to locate the database
services: Volume Location, Protection, etc.  DNS cannot be used by
servers to locate other servers.  Nor can DNS be used by clients to
locate file servers.

When AuriStor added IPv6 support to its product one of the goals was
support for 6to4 gateways such as Microsoft Direct Access.  These
gateways rely upon DNS queries to obtain a AAAA record for a host that
only has an IPv4 address.  The AFS3 model of embedding IPv4 addresses in
the VL_GetAddrsU RPC response posed a challenge because an IPv6 only
client must implement the necessary logic to identify the correct ipv6
prefix to use in converting the private IPv4 address to an IPv6 address
that will be routed through the 6to4 gateway. Other protocols such as
FTP that embed addresses in the control stream have similar problems.

Although it might be tempting to extend the VL RPCs to permit
registration of file servers by DNS host name instead of by IP address,
a better approach might be to add split horizon support to the VL
service such that file servers can register both the public and private
addresses with the VL service but the VL service would only disclose one
based upon which address the client's query is received from.

It would not be safe to convert the Ubik servers to rely upon DNS host
names.  Not with the current Ubik protocol specification.

Jeffrey Altman

--------------020A75994817685A50B760DD
Content-Type: text/x-vcard; charset=utf-8;
 name="jaltman.vcf"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
 filename="jaltman.vcf"

begin:vcard
fn:Jeffrey Altman
n:Altman;Jeffrey
org:AuriStor, Inc.
adr:Suite 6B;;255 West 94Th Street;New York;New York;10025-6985;United St=
ates
email;internet:jaltman@auristor.com
title:Founder and CEO
tel;work:+1-212-769-9018
note;quoted-printable:LinkedIn: https://www.linkedin.com/in/jeffreyaltman=
=3D0D=3D0A=3D
	Skype: jeffrey.e.altman=3D0D=3D0A=3D
=09
url:https://www.auristor.com/
version:2.1
end:vcard

--------------020A75994817685A50B760DD--

--------------ms030403090802090204090003
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCC
DEkwggYFMIIE7aADAgECAhAxSdDYnMC0s7K+ddH7ywANMA0GCSqGSIb3DQEBCwUAMIGmMQsw
CQYDVQQGEwJVUzEdMBsGA1UEChMUU3ltYW50ZWMgQ29ycG9yYXRpb24xHzAdBgNVBAsTFlN5
bWFudGVjIFRydXN0IE5ldHdvcmsxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3
MDUGA1UEAxMuU3ltYW50ZWMgQ2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBH
NTAeFw0xNTExMDEwMDAwMDBaFw0xNjExMDEyMzU5NTlaMIGnMS4wLAYDVQQDDCVQZXJzb25h
IE5vdCBWYWxpZGF0ZWQgLSAxNDQ2NDA0MDI1NjI1MSMwIQYJKoZIhvcNAQkBFhRqYWx0bWFu
QGF1cmlzdG9yLmNvbTEPMA0GA1UECwwGUy9NSU1FMR4wHAYDVQQLDBVQZXJzb25hIE5vdCBW
YWxpZGF0ZWQxHzAdBgNVBAsMFlN5bWFudGVjIFRydXN0IE5ldHdvcmswggEiMA0GCSqGSIb3
DQEBAQUAA4IBDwAwggEKAoIBAQC2cv6bENZULsb1FNyEBI47G2kA7Rogocg5u0qnQuDMCCNH
DnXkI62H2z/464AS8AGp4FcZIdvCPYp0POFlOl2XEiyA59FEDi+s3b33nLrnVOa4esw2NFBi
ZvwHvniUjgwWSUcS5V5+VhXeIKBL1U1NtMtB2XEVnc72RiQZB3KqzlS9GUvtSTdmCc6ULD/t
P009yCsBqY6NR/nug5NtUja2N+xUC2l8coYU1ingj/M4+fT8KcSq5t8laj0E+3X9ZPYlN9in
L364hXB1b+EHfxp0F0wZdsCapkEKV2VN16S1Ee5rccJIMaAJxrybK7NdI36Es/XqWlQzSeVN
wUYdwejrAgMBAAGjggIqMIICJjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIFoDAgBgNV
HSUBAf8EFjAUBggrBgEFBQcDBAYIKwYBBQUHAwIwHQYDVR0OBBYEFInhroYG7TrlBeOJmwRc
a0rfA3jtMB8GA1UdEQQYMBaBFGphbHRtYW5AYXVyaXN0b3IuY29tMGwGA1UdIARlMGMwYQYL
YIZIAYb4RQEHFwEwUjAmBggrBgEFBQcCARYaaHR0cDovL3d3dy5zeW1hdXRoLmNvbS9jcHMw
KAYIKwYBBQUHAgIwHBoaaHR0cDovL3d3dy5zeW1hdXRoLmNvbS9ycGEwXQYDVR0fBFYwVDBS
oFCgToZMaHR0cDovL3BraS1jcmwuc3ltYXV0aC5jb20vY2FfNTdkZTdhMjM4ZDQ1ZDhkNGZj
NzFmOGE1YzZiODFjOTMvTGF0ZXN0Q1JMLmNybDBOBggrBgEFBQcBAQRCMEAwPgYIKwYBBQUH
MAKGMmh0dHA6Ly9jYWNlci5zeW1hdXRoLmNvbS9tcGtpL3N5bWNjMWluZHN1YmNhZzUuY3J0
MB8GA1UdIwQYMBaAFGcZtj2lebszYNgtU9OMCT0HrBhwMCsGCmCGSAGG+EUBEAMEHTAbBhJg
hkgBhvhFARABAgIEAa3ukhMWBTEwOTIyMDkGCmCGSAGG+EUBEAUEKzApAgEAFiRhSFIwY0hN
Nkx5OXdhMmt0Y21FdWMzbHRZWFYwYUM1amIyMD0wDQYJKoZIhvcNAQELBQADggEBAHhpoe3z
j0uxbWFz4Q7F2KyRJTgREbQ+imVv3ibbd2uvnckJ+vTNJuFoaORXKTt3B8TUT8rBTwXm35D5
K4DOrKMVS0iyR9PDobLpjQM8FcIGUdbGWeZZq/1zoWhi3RtFNzZaSKXjwiQUUWYTZUE7rUmj
qu7fiACksPAAdyG12FJtk1pdVByqmaFC75/z2Qs/jVjyySpy2SRg8wlpqM2h8tl9SGska/BT
gXLHJMhKrHQvBi+9Xo3MrmJA5Z3f5vcheoOAoM5/ScHBUXWDG0+NnEIb340By0adU823pF9c
K4gxI0ZJNID+eaT0XCqg47QFOZYZUlm3s4rDi1l1iPIHekwwggY8MIIFJKADAgECAhAHAqIa
hbhLZZ4YCm7m9aNlMA0GCSqGSIb3DQEBCwUAMIHKMQswCQYDVQQGEwJVUzEXMBUGA1UEChMO
VmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlTaWduIFRydXN0IE5ldHdvcmsxOjA4BgNV
BAsTMShjKSAxOTk5IFZlcmlTaWduLCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkx
RTBDBgNVBAMTPFZlcmlTaWduIENsYXNzIDEgUHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlv
biBBdXRob3JpdHkgLSBHMzAeFw0xNTEwMDEwMDAwMDBaFw0yNTA5MzAyMzU5NTlaMIGmMQsw
CQYDVQQGEwJVUzEdMBsGA1UEChMUU3ltYW50ZWMgQ29ycG9yYXRpb24xHzAdBgNVBAsTFlN5
bWFudGVjIFRydXN0IE5ldHdvcmsxHjAcBgNVBAsTFVBlcnNvbmEgTm90IFZhbGlkYXRlZDE3
MDUGA1UEAxMuU3ltYW50ZWMgQ2xhc3MgMSBJbmRpdmlkdWFsIFN1YnNjcmliZXIgQ0EgLSBH
NTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANb7DTFJIsAMuu9OLRe4tDDlNNKs
Ce7JiQBd9K4TEbMTjlXPIhioIAam/SXLaJS0oIKQDm6WkkTuFbm07PrPeT1p1Jwf3CCaDijV
DbXGnjrx1GUZwIQzTY7aivHWJ7kDoNH6KJyCk2z3DYV9W+lOmWL+k0LS7j6zcVeGl0bL3w3h
xoRasw2P9fQQigVdn2hG7AiwWEKC9r4tEEamJAsn/pgUU4OTgtvqwD9PolhhtUtyaRJfM1n2
+bNMAGTOhcWGkgxuHOsoz3GpkKl0mXQk60jhDl1oEqgBZujumrIv+D3Nt3gkzqVgfOgWPUnx
B7ozvjIrwmejFsdvwNJalATCa0UCAwEAAaOCAj4wggI6MDcGCCsGAQUFBwEBBCswKTAnBggr
BgEFBQcwAYYbaHR0cDovL3BraS1vY3NwLnN5bWF1dGguY29tMBIGA1UdEwEB/wQIMAYBAf8C
AQAwbAYDVR0gBGUwYzBhBgtghkgBhvhFAQcXATBSMCYGCCsGAQUFBwIBFhpodHRwOi8vd3d3
LnN5bWF1dGguY29tL2NwczAoBggrBgEFBQcCAjAcGhpodHRwOi8vd3d3LnN5bWF1dGguY29t
L3JwYTAvBgNVHR8EKDAmMCSgIqAghh5odHRwOi8vcy5zeW1jYi5jb20vcGNhMS1nMy5jcmww
DgYDVR0PAQH/BAQDAgEGMCkGA1UdEQQiMCCkHjAcMRowGAYDVQQDExFTeW1hbnRlY1BLSS0y
LTIxNzAdBgNVHQ4EFgQUZxm2PaV5uzNg2C1T04wJPQesGHAwgfEGA1UdIwSB6TCB5qGB0KSB
zTCByjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZW
ZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMTk5OSBWZXJpU2lnbiwgSW5j
LiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxWZXJpU2lnbiBDbGFzcyAx
IFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5IC0gRzOCEQCLW3VWhFSF
CwDPrzhIzrGkMA0GCSqGSIb3DQEBCwUAA4IBAQBGGeQndTu+r+LaohRgjx5EkIFpK2NJNY3p
drqfmI8TgdfL/GUb+A5j3pQodPvjb9jJKnoOVKaDrilK9itR/T04ErvdY0X7ZMw+VIZ/TkJt
I7cdC/36zA2OkzXK5UX5zy9/eT1jGMdHI0r2qRQArX5ZVRqJJ9uUoJE4xv5AlaNg9l24yMUW
7ZxmaRRGEErKcCpv0VDgJhrTUrRHcotF0r0DuaXc2QjzkKt0cKvKoE7wwE7k4L5PkBFgJwwr
HN/nbMp1tCXnkUiqkrRRdV8pm0cXHL3J6s91rXUjz/LF31qrt2vKu7heq9WjcNRo8xd6mwug
FDz76IFWaOjPXXxzuC68MYIEYjCCBF4CAQEwgbswgaYxCzAJBgNVBAYTAlVTMR0wGwYDVQQK
ExRTeW1hbnRlYyBDb3Jwb3JhdGlvbjEfMB0GA1UECxMWU3ltYW50ZWMgVHJ1c3QgTmV0d29y
azEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5TeW1hbnRlYyBD
bGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQSAtIEc1AhAxSdDYnMC0s7K+ddH7ywAN
MA0GCWCGSAFlAwQCAQUAoIICdzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3
DQEJBTEPFw0xNjA4MzExNDU1MDdaMC8GCSqGSIb3DQEJBDEiBCDVWZuDxNwtKThhqZXBQLSO
b3Q4BcBAXw39TjvYWCiAdjBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjALBglghkgB
ZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsO
AwIHMA0GCCqGSIb3DQMCAgEoMIHMBgkrBgEEAYI3EAQxgb4wgbswgaYxCzAJBgNVBAYTAlVT
MR0wGwYDVQQKExRTeW1hbnRlYyBDb3Jwb3JhdGlvbjEfMB0GA1UECxMWU3ltYW50ZWMgVHJ1
c3QgTmV0d29yazEeMBwGA1UECxMVUGVyc29uYSBOb3QgVmFsaWRhdGVkMTcwNQYDVQQDEy5T
eW1hbnRlYyBDbGFzcyAxIEluZGl2aWR1YWwgU3Vic2NyaWJlciBDQSAtIEc1AhAxSdDYnMC0
s7K+ddH7ywANMIHOBgsqhkiG9w0BCRACCzGBvqCBuzCBpjELMAkGA1UEBhMCVVMxHTAbBgNV
BAoTFFN5bWFudGVjIENvcnBvcmF0aW9uMR8wHQYDVQQLExZTeW1hbnRlYyBUcnVzdCBOZXR3
b3JrMR4wHAYDVQQLExVQZXJzb25hIE5vdCBWYWxpZGF0ZWQxNzA1BgNVBAMTLlN5bWFudGVj
IENsYXNzIDEgSW5kaXZpZHVhbCBTdWJzY3JpYmVyIENBIC0gRzUCEDFJ0NicwLSzsr510fvL
AA0wDQYJKoZIhvcNAQEBBQAEggEAFUg+/qChkNEICn5K0xsO+4SPz9o2WdaaonEwbyorQXhY
6xHztnwKwGH3loCl9h99NlT+jes0kUG/AVi2Flv7aooNh4VYL8X1tbqTrf1ZeB5xotvunPLl
7C0IUfHLBLLxTo9rnAsXYLNxsZxD+WYY8sTAZK5r30hlYNVK52Xb1cZ5n7ukYl6zT4SwJl7G
c8z5OmiODIlh8NJXH4f701eSO6OR5O/HRbQME204Smsk0gGKohXGOFN3mtX5ppH8C9qJqBOn
AxzWtA2WLnxbh4yKZEGPVSKtRbdyt0aat8KwHIPAvj10JB2wb6q03cUUwzGAKJdB+imqA7O6
hv+oak8twAAAAAAAAA==
--------------ms030403090802090204090003--