[OpenAFS] Strange DNS SRV traffic resulting from stat() in 1.8.13.2

Mon, 25 Aug 2025 09:41:03 +0200

Hi Ryan, hi Jeffrey,

after seeing this issue, I looked into our nameserver log files and 
found similar entries.

There are many queries for _afs3-vlserver including a filename. 
Sometimes the full path is included but in that case the "/afs/" prefix 
is omitted.
The @sys is not involved here.

The queries are made for each domain suffix that is configured in 
/etc/resolv.conf with the "search" directive.

Here are some selected queries:

* queries from Debian Bookworm with OpenAFS Client 1.8.13-1, Kernel 
6.1.0-38-amd64
24-Aug-2025 19:30:56.916 client @0x7fa6b6f11168 
2001:638:60c:310::XXX:XXX#34611 (_afs3-vlserver._udp.htaccess): query: 
_afs3-vlserver._udp.htaccess IN SRV + (2001:638:60c:310::81:212)
25-Aug-2025 08:13:19.303 client @0x7fa6b793c168 
2001:638:60c:310::XXX:XXX#52502 
(_afs3-vlserver._udp.htaccess.informatik.uni-goettingen.de): query: 
_afs3-vlserver._udp.htaccess.informatik.uni-goettingen.de IN SRV + 
(2001:638:60c:310::81:212)

* queries from Ubuntu 24.04 with OpenAFS Client 1.8.13.2-1ubuntu1, 
Kernel 6.14.0-28-generic
24-Aug-2025 01:26:34.718 client @0x7fa6b794e168 172.27.1.150#36732 
(_afs3-vlserver._udp.timewarrior.json.informatik.uni-goettingen.de): 
query: _afs3-vlserver._udp.timewarrior.json.informatik.uni-goettingen.de 
IN SRV +E(0) (134.76.81.212)
24-Aug-2025 17:46:05.241 client @0x7fa6b783f168 172.27.1.155#41379 
(_afs3-vlserver._udp.informatik.uni-goettingen.de/user/b/xxxx.xxxx/ulsi-prefix/usr/share/desktop-directories.informatik.uni-goettingen.de): 
query: 
_afs3-vlserver._udp.informatik.uni-goettingen.de/user/b/xxxx.xxxx/ulsi-prefix/usr/share/desktop-directories.informatik.uni-goettingen.de 
IN SRV +E(0) (134.76.81.212)
25-Aug-2025 07:15:47.756 client @0x7fa6b79fe168 172.27.2.4#34129 
(_afs3-vlserver._udp.informatik.uni-goettingen.de/user/a/xxxxxx/.xxxlogin/.google_authenticator.informatik.uni-goettingen.de): 
query: 
_afs3-vlserver._udp.informatik.uni-goettingen.de/user/a/xxxxxx/.xxxlogin/.google_authenticator.informatik.uni-goettingen.de 
IN SRV +E(0) (134.76.81.212)
25-Aug-2025 07:50:09.586 client @0x7fa6b6e9e168 172.21.0.1#35881 
(_afs3-vlserver._udp.kateconfig.informatik.uni-goettingen.de): query: 
_afs3-vlserver._udp.kateconfig.informatik.uni-goettingen.de IN SRV +E(0) 
(134.76.81.212)
25-Aug-2025 07:50:09.626 client @0x7fa6b6e9e168 172.21.0.1#38919 
(_afs3-vlserver._udp.editorconfig.informatik.uni-goettingen.de): query: 
_afs3-vlserver._udp.editorconfig.informatik.uni-goettingen.de IN SRV 
+E(0) (134.76.81.212)
25-Aug-2025 07:50:10.098 client @0x7fa6b78c5168 172.21.0.1#51962 
(_afs3-vlserver._udp.kateproject.informatik.uni-goettingen.de): query: 
_afs3-vlserver._udp.kateproject.informatik.uni-goettingen.de IN SRV 
+E(0) (134.76.81.212)

The OpenAFS servers are running on Debian Bookworm with version 1.8.13-1.

The DNS SRV records for afs are configured on our nameservers.

Regards,
Gunnar

On 22/08/2025 19:08, Jeffrey Altman wrote:
> Ryan,
> 
> Since no one else has replied, I will try to explain what I think is happening although I have no idea why or how it is happening.   I will also ask a few questions.
> 
>> On Aug 17, 2025, at 7:15 AM, Ryan C. Underwood <nemesis-lists@icequake.net> wrote:
> 
>> I recently tried out the 1.8.13.2 client on my Ubuntu laptop,
>> upgrading from a somewhat older client version.
> 
> What is the version of Ubuntu and most importantly, what is the kernel version?
> 
>>   Historically, I have
>> used @sys-based components in PATH as a way of centrally managing
>> CLI tools without having to distribute them to every system I use.
>> However, this has become problematic in the following way.
> 
> I’m not sure that @sys is involved.
>>
>> Immediately after system startup, all is well, my shell profile loads
>> immediately and CLI commands also return immediately.  However, after
>> some time of system utilization, not necessarily including any heavy
>> AFS usage, doing anything within my shell profile slows to a crawl.
>> The reason is because of the AFS-based PATH component.  Instead of
>> stat() calls returning immediately, at this point each one creates
>> many DNS roundtrips fetching some very strange SRV records that I do
>> not recall seeing before, and which are perhaps malformed:
>>
>> 12:52:12.101148 IP 192.168.8.248.46167 > 192.168.8.1.53: 25095+ AFSDB?  icequake.net/users/nemesis/bin/bash. (53)
>> 12:52:12.245481 IP 192.168.8.1.53 > 192.168.8.248.46167: 25095 NXDomain 0/1/0 (128)
>> 12:52:12.246822 IP 192.168.8.248.57487 > 192.168.8.1.53: 46166+ AFSDB?  icequake.net/users/nemesis/bin/bash.icequake.net. (66)
>> 12:52:12.407925 IP 192.168.8.1.53 > 192.168.8.248.57487: 46166 NXDomain* 0/1/0 (134)
>> 12:52:12.408765 IP 192.168.8.248.60842 > 192.168.8.1.53: 45806+ SRV?  _afs3-vlserver._udp.icequake.net/users/nemesis/bin/bash. (73)
>> 12:52:12.612530 IP 192.168.8.1.53 > 192.168.8.248.60842: 45806 NXDomain 0/1/0 (148)
>> 12:52:12.613264 IP 192.168.8.248.33038 > 192.168.8.1.53: 54461+ SRV?  _afs3-vlserver._udp.icequake.net/users/nemesis/bin/bash.icequake.net.  (86)
>> 12:52:12.817311 IP 192.168.8.1.53 > 192.168.8.248.33038: 54461 NXDomain* 0/1/0 (154)
>> 12:52:12.817829 IP 192.168.8.248.33177 > 192.168.8.1.53: 53983+ SRV?  _afs3-vlserver._udp.icequake.net/users/nemesis/bin/bash. (73)
>> 12:52:12.958650 IP 192.168.8.1.53 > 192.168.8.248.33177: 53983 NXDomain 0/1/0 (148)
>> 12:52:12.959152 IP 192.168.8.248.50256 > 192.168.8.1.53: 12967+ SRV?  _afs3-vlserver._udp.icequake.net/users/nemesis/bin/bash.icequake.net.  (86)
>> 12:52:13.124390 IP 192.168.8.1.53 > 192.168.8.248.50256: 12967 NXDomain* 0/1/0 (154)
>> 12:52:13.125189 IP 192.168.8.248.47935 > 192.168.8.1.53: 56862+ AFSDB?  icequake.net/users/nemesis/bin/bash. (53)
>> 12:52:13.329797 IP 192.168.8.1.53 > 192.168.8.248.47935: 56862 NXDomain 0/1/0 (128)
>> 12:52:13.330370 IP 192.168.8.248.58386 > 192.168.8.1.53: 55085+ AFSDB?  icequake.net/users/nemesis/bin/bash.icequake.net. (66)
>> 12:52:13.537502 IP 192.168.8.1.53 > 192.168.8.248.58386: 55085 NXDomain* 0/1/0 (134)
>> 12:52:13.540776 IP 192.168.8.248.41895 > 192.168.8.1.53: 6729+ SRV?  _afs3-vlserver._udp.icequake.net/users/nemesis/@sys/bin/bash. (78)
>> 12:52:13.740662 IP 192.168.8.1.53 > 192.168.8.248.41895: 6729 NXDomain 0/1/0 (153)
>> 12:52:13.741143 IP 192.168.8.248.37102 > 192.168.8.1.53: 33110+ SRV?  _afs3-vlserver._udp.icequake.net/users/nemesis/@sys/bin/bash.icequake.net.  (91)
>> 12:52:14.159513 IP 192.168.8.1.53 > 192.168.8.248.37102: 33110 NXDomain* 0/1/0 (159)
>> 12:52:14.160055 IP 192.168.8.248.58652 > 192.168.8.1.53: 64126+ SRV?  _afs3-vlserver._udp.icequake.net/users/nemesis/@sys/bin/bash. (78)
>> 12:52:14.731726 IP 192.168.8.1.53 > 192.168.8.248.58652: 64126 NXDomain 0/1/0 (153)
>> 12:52:14.732197 IP 192.168.8.248.41774 > 192.168.8.1.53: 19149+ SRV?  _afs3-vlserver._udp.icequake.net/users/nemesis/@sys/bin/bash.icequake.net.  (91)
>> [...]
> 
> There are two types of DNS queries being issued.  The first type are DNS SRV queries for service “afs3-vlserver” using protocol “udp”.   The second type of query are the deprecated DNS AFSDB queries the DNS SRV queries are intended to replace.
> 
> However, the queries are malformed and are leaking to the world every path that is traversed.
> 
>> Can someone explain what is going on here?  Whatever is happening, it
>> is bypassing the cache manager because the same strange queries are
>> issued repeatedly as the shell profile executes.  It's additionally
>> painful in the case of my laptop, since the ultimate DNS server is on
>> the other end of a VPN link.
> 
> Unless the afsd dynamic root functionality is disabled, DNS SRV or DNS AFSDB queries would be expected if the cache manager’s CellServDB does not contain a list of database servers for the cell “ice quake.net”.  In particular we would expect to see a DNS SRV query for  :_afs3-vlserver._udp.icequake.net and if that query failed, then a fallback query for DNS AFSDB “icequake.net”.   Those queries are not present in the above trace.  As far as I can tell there are neither DNS SRV nor DNS AFSDB records published for ice quake.net.
> 
> What is odd about the above queries is that they contain the full path with the exclusion of the “/afs/“ prefix.  This is odd because it appears that the /afs/some-cell-name/path query is being treated as a single path component.  For example:
> 
>    icequake.net/users/nemesis/bin/bash
>    icequake.net/users/nemesis/bin/bash.icequake.net
>    icequake.net/users/nemesis/@sys/bin/bash
>    icequake.net/users/nemesis/@sys/bin/bash.icequake.net
> 
> The full path is usually not visible to a filesystem.  One question to answer is whether these paths are being delivered to the afs lookup function or whether they are being constructed internally?  Filesystems do not normally have visibility to full paths so I wouldn’t expect them to be constructed internally.  If the paths are being passed to the afs lookup function, then the failure to separate the components for lookup is somewhere higher in the stack.
> 
>> I believe the previous version I was using (without this new behavior)
>> was 1.8.10, but don't quote me on that.
> 
> What else changed between when 1.8.10 was installed and 1.8.13.2?
> 
> I am curious if you experience the same behavior problems with the AuriStorFS client.  If you do, that is a strong indication that the problem is external to OpenAFS.
> 
> Jeffrey Altman
> 
> 
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info