[OpenAFS-devel] is multihoming supported under openafs these
days?
Martin MOKREJŠ
mmokrejs@ribosome.natur.cuni.cz
Fri, 02 Sep 2005 16:45:18 +0200
Hi Roland,
Roland Kuhn wrote:
> Hi Martin!
>
> You mention a very dark corner of AFS on Linux. I don't know the code,
> but I also had to fight it. My findings were:
>
> - AFS uses all addresses by enumerating the network devices found by
> the kernel
> - The smallest IP number _must_ be on the first device, otherwise
> nothing works
> - It depends on pure luck if the internal cluster IPs are published to
> the outside, causing longish timeouts for client boot procedures.
>
> It would be nice to be able to tell AFS exactly which IPs to use for what.
>
> Concerning your clients question: Linux follows the weak host model,
> accepting all IPs on all interfaces. The assignment of IPs to
> interfaces only affects routing and the selection of source IPs to put
> in new packets (not responses).
>
> On Sep 2, 2005, at 3:50 , Martin MOKREJŠ wrote:
>
>> Hi,
>> I have a server with 3 network interfaces. Can I use the server 3
>> interfaces
>> and put for some clients into CellServDB IP address of eth0 or eth1
>> or eth2 interface
>> respectively?
>>
>> At the moment I thought I use eth2, which corresponds to the
>> hostname.domainname
>> but what happened is that "vos listvldb" shows IP address of wrong
>> interface (at the
>> moment not even connected to the network)
>>
>> phylo ~ # vos listvldb
>> VLDB entries for all servers
>>
>> home
>> RWrite: 536870918
>> number of sites -> 1
>> server 192.168.1.254 partition /vicepa RW Site
>>
> [snip]
>
>> Total entries: 8
>> phylo ~ # ifconfig -a
>> eth0 Link encap:Ethernet HWaddr 00:11:09:B6:C1:7A
>> inet addr:192.168.1.254 Bcast:192.168.1.255 Mask:
>> 255.255.255.0
>> UP BROADCAST MULTICAST MTU:1500 Metric:1
>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
>> Base address:0x2400 Memory:d0140000-d0160000
>>
>> eth1 Link encap:Ethernet HWaddr 00:11:09:B6:C1:7B
>> inet addr:192.168.2.254 Bcast:192.168.2.255 Mask:
>> 255.255.255.0
>> UP BROADCAST MULTICAST MTU:1500 Metric:1
>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
>> Base address:0x2440 Memory:d0180000-d01a0000
>>
>> eth2 Link encap:Ethernet HWaddr 00:0E:0C:84:83:71
>> inet addr:195.113.57.18 Bcast:195.113.57.255 Mask:
>> 255.255.255.0
>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>> RX packets:9595 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:1157 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:699461 (683.0 Kb) TX bytes:182556 (178.2 Kb)
>> Base address:0x3000 Memory:d0220000-d0240000
>>
>> lo Link encap:Local Loopback
>> inet addr:127.0.0.1 Mask:255.0.0.0
>> UP LOOPBACK RUNNING MTU:16436 Metric:1
>> RX packets:649 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:649 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:0
>> RX bytes:123551 (120.6 Kb) TX bytes:123551 (120.6 Kb)
>>
>> phylo ~ # cat /usr/vice/etc/CellServDB
>>
>>> phylo.natur.cuni.cz #Cell name
>>>
>> 195.113.57.18 #phylo.natur.cuni.cz
>
>
> This is only what the clients try to connect to, right?
Will try to, yes. ;) But if possible, some clients would be pointed
in CellServDB to 192.168.1.254 or 192.168.2.254.
BTW:
/usr/afs/logs/FileLog:
Fri Sep 2 16:39:04 2005 File server starting
Fri Sep 2 16:39:05 2005 VL_RegisterAddrs rpc failed; will retry periodically (code=5376, err=2)
Fri Sep 2 16:39:05 2005 Set thread id 14 for FSYNC_sync
Fri Sep 2 16:39:05 2005 Partition /vicepa: attaching volumes
Fri Sep 2 16:39:05 2005 Partition /vicepa: attached 10 volumes; 0 volumes not attached
Fri Sep 2 16:39:05 2005 Getting FileServer name...
Fri Sep 2 16:39:05 2005 FileServer host name is 'phylo'
Fri Sep 2 16:39:05 2005 Getting FileServer address...
Fri Sep 2 16:39:05 2005 FileServer phylo has address 195.113.57.18 (0x123971c3 or 0xc3713912 in host byte order)
Fri Sep 2 16:39:05 2005 File Server started Fri Sep 2 16:39:05 2005
Fri Sep 2 16:39:05 2005 Set thread id 15 for 'FiveMinuteCheckLWP'
Fri Sep 2 16:39:05 2005 Set thread id 16 for 'HostCheckLWP'
Fri Sep 2 16:39:05 2005 Set thread id 17 for 'FsyncCheckLWP'
Please note the 195.113.57.18 address picked up (probably because I always used "bos -server phylo" which
translates to the eth2 based IP address?).
/usr/afs/logs/PtLog:
Fri Sep 2 16:39:04 2005 Using 195.113.57.18 as my primary address
/usr/afs/logs/VLLog:
Fri Sep 2 16:39:04 2005 Using 195.113.57.18 as my primary address
Fri Sep 2 16:39:04 2005 Starting AFS vlserver 4 (/usr/afs/bin/vlserver)
/usr/afs/logs/VolserLog:
Fri Sep 2 16:39:08 2005 Starting AFS Volserver 2.0 (/usr/afs/bin/volserver)
Why does not VolserLog show what IP address is in use? It should have revealed the 192.168.1.254
of eth0 right? I just guess so because the "vos" command shows that IP address.
It seems everything except the Volserver picked up the IP address use during
"bos -server phylo" configuration steps.
>> phylo ~ # host phylo
>> -bash: host: command not found
>> phylo ~ # cat /etc/hosts
>> 127.0.0.1 localhost
>> # IPV6 versions of localhost and co
>> ::1 ip6-localhost ip6-loopback
>> fe00::0 ip6-localnet
>> ff00::0 ip6-mcastprefix
>> ff02::1 ip6-allnodes
>> ff02::2 ip6-allrouters
>> ff02::3 ip6-allhosts
>>
>> 195.113.57.18 phylo.natur.cuni.cz phylo
>> [cut]
>>
>>
>> The hostname is phylo.
>> The DNS domain is natur.cuni.cz.
>> The REALM is PHYLO.NATUR.CUNI.CZ (yes, NATUR.CUNI.CZ is used on other
>> machines not under my control).
>> Therefore, cellname is phylo.natur.cuni.cz.
>>
>> Could these problem be result of this a bit unusal setup?
>> Why does "vos listvldb" show wrong interface IP address?
>> Or does it just blindly use eth0? What if I wouldn't have no eth0 at all
>> but rather have ra0 (WiFi network card interface)?
>>
>> phylo ~ # vos listaddrs
>> 192.168.1.254
>> 192.168.2.254
>> phylo.natur.cuni.cz
>>
> Notice how the addresses are listed in enumeration order of the
> interfaces.