[OpenAFS] BIG Problem Setup AFS Multihomed Server

Sven Oehme oehmes@de.ibm.com
Wed, 29 Jan 2003 10:00:15 +0100


This is a multipart message in MIME format.
--=_alternative 0031715EC1256CBD_=
Content-Type: text/plain; charset="US-ASCII"

Hy, 

i am working now since a week on a Problem setting up a Multihomed file 
and Database Server and become crazy ..

my Scenario is the following :




     |-----| 126.X |---------|192.X |-----| 192.X |------------|  9.X 
|-----| 
     |     | ----- |CellServ1| ---- |     | ----- |AFS Client 1| ---- |  | 

     | B   |       |---------|      |  A  |       |------------|       | P 
 |
     | A   |                        |  F  |                            | L 
 |  9.X         |-------------|
     | C   | 126.X |---------|192.X |  S  | 192.X |------------|  9.X  | A 
 |  ------------|Samba clients|
     | K   | ----- |CellServ1| ---- |     | ----- |AFS Client 1| ---- |  N 
 |              |-------------|
     | U   |       |---------|      |  L  |       |------------|       | T 
 |
     | P   |                        |  A  |                            |  
| 
     |     |                        |  N  |                            | L 
 | 
     | L   | 126.X |---------|192.X |     |                            | A 
 | 
     | A   | ----- |FileServ1| ---- |     |                            | N 
 | 
     | N   |       |---------|      |     |                            |  
| 
     |     |                        |     |                            |  
| 
     |126.X|                        |192.X|                            | 
9.X | 
     |-----|                        |-----| |-----| 



so for Explanation , 

i like to setup the CellServers and Fileserver connected to the 126.X and 
the 192.X Lan (but only use the 192.X Range for AFS)
the Clients read the Data from the FileServer and share the Data with 
Samba to the Plant Lan .

The CellServDB look like this for all Clients and Servers in /usr/afs/etc/ 
and /usr/vice/etc:

>testme.org             #test Cell
192.168.180.101 #Cellserv1.afslan.com
192.168.180.102 #Cellserv2.afslan.com

my /etc/hosts file is empty (only the 127.0.0.1 with localhost) , but 
forward and reverse 
look-up's are possible and correct for 126.x and 192.x network trough DNS

so 'host cellserv1' returns 192.168.180.101, also 'host 
cellserv1.afslan.com' is pointing to 192.168.180.101
and reverse is also working 'host 192.168.180.101' is pointing to 
cellserv1.afslan.com 

so name resolution is not a Problem (and i also tried to put everything in 
the /etc/hosts with the same result ).

the Interface configuration of cellserv1 for examples is  :

en0 empty (not used and down)
en1 empty (not used and down)
en2 has ip 126.201.100.241 subnetmask 255.255.0.0 and gateway 
126.201.100.9
en3 has ip 192.168.180.11 subnetmask 255.255.0.0 and no gateway

so , when i now try to start only this one host  i get a lot off ubik 
errors .

looking in the logs shows me , that 192.168.180.11 is not the primary ip 
address 
buserver , Fileserver , salvager ...  are dieing every 10 -15 sec. and 
restart 

when is bos shutdown the server remove the 192.X ip , change the entry in 
the Cellservdb to 126.X bos startup the Server , everything is working , 
but with the 126.X  ip.
then i tried to use /usr/afs/local/NetRestict  and /usr/afs/local/NetInfo 
, same result , with the 192.X configured 

cat /usr/afs/local/NetRestrict  reports 126.201.100.241  and cat 
/usr/afs/local/NetInfo reports 192.168.180.11 

no change , he only reports me every time 192.X is not the primary address 
.

when i remove the 126.X ip everything is back working .

it is also working , when i setup /etc/resolv.conf that 'host cellserv1' 
and reports the 126.X address and change the CellServDB to the 126.X ip's 
and start the Server 
than starts correct , but only registers the 126.X ip in the VLDB .

so is it possible , that this is a bug in the OPENAFS AIX code ? because i 
installed with the same setup a Linux Box and everything works fine .
I also reinstalled the whole AIX Box , so i think there is no Problem with 
the installation itself .
it also looks like /usr/afs/local/NetRestict is not used for IP exclusions 
.

i am running AIX 4.3.3 ML10 

Some Hints or Help would be Great ..

Sven

--=_alternative 0031715EC1256CBD_=
Content-Type: text/html; charset="US-ASCII"


<br><font size=2 face="sans-serif">Hy, </font>
<br>
<br><font size=2 face="sans-serif">i am working now since a week on a Problem
setting up a Multihomed file and Database Server and become crazy ..</font>
<br>
<br><font size=2 face="sans-serif">my Scenario is the following :</font>
<br>
<br>
<br>
<br>
<br><font size=2 face="Courier">&nbsp; &nbsp; &nbsp;|-----| 126.X |---------|192.X
|-----| 192.X |------------| &nbsp;9.X &nbsp;|-----| &nbsp; </font>
<br><font size=2 face="Courier">&nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; | -----
|CellServ1| ---- | &nbsp; &nbsp; | ----- |AFS Client 1| &nbsp; &nbsp; &nbsp;
&nbsp;---- | &nbsp; &nbsp; | </font>
<br><font size=2 face="Courier">&nbsp; &nbsp; &nbsp;| B &nbsp; | &nbsp;
&nbsp; &nbsp; |---------| &nbsp; &nbsp; &nbsp;| &nbsp;A &nbsp;| &nbsp;
&nbsp; &nbsp; |------------| &nbsp; &nbsp; &nbsp; | &nbsp;P &nbsp;|</font>
<br><font size=2 face="Courier">&nbsp; &nbsp; &nbsp;| A &nbsp; | &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|
&nbsp;F &nbsp;| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp;L &nbsp;| &nbsp;9.X &nbsp;
&nbsp; &nbsp; &nbsp; |-------------|</font>
<br><font size=2 face="Courier">&nbsp; &nbsp; &nbsp;| C &nbsp; | 126.X
|---------|192.X | &nbsp;S &nbsp;| 192.X |------------| &nbsp;9.X &nbsp;|
&nbsp;A &nbsp;| &nbsp;------------|Samba clients|</font>
<br><font size=2 face="Courier">&nbsp; &nbsp; &nbsp;| K &nbsp; | -----
|CellServ1| ---- | &nbsp; &nbsp; | ----- |AFS Client 1| &nbsp; &nbsp; &nbsp;
&nbsp;---- | &nbsp;N &nbsp;| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp;|-------------|</font>
<br><font size=2 face="Courier">&nbsp; &nbsp; &nbsp;| U &nbsp; | &nbsp;
&nbsp; &nbsp; |---------| &nbsp; &nbsp; &nbsp;| &nbsp;L &nbsp;| &nbsp;
&nbsp; &nbsp; |------------| &nbsp; &nbsp; &nbsp; | &nbsp;T &nbsp;|</font>
<br><font size=2 face="Courier">&nbsp; &nbsp; &nbsp;| P &nbsp; | &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|
&nbsp;A &nbsp;| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; | </font>
<br><font size=2 face="Courier">&nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; | &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|
&nbsp;N &nbsp;| &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp;L &nbsp;| </font>
<br><font size=2 face="Courier">&nbsp; &nbsp; &nbsp;| L &nbsp; | 126.X
|---------|192.X | &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp;A &nbsp;|
</font>
<br><font size=2 face="Courier">&nbsp; &nbsp; &nbsp;| A &nbsp; | -----
|FileServ1| ---- | &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp;N &nbsp;|
&nbsp; &nbsp;<br>
 &nbsp; &nbsp; | N &nbsp; | &nbsp; &nbsp; &nbsp; |---------| &nbsp; &nbsp;
&nbsp;| &nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; | &nbsp;
&nbsp;</font>
<br><font size=2 face="Courier">&nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; | &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|
&nbsp; &nbsp; | &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;| &nbsp; &nbsp; | &nbsp; &nbsp;</font>
<br><font size=2 face="Courier">&nbsp; &nbsp; &nbsp;|126.X| &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|192.X|
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp;| 9.X | </font>
<br><font size=2 face="Courier">&nbsp; &nbsp; &nbsp;|-----| &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|-----|
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp;|-----| &nbsp; &nbsp; </font>
<br>
<br>
<br>
<br><font size=2 face="Courier">so for Explanation , </font>
<br>
<br><font size=2 face="Courier">i like to setup the CellServers and Fileserver
connected to the 126.X and the 192.X Lan (but only use the 192.X Range
for AFS)</font>
<br><font size=2 face="Courier">the Clients read the Data from the FileServer
and share the Data with Samba to the Plant Lan .</font>
<br>
<br><font size=2 face="Courier">The CellServDB look like this for all Clients
and Servers in /usr/afs/etc/ and /usr/vice/etc:</font>
<br>
<br><font size=2 face="Courier">&gt;testme.org &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp;#test Cell</font>
<br><font size=2 face="Courier">192.168.180.101 &nbsp; &nbsp; &nbsp; &nbsp;#Cellserv1.afslan.com</font>
<br><font size=2 face="Courier">192.168.180.102 &nbsp; &nbsp; &nbsp; &nbsp;#Cellserv2.afslan.com</font>
<br>
<br><font size=2 face="Courier">my /etc/hosts file is empty (only the 127.0.0.1
with localhost) , but forward and reverse </font>
<br><font size=2 face="Courier">look-up's are possible and correct for
126.x and 192.x network trough DNS</font>
<br>
<br><font size=2 face="Courier">so 'host cellserv1' returns 192.168.180.101,
also 'host cellserv1.afslan.com' is pointing to 192.168.180.101</font>
<br><font size=2 face="Courier">and reverse is also working 'host 192.168.180.101'
is pointing to cellserv1.afslan.com </font>
<br>
<br><font size=2 face="sans-serif">so name resolution is not a Problem
(and i also tried to put everything in the /etc/hosts with the same result
).</font>
<br>
<br><font size=2 face="sans-serif">the Interface configuration of cellserv1
for examples is &nbsp;:</font>
<br>
<br><font size=2 face="sans-serif">en0 empty (not used and down)</font>
<br><font size=2 face="sans-serif">en1 empty (not used and down)</font>
<br><font size=2 face="sans-serif">en2 has ip 126.201.100.241 subnetmask
255.255.0.0 and gateway 126.201.100.9</font>
<br><font size=2 face="sans-serif">en3 has ip 192.168.180.11 subnetmask
255.255.0.0 and no gateway</font>
<br>
<br><font size=2 face="sans-serif">so , when i now try to start only this
one host &nbsp;i get a lot off ubik errors .</font>
<br>
<br><font size=2 face="sans-serif">looking in the logs shows me , that
192.168.180.11 is not the primary ip address </font>
<br><font size=2 face="sans-serif">buserver , Fileserver , salvager ...
&nbsp;are dieing every 10 -15 sec. and restart </font>
<br>
<br><font size=2 face="sans-serif">when is bos shutdown the server remove
the 192.X ip , change the entry in the Cellservdb to 126.X bos startup
the Server , everything is working , but with the 126.X &nbsp;ip.</font>
<br><font size=2 face="sans-serif">then i tried to use /usr/afs/local/NetRestict
&nbsp;and /usr/afs/local/NetInfo &nbsp;, same result , with the 192.X configured
</font>
<br>
<br><font size=2 face="sans-serif">cat /usr/afs/local/NetRestrict &nbsp;reports
126.201.100.241 &nbsp;and cat /usr/afs/local/NetInfo reports 192.168.180.11
</font>
<br>
<br><font size=2 face="sans-serif">no change , he only reports me every
time 192.X is not the primary address .</font>
<br>
<br><font size=2 face="sans-serif">when i remove the 126.X ip everything
is back working .</font>
<br>
<br><font size=2 face="sans-serif">it is also working , when i setup /etc/resolv.conf
that 'host cellserv1' and reports the 126.X address and change the CellServDB
to the 126.X ip's and start the Server </font>
<br><font size=2 face="sans-serif">than starts correct , but only registers
the 126.X ip in the VLDB .</font>
<br>
<br><font size=2 face="sans-serif">so is it possible , that this is a bug
in the OPENAFS AIX code ? because i installed with the same setup a Linux
Box and everything works fine .</font>
<br><font size=2 face="sans-serif">I also reinstalled the whole AIX Box
, so i think there is no Problem with the installation itself .</font>
<br><font size=2 face="sans-serif">it also looks like /usr/afs/local/NetRestict
is not used for IP exclusions .</font>
<br>
<br><font size=2 face="sans-serif">i am running AIX 4.3.3 ML10 </font>
<br>
<br><font size=2 face="sans-serif">Some Hints or Help would be Great ..</font>
<br>
<br><font size=2 face="sans-serif">Sven</font>
<br>
--=_alternative 0031715EC1256CBD_=--