[OpenAFS] Active Directory 2003, kerberos 5, openAFS - rxkad
error=19270407, arghhhh
Douglas E. Engert
deengert@anl.gov
Fri, 12 Jan 2007 09:06:46 -0600
The KB article you found:
http://support.microsoft.com/kb/919557
Sounds like what Jeff was talking about as part of Vista. I guess
it is available seperatly.
"The Ktpass tool uses the host name part of the servicePrincipalName attribute
instead of the samAccountName attribute that the Key Distribution Center (KDC)
uses to salt the password."
This indicates the salt that should be used is MSE.UNCCS.TESTAFS.CS.UNC.EDU
as was indicated by the krb5_error e-data. It would also indicate that
you should be able to use the bos_util adddes to create the key.
P.S. You use +DesOnly and not -DesOnly on your ktpass when you changed
the password last?
John W. Sopko Jr. wrote:
>
>
> Douglas E. Engert wrote:
>> I don't see anything wrong. But there are some situations
>> that could have caused problems in testing:
>>
>> aklog will look in your ticket cache to see if there is an
>> existing ticket for AFS and use it to create the AFS token.
>> Thus after each attempt to change the key on the AD side,
>> you have to kinit (to destroy the old ticket) so aklog will have
>> to contact the KDC to get a new service ticket with the key you
>> just changed. So during your testing you may have had a good key
>> in AD and the KeyFile but not know it.
>
> Did kdestroy many times..
>
>>
>> If you have more then one DC in the domain, there is a propagation
>> delay, and it could be that the aklog contacts the other DC, and
>> thus might get a ticket with the old key. Doing all you AD updates
>> to one DC and having it listed as the only KDC in the realm, can
>> get around this. You said you where using dns_lookup_kdc = true and
>> the domain had two DCs if I recall. So this could be a problem,
>
> set dns_lookup_kdc = false
> kdc = madison-cs.mse.unccs.test
>
> did a trace, I am only going to madison-cs
>
>>
>> I am running out of ideas.
>>
>> We use AD for our Kerberos KDCs, and have all employees registered.
>> 200 or so use AFS, many from Windows, via aklog, ak5log and gssklog.
>
> I found this and am going to try to get the hotfix for ktpassword:
>
> http://support.microsoft.com/kb/919557
>
>
>
>>
>>
>> John W. Sopko Jr. wrote:
>>>
>>>
>>> Douglas E. Engert wrote:
>>>> You said your test cell was at one time working with MIT k5
>>>> realm of CSX.UNC.EDU
>>>>
>>>> Did you change the server realm of cell file?
>>>> /usr/afs/etc/krb.conf
>>>> to have the line
>>>> MSE.UNCC.TEST
>>>
>>> yes
>>>
>>>>
>>>> (The aklog use the K5 realm of one of the AFS servers,
>>>> to figure out what realm it things the servers are in.
>>>> The servers use the /usr/afs/etc/krb.conf.
>>>> or assume the realm is upper(cellname)
>>>>
>>>> Clock skew between the AD and the AFS server or client?
>>>
>>> I manage ntp, skey is less the 100mSec
>>>
>>>>
>>>> The RXKADBADTICKET can be produced for more then a bad key,
>>>> maybe something else is going on here.
>>>>
>>>> I am runing out if ideas, other then to try and use the
>>>> token with some service running under a debugger, to see
>>>> where the RXKADBADTICKET is coming from.
>>>
>>>
>>> I am going to pursue ktpass for now and see if I can
>>> figue out the problems with it. There is a new ktpass for
>>> windows XP. I built mskutil and took a quick look,
>>> I would prefer not to use that.
>>>
>>>
>>> According to the afs docs you are supposed to change your
>>> afs principal key once a month. I change ours 2 times
>>> a year. I am still using kaserver.
>>>
>>> If afs users are to use windows ad we need to figure
>>> out a good easy documented way to generate the service
>>> key.
>>>
>>>>
>>>>
>>>>
>>>> John W. Sopko Jr. wrote:
>>>>>
>>>>>
>>>>> Douglas E. Engert wrote:
>>>>>> Not sure what's wrong. Your system is saying the salt is the
>>>>>> sAMAccountName. My W2K3 system says it is the
>>>>>> UserPrincipalName (UPN). But then again we have not changed the
>>>>>> password since we went from W2K to W2K3.
>>>>>>
>>>>>> Since all the other methods (ktpass and asetkey) are failing
>>>>>> too, there must be something else going on here. I assume you
>>>>>> have deleted the account, and started over each time.
>>>>>>
>>>>>> Is there more then one DC in the domain?
>>>>>
>>>>> Yes, 2 in the MSE.UNCCS.TEST domain.
>>>>>
>>>>>>
>>>>>> Is this a W2K or w2K3 domain?
>>>>>
>>>>> This is a w2003 server running in w2k mixed mode for
>>>>> now, our windows admins are going to be upgrading
>>>>> our production Windows CS.UNC.EDU REALM soon, they
>>>>> will still use mixed mode then move to 2003 mode
>>>>> only.
>>>>>
>>>>>>
>>>>>> Is the more then one MSE.UNCCS.TEST realm and kdc?
>>>>>
>>>>> There are 2 test AD's servers
>>>>> in the MSE.UNCCS.TEST realm, are production REALM
>>>>> is CS.UNC.EDU.
>>>>>
>>>>>> Domain names are usually based on DNS names, so
>>>>>> why did you not use TEST.CS.UNC.EDU as the AD domain?
>>>>>
>>>>> I did not choose the name, this is a test server our
>>>>> Wndows group uses. They were testing Microsoft Exchange
>>>>> at one time and that is where the MSE came from. We run
>>>>> our own linux DNS server so can do as we wish. This server
>>>>> is only routable on our internal network.
>>>>>
>>>>>
>>>>>> What actually fails? Getting an AFS token with aklog?
>>>>>> What is the error message?
>>>>>
>>>>> My test afs server, host eagle, is running openafs 1.4.2,
>>>>> it has the same cell name, cs.unc.edu as our production
>>>>> server but they do not share any other afs info. I wanted
>>>>> to simulate authentication to our production cell name.
>>>>> I am using eagle as a test client also.
>>>>>
>>>>> I got this test cell/server working to my linux/mit krb5
>>>>> server using the realm name CSX.UNC.EDU since our windows
>>>>> servers have taken over CS.UNC.EDU kerberos realm. I had no
>>>>> problems generating a service key under mit k5
>>>>> and this test server and getting afs perissions. My goal
>>>>> was for us to run one K5 server for authentication, a Windows
>>>>> server, if I can get comfortable doing that.
>>>>>
>>>>>
>>>>> I have the linux/afs krb5.conf configured like this:
>>>>>
>>>>> [libdefaults]
>>>>> default_realm = MSE.UNCCS.TEST
>>>>> dns_lookup_realm = false
>>>>> dns_lookup_kdc = true
>>>>> renew_lifetime = 14d
>>>>>
>>>>> [domain_realm]
>>>>> .cs.unc.edu = MSE.UNCCS.TEST
>>>>> cs.unc.edu = MSE.UNCCS.TEST
>>>>>
>>>>> [realms]
>>>>>
>>>>> MSE.UNCCS.TEST = {
>>>>> master_kdc = madison-cs.mse.unccs.test
>>>>> kpasswd_server = madison-cs.mse.unccs.test
>>>>> ticket_lifetime = 604800
>>>>> }
>>>>>
>>>>> Host madison-cs is the AD server I have been working on.
>>>>> We have kerberos DNS records so dns_lookup_kdc
>>>>> works fine.
>>>>>
>>>>> kinit works fine, aklog works fine, get tokens fine.
>>>>> Get "permission denied" when trying to access a protected
>>>>> directory and the following error in the syslog /var/log/messages:
>>>>>
>>>>> Jan 11 12:14:05 eagle kernel: afs: Tokens for user of AFS id 3903
>>>>> for cell cs.unc.edu are discarded (rxkad error=19270407)
>>>>>
>>>>> eagle/root [/usr/afs/bin] # /usr/bin/translate_et 19270407
>>>>> 19270407 (rxk).7 = security object was passed a bad ticket
>>>>>
>>>>> For example:
>>>>>
>>>>> [sopko@eagle /]$ whoami
>>>>> sopko
>>>>> [sopko@eagle /]$ tokens
>>>>>
>>>>> Tokens held by the Cache Manager:
>>>>>
>>>>> --End of list--
>>>>> [sopko@eagle /]$ klist
>>>>> klist: No credentials cache found (ticket cache
>>>>> FILE:/tmp/krb5cc_3903_l8r6DK)
>>>>>
>>>>>
>>>>> Kerberos 4 ticket cache: /tmp/tkt3903
>>>>> klist: You have no tickets cached
>>>>> [sopko@eagle /]$ kinit
>>>>> Password for sopko@MSE.UNCCS.TEST:
>>>>> [sopko@eagle /]$ aklog -d
>>>>> Authenticating to cell cs.unc.edu (server eagle.cs.unc.edu).
>>>>> We've deduced that we need to authenticate to realm MSE.UNCCS.TEST.
>>>>> Getting tickets: afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>> Using Kerberos V5 ticket natively
>>>>> About to resolve name sopko to id in cell cs.unc.edu.
>>>>> Id 3903
>>>>> Set username to AFS ID 3903
>>>>> Setting tokens. AFS ID 3903 / @ MSE.UNCCS.TEST
>>>>>
>>>>> [sopko@eagle /]$ klist -e
>>>>> Ticket cache: FILE:/tmp/krb5cc_3903_l8r6DK
>>>>> Default principal: sopko@MSE.UNCCS.TEST
>>>>>
>>>>> Valid starting Expires Service principal
>>>>> 01/11/07 12:28:05 01/11/07 22:28:08
>>>>> krbtgt/MSE.UNCCS.TEST@MSE.UNCCS.TEST
>>>>> renew until 01/18/07 12:28:05, Etype (skey, tkt): ArcFour
>>>>> with HMAC/md5, ArcFour with HMAC/md5
>>>>> 01/11/07 12:28:13 01/11/07 22:28:08 afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>> renew until 01/18/07 12:28:05, Etype (skey, tkt): DES cbc
>>>>> mode with CRC-32, DES cbc mode with RSA-MD5
>>>>>
>>>>>
>>>>> Kerberos 4 ticket cache: /tmp/tkt3903
>>>>> klist: You have no tickets cached
>>>>>
>>>>> [sopko@eagle /]$ tokens
>>>>>
>>>>> Tokens held by the Cache Manager:
>>>>>
>>>>> User's (AFS ID 3903) tokens for afs@cs.unc.edu [Expires Jan 11 22:28]
>>>>> --End of list--
>>>>> [sopko@eagle /]$ ls /afs/cs.unc.edu/home
>>>>> ls: /afs/cs.unc.edu/home: Permission denied
>>>>> [sopko@eagle /]$
>>>>>
>>>>>
>>>>>>
>>>>>> John W. Sopko Jr. wrote:
>>>>>>>
>>>>>>>
>>>>>>> Douglas E. Engert wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> John W. Sopko Jr. wrote:
>>>>>>>>> I tried and it did not work, am I missing something?
>>>>>>>>>
>>>>>>>>> My linux server is a test server, I can do anything on it:
>>>>>>>>>
>>>>>>>>> Verify kvno of service principal:
>>>>>>>>>
>>>>>>>>> [sopko@eagle /]$ kvno afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>>>>>> afs/cs.unc.edu@MSE.UNCCS.TEST: kvno = 2
>>>>>>>>>
>>>>>>>>> I have admin rights on the AD, it also is a test
>>>>>>>>> server. I loaded the Active Directory ADSI Edit snapin
>>>>>>>>> and examined the Domain user AFS.CS.UNC.EDU everything
>>>>>>>>> looks in order, I think:
>>>>>>>>>
>>>>>>>>> sAMAccountName AFS.CS.UNC.EDU
>>>>>>>>> servicePrincipalName afs/cs.unc.edu
>>>>>>>>> msDS-KeyVersionNumber 2
>>>>>>>>> userPrincipalName afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>>>>>
>>>>>>>> The only thing I can see is that in the trace you ran the other day
>>>>>>>> you where referring to kvno 7. Today it says kvno 2. Did you
>>>>>>>> recreate
>>>>>>>> the account and reset the password? If so can you run the trace
>>>>>>>> again
>>>>>>>> and make sure it is not using a different salt?
>>>>>>>
>>>>>>> Originally the sAMAccountName was afs, I deleted
>>>>>>> that account and started over using sAMAccountName=AFS.CS.UNC.EDU,
>>>>>>> the trace yesterday was using the AFS.CS.UNC.EDU account
>>>>>>> and that is what the tshark trace showed. The kvno is 2 for sure
>>>>>>> as proved aboce.
>>>>>>>
>>>>>>> I just ran this, the -x option prints the the frame in ascii:
>>>>>>>
>>>>>>> eagle/root [/usr/afs/etc] # /usr/sbin/tshark -V -x port 88
>>>>>>>
>>>>>>> Value: 304E3025A003020103A11E041C4D53452E554E4343532E54...
>>>>>>> des-cbc-md5 des-cbc-crc
>>>>>>> Encryption type: des-cbc-md5 (3)
>>>>>>> Salt:
>>>>>>> 4D53452E554E4343532E544553544146532E43532E554E43...
>>>>>>> Encryption type: des-cbc-crc (1)
>>>>>>> Salt:
>>>>>>> 4D53452E554E4343532E544553544146532E43532E554E43...
>>>>>>> Type: PA-ENC-TIMESTAMP (2)
>>>>>>> Type: PA-PK-AS-REP (15)
>>>>>>>
>>>>>>> 0000 00 02 55 e1 d4 3a 00 d0 00 0f 34 00 08 00 45 00
>>>>>>> ..U..:....4...E.
>>>>>>> 0010 00 f8 c1 08 00 00 7f 11 ad 7d c0 a8 f3 0a 98 02
>>>>>>> .........}......
>>>>>>> 0020 80 b9 00 58 ca 18 00 e4 97 2c 7e 81 d9 30 81 d6
>>>>>>> ...X.....,~..0..
>>>>>>> 0030 a0 03 02 01 05 a1 03 02 01 1e a4 11 18 0f 32 30
>>>>>>> ..............20
>>>>>>> 0040 30 37 30 31 31 31 31 36 32 34 32 32 5a a5 04 02
>>>>>>> 070111162422Z...
>>>>>>> 0050 02 4d 30 a6 03 02 01 19 a9 10 1b 0e 4d 53 45 2e
>>>>>>> .M0.........MSE.
>>>>>>> 0060 55 4e 43 43 53 2e 54 45 53 54 aa 23 30 21 a0 03
>>>>>>> UNCCS.TEST.#0!..
>>>>>>> 0070 02 01 00 a1 1a 30 18 1b 06 6b 72 62 74 67 74 1b
>>>>>>> .....0...krbtgt.
>>>>>>> 0080 0e 4d 53 45 2e 55 4e 43 43 53 2e 54 45 53 54 ac
>>>>>>> .MSE.UNCCS.TEST.
>>>>>>> 0090 75 04 73 30 71 30 59 a1 03 02 01 0b a2 52 04 50
>>>>>>> u.s0q0Y......R.P
>>>>>>> 00a0 30 4e 30 25 a0 03 02 01 03 a1 1e 04 1c 4d 53 45
>>>>>>> 0N0%.........MSE
>>>>>>> 00b0 2e 55 4e 43 43 53 2e 54 45 53 54 41 46 53 2e 43
>>>>>>> .UNCCS.TESTAFS.C
>>>>>>> 00c0 53 2e 55 4e 43 2e 45 44 55 30 25 a0 03 02 01 01
>>>>>>> S.UNC.EDU0%.....
>>>>>>> 00d0 a1 1e 04 1c 4d 53 45 2e 55 4e 43 43 53 2e 54 45
>>>>>>> ....MSE.UNCCS.TE
>>>>>>> 00e0 53 54 41 46 53 2e 43 53 2e 55 4e 43 2e 45 44 55
>>>>>>> STAFS.CS.UNC.EDU
>>>>>>> 00f0 30 09 a1 03 02 01 02 a2 02 04 00 30 09 a1 03 02
>>>>>>> 0..........0....
>>>>>>> 0100 01 0f a2 02 04 00 ......
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I then did this on the linux/afs server:
>>>>>>>>>
>>>>>>>>> eagle/root [/usr/afs/etc] # /bin/rm KeyFile
>>>>>>>>> eagle/root [/usr/afs/etc] # ../bin/bos_util adddes 2
>>>>>>>>> input key:
>>>>>>>>> Retype input key:
>>>>>>>>> eagle/root [/usr/afs/etc] # /etc/init.d/openafs-server restart
>>>>>>>>> Stopping openafs-server: [ OK ]
>>>>>>>>> Starting openafs-server:
>>>>>>>>>
>>>>>>>>> Using an input key of:
>>>>>>>>>
>>>>>>>>> win_passwordMSE.UNCCS.TESTAFS.CS.UNC.EDU
>>>>>>>>
>>>>>>>> That looks correct, based on the trace from the other day
>>>>>>>> I assume using:
>>>>>>>>
>>>>>>>> kinit afs/cs.unc.edu@MSE.UNCC.TEST
>>>>>>>>
>>>>>>>> But it looks like you recreated the account, I would run
>>>>>>>> the trace again.
>>>>>>>>
>>>>>>>> On my system I see the salt as derived from the UPN,
>>>>>>>> ANL.GOVafsanl.gov
>>>>>>>>
>>>>>>>> SO try:
>>>>>>>>
>>>>>>>> win_passwordMSE.UNCCS.TESTafscs.unc.edu
>>>>>>>
>>>>>>> Just tried no luck. What is UPN please, User Principal Name?
>>>>>>
>>>>>> Yes.
>>>>>>
>>>>>>> As shown above, I got these from ADSI edit:
>>>>>>>
>>>>>>> sAMAccountName AFS.CS.UNC.EDU
>>>>>>> servicePrincipalName afs/cs.unc.edu
>>>>>>> msDS-KeyVersionNumber 2
>>>>>>> userPrincipalName afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>>>>
>>>>>>> I cut/paste the win_passord to make sure they are
>>>>>>> the same, I can successfully kinit to the service principal:
>>>>>>>
>>>>>>> [sopko@eagle /]$ kinit afs/cs.unc.edu
>>>>>>> Password for afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> (Case sensitive, no @ or / with the realm first.)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> win_password = password set for Domain user AFS.CS.UNC.EDU
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Douglas E. Engert wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> John W. Sopko Jr. wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I really did not think it would be this complex to
>>>>>>>>>>> generate a Windows service principal and corresponding
>>>>>>>>>>> /usr/afs/etc/KeyFile.
>>>>>>>>>>
>>>>>>>>>> No it should not be, but then again Microsoft did some
>>>>>>>>>> things early on that did not follow the Kerberos
>>>>>>>>>> conversions, including not using kvnos. We are living with
>>>>>>>>>> thuse early decisions.
>>>>>>>>>>
>>>>>>>>>>> think the best route is to
>>>>>>>>>>> try windows ktpass again. The thing I do not understand
>>>>>>>>>>> is; you create a Windows domain account and then use
>>>>>>>>>>> setspn to add the afs/cs.unc.edu service principal.
>>>>>>>>>>> (Or I found the service principal gets added if you
>>>>>>>>>>> use the "-mapuser" option to ktpass). Then if I use
>>>>>>>>>>> ktpass to set the password, kvno and generate a
>>>>>>>>>>> keytab, neither the password or the kvno change for
>>>>>>>>>>> afs/cs.unc.edu service principal. I thought ktpass was
>>>>>>>>>>> supposed to do what MIT "kadmin ktadd" does, change the
>>>>>>>>>>> principals password and dump the key to a keytab so
>>>>>>>>>>> they matched. Maybe this is the problem with the "bad"
>>>>>>>>>>> ktpass version.
>>>>>>>>>>
>>>>>>>>>> You have to have admin privilages in AD to change the
>>>>>>>>>> enties, although a user can change thier own password,
>>>>>>>>>> but I don't thin ktpass is doing that.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> At this point I am confused with ktpass, I read the microsoft
>>>>>>>>>>> docs:
>>>>>>>>>>>
>>>>>>>>>>> http://technet2.microsoft.com/WindowsServer/en/library/a412a943-4567-4674-9015-fe1b7bf0228c1033.mspx?mfr=true
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> The only comment on this is the statememt:
>>>>>>>>>>
>>>>>>>>>> "You cannot map multiple service instances to the same user
>>>>>>>>>> account."
>>>>>>>>>>
>>>>>>>>>> You can have multiple SPNs, but ktpass appearently can not
>>>>>>>>>> handle it.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> For example I re-created my domain account as AFS.CS.UNC.EDU
>>>>>>>>>>> and the ran ktpass like this:
>>>>>>>>>>>
>>>>>>>>>>> ktpass -out keytab -princ afs/cs.unc.edu@MSE.UNCCS.TEST -kvno
>>>>>>>>>>> 7 -crypto DES-CBC-CRC -DesOnly -pass * -mapuser AFS.CS.UNC.EDU
>>>>>>>>>>>
>>>>>>>>>>> the -mapuser option adds the afs/cs.unc.edu service to the
>>>>>>>>>>> user account.
>>>>>>>>>>> You can verify this with setspn -L AFS.CS.UNC.EDU, I did not
>>>>>>>>>>> need to run the Windows "setspn -A afs/cs.unc.edu
>>>>>>>>>>> AFS.CS.UNC.EDU"
>>>>>>>>>>> command, (FYI).
>>>>>>>>>>>
>>>>>>>>>>> The kvno will always default to 1 if you do not specify it.
>>>>>>>>>>> But after I run this I run "kvno afs/cs.unc.edu @MSE.UNCCS.TEST"
>>>>>>>>>>> and the kvno is not 7. The kvno only increments if you change
>>>>>>>>>>> the password for the domain user, in this case AFS.CS.UNC.EDU.
>>>>>>>>>>
>>>>>>>>>> Yes, one password per account, and the kvno is stored in the
>>>>>>>>>> account
>>>>>>>>>> as msDS-KeyVersionNumber with ( mmc and the ADSI plugin you
>>>>>>>>>> can look
>>>>>>>>>> at the AD entries.)
>>>>>>>>>>
>>>>>>>>>>> At this point I am not sure if there is one password for
>>>>>>>>>>> the user accunt and one for the service principal or if they
>>>>>>>>>>> are one and the same.
>>>>>>>>>>
>>>>>>>>>> They are the same.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I do not know where to get a "good" version of ktpass. I will
>>>>>>>>>>> ask our Windows guy, I searched Microsoft and they only have
>>>>>>>>>>> the SP1 version.
>>>>>>>>>>
>>>>>>>>>> Asw I said there is the other way, to avoid ktpass and asetkey.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Another way is to create the AD account, and add the SPN,
>>>>>>>>>>>> and set the password to something known. The salt can then
>>>>>>>>>>>> be determened from the UPN,
>>>>>>>>>>>
>>>>>>>>>>> I can try this for fun but we really need simpler solution.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> msktutil we used this for unix hosts too.
>>>>>>>>>>
>>>>>>>>>>> What do I run to determine this?
>>>>>>>>>>
>>>>>>>>>> You AD dmin can look at te entry. You as a user can run
>>>>>>>>>> on the mmc command, then use the ADSI Edit plugin to look at your
>>>>>>>>>> domain.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> of to *really* be sure, do a network
>>>>>>>>>>>> trace of the KRB5_ERROR packet returned when you do a
>>>>>>>>>>>> kinit afs/your.cell@REALM. Then drill down in the e-data
>>>>>>>>>>>> till you find the salt for DES.
>>>>>>>>>>>
>>>>>>>>>>> Here it is, don't know how to decode the salt info, I got
>>>>>>>>>>> this using the command:
>>>>>>>>>>>
>>>>>>>>>>> /usr/sbin/tshark -V host eagle or host madison-cs and port 88
>>>>>>>>>>>
>>>>>>>>>>> host eagle is the afs server, host madison-cs the Windows AD
>>>>>>>>>>> server.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> e-data
>>>>>>>>>>> padata: PA-ENCTYPE-INFO
>>>>>>>>>>> Type: PA-ENCTYPE-INFO (11)
>>>>>>>>>>> Value:
>>>>>>>>>>> 304E3025A003020103A11E041C4D53452E554E4343532E54...
>>>>>>>>>>> des-cbc-md5 des-cbc-crc
>>>>>>>>>>> Encryption type: des-cbc-md5 (3)
>>>>>>>>>>> Salt:
>>>>>>>>>>> 4D53452E554E4343532E544553544146532E43532E554E43...
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> OK this is the salt! Its in asci, so:
>>>>>>>>>> 4D 53 45 2E 55 4E 43 43 53 2E 54 45 53 54 41 46 53 2E 43 53
>>>>>>>>>> 2E 55 4E 43...
>>>>>>>>>> M S E . U N C C S . T E S T A F S . C S
>>>>>>>>>> . U N C ...
>>>>>>>>>>
>>>>>>>>>> So this look like it is taking the realm(MSE.UNCCS.TEST) and the
>>>>>>>>>> SamAccountName AFS.CS.UNC... and using these as the salt. (I
>>>>>>>>>> assume the ...
>>>>>>>>>> would be .EDU from uyourother messages.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Encryption type: des-cbc-crc (1)
>>>>>>>>>>> Salt:
>>>>>>>>>>> 4D53452E554E4343532E544553544146532E43532E554E43...
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Then forget about asetkey, and use the AFS "bos_util adddes"
>>>>>>>>>>>> command instead, to add the key to the keytab. Since bos_util
>>>>>>>>>>>> was designed for Krb4 where no salt was used for DES we can in
>>>>>>>>>>>> effect concatenate the password and the salt used in AD and use
>>>>>>>>>>>> this as the password to bos_util adddes!
>>>>>>>>>>>>
>>>>>>>>>>>> Best to test this first on a non-server machine as bos_util
>>>>>>>>>>>> wants to update the /usr/afs/etc/KeyTab file.
>>>>>>>>>>>
>>>>>>>>>>> If you can give me the password string to give
>>>>>>>>>>> bos_util I will try it, that is I will enter my known password
>>>>>>>>>>> I just need the other string/salt info.
>>>>>>>>>>
>>>>>>>>>> From the above message it looks like you would concatenate to
>>>>>>>>>> the
>>>>>>>>>> known password MSE.UNCCS.TESTAFS.CS.UNC.EDU
>>>>>>>>>>
>>>>>>>>>> I am not sure what the kvno is, but you can find that with the
>>>>>>>>>> mmc ADSI Edit and look in the entry for the msDS-KeyVersionNumber
>>>>>>>>>>
>>>>>>>>>> Make sure the knvo does not match any existing keys in the
>>>>>>>>>> KeyTable file.
>>>>>>>>>> If so change the passwrod in AD again, and it will increment the
>>>>>>>>>> kvno.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks for all the input.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> John W. Sopko Jr. wrote:
>>>>>>>>>>>>> Getting close, I can feel it:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Verify Windows service account:
>>>>>>>>>>>>> -------------------------------
>>>>>>>>>>>>> C:\temp>setspn -L afs
>>>>>>>>>>>>> Registered ServicePrincipalNames for CN=afs service
>>>>>>>>>>>>> principal,CN=Users,DC=mse,DC
>>>>>>>>>>>>> =unccs,DC=test:
>>>>>>>>>>>>> afs/cs.unc.edu
>>>>>>>>>>>>>
>>>>>>>>>>>>> Change the Windows afs domain user password to a known
>>>>>>>>>>>>> password, this
>>>>>>>>>>>>> increments the kvno from 4 to 5. This is verified below.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Create /usr/afs/etc/KeyFile with kvno 5:
>>>>>>>>>>>>> ----------------------------------------
>>>>>>>>>>>>> ktutil: add_entry -password -p afs/cs.unc.edu -k 5 -e
>>>>>>>>>>>>> des-cbc-crc
>>>>>>>>>>>>> Password for afs/cs.unc.edu@MSE.UNCCS.TEST:
>>>>>>>>>>>>> ktutil: wkt keytab.ktutil
>>>>>>>>>>>>>
>>>>>>>>>>>>> eagle/root [/usr/afs/etc] # asetkey add 5 keytab.ktutil
>>>>>>>>>>>>> afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>>>>>>>>>>
>>>>>>>>>>>>> eagle/root [/usr/afs/etc] # bos listkeys eagle -localauth
>>>>>>>>>>>>> key 5 has cksum 509175897
>>>>>>>>>>>>> Keys last changed on Wed Jan 10 08:53:50 2007.
>>>>>>>>>>>>> All done.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Get afs token and try afs access:
>>>>>>>>>>>>> ---------------------------------
>>>>>>>>>>>>> [sopko@eagle /]$ klist
>>>>>>>>>>>>> klist: No credentials cache found (ticket cache
>>>>>>>>>>>>> FILE:/tmp/krb5cc_3903_015mRF)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Kerberos 4 ticket cache: /tmp/tkt3903
>>>>>>>>>>>>> klist: You have no tickets cached
>>>>>>>>>>>>>
>>>>>>>>>>>>> [sopko@eagle /]$ kinit
>>>>>>>>>>>>> Password for sopko@MSE.UNCCS.TEST:
>>>>>>>>>>>>>
>>>>>>>>>>>>> [sopko@eagle /]$ klist
>>>>>>>>>>>>> Ticket cache: FILE:/tmp/krb5cc_3903_015mRF
>>>>>>>>>>>>> Default principal: sopko@MSE.UNCCS.TEST
>>>>>>>>>>>>>
>>>>>>>>>>>>> Valid starting Expires Service principal
>>>>>>>>>>>>> 01/10/07 08:56:02 01/10/07 18:56:06
>>>>>>>>>>>>> krbtgt/MSE.UNCCS.TEST@MSE.UNCCS.TEST
>>>>>>>>>>>>> renew until 01/17/07 08:56:02
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Kerberos 4 ticket cache: /tmp/tkt3903
>>>>>>>>>>>>> klist: You have no tickets cached
>>>>>>>>>>>>>
>>>>>>>>>>>>> [sopko@eagle /]$ kvno afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>>>>>>>>>> afs/cs.unc.edu@MSE.UNCCS.TEST: kvno = 5
>>>>>>>>>>>>>
>>>>>>>>>>>>> [sopko@eagle /]$ klist
>>>>>>>>>>>>> Ticket cache: FILE:/tmp/krb5cc_3903_015mRF
>>>>>>>>>>>>> Default principal: sopko@MSE.UNCCS.TEST
>>>>>>>>>>>>>
>>>>>>>>>>>>> Valid starting Expires Service principal
>>>>>>>>>>>>> 01/10/07 08:56:02 01/10/07 18:56:06
>>>>>>>>>>>>> krbtgt/MSE.UNCCS.TEST@MSE.UNCCS.TEST
>>>>>>>>>>>>> renew until 01/17/07 08:56:02
>>>>>>>>>>>>> 01/10/07 08:56:28 01/10/07 18:56:06
>>>>>>>>>>>>> afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>>>>>>>>>> renew until 01/17/07 08:56:02
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Kerberos 4 ticket cache: /tmp/tkt3903
>>>>>>>>>>>>> klist: You have no tickets cached
>>>>>>>>>>>>>
>>>>>>>>>>>>> [sopko@eagle /]$ aklog -d
>>>>>>>>>>>>> Authenticating to cell cs.unc.edu (server eagle.cs.unc.edu).
>>>>>>>>>>>>> We've deduced that we need to authenticate to realm
>>>>>>>>>>>>> MSE.UNCCS.TEST.
>>>>>>>>>>>>> Getting tickets: afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>>>>>>>>>> Using Kerberos V5 ticket natively
>>>>>>>>>>>>> About to resolve name sopko to id in cell cs.unc.edu.
>>>>>>>>>>>>> Id 3903
>>>>>>>>>>>>> Set username to AFS ID 3903
>>>>>>>>>>>>> Setting tokens. AFS ID 3903 / @ MSE.UNCCS.TEST
>>>>>>>>>>>>> [sopko@eagle /]$ tokens
>>>>>>>>>>>>>
>>>>>>>>>>>>> Tokens held by the Cache Manager:
>>>>>>>>>>>>>
>>>>>>>>>>>>> User's (AFS ID 3903) tokens for afs@cs.unc.edu [Expires Jan
>>>>>>>>>>>>> 10 18:56]
>>>>>>>>>>>>> --End of list--
>>>>>>>>>>>>>
>>>>>>>>>>>>> [sopko@eagle /]$ ls /afs/cs.unc.edu/home
>>>>>>>>>>>>> ls: /afs/cs.unc.edu/home: Permission denied
>>>>>>>>>>>>>
>>>>>>>>>>>>> Jan 10 08:56:39 eagle kernel: afs: Tokens for user of AFS
>>>>>>>>>>>>> id 3903 for cell cs.unc.edu are discarded (rxkad
>>>>>>>>>>>>> error=19270407)
>>>>>>>>>>>>>
>>>>>>>>>>>>> eagle/root [/usr/afs/etc] # translate_et 19270407
>>>>>>>>>>>>> 19270407 (rxk).7 = security object was passed a bad ticket
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Jeffrey Altman wrote:
>>>>>>>>>>>>>> Even assuming you wanted to kinit to your service principal
>>>>>>>>>>>>>> you would have to so with the correct principal name
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> afs/cs.unc.edu@CSX.UNC.EDU != afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Your default realm name is CSX.UNC.EDU, not MSE.UNCCS.TEST.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> However, you don't want to be able to kinit to that service
>>>>>>>>>>>>>> principal. What you want is to be able to obtain a service
>>>>>>>>>>>>>> ticket for it using a client principal
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> kvno afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> obtains a service ticket for the specified principal name.
>>>>>>>>>>>>>> Assuming the kvno is still 4 after you set the service
>>>>>>>>>>>>>> principal
>>>>>>>>>>>>>> name. You should try to authenticate to your AFS servers
>>>>>>>>>>>>>> again.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> John W. Sopko Jr. wrote:
>>>>>>>>>>>>>>> Jeffrey Altman wrote:
>>>>>>>>>>>>>>>> afs@MSE.UNCCS.TEST != afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> choose one and stick with it.
>>>>>>>>>>>>>>> I am confused with Windows principals:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [sopko@eagle /]$ kinit afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>>>>>>>>>>>> kinit(v5): Client not found in Kerberos database while
>>>>>>>>>>>>>>> getting initial
>>>>>>>>>>>>>>> credentials
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> That is why I did:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [sopko@eagle /]$ kinit afs
>>>>>>>>>>>>>>> Password for afs@MSE.UNCCS.TEST:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So I have a Windows "afs" user account that I ran:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> setspn -A afs/cs.unc.edu afs
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To Add/Associate a service principal to the Windows login
>>>>>>>>>>>>>>> name.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> But I cannot kinit to afs/cs.unc.edu like I can
>>>>>>>>>>>>>>> under MIT KRB5, (my CSX.UNC.EDU linux server):
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> |sopko@lark:19% kinit afs/cs.unc.edu
>>>>>>>>>>>>>>> Password for afs/cs.unc.edu@CSX.UNC.EDU:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> John W. Sopko Jr. wrote:
>>>>>>>>>>>>>>>>> Jeffrey Altman wrote:
>>>>>>>>>>>>>>>>>> John W. Sopko Jr. wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> In C:\Program Files\Support Tools\ktpass
>>>>>>>>>>>>>>>>>>> right click properties "version tab" shows 5.2.3790.1830
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> So use ktutil on the linux openafs server, setting the
>>>>>>>>>>>>>>>>>>> password the same as the afs users Windows password:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> eagle/root [/usr/afs/etc] # ktutil
>>>>>>>>>>>>>>>>>>> ktutil: add_entry -password -p
>>>>>>>>>>>>>>>>>>> afs/cs.unc.edu@MSE.UNCCS.TEST -k 1 -e
>>>>>>>>>>>>>>>>>>> des-cbc-crc
>>>>>>>>>>>>>>>>>> Where did you get the key version number of 1 from?
>>>>>>>>>>>>>>>>> When I ran the "bad" ktpass command on windows it
>>>>>>>>>>>>>>>>> always generates
>>>>>>>>>>>>>>>>> kvno 1
>>>>>>>>>>>>>>>>> by default. The ktpass /? (help) says:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> kvno : Override Key Version Number
>>>>>>>>>>>>>>>>> Default: query DC for kvno. Use /kvno 1 for
>>>>>>>>>>>>>>>>> Win2K compat.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Since this Windows 2003 server is running in 2000 mixed
>>>>>>>>>>>>>>>>> mode I thought
>>>>>>>>>>>>>>>>> it forced/kept the kvno at 1 for w2k compatability.
>>>>>>>>>>>>>>>>> Below is the
>>>>>>>>>>>>>>>>> output of
>>>>>>>>>>>>>>>>> the ktpass, no matter how many times I run it it keeps
>>>>>>>>>>>>>>>>> the "vno"
>>>>>>>>>>>>>>>>> at 1. I check the keytab.mse file with ktutil and it is
>>>>>>>>>>>>>>>>> at 1.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> But you are right I do not know what is in the server.
>>>>>>>>>>>>>>>>> I did not
>>>>>>>>>>>>>>>>> think hard enough about this.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The key version number must match the number that is
>>>>>>>>>>>>>>>>>> actually
>>>>>>>>>>>>>>>>>> issued by the KDC. You can identify the version
>>>>>>>>>>>>>>>>>> number using
>>>>>>>>>>>>>>>>>> the MIT Kerberos utility
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> kvno <principal>
>>>>>>>>>>>>>>>>> I tried this to get the kvno:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> eagle/root [/usr/afs/etc] # kinit afs
>>>>>>>>>>>>>>>>> Password for afs@MSE.UNCCS.TEST:
>>>>>>>>>>>>>>>>> eagle/root [/usr/afs/etc] # kvno afs
>>>>>>>>>>>>>>>>> afs@MSE.UNCCS.TEST: kvno = 4
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I then ran:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ktutil> add_entry -password -p
>>>>>>>>>>>>>>>>> afs/cs.unc.edu@MSE.UNCCS.TEST -k 4 -e
>>>>>>>>>>>>>>>>> des-cbc-crc
>>>>>>>>>>>>>>>>> ktutil> write_kt keytab.ktutil_generated
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> /usr/sbin/asetkey add 4 keytab.ktutil_generated
>>>>>>>>>>>>>>>>> afs/cs.unc.edu
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> /etc/init.d/openafs-server restart
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I now get the same error as Eric had:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Jan 9 17:14:27 eagle kernel: afs: Tokens for user of
>>>>>>>>>>>>>>>>> AFS id 3903 for
>>>>>>>>>>>>>>>>> cell cs.unc.edu are discarded (rxkad error=19270407)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Do I need to map an account like Eric did with the
>>>>>>>>>>>>>>>>> "mapuser" option
>>>>>>>>>>>>>>>>> to ktpass?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The key version number must match the number that is
>>>>>>>>>>>>>>>>>> actually
>>>>>>>>>>>>>>>>>> issued by the KDC. You can identify the version
>>>>>>>>>>>>>>>>>> number using
>>>>>>>>>>>>>>>>>> the MIT Kerberos utility
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> kvno <principal>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> cell cs.unc.edu are discarded (rxkad error=19270408)
>>>>>>>>>>>>>>>>>> The OpenAFS translate_et <error_code> command will
>>>>>>>>>>>>>>>>>> tell you this
>>>>>>>>>>>>>>>>>> is because
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 19270408 = ticket contained unknown key version number
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Windows Event Viewer, System log shows this, sometimes:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> While processing a TGS request for the target server
>>>>>>>>>>>>>>>>>>> afs/cs.unc.edu, the
>>>>>>>>>>>>>>>>>>> account sopko@MSE.UNCCS.TEST did not have a suitable
>>>>>>>>>>>>>>>>>>> key for
>>>>>>>>>>>>>>>>>>> generating
>>>>>>>>>>>>>>>>>>> a Kerberos ticket (the missing key has an ID of 8).
>>>>>>>>>>>>>>>>>>> The requested
>>>>>>>>>>>>>>>>>>> etypes
>>>>>>>>>>>>>>>>>>> were 2. The accounts available etypes were 3 1.
>>>>>>>>>>>>>>>>>> What in the world is requesting a ticket with
>>>>>>>>>>>>>>>>>> DES-CBC-MD4 ?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> AM I CRAZY?
>>>>>>>>>>>>>>>>>>> -----------
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Once I get Windows AD working can I run both our
>>>>>>>>>>>>>>>>>>> current kaserver and
>>>>>>>>>>>>>>>>>>> Windows AD authentication against our production
>>>>>>>>>>>>>>>>>>> cs.unc.edu openafs
>>>>>>>>>>>>>>>>>>> cell
>>>>>>>>>>>>>>>>>>> at the same time? If I can generate afs/cs.unc.edu
>>>>>>>>>>>>>>>>>>> service pincipals
>>>>>>>>>>>>>>>>>>> with the same password on the kaserver and the AD
>>>>>>>>>>>>>>>>>>> server will this
>>>>>>>>>>>>>>>>>>> work?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This may be a good migration path for us. We
>>>>>>>>>>>>>>>>>>> currently have 2 password
>>>>>>>>>>>>>>>>>>> databases, kaserver and Windows AD. When we create
>>>>>>>>>>>>>>>>>>> accounts we use the
>>>>>>>>>>>>>>>>>>> same user login name for both wndows and linux. Most
>>>>>>>>>>>>>>>>>>> users keep their
>>>>>>>>>>>>>>>>>>> passwords the same so logging into Windows gives them
>>>>>>>>>>>>>>>>>>> an afs token.
>>>>>>>>>>>>>>>>>>> Even if they don't we just tell them to use their
>>>>>>>>>>>>>>>>>>> Windows password
>>>>>>>>>>>>>>>>>>> as we migrate machine configurations.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This way we can migrate machines to authenticate to
>>>>>>>>>>>>>>>>>>> "Windows AD only"
>>>>>>>>>>>>>>>>>>> over a short period of time and start testing real
>>>>>>>>>>>>>>>>>>> live systems.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> First I have to get Windows AD afs service pricnipal
>>>>>>>>>>>>>>>>>>> working.
>>>>>>>>>>>>>>>>>> AFS only stores DES keys by key version number.
>>>>>>>>>>>>>>>>>> Ensure that your
>>>>>>>>>>>>>>>>>> kaserver key and your AD key have different version
>>>>>>>>>>>>>>>>>> numbers and
>>>>>>>>>>>>>>>>>> you will be just fine.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Jeffrey Altman
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
--
Douglas E. Engert <DEEngert@anl.gov>
Argonne National Laboratory
9700 South Cass Avenue
Argonne, Illinois 60439
(630) 252-5444