[OpenAFS] Active Directory 2003, kerberos 5, openAFS - rxkad error=19270407, arghhhh

John W. Sopko Jr. sopko@cs.unc.edu
Wed, 10 Jan 2007 15:02:17 -0500


Douglas E. Engert wrote:
> Its all about the salt -- When DES is used with Krb5 a des key is generated
> by concatenating the password with the salt, and calling des_string_to_key
> 
> So the only way to really know what the key is is to know thew password
> and the salt.
> 
> The salt is normally derived from the principal by concatenating the
> realm and the components of the name without the /or @.
> 
> Thus if the principal name was afs/cs.unc.edu@MSE.UNCCS.TEST
> and the password was newpasswd the DES key would be generated
> from the string "newpasswdMSE.UNCCS.TESTafscs.unc.edu"
> 
> Most of the Krb5 commands take care of the salt for you.
> Each principal has a key stored in the KDC, generated when
> the password was changed.
> 
> *BUT* Microsoft implemented Kerberos a little differently:
> 
>  * They do not store a key, but store a password and generate the
>    key on the fly as needed. With DES this requires the salt.
> 
>  * They store the password with the account.
> 
>  * Each account can have a UserPrincipalName and several
>    ServicePrincipalNames.
> 
> 
> They did not use the correct salt from the principal,
> with W2k but used the SamAccountName (without the $) and the upper
> case DomainName. For normal users this would match krb5 way of using
> the salt, but not for a service principal. Thus all the service
> principals on an account have the same key. I believe in W2k3 they
> started generating the salt from the UPN.
> 
> 
> So where does this leave us,and how can we get around it?
> (msktutil is one way.)

I really did not think it would be this complex to
generate a Windows service principal and corresponding
/usr/afs/etc/KeyFile. I think the best route is to
try windows ktpass again. The thing I do not understand
is; you create a Windows domain account and then use
setspn to add the afs/cs.unc.edu service principal.
(Or I found the service principal gets added if you
use the "-mapuser" option to ktpass). Then if I use
ktpass to set the password, kvno and generate a
keytab, neither the password or the kvno change for
afs/cs.unc.edu service principal. I thought ktpass was
supposed to do what MIT "kadmin ktadd" does, change the
principals password and dump the key to a keytab so
they matched. Maybe this is the problem with the "bad"
ktpass version.

At this point I am confused with ktpass, I read the microsoft docs:

http://technet2.microsoft.com/WindowsServer/en/library/a412a943-4567-4674-9015-fe1b7bf0228c1033.mspx?mfr=true

For example I re-created my domain account as AFS.CS.UNC.EDU
and the ran ktpass like this:

ktpass -out keytab -princ afs/cs.unc.edu@MSE.UNCCS.TEST -kvno 7 -crypto 
DES-CBC-CRC  -DesOnly -pass * -mapuser AFS.CS.UNC.EDU

the -mapuser option adds the afs/cs.unc.edu service to the user account.
You can verify this with setspn -L AFS.CS.UNC.EDU, I did not
need to run the Windows "setspn -A afs/cs.unc.edu  AFS.CS.UNC.EDU"
command, (FYI).

The kvno will always default to 1 if you do not specify it.
But after I run this I run "kvno afs/cs.unc.edu @MSE.UNCCS.TEST"
and the kvno is not 7. The kvno only increments if you change
the password for the domain user, in this case AFS.CS.UNC.EDU.
At this point I am not sure if there is one password for
the user accunt and one for the service principal or if they
are one and the same.

I do not know where to get a "good" version of ktpass. I will
ask our Windows guy, I searched Microsoft and they only have
the SP1 version.

> 
> Another way is to create the AD account, and add the SPN,
> and set the password to something known. The salt can then
> be determened from the UPN,

I can try this for fun but we really need simpler solution.
What do I run to determine this?

> of to *really* be sure, do a network
> trace of the KRB5_ERROR packet returned when you do a
> kinit afs/your.cell@REALM.  Then drill down in the e-data
> till you find the salt for DES.

Here it is, don't know how to decode the salt info, I got
this using the command:

/usr/sbin/tshark  -V host eagle or host madison-cs and port 88

host eagle is the afs server, host madison-cs the Windows AD server.


e-data
         padata: PA-ENCTYPE-INFO
             Type: PA-ENCTYPE-INFO (11)
                 Value: 304E3025A003020103A11E041C4D53452E554E4343532E54... 
des-cbc-md5 des-cbc-crc
Encryption type: des-cbc-md5 (3)
                     Salt: 4D53452E554E4343532E544553544146532E43532E554E43...
                     Encryption type: des-cbc-crc (1)
                     Salt: 4D53452E554E4343532E544553544146532E43532E554E43...

> 
> Then forget about asetkey, and use the AFS "bos_util adddes"
> command instead, to add the key to the keytab.  Since bos_util
> was designed for Krb4 where no salt was used for DES we can in
> effect concatenate the password and the salt used in AD and use
> this as the password to bos_util adddes!
> 
> Best to test this first on a non-server machine as bos_util
> wants to update the /usr/afs/etc/KeyTab file.

If you can give me the password string to give
bos_util I will try it, that is I will enter my known password
I just need the other string/salt info.

Thanks for all the input.

> 
> 
> 
> 
> 
> John W. Sopko Jr. wrote:
>> Getting close, I can feel it:
>>
>> Verify Windows service account:
>> -------------------------------
>> C:\temp>setspn -L afs
>> Registered ServicePrincipalNames for CN=afs service 
>> principal,CN=Users,DC=mse,DC
>> =unccs,DC=test:
>>     afs/cs.unc.edu
>>
>> Change the Windows afs domain user password to a known password, this
>> increments the kvno from 4 to 5. This is verified below.
>>
>> Create /usr/afs/etc/KeyFile with kvno 5:
>> ----------------------------------------
>> ktutil:  add_entry -password -p afs/cs.unc.edu -k 5 -e des-cbc-crc
>> Password for afs/cs.unc.edu@MSE.UNCCS.TEST:
>> ktutil:  wkt keytab.ktutil
>>
>> eagle/root [/usr/afs/etc] # asetkey add 5 keytab.ktutil 
>> afs/cs.unc.edu@MSE.UNCCS.TEST
>>
>> eagle/root [/usr/afs/etc] # bos listkeys eagle -localauth
>> key 5 has cksum 509175897
>> Keys last changed on Wed Jan 10 08:53:50 2007.
>> All done.
>>
>> Get afs token and try afs access:
>> ---------------------------------
>> [sopko@eagle /]$ klist
>> klist: No credentials cache found (ticket cache 
>> FILE:/tmp/krb5cc_3903_015mRF)
>>
>>
>> Kerberos 4 ticket cache: /tmp/tkt3903
>> klist: You have no tickets cached
>>
>> [sopko@eagle /]$ kinit
>> Password for sopko@MSE.UNCCS.TEST:
>>
>> [sopko@eagle /]$ klist
>> Ticket cache: FILE:/tmp/krb5cc_3903_015mRF
>> Default principal: sopko@MSE.UNCCS.TEST
>>
>> Valid starting     Expires            Service principal
>> 01/10/07 08:56:02  01/10/07 18:56:06  
>> krbtgt/MSE.UNCCS.TEST@MSE.UNCCS.TEST
>>         renew until 01/17/07 08:56:02
>>
>>
>> Kerberos 4 ticket cache: /tmp/tkt3903
>> klist: You have no tickets cached
>>
>> [sopko@eagle /]$ kvno afs/cs.unc.edu@MSE.UNCCS.TEST
>> afs/cs.unc.edu@MSE.UNCCS.TEST: kvno = 5
>>
>> [sopko@eagle /]$ klist
>> Ticket cache: FILE:/tmp/krb5cc_3903_015mRF
>> Default principal: sopko@MSE.UNCCS.TEST
>>
>> Valid starting     Expires            Service principal
>> 01/10/07 08:56:02  01/10/07 18:56:06  
>> krbtgt/MSE.UNCCS.TEST@MSE.UNCCS.TEST
>>         renew until 01/17/07 08:56:02
>> 01/10/07 08:56:28  01/10/07 18:56:06  afs/cs.unc.edu@MSE.UNCCS.TEST
>>         renew until 01/17/07 08:56:02
>>
>>
>> Kerberos 4 ticket cache: /tmp/tkt3903
>> klist: You have no tickets cached
>>
>> [sopko@eagle /]$ aklog -d
>> Authenticating to cell cs.unc.edu (server eagle.cs.unc.edu).
>> We've deduced that we need to authenticate to realm MSE.UNCCS.TEST.
>> Getting tickets: afs/cs.unc.edu@MSE.UNCCS.TEST
>> Using Kerberos V5 ticket natively
>> About to resolve name sopko to id in cell cs.unc.edu.
>> Id 3903
>> Set username to AFS ID 3903
>> Setting tokens. AFS ID 3903 /  @ MSE.UNCCS.TEST
>> [sopko@eagle /]$ tokens
>>
>> Tokens held by the Cache Manager:
>>
>> User's (AFS ID 3903) tokens for afs@cs.unc.edu [Expires Jan 10 18:56]
>>    --End of list--
>>
>> [sopko@eagle /]$ ls /afs/cs.unc.edu/home
>> ls: /afs/cs.unc.edu/home: Permission denied
>>
>> Jan 10 08:56:39 eagle kernel: afs: Tokens for user of AFS id 3903 for 
>> cell cs.unc.edu are discarded (rxkad error=19270407)
>>
>> eagle/root [/usr/afs/etc] # translate_et 19270407
>> 19270407 (rxk).7 = security object was passed a bad ticket
>>
>>
>> Jeffrey Altman wrote:
>>> Even assuming you wanted to kinit to your service principal
>>> you would have to so with the correct principal name
>>>
>>>   afs/cs.unc.edu@CSX.UNC.EDU != afs/cs.unc.edu@MSE.UNCCS.TEST
>>>
>>> Your default realm name is CSX.UNC.EDU, not MSE.UNCCS.TEST.
>>>
>>> However, you don't want to be able to kinit to that service
>>> principal.  What you want is to be able to obtain a service
>>> ticket for it using a client principal
>>>
>>>   kvno afs/cs.unc.edu@MSE.UNCCS.TEST
>>>
>>> obtains a service ticket for the specified principal name.
>>> Assuming the kvno is still 4 after you set the service principal
>>> name. You should try to authenticate to your AFS servers again.
>>>
>>>
>>> John W. Sopko Jr. wrote:
>>>> Jeffrey Altman wrote:
>>>>> afs@MSE.UNCCS.TEST != afs/cs.unc.edu@MSE.UNCCS.TEST
>>>>>
>>>>> choose one and stick with it.
>>>> I am confused with Windows principals:
>>>>
>>>> [sopko@eagle /]$ kinit afs/cs.unc.edu@MSE.UNCCS.TEST
>>>> kinit(v5): Client not found in Kerberos database while getting initial
>>>> credentials
>>>>
>>>> That is why I did:
>>>>
>>>> [sopko@eagle /]$ kinit afs
>>>> Password for afs@MSE.UNCCS.TEST:
>>>>
>>>> So I have a Windows "afs" user account that I ran:
>>>>
>>>> setspn -A afs/cs.unc.edu afs
>>>>
>>>> To Add/Associate a service principal to the Windows login
>>>> name.
>>>>
>>>> But I cannot kinit to afs/cs.unc.edu like I can
>>>> under MIT KRB5, (my CSX.UNC.EDU linux server):
>>>>
>>>>
>>>> |sopko@lark:19% kinit afs/cs.unc.edu
>>>> Password for afs/cs.unc.edu@CSX.UNC.EDU:
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>> John W. Sopko Jr. wrote:
>>>>>> Jeffrey Altman wrote:
>>>>>>> John W. Sopko Jr. wrote:
>>>>>>>
>>>>>>>> In C:\Program Files\Support Tools\ktpass
>>>>>>>> right click properties "version tab" shows 5.2.3790.1830
>>>>>>>>
>>>>>>>> So use ktutil on the linux openafs server, setting the
>>>>>>>> password the same as the afs users Windows password:
>>>>>>>>
>>>>>>>> eagle/root [/usr/afs/etc] # ktutil
>>>>>>>> ktutil:  add_entry -password -p afs/cs.unc.edu@MSE.UNCCS.TEST -k 
>>>>>>>> 1 -e
>>>>>>>> des-cbc-crc
>>>>>>> Where did you get the key version number of 1 from?
>>>>>> When I ran the "bad" ktpass command on windows it always generates
>>>>>> kvno 1
>>>>>> by default. The ktpass /? (help) says:
>>>>>>
>>>>>> kvno : Override Key Version Number
>>>>>>        Default: query DC for kvno.  Use /kvno 1 for Win2K compat.
>>>>>>
>>>>>> Since this Windows 2003 server is running in 2000 mixed mode I 
>>>>>> thought
>>>>>> it forced/kept the kvno at 1 for w2k compatability. Below is the
>>>>>> output of
>>>>>> the ktpass, no matter how many times I run it it keeps the "vno"
>>>>>> at 1. I check the keytab.mse file with ktutil and it is at 1.
>>>>>>
>>>>>> But you are right I do not know what is in the server. I did not
>>>>>> think hard enough about this.
>>>>>>
>>>>>>> The key version number must match the number that is actually
>>>>>>> issued by the KDC.  You can identify the version number using
>>>>>>> the MIT Kerberos utility
>>>>>>>
>>>>>>>   kvno <principal>
>>>>>> I tried this to get the kvno:
>>>>>>
>>>>>> eagle/root [/usr/afs/etc] # kinit afs
>>>>>> Password for afs@MSE.UNCCS.TEST:
>>>>>> eagle/root [/usr/afs/etc] # kvno afs
>>>>>> afs@MSE.UNCCS.TEST: kvno = 4
>>>>>>
>>>>>> I then ran:
>>>>>>
>>>>>> ktutil> add_entry -password -p afs/cs.unc.edu@MSE.UNCCS.TEST -k 4 -e
>>>>>> des-cbc-crc
>>>>>> ktutil> write_kt keytab.ktutil_generated
>>>>>>
>>>>>> /usr/sbin/asetkey add 4 keytab.ktutil_generated afs/cs.unc.edu
>>>>>>
>>>>>> /etc/init.d/openafs-server restart
>>>>>>
>>>>>> I now get the same error as Eric had:
>>>>>>
>>>>>> Jan  9 17:14:27 eagle kernel: afs: Tokens for user of AFS id 3903 for
>>>>>> cell cs.unc.edu are discarded (rxkad error=19270407)
>>>>>>
>>>>>> Do I need to map an account like Eric did with the "mapuser" option
>>>>>> to ktpass?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> The key version number must match the number that is actually
>>>>>>> issued by the KDC.  You can identify the version number using
>>>>>>> the MIT Kerberos utility
>>>>>>>
>>>>>>>   kvno <principal>
>>>>>>>
>>>>>>>> cell cs.unc.edu are discarded (rxkad error=19270408)
>>>>>>> The OpenAFS translate_et <error_code> command will tell you this
>>>>>>> is because
>>>>>>>
>>>>>>>   19270408 = ticket contained unknown key version number
>>>>>>>
>>>>>>>> Windows Event Viewer, System log shows this, sometimes:
>>>>>>>>
>>>>>>>> While processing a TGS request for the target server
>>>>>>>> afs/cs.unc.edu, the
>>>>>>>> account sopko@MSE.UNCCS.TEST did not have a suitable key for
>>>>>>>> generating
>>>>>>>> a Kerberos ticket (the missing key has an ID of 8). The requested
>>>>>>>> etypes
>>>>>>>> were 2.  The accounts available etypes were 3  1.
>>>>>>> What in the world is requesting a ticket with DES-CBC-MD4 ?
>>>>>>>
>>>>>>>> AM I CRAZY?
>>>>>>>> -----------
>>>>>>>>
>>>>>>>> Once I get Windows AD working can I run both our current 
>>>>>>>> kaserver and
>>>>>>>> Windows AD authentication against our production cs.unc.edu openafs
>>>>>>>> cell
>>>>>>>> at the same time? If I can generate afs/cs.unc.edu service 
>>>>>>>> pincipals
>>>>>>>> with the same password on the kaserver and the AD server will this
>>>>>>>> work?
>>>>>>>>
>>>>>>>> This may be a good migration path for us. We currently have 2 
>>>>>>>> password
>>>>>>>> databases, kaserver and Windows AD. When we create accounts we 
>>>>>>>> use the
>>>>>>>> same user login name for both wndows and linux. Most users keep 
>>>>>>>> their
>>>>>>>> passwords the same so logging into Windows gives them an afs token.
>>>>>>>> Even if they don't we just tell them to use their Windows password
>>>>>>>> as we migrate machine configurations.
>>>>>>>>
>>>>>>>> This way we can migrate machines to authenticate to "Windows AD 
>>>>>>>> only"
>>>>>>>> over a short period of time and start testing real live systems.
>>>>>>>>
>>>>>>>> First I have to get Windows AD afs service pricnipal working.
>>>>>>> AFS only stores DES keys by key version number.  Ensure that your
>>>>>>> kaserver key and your AD key have different version numbers and
>>>>>>> you will be just fine.
>>>>>>>
>>>>>>> Jeffrey Altman
>>>
>>
> 

-- 
John W. Sopko Jr.               University of North Carolina
email: sopko AT cs.unc.edu      Computer Science Dept., CB 3175
Phone: 919-962-1844             Sitterson Hall; Room 044
Fax:   919-962-1799             Chapel Hill, NC 27599-3175