[OpenAFS-devel] Potential write-consistency race

[Derrick J Brashear]

On Thu, 10 Oct 2002, David Howells wrote:

> Furthermore, the reason for the push every write direct to the server was to
> avoid three problems:
> 
>  (*) dealing with cached writes over a reboot.
>  (*) dealing with unnotified access changes (group memberships changing).
>  (*) dealing with unsent modifications in the cache being deprecated.
>
> I think, then, I have to take a policy of:
> 
>  (1) Open a file with O_SYNC (or if there's no cache at all) then every write
>      to that file gets pushed direct to the server and written into the cache
>      (if there is one).
> 
>  (2) Open a file without O_SYNC and writes get stuffed into the cache and
>      written back only on close or sync with the credentials of the whoever
>      does that operation. Making space in the cache for some other file can
>      then be tricky since you can't send dirty pages to the server without
>      a set credentials to use.
> 
> I don't much like (2) as there's no way to tell whether a user has got write
> permission short of trying StoreData().

2 is traditionally how AFS semantics work (essentially). 

> > and I think Jan has explained well why at least at this point the world
> > isn't ready for that.
> 
> I don't think he explained _why_ it's necessary. He just stated he thought
> that that's what ought to be done (and that that's how it's done in Coda).

Because modulo cache-full issues none of the other clients expect
something other than open-close consistency; The worlds isn't ready for
what boils down to multiple writers. The bizarre semantics you describe
are what people know will happen, whereas a file which may end up being
not what either client wrote but instead some amalgam of the two will be a
surprise and in some cases a problem.

> > I should also mention that perhaps linux-kernel isn't the best place to
> > discuss AFS3 protocol. The best place, sadly, hasn't been well-advertised or
> > particularly active. That would be afs3-protocol@stacken.kth.se (subscribe
> > by mailing afs3-protocol-request@stacken.kth.se).
> 
> Okay. Is there an archive?

I don't know of one; My own archive has one real message in it, but I
don't know what was missed before that. Admittedly much of the protocol
discussion has been done offline, for instance at the Arla hackers meeting
last September, as well as at such events as USENIX and LISA.

> > I may have missed a reply, but I should also note that OpenAFS doesn't have
> > any krb5 code in the kernel, nor real krb4; Since I know one of the messages
> > on linux-kernel implied or stated we did, I thought I should point that out.
> 
> I didn't say it did. Jan suggested that we might need kerberos in the kernel
> and I pointed out that OpenAFS, if it had any, would have a minimal amount.

Nothing had been decided for certain but it looked like for Kerberos 5
we'd have some form of upcall, but none of the options seemed pleasant.

> The Linux NFSv4 person confirmed that NFSv4 is going to require Kerberos
> support though.

They need Kerberos 5, and we aren't (quite) that far along yet.