[OpenAFS] rxk5 Mainline Issues?

Sun, 8 Nov 2009 11:01:18 -0800

--0016368e270cb49ab70477e0b368
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Hope you update the manuals prior to coding...

On Sun, Nov 8, 2009 at 10:11 AM, Simon Wilkinson <sxw@inf.ed.ac.uk> wrote:

>
>  Could you expand a bit on the =93number of issues=94 that remain if it=
=92s
>> functionally complete? Is there a published list of things that need to =
be
>> in place for something to be mainlined, in general, or are these issues =
only
>> for Rxk5?
>>
>
> Marcus has pointed out that you were asking a more general question, and =
it
> occurred to me that, whilst I've talked about these issues at both the
> Stanford and Rome Workshops, there are folks who won't have been to eithe=
r.
> So, here's a rough summary of what I think the process of getting code
> upstream is. Bear in mind that I'm very much a single voice here, and wha=
t I
> say has no more weight they anyone else - but I've said this lot a few ti=
mes
> now without getting shouted down...
>
> Firstly, any changes to the protocol have to go past the
> afs3-standardisation list. This effectively means anything on the wire, b=
ut
> may also extend to (for example) the pioctl interfaces when there are
> multiple implementations of these. Proposals should be presented in the f=
orm
> of an internet draft. We're still coming up to speed on how these get dea=
lt
> with - at present, the level of comment generally depends on the scale of
> the change. If the proposal is for a minor tweak to a debugging interface=
,
> it's likely to get through easily. If it's a wholescale change to a
> fundamental bit of AFS, then things will take longer, and probably involv=
e
> actively soliciting responses.
>
> Secondly, any major changes should be discussed before they're implemente=
d.
> The openafs-devel list is the best place to do this. It is of course up t=
o
> an author what to do with the feedback they receive. My general advice wo=
uld
> be to listen carefully if there are suggestions that the fundamental
> approach is wrong, but to be cautious of demands for feature creep. One k=
ey
> point is that the people commenting now are the people who'll be reviewin=
g
> the finished code. If nobody agrees that an approach is the best solution
> now, finding reviewers when it comes to getting it into the tree is likel=
yto
> be difficult. It's at this point that it'll also be possible to get an id=
ea
> of when merge windows are available for this kind of change. Typically,
> large scale changes won't be accepted, even into the development branch,
> around branch points for stable releases.
>
> Thirdly, code gets written, with steps one and two repeating as required.
> If it's a large project, please keep the community informed, especially i=
f
> you are significantly changing previously discussed designs. When coding,
> bear in mind that the tree has a documented coding style, which should be
> followed as far as possible. Try and keep your code as small modular chun=
ks.
> Consider, as you write it, how it would make sense to split it for an
> upstream contribution. If your code requires interface changes, then spli=
t
> off these changes, and make them separate commits. Avoid bundling unrelat=
ed
> changes in with your core work - if you need to fix bugs in existing code=
,
> then make those changes as separate patches, and contribute them as you
> work. Cutting down the size of you main code drop will make the final
> integration steps much much easier.
>
> At the end of all of this, there will be a collection of code to contribu=
te
> as a set of patches. The rough guidance here is that there should be one
> patch per change, and one change per patch - but good judgement is necess=
ary
> as 10,000 individual 10 line patches are no more manageable than a single
> 100,000 line patch.
>
> The key thing to consider when splitting code into patches is that the
> people who review them are volunteering their time to do so - there's no
> commitment upon them to spend time evaluating any code, so contributors
> should make it as easy for them as possible. This is one reason why break=
ing
> large changes up makes life easier - it's a lot easier for reviewers to f=
ind
> spare hours here and there, than it is to contemplate looking at somethin=
g
> that's obviously going to take days to get through. Also, a set of sensib=
ly
> created patches is far easier to review, and to test, than a single
> monolithic change. Providing reviewers with a guide to the code is hugely
> helpful - either through commit comments, in-tree design documentation, o=
r a
> separate document.
>
> Contributors should expect that their code won't be accepted first time
> round. There are likely to be nits picked up by the review process, altho=
ugh
> the more communication that's occurred at the beginning, the less
> significant those nits are likely to be.
>
> The final issue I'd like to address is that of _when_ code should make it=
s
> way into the development branches. Maintaining code out of tree is hard -
> keeping it up to date is a pain - and people  will often break code that'=
s
> not in the tree (or even that is in the tree, but on a different branch)
> simply because they don't realise its there, or don't test against it. Th=
is
> leads to a general desire by contributors to get code in as quicky as
> possible. However, there is a flip side. There are a lot of developers
> working off the main branch, and committing code that breaks the tree or =
the
> build is a bad thing. Once code has gone in, the longer it remains in the
> tree, the harder it becomes to remove it. This is a real problem when a n=
ew
> 'stable' branch is required - if the development branch is broken, how do
> you create the new stable branch. This is exactly the state of affairs th=
at
> demand attach left us in - with a development branch that was too broken =
to
> even consider creating a 1.6 from, but with a set of changes that it woul=
d
> be hugely timeconsuming to unpick from the rest of the code. There is, of
> course, the question of branching earlier - you have 'stable', 'developme=
nt'
> and 'very bleeding edge'. But this just shifts the problem slightly
> sideways. Additionally, each new branch creates significantly more work f=
or
> everyone - as patches need to be created, tested, and committed against e=
ach
> new branch.
>
> So, I think the best answer is 'when its ready' and 'not near proposed
> branch points'. Lots of projects have clearer answers to these particular
> questions - for example, Linux has its merge windows, and Mozilla has cod=
e
> freezes around releases, where no feature commits are permitted. I'd argu=
e
> that OpenAFS probably wants to adopt something along these lines too, alo=
ng
> with a clearer set of data driven targets for stable release. But that's =
an
> argument for another email.
>
> I should round off by saying that absolutely none of this is specific to
> OpenAFS - they're all just part of a functioning Open Source eco system. =
If
> you're interested, I'd highly recommend reading:
>
> How to Participate in the Linux Community (A Guide to the Kernel
> Development Process)
> http://ldn.linuxfoundation.org/how-participate-linux-community
>
> Mozilla Hacker's Getting Started Guide
> https://developer.mozilla.org/en/Mozilla_Hacker's_Getting_Started_Guide<h=
ttps://developer.mozilla.org/en/Mozilla_Hacker%27s_Getting_Started_Guide>
>
> or any other big projects' developer guidelines - we're all struggling wi=
th
> the same problems!
>
> Hope that helps,
>
> Simon.
>
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info
>

--0016368e270cb49ab70477e0b368
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Hope you update the manuals prior to coding... <div class=3D"gmail_q=
uote">On Sun, Nov 8, 2009 at 10:11 AM, Simon Wilkinson &l=
t;<a href=3D"mailto:sxw@inf.ed.ac.uk">sxw@inf.ed.ac.uk</a>&gt; wrote=
: 
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> 
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Could you expand a bit on the =93number of issues=94 that remain if it=92s =
functionally complete? Is there a published list of things that need to be =
in place for something to be mainlined, in general, or are these issues onl=
y for Rxk5?

</blockquote>
 
Marcus has pointed out that you were asking a more general question, and it=
 occurred to me that, whilst I&#39;ve talked about these issues at both the=
 Stanford and Rome Workshops, there are folks who won&#39;t have been to ei=
ther. So, here&#39;s a rough summary of what I think the process of getting=
 code upstream is. Bear in mind that I&#39;m very much a single voice here,=
 and what I say has no more weight they anyone else - but I&#39;ve said thi=
s lot a few times now without getting shouted down...

Firstly, any changes to the protocol have to go past the afs3-standardisati=
on list. This effectively means anything on the wire, but may also extend t=
o (for example) the pioctl interfaces when there are multiple implementatio=
ns of these. Proposals should be presented in the form of an internet draft=
. We&#39;re still coming up to speed on how these get dealt with - at prese=
nt, the level of comment generally depends on the scale of the change. If t=
he proposal is for a minor tweak to a debugging interface, it&#39;s likely =
to get through easily. If it&#39;s a wholescale change to a fundamental bit=
 of AFS, then things will take longer, and probably involve actively solici=
ting responses.

Secondly, any major changes should be discussed before they&#39;re implemen=
ted. The openafs-devel list is the best place to do this. It is of course u=
p to an author what to do with the feedback they receive. My general advice=
 would be to listen carefully if there are suggestions that the fundamental=
 approach is wrong, but to be cautious of demands for feature creep. One ke=
y point is that the people commenting now are the people who&#39;ll be revi=
ewing the finished code. If nobody agrees that an approach is the best solu=
tion now, finding reviewers when it comes to getting it into the tree is li=
kelyto be difficult. It&#39;s at this point that it&#39;ll also be possible=
 to get an idea of when merge windows are available for this kind of change=
. Typically, large scale changes won&#39;t be accepted, even into the devel=
opment branch, around branch points for stable releases.

Thirdly, code gets written, with steps one and two repeating as required. I=
f it&#39;s a large project, please keep the community informed, especially =
if you are significantly changing previously discussed designs. When coding=
, bear in mind that the tree has a documented coding style, which should be=
 followed as far as possible. Try and keep your code as small modular chunk=
s. Consider, as you write it, how it would make sense to split it for an up=
stream contribution. If your code requires interface changes, then split of=
f these changes, and make them separate commits. Avoid bundling unrelated c=
hanges in with your core work - if you need to fix bugs in existing code, t=
hen make those changes as separate patches, and contribute them as you work=
. Cutting down the size of you main code drop will make the final integrati=
on steps much much easier.

At the end of all of this, there will be a collection of code to contribute=
 as a set of patches. The rough guidance here is that there should be one p=
atch per change, and one change per patch - but good judgement is necessary=
 as 10,000 individual 10 line patches are no more manageable than a single =
100,000 line patch.

The key thing to consider when splitting code into patches is that the peop=
le who review them are volunteering their time to do so - there&#39;s no co=
mmitment upon them to spend time evaluating any code, so contributors shoul=
d make it as easy for them as possible. This is one reason why breaking lar=
ge changes up makes life easier - it&#39;s a lot easier for reviewers to fi=
nd spare hours here and there, than it is to contemplate looking at somethi=
ng that&#39;s obviously going to take days to get through. Also, a set of s=
ensibly created patches is far easier to review, and to test, than a single=
 monolithic change. Providing reviewers with a guide to the code is hugely =
helpful - either through commit comments, in-tree design documentation, or =
a separate document.

Contributors should expect that their code won&#39;t be accepted first time=
 round. There are likely to be nits picked up by the review process, althou=
gh the more communication that&#39;s occurred at the beginning, the less si=
gnificant those nits are likely to be.

<br>
The final issue I&#39;d like to address is that of _when_ code should make =
its way into the development branches. Maintaining code out of tree is hard=
 - keeping it up to date is a pain - and people =A0will often break code th=
at&#39;s not in the tree (or even that is in the tree, but on a different b=
ranch) simply because they don&#39;t realise its there, or don&#39;t test a=
gainst it. This leads to a general desire by contributors to get code in as=
 quicky as possible. However, there is a flip side. There are a lot of deve=
lopers working off the main branch, and committing code that breaks the tre=
e or the build is a bad thing. Once code has gone in, the longer it remains=
 in the tree, the harder it becomes to remove it. This is a real problem wh=
en a new &#39;stable&#39; branch is required - if the development branch is=
 broken, how do you create the new stable branch. This is exactly the state=
 of affairs that demand attach left us in - with a development branch that =
was too broken to even consider creating a 1.6 from, but with a set of chan=
ges that it would be hugely timeconsuming to unpick from the rest of the co=
de. There is, of course, the question of branching earlier - you have &#39;=
stable&#39;, &#39;development&#39; and &#39;very bleeding edge&#39;. But th=
is just shifts the problem slightly sideways. Additionally, each new branch=
 creates significantly more work for everyone - as patches need to be creat=
ed, tested, and committed against each new branch.<br>

So, I think the best answer is &#39;when its ready&#39; and &#39;not near p=
roposed branch points&#39;. Lots of projects have clearer answers to these =
particular questions - for example, Linux has its merge windows, and Mozill=
a has code freezes around releases, where no feature commits are permitted.=
 I&#39;d argue that OpenAFS probably wants to adopt something along these l=
ines too, along with a clearer set of data driven targets for stable releas=
e. But that&#39;s an argument for another email.

I should round off by saying that absolutely none of this is specific to Op=
enAFS - they&#39;re all just part of a functioning Open Source eco system. =
If you&#39;re interested, I&#39;d highly recommend reading: 
 
How to Participate in the Linux Community (A Guide to the Kernel Developmen=
t Process) 
<a href=3D"http://ldn.linuxfoundation.org/how-participate-linux-community" =
target=3D"_blank">http://ldn.linuxfoundation.org/how-participate-linux-comm=
unity</a> 
 
Mozilla Hacker&#39;s Getting Started Guide 
<a href=3D"https://developer.mozilla.org/en/Mozilla_Hacker%27s_Getting_Star=
ted_Guide" target=3D"_blank">https://developer.mozilla.org/en/Mozilla_Hacke=
r&#39;s_Getting_Started_Guide</a> 
 
or any other big projects&#39; developer guidelines - we&#39;re all struggl=
ing with the same problems! 
 
Hope that helps, 
 
Simon. 
 
_______________________________________________ 
OpenAFS-info mailing list 
<a href=3D"mailto:OpenAFS-info@openafs.org" target=3D"_blank">OpenAFS-info@=
openafs.org</a> 
<a href=3D"https://lists.openafs.org/mailman/listinfo/openafs-info" target=
=3D"_blank">https://lists.openafs.org/mailman/listinfo/openafs-info</a> 
</blockquote></div>

--0016368e270cb49ab70477e0b368--