appeal mechanisms was Re: Ombuds-process

[John C Klensin]

--On Tuesday, 01 July, 2003 09:42 +0900 avri 
<avri at apocalypse.org> wrote:

>>> i certainly do not believe the existing process is a
>>> mechanism that aids people in making appeals.
>>
>> Mumble.  Ok, I have filed an appeal, and it was successful in
>> getting  an action changed.
>
> And you are most definitely not a regular participant and are
> most adept at the processes of the organization.  You also
> have excellent command of the english language.  And due to
> long term association with the I* know exactly what is
> relevant and what is not.  You also share a cultural
> similarity to the majority of those who read and adjudicate
> the appeals.
>
> the fact that you think this is trivial easy and thatanyone
> can do what you can do is indeed a significant part of the
> problem in my opinion.  Those of us who can command the
> language to do what we want it to do (well I don't always
> succeed) and those of us who can play the process game, often
> think that everyone can do what we can do.  It is not the case.

Avri,

I didn't say, and didn't mean to suggest, that there was 
anything trivial about it.  I even said "This is often not an 
easy writing  job, but it is necessary".   And I understand 
that, for a non-native speaker of English, it may be 
exceptionally hard.  But this really is an IETF community 
matter.   If someone who wants to make an appeal finds the 
writing difficult, find someone else who might agree, get him or 
her to help with the writing, and then launch a joint appeal. 
If no one else can be found, it is questionable whether the IESG 
will be persuaded either.

I have, personally, spent a good deal of time working with 
authors to improve the presentation and clarity of statements 
and I-Ds which I consider important -- sometimes many weeks on a 
single draft and occasionally even on drafts with which I 
disagree technically -- before, during, and after my time on the 
IESG and IAB.  I don't think I am unique in that regard.

[Excursion toward solution-space: as with recalls, there is a 
case to be made that appeals should be required to have at least 
a few "endorsers" or co-sponsors before getting past the 
"discussion with AD" or "discussion with IETF Chair" stages. 
The problem with presentation and "playing the process game" you 
identify points out what might be an additional advantage of 
that approach, since trying to gather endorsers might well 
gather up people who could help with presentation and process 
and would give them an extra incentive to help.  On the other 
hand, it does raise the bar a bit.]

Further, with regard to "playing the process"... _Any_ process 
can be "played", and some will be better at it than others. 
Even if we abandoned an appeal model and permitted anyone who 
could stand up and shout "I disagree" and thereby force an 
additional review (a terrible idea, but, perhaps, a useful 
example), some people would be better at shouting, while others 
--by personality or cultural norms-- would be reluctant to call 
that much attention to themselves.  What, IMO, we should strive 
for is a process whose workings are sufficiently clear and 
public to make it usable by those who are not "insiders".  How 
do we do that?  I think we need to look as much at education 
about how the process actually works in practice and how to 
"work" it as to changing it for the sake of change.

My note, which you believe is a symptom of the problem, was an 
attempt to help with exactly that explanation problem.  I don't 
know how often it has been said explicitly, but it has long been 
understood within the IESG and IAB that appeals that are 
incoherent and that lack clear statements of requested remedies 
are unlikely to be useful.  That doesn't imply they aren't 
considered carefully -- I have been amazed and pleased by the 
amount of time and energy both groups have put into trying to 
understand and respond carefully to appeals, even (or 
especially) the incoherent ones.  But they are less likely to be 
"successful", largely because no one can figure out what 
"success" means (or what was intended by the author).  So, now, 
with Keith's supplement to my not (with which I agree) I hope 
that the process is a bit less opaque and requires a bit less 
inside knowledge and, hence, "process working" skills, than it 
did before.    I would have hoped that would have been 
considered progress and a positive move.

Why haven't the IAB or IESG written those "steps" up as an I-D? 
I don't know.  Perhaps one reason is that those steps look to 
them like common sense.  Perhaps that is a misjudgment, 
especially when compared to cultural contexts in which it is 
normally inappropriate to ask for something in a direct way. 
If so, that is a problem/issue and someone should try to take on 
a writing job... and, if it is useful, my few paragraphs and 
Keith's might be a starting point for a draft.

>> I've also been in the middle of the IAB review process for
>> several of  them.  So, with your indulgence (I don't think
>> this is a problem, and  what I'm about to say isn't a
>> solution), let me explain how to mount a  successful appeal
>> so it isn't a secret any more.
>>
>> 	Step 1: Produce a coherent and focused statement about
>> 	what is being appealed, what the issues are, and why the
>> 	action that was taken was wrong.  Leave out any random
>> 	musings or accusations about, e.g., the eating habits or
>> 	parentage of IESG members and assumptions about their
>> 	evil motivations.  This is often not an easy writing
>> 	job, but it is necessary.  For better or worse, if the
>> 	IESG and IAB can't figure out what you are complaining
>> 	about, or if the appeal seems to be an excuse for a
>> 	personal attack, the appeal won't go anywhere unless the
>> 	problem is really, obviously, an abuse of judgment or
>> 	authority.
>
> Coherency is a value laden judgment and is often difficult for
> people. What is a lack of coherence to one is lack of
> understanding to another. I have often found that it takes me
> a lot of work to find the coherency in an argument.  And the
> fault is often mine.  If I look at the world one way, and the
> writer looks at it another, the language may be such that the
> beauty and coherence of the argument is lost on me.

I think we all have that problem, albeit possibly to different 
degrees.   Some philosophers and linguists have, of course, 
argued that it is fundamental to human communications.  In an 
IETF context, one remedy is a bit of discussion about whether an 
appeal is required and rational and whether one's understanding 
and presentation of the issue are adequate.  Perhaps I should 
have included a "step 0", which might have been

	Discuss the perceived issue with colleagues and get
	their opinions as to whether the issue is worth
	appealing and, if so, what the key issues in the appeal
	really are.

Again, if you can't convince anyone that the appeal is 
desirable, you probably won't be able to convince the IESG or 
IAB either.  Speaking from personal experience, that has 
happened to me several times: I've seen actions that I am 
convinced are inappropriate or unjust, have contemplated filing 
an appeal, have discussed it with others, and have ultimately 
concluded that the amount of trauma associated with an appeal 
would not by justified by the possible outcome or remedy.  Of 
course, one can't win sometimes -- I've been accused of abusing 
authority, or my "power" as a member of some in-group, by 
raising the possibility of an appeal and then not following 
through by actually initiating one.

> And sometimes a complaint is covered by some vitriol that is
> emotionally justifiable as it does happen that sometimes WG
> chairs and AD are not paragons of impartiality and virtue -
> even though I know we try to claim that all problems are
> systemic and have nothing to do with the people themselves or
> human  nature.  And yes, it would be better if they could put
> all hard feelings aside and write like lawyers but sometimes
> they can't. So, underneath there may be a real issue that
> requires a real solution and to ignore it would do harm to the
> complainant and sometimes even to the Internet itself.

We are in complete agreement about this.    I just don't see a 
process change --as distinct from a human nature change-- that 
would fix it... although discussion with colleagues and a bit of 
group effort in shaping the appeal might, again, help.

> These questions are indeed part of my reason for believing
> that the IETF needs some mechanism to make sure that all valid
> complaints do get a just and complete airing.

Again, I think we agree about this.   I am just hesitant about 
piling on more process.

And, as a reminder, my note wasn't about the belief that better 
ways to get valid complaints aired were unnecessary.  I think 
they are necessary (although, personally, I hope they can be 
done informally -- perhaps using some of the methods hinted at 
above-- rather than requiring a new position with a 
potentially-complex authority relationship with the IESG).

>> 	Step 2: Be absolutely clear about what remedy is desired
>> 	and expected, and be sure that it is within the
>> 	authority of the AD or IESG to grant and execute.
>> 	Leaving the IAB or IESG sitting around and saying to
>> 	each other "well, she is probably right, but what on
>> 	earth does she want us to do about it?" is not likely to
>> 	produce a useful and positive outcome.
>
> I also question whether the appellant must also have the
> solution.  Often one see something and 'knows' it is wrong,
> but does not know the right solution.  Or may have a valid
> complaint but an invalid solution.  Is the complaint therefore
> worthless?  And should the problem be ignored in that case?

And the current procedures don't require that a proposed remedy 
be stated, perhaps for that reason.   But I dread --and my 
perception has been that the IESG and IAB dread-- getting into a 
situation in which the proper response to an appeal is "Yes, we 
agree that Vlad is an often-abusive character who isn't gentle 
with those who disagree with him.  So?"

I also deliberately said "remedy" and not "solution".  Suppose 
we have an appeal whose substance is "the solution the WG came 
up with is bad, because..., and doesn't really represent 
consensus, so the IESG shouldn't make it a standard".  "XYZ 
should be the standard instead" might be a "solution", but the 
IESG would be on thin ice if it substituted its protocol 
judgment (and that of the appeal) for that of the WG.  By 
contrast, "the IESG should return this matter to the WG, insist 
that it reconsider the technical matter and gets its act 
together procedurally, and consider whether the current chair is 
the right person to steer that process".   That is a "remedy" 
for the particular problem -- it doesn't require knowing the 
solution.

>> That is really all there is to it.  Most, if not all, of the
>> appeals  that have dragged out for a long time and then
>> produced results that  either upheld the IESG in some
>> lukewarm way or were satisfactory to no  one showed symptoms
>> consequent of omitting one of the steps above.     If the IAB
>> (or even the IESG) get an appeal that says "the problem is
>> obvious, you can read all about it on the following 400 web
>> pages and  many megabytes of logs, AD Jones is a fool and has
>> it in for me, and  I'd like you to un-invent the wheel so I
>> can patent it", that appeal  probably isn't going anywhere
>> even if the underlying issue is a  problem that really does
>> need fixing.
>
> Have there really been that many appeals asking that the wheel
> be un-invented?

There have not been that many appeals, total.   But a 
non-trivial fraction of them have seemed to ask for turning back 
the clock and having a WG start over with a different set of 
assumptions and none of the intervening experience and 
discussions.  That isn't ultimately much different from wanting 
to un-invent the wheel -- the same collective time machine or 
similar process would be needed.

> I think it is is incumbent on those resolving appeals to find
> the underlying problem. Or rather it is the responsibility of
> the IETF leadership, in some way and form, to make sure that
> the underlying issue is always brought forward.

And I think that, in some cases at least, imposing that 
requirement on the "leadership" requires that they learn to read 
minds to understand the real intent of the appeal.   I don't 
think that is a reasonable requirement.  Worse, given the small 
number of people in the community who I think Keith referred to 
as "disturbed personalities", it increases the possibility of 
appeals as DoS attacks: I can imagine a cycle in which the IESG 
and IAB make a guess at the underlying issue and process the 
appeal, only to be faced with a second appeal whose substance is 
"you guessed wrong about the underlying issue, guess again".

Everyone loses in such a process -- especially those who are 
trying to get unrelated work through the IESG or IAB. 
Regardless of how we characterize overload or the absence 
thereof, there really are only so many IESG or IAB cycles 
available in a given month, and appeals, handled carefully, take 
up a lot of them.

>> This isn't either a complex procedure or rocket science... it
>> is just  recognizing that presenting a clear case to the IESG
>> and IAB is much  more likely to get careful attention and
>> reasonable results.
>
> I disagree.  Construction of a valid appeal is a complex
> problem for many. And the process with its nuance of who get
> which  appeal when and in in which order can run people ragged.

Again, if the function of some sort of Ombuds-process is to help 
people understand whether an appeal if plausible and to help in 
framing it if it is, then we are probably in agreement that one 
is needed.  My only concerns on that front are that the level of 
topical understanding needed is such that one might require one 
or two such people per area, not just one for the IETF.  And I 
think it is better done informally, precisely to encourage 
people to "shop" for a sympathetic listener.  But those are 
issues about possible solutions.  If the problem statement can 
be kept to "the appeals process is less effective than it should 
be because some people need assistance in preparing and 
shepherding effective appeals", I'd be in complete agreement 
with it.

regards,
      john