Sampling

Harald Tveit Alvestrand harald at alvestrand.no
Mon Jul 28 10:22:04 CEST 2003



--On 27. juli 2003 12:22 -0700 Dave Crocker <dhc at dcrocker.net> wrote:

> Folks,
>
> The latter part of a posting by John cites statistic issues.  I'd like
> to explore that issue separately and carefully:
>
> I believe that both the working group participation and the IETF Vienna
> plenary presentation represent very, very highly biased samples of the
> IETF population.

Yes. In both cases, they consist of the people that show up.

> Sampling bias does not require intentional "packing". It requires
> selection methods that make the group unrepresentative of the total
> population.
>
> In this case, we had a smaller-than-expected total IETF meeting
> attendance, in a venue that prevented significant numbers of regular
> participants from attending. And not all of the meeting attendees were
> Plenary attendees.
>
> Notably, we have not paid much enough attention to just how small the
> participation in the Problems working group has been. And the recent
> Plenary had all sorts of confusion to this discussion. Overall, we need
> to be very circumspect in making decisions based on either bits of
> input.

Actually I was pleasantly surprised that about a quarter of the plenary 
actually claimed to have read the problem drafts. This is significantly 
higher than many working groups, and indicates that the part of the 
community that shows up at plenaries outside the US *does* care about the 
issues, enough to do prep work.

We trust WG meetings with less prepwork made with decisions, too.

>
> However the call for Last Call review can (and does) lead to
> re-synchronization far too late, I believe. We need to make sure that
> there are ways of doing synchronization with the community early and
> often.
>
> Even waiting for a Plenary presentation might be too infrequently.
> Perhaps we need tight, summary "presentations" on the Announce-list,
> occasionally.  We might think of these as "phase" Last Calls, or
> somesuch.

actually the participants on ietf-announce (about 2500 of them) are a 
*third* statistically skewed sample of our participants - and the probably 
smaller number who actually read and think about responding to Last Calls 
(of any sort) is even more statistically skewed.

I think of this as a problem, in general.....

                       Harald



More information about the Problem-statement mailing list