Competition formats resistant to bad prior rating information

[PeterGustafsson]

Hi!


Now, as many times before, the issue of non-perfect ratings causing uneven pools - and therefore flaky end results - is being discussed again.

If one uses straight DE from the beginning good rating is very important to get the best possible rating, but things go fast.

If one uses a giant pool which includes all fencers in the competition the rating information becomes moot, but it takes a loooong time.

The currently most common format, pools->straight DE, is a compromise in both regards. However, it does not include any correcting measure where effects of bad initial seeding are counteracted. One good side of it is that it is reasonably easy on the bout committe workload.

The format pool1->pool2->straight DE takes for time, and is less prone to faulty ratings. The latter is done by a dilution effect, however

If one uses a list of fencers with rating information, such as it is, one can use it as a starting point for a shell-sort algorithm. The shell-sort algorithm is quite good for reasonably well sorted lists. It is described here: http://en.wikipedia.org/wiki/Shell_sort
OTOH, it requires that the bout committee to print out a new bout list between each set of bouts - no fencer can know which their next bout - after the one coming up - is.

So, I put up for discussion these points:
1. What are good ways to introduce ratings error corrections inside a competition format (i.e. sorting algorithm) so that a few faulty ratings are not just diluted, but actively counteracted?
2. What are in general good competition format which combine both robust behavior (the produced results list is reasonably close to the "true" strength of the fencers, at almost all tries) and are reasonably fast?
3. Which, if any, competition format is at least as good as the conventional one in both respects (robustness and speed), and on average better than in in at least one respect?

For the sake of the discussion, assume that the bout committee has access to a strong computer, and that its members are competent to use whatever format one can devise.


Have a nice time!

Peter Gustafsson

[KD5MDK]

(the produced results list is reasonably close to the "true" strength of the fencers, at almost all tries)

There's a flaw in this, specifically that fencer results will be consistant in competitions.

[the doc]

personally, i don't think you should even try to get some kind of perfect distribution of seeding. sometimes, you get an easy poule, sometimes you get a hard poule. just live with it. whatever format you choose the best people will always win, the rubbish people will come at the bottom and it gets a bit fuzzy in the middle. no cause for alarm. everything evens itself out if you compete enough.

[glowstix]

Originally Posted by PeterGustafsson

2. What are in general good competition format which combine both robust behavior (the produced results list is reasonably close to the "true" strength of the fencers, at almost all tries) and are reasonably fast?

the current one as far as i'm concerned...and i agree with the previous poster..

[mrbiggs]

As far as I'm concerned, if a high school graduate can't do it with index cards, it's not a good system.

And I also agree that there is little to nothing wrong with the current system.

[fencerX]

Yipee. I was wondering if anyone would ever make it past the ever popular topic of whining about classifications and seeding and even mention how event formats affect results. Discussing seeding while ignoring format is just silly.

The answer to the question "can classifications and seeding affect the results of fencing competitions" is it depends. Sometimes the answer is it can be proved that the seeding doesn't matter. Sometimes seeding can matter.


Before complaining about classifications affecting seeding which could possibly affect results, you really need to go back and figure out what do you want the results to do for you in the first place. Is the purpose to identify (solely) the best fencer that day's event (to hand them the prize, etc)? The best X top fencers? Is that the best X fencers in that day's event or over time and multiple events? The best Y% fencers to qualify them for something or other? Do you require or want all of these top X or Y fencers in order? Do you really wish or expect or is it necessary to be able to place every competitor in the "true" position of their "relative performance" in the event?

Different event (and qualification and seeding) approaches can help you do different things, but first you need to figure out and agree on the point of the entire exercise.

[KD5MDK]

However, the format of competitions in general is up to organizers: Any approved format may be used, and there are many, including pools to pools to pools, pools to DEs to pool, etc. However, all international competitions follow pretty much one format (excepting the Olympic Games). Also, as long as the format is flexible, the seeding system needs to be able to accomodate that.

[PeterGustafsson]

Hi!

Originally Posted by KD5MDK

There's a flaw in this, specifically that fencer results will be consistant in competitions.

Of course fencers are inconsistent, to lesser or higher degree! However, if a set of fencers would compete many times, the average result for any one fencer would approach some value, which would then be the true strength of that fencer within that group.

One feature of a good competition format is that it tends to produce results lists which are reasonably close to rank of the true strengths, with fencer inconsistency thrown in as statistical noise. However, inconsistency due to competition format effects should be counteracted. Learn to separate those two concepts.


Have a nice time!

Peter Gustafsson

[PeterGustafsson]

Hi!

Originally Posted by the doc

personally, i don't think you should even try to get some kind of perfect distribution of seeding.

The point of this thread was to discuss competition formats which work well with bad seeding - not how to improve upon seeding itself. Please try to read the topic.


Have a nice time!

Peter Gustafsson

[PeterGustafsson]

Hi!

Originally Posted by PeterGustafsson

2. What are in general good competition format which combine both robust behavior (the produced results list is reasonably close to the "true" strength of the fencers, at almost all tries) and are reasonably fast?

Originally Posted by glowstix

the current one as far as i'm concerned...

OK then, we have a response to the posed questions. At last!
So, if you think this format is good - do you claim to have any proof that it is the best? Any argument for plausibility?


Have a nice time!

Peter Gustafsson

[PeterGustafsson]

Hi!


From the threadstart:

Originally Posted by PeterGustafsson

For the sake of the discussion, assume that the bout committee has access to a strong computer, and that its members are competent to use whatever format one can devise.

Originally Posted by mrbiggs

As far as I'm concerned, if a high school graduate can't do it with index cards, it's not a good system.

Please - If you really think that a premise is faulty, then argue for that position. If not, follow the premise - or start another thread where you state the premises.

That said, there are 3 features that a competition format might have and are under discussion in this thread.
1. Fast - reaches a final results list in few bouts
2. Robust - gives good results list even when bad or nonexistent ratings information is used.
3. Simple for the BC - low workload in figuring out which bouts should be fought, and when.

The Olympic format (straight DE from beginning) combines features 1&3, but it is not robust. The maxipool format combines features 2&3, but it takes a long time.

The whole point of this thread is to discuss formats which combine features 1&2, while feature #3 - while not bad to have - is not a deciding factor.


Have a nice time!

Peter Gustafsson

[edew]

The Olympics format is adequate for the Olympics because the seeding is a fairly accurate representation of the skills of the fencers (modulo the randomness in deciding who among the top-4 will be first, second, third and fourth). These fencers have gone through 1+ season of many events to derive their rankings, and hence, their seeding into the Olympics. It works because there isn't any "bad prior rating information", as premised by the start of your thread.

I wouldn't recommend it for non-Olympics events, mainly because people ain't going to have too much fun knowing they're seeded into a straight DE. People aren't going to pay $$ to fence probably one bout, or two, maybe.

[Arbe]

Originally Posted by PeterGustafsson
2. What are in general good competition format which combine both robust behavior (the produced results list is reasonably close to the "true" strength of the fencers, at almost all tries) and are reasonably fast?

What is the "true" strength of the fencers? There is the rock paper scissors issue: Fencer A beats Fencer B, Fencer B beats Fencer C, Fencer C beats Fencer A. This could happen because of different styles, or different strengths and weaknesses. So which fencer is strongest?

Is this a good definition of true strength: the result from a round robin tournament.

[mrbiggs]

Originally Posted by PeterGustafsson

Please - If you really think that a premise is faulty, then argue for that position.

Yes, I think the premise is faulty. If the tournament must be run by a computer, there's too much room for error. How does a fencer know if a mistake is made, by the scorekeeper, whoever input the scores into the computer, or even the program?

Using the method right now, I can tell who exactly I am going to fence at any given time. And this has helped me several times in figuring out where I'm fencing, when I'm fencing, and if a mistake has been made. This is, of course, on top of the convenience for whoever is running the tournament.

[KD5MDK]

Peter - Have you ever encountered a fencing problem (off-strip) for which you haven't tried to construct an algorithmic solution?

[PeterGustafsson]

Hi!

Originally Posted by KD5MDK

Peter - Have you ever encountered a fencing problem (off-strip) for which you haven't tried to construct an algorithmic solution?

In general, it is my second resort, used if I do not know any tried-and-true solution that does not have any drawbacks. In the case that I do not attempt to construct an algorithmic solution, I do not post that non-attempt here, so your perception of me becomes a little skewed. (Surely, my perception of you is likewise skewed by what you choose to post here.)

In general, contruction of algorithmic solutions brings me great joy.


Have a nice time!

Peter Gustafsson

[PeterGustafsson]

Hi!

Originally Posted by mrbiggs

Using the method right now, I can tell who exactly I am going to fence at any given time.

Once you reach the DE stage, you can only be sure of your upcoming opponent - the next one is decided by a bout outside your control, and often not yet fenced.


Have a nice time!

Peter Gustafsson

[Wafath]

Interesting topic. I have been thinking about something similar for USACFC.

There is an advantage to the current, relatively simple formats we use today: it passes the blame to the fencer. Fencers know what it is well in advance, and can train for it. Got a rough pool, or an unfortuneate DE? Tough. Do better, get a better seed, and prove that you deserve it.

I think various two pool round formats may offer better protection against poor seeding information. The second round of pools should be better balanced to reflect how fencers are doing that day. But even they aren't perfect, as if you had a fairly easy first round, you are going to have a fairly easy second round.

If one wanted to go completely hog wild with the format, have an infinate organizing budget, and want the fencers to have no clue who they are fencing next, how about something like:

round 1: v. small pools. Have everyone fence in pools of 4, seeded as normal.
round 2: more 5 touch bouts. After round 1, mathematically calculate every fencer's strength, which should be a combination of fencer's results, and who they fenced. Have each fencer fence other opponents s/t each fencer has competed against an equal number of opponents, and a near-equal average opponent strength. After every round of bouts (max one bout per fencer) assign the next round based on the total results of the pools and all previous bouts.

(the only way this could work with more than about 16 fencers is to have the referees electronically enter in each bout immediately after at the strip, and have some vastly distributed information on where fencers are to go next. Like, perhaps, a palm like device for each ref, plus a lookup terminal per strip. As I said, inifinate budget.)

Round 3: after each fencer has fenced a total of 6-8 bouts, seed a DE based on fencer strength.

Note, I don't recommend anyone actually doing this. It would be pure chaos. But I think it meets the desired requirements.

W

[Arbe]

2. What are in general good competition format which combine both robust behavior (the produced results list is reasonably close to the "true" strength of the fencers, at almost all tries) and are reasonably fast?

If I were given the task of generating an accurate results list, then the bout I would be least interested in would be some A-rated fencer facing some U-rated fencer. I would be most interested in some A-rated fencer facing some other A-rated fencer. Yet our current system actively avoids such pool bouts between equally rated opponents.

There are several possibly valid reasons for doing this. Maybe tournaments have other goals beyond producing accurate results lists. Maybe the ratings aren't accurate enough to be given much strength in determining DE seeding.


One alternative pool assignment system would just put the top 7 rated fencers in pool 1, the next 7 in pool 2, and so on. The top 7 rated fencers would get the top 7 DE seeds, with the order determined by the pool results. This system gives ratings a lot more strength than the current system. Maybe too much?


To get a handle on this, I looked at how A-rated fencers did in this year's senior sectional (North Atlantic):
There were 64 fencers: 5 A's, 20 B's, 14 C's, 7 D's, 11 E's, and 7 U's.
The five A-rated fencers were put into 5 different pools.
Their pool percentage wins were: 40%, 50%, 67%, 83%, and 100%.
Their DE seedings were: 41st, 34th, 16th, 4th, and 2nd. (out of 64)
Their final results were: 46th, 42nd, 12th, 3rd, and 1st. (out of 64)

(The results are in order. That it, the fencer who won 100% of his pool bouts was seeded 2nd and finished 1st. The fencer who won 40% of his pool bouts was seeded 41st and finished 46th.)


The 5 A-rated fencers had widely different results. So, basing DE seeding mostly on rating appears to be a bad idea.

The current pool system did a good job in this tournament. The DE seedings were very close to the final results for the A-rated fencers.

[Poulet]

Originally Posted by Arbe

One alternative pool assignment system would just put the top 7 rated fencers in pool 1, the next 7 in pool 2, and so on. The top 7 rated fencers would get the top 7 DE seeds, with the order determined by the pool results. This system gives ratings a lot more strength than the current system. Maybe too much?

What's the incentive to try hard in pools under that system? Being seeded 1st or 6th doesn't really ever matter, except in tiny competitions. Likewise, being 40th or 45th makes virtually no difference, you're just about as likely to have a harder bout being seeded 40th as 45th. Pools would end up being some sort of warmup, where everyone is a little tentative (especially the good fencers, who might not want to familiarize all their toughest competition, who they're likely to face later in the day, with how they're fencing that specific day, what moves they're relying on, etc.) and would really serve no purpose.