Desired number of competitive results affecting rating?

[PeterGustafsson]

Hi!


In his recent ratings thread, PEET wrote (among other things):

Originally Posted by peet

A rating/ranking system should:

* Accurately reflect a fencer's skill
* Motivate fencers to compete
* Be fair
* Be easy to implement - resistant to error (human & other)
* Be consistent across geographic regions
* Depend on the strength of the event
* Scale well from the top to the bottom (effectively categorize/rank all levels of fencers)
* Appropriately incorporate international results


A rating/ranking system should NOT

* Provide dis-incentive to higher ranked fencers to compete in "lower" events
* Penalize a fencer for "working on something" in a local tournament rather than fence every bout to win.
* Rank a fencer who competes in tons of events (but loses every bout) higher than a better fencer who merely competes a reasonable amount
* Reward "fluke" results
* Allow people to cheat or game the system

peet had other objectives on his lists, but I have deleted in the quote those that did not directly pertain to the number of competitive results which affect the rating of a fencer.

Under the USFA U->A system, only one result determines your rating. Under the NRPS a few more results are important. Under the FIE rating system, lots of world cup results can go into your rating.

So, what is the optimum number (and type of) of rating-affecting competitive results? If the number is too low, flukes can be overrated, if the number is too high, the rating system will reflect resource investment, rather than inherent ability, far too much. Rich parents can send their kid to gobs of tournaments where the kid picks up some points here and there, surpassing the good - but poor - competitor who can only go to one competition but places high there.

OTOH, if one specifies (is is common in most rating systems today) that only the best competitive results (until they time out) of a given fencer affect his rating, then many of the points in peet´s lists are fixed. I have underlined those points in the quote above. Such a limitation will unduly reward the fencer with big ups and downs over a more consistent one of the same average ability, but I think that is price that we have to pay to have system without lots of the problems in peet´s "avoid" list.

So, what is the number of rating-affecting results that you consider appropriate?


Have a nice time!

Peter Gustafsson

[Sciurus-Rex]

((self-edited to reduce unnecessary sarcasm))

Actually, P.G., it's not a single event that determines your rating, but rather a single high mark in addition to a time element -- the year following a letter. And the time element is regularly downgraded. That probably needs to be referenced as well.

[wbowman]

Does that mean the poll is flawed?
(Sorry, couldn't resist)

[downunder]

Originally Posted by wbowman

Does that mean the poll is flawed?
(Sorry, couldn't resist)

(sorry, you really didn't have to post this)

Its not really that funny anymore.



Pete, we seem to have a ratings thread every week, and we've had 3 or 4 this week alone.

It isn't really broken. As far as i'm aware, there is a rolling ranking at the top for the elite athletes in the US, and at the lower level, the smallest amount of rating inflation never really hurts anyone.

We have a comprension ranking in both the UK and Aus for every competition I enter. I still get a sh*t poule some of the time and i get an easy poule some of the time. Sometime i get a fair poule. Its part of the sport. No system is perfect, but i think the US system does a pretty good job of handling a massive number of fencers with a pretty fair result.

[fencerbill]

Originally Posted by PeterGustafsson

Hi!


In his recent ratings thread, PEET wrote (among other things):


peet had other objectives on his lists, but I have deleted in the quote those that did not directly pertain to the number of competitive results which affect the rating of a fencer.

Under the USFA U->A system, only one result determines your rating. Under the NRPS a few more results are important. Under the FIE rating system, lots of world cup results can go into your rating.

So, what is the optimum number (and type of) of rating-affecting competitive results? If the number is too low, flukes can be overrated, if the number is too high, the rating system will reflect resource investment, rather than inherent ability, far too much. Rich parents can send their kid to gobs of tournaments where the kid picks up some points here and there, surpassing the good - but poor - competitor who can only go to one competition but places high there.

OTOH, if one specifies (is is common in most rating systems today) that only the best competitive results (until they time out) of a given fencer affect his rating, then many of the points in peet´s lists are fixed. I have underlined those points in the quote above. Such a limitation will unduly reward the fencer with big ups and downs over a more consistent one of the same average ability, but I think that is price that we have to pay to have system without lots of the problems in peet´s "avoid" list.

So, what is the number of rating-affecting results that you consider appropriate?


Have a nice time!

Peter Gustafsson

I hit quick reply but got quote anyway (?).

I think P.G is mixing up ratings and rankings. Ratings are single result which tarnishes each year and fades after 4 if not renewed. Rankings are used for FIE World Cup sorting and, for US team membership, also use domestic as well as foreign results.

The point about people affording to go to more events, particularly foreign events, having an advantage about those less economically fortunate is quite valid. With less money, it typically could be 4 out of 4, for example. With more resources it could be best 4 of 7 opportunities (the numbers are not necessarily those actually used, just examples).

How do we balance this? Use the first four of the year or the last four of the year? One possibility: Have the fencer prioritize the possible point events they go to. They could go to seven, for example, but only use the results of the four they designated before first event. There would be a board they would have to satisfy to change the events designated in case of injury usw.

So far US Veterans Fencing has steered clear of this, ONLY best 2 of 3 qualifiers/Summer Champs are used, no bonus from Div 1 or foreign placements or from prior year World placements.

[PeterGustafsson]

Hi!

Originally Posted by fencerbill

I think P.G is mixing up ratings and rankings.

PEET wrote about rankings in the text I cited, but Peter did not write about rankings at all in the threadstart. The similarity in names could have gotten you mixed up.

Originally Posted by fencerbill

Ratings are single result which tarnishes each year and fades after 4 if not renewed. Rankings are used for FIE World Cup sorting and, for US team membership, also use domestic as well as foreign results.

That is not the way I have thought of the distinction. From Wikipedia:

Ranking is the process of positioning individuals, groups or businesses on an ordinal scale in relation to others. A list arranged in this way is said to be in rank order. Ranking is a common technique used in non-parametric statistics.

Rating is a means of classifying things in different categories.
* In online communities, rating means a reputation system, it is a type of collaborative filtering algorithm which attempts to determine ratings for a collection of entities, given a collection of opinions that those entities hold about each other. This is similar to a recommender system, but with the purpose of entities recommending each other, rather than some external set of entities (such as books, movies, or music).
* ELO rating system, for calculating the skill level of a player in a two-player game

So, ranking is the number that is derived from the global (in the mathematical sense) rating for a local (in the mathematical sense) competition. An E will be ranked 1st if all other competitiors are rated U´s, while he will be ranked last if the all are rated A´s. Same rating, different ranking dependent on the competition.

Originally Posted by fencerbill

The point about people affording to go to more events, particularly foreign events, having an advantage about those less economically fortunate is quite valid. With less money, it typically could be 4 out of 4, for example. With more resources it could be best 4 of 7 opportunities (the numbers are not necessarily those actually used, just examples).

How do we balance this? Use the first four of the year or the last four of the year? One possibility: Have the fencer prioritize the possible point events they go to. They could go to seven, for example, but only use the results of the four they designated before first event. There would be a board they would have to satisfy to change the events designated in case of injury usw.

I see several problems with this system:
1. There can be a "chicken-race" situation where fencers wait each other out to designate their events as late as possible, since the fencer who knows the others designations is at an advantage.
2. In an event, it is quite possible that there will be one or more bouts between fencers where one has that event as one of his designated ones, while the other fencer has not. Those fencers will be quite differently motivated, which can create all sorts of quirky results. Such results can also affect other fencers, who can not do anything to counteract the situation.
3. If there is a board which is tasked with allowing or disallowing a change in designation status, then we create yet another non-automatable task for the USFA headquarters. Such descisions are also uinherently subjective, opening up for accusations of favoritism, politicing, etc. That is not a good situation.

Another way to level the field, economically, would be to have a two-tier qualification system for national teams. Those who make top-10 (or so) of the NRPS list become eligble for a (or at most) run-off competition which is the ultimate team decider. Rich, not so talented, fencers which have managed to reach the top-10 by scraping together enough points at lots of competitions, will then have to face the better, but less well funded fencer in the run-off. There, money will be at most a minor factor.


Have a nice time!

Peter Gustafsson

[PeterGustafsson]

Hi!

Originally Posted by downunder

Pete, we seem to have a ratings thread every week, and we've had 3 or 4 this week alone.

It isn't really broken. As far as i'm aware, there is a rolling ranking at the top for the elite athletes in the US, and at the lower level, the smallest amount of rating inflation never really hurts anyone.

Well, if you find those threads boring, I invite you to not read them. We might even petition Craig and Gav to create a new "ratings discussion" subfolder of the fencing discussion board. Even easier for you to avoid them!

Originally Posted by downunder

We have a comprension ranking in both the UK and Aus for every competition I enter. I still get a sh*t poule some of the time and i get an easy poule some of the time. Sometime i get a fair poule. Its part of the sport. No system is perfect, but i think the US system does a pretty good job of handling a massive number of fencers with a pretty fair result.

So, you think the rating system is good enough, and that it should not be changed for that reason. Fine, you are entitled to your opinion. However, my opinion is that one should strive for as perfect a rating system as is humanly possible, within reasonably economic constraints. Furthermore, it has been amply showed, with numerous examples, on this board that the USFA rating system is far from that level.


Have a nice time!

Peter Gustafsson

[glowstix]

Originally Posted by downunder

(sorry, you really didn't have to post this)

Its not really that funny anymore.

"not really" funny anymore?? its not funny at all!!!!!!!! sh*t has gotten so old its ridiculous..

[oiuyt]

Originally Posted by PeterGustafsson

Another way to level the field, economically, would be to have a two-tier qualification system for national teams. Those who make top-10 (or so) of the NRPS list become eligble for a (or at most) run-off competition which is the ultimate team decider. Rich, not so talented, fencers which have managed to reach the top-10 by scraping together enough points at lots of competitions, will then have to face the better, but less well funded fencer in the run-off. There, money will be at most a minor factor.

Our very own "Race for the Chase".

Interesting idea.

-B

[Inquartata]

Originally Posted by glowstix

"not really" funny anymore?? its not funny at all!!!!!!!! sh*t has gotten so old its ridiculous..

Dissent! It IS funny, at least outside of the British Commonwealth.

Luckily neither of you is arbiter of humor around here.

[KD5MDK]

I'm pleased to have taken that role myself.

I don't see any reason why best X of Y isn't suitable, with X and Y determined by the category of event (Junior, Senior, Vet) etc.

[glowstix]

Originally Posted by Inquartata

Dissent! It IS funny, at least outside of the British Commonwealth.

Luckily neither of you is arbiter of humor around here.

aiight, inqy-poo, have it your way, it WAS funny but clearly it's run its course and should be retired.

[Inquartata]

That's like saying "Everyone has heard all the Monty Python lines a hundred times, so they aren't funny any more and should no longer be quoted".

Just isn't going to happen.

[downunder]

Originally Posted by Inquartata

That's like saying "Everyone has heard all the Monty Python lines a hundred times, so they aren't funny any more and should no longer be quoted".

Yeah thats pretty much right too.

[fencerbill]

Originally Posted by KD5MDK

I'm pleased to have taken that role myself.

I don't see any reason why best X of Y isn't suitable, with X and Y determined by the category of event (Junior, Senior, Vet) etc.

So:

Having deeper financial pockets is more important for being selected to world teams than how well the person fences.

[KD5MDK]

That depends on the size of X and Y, doesn't it? I don't think best 1 of 2 is all that unfair to people who can't afford to go to many tournaments.

[RoninX]

"How many non-timedout results should affect fencer ratings?"

So, just to be clear: if I liked the current system I would say "1", correct?

If I thought you should have to "earn your A" twice for you to acheive that ranking I would say "2"?

But I could not acheive those results 5 years apart under the current system (4 year time out) because the first rating would have timed out by then?


If the above assumptions are correct then when does the rating "time out"? 4 years after you qualify for the rating (with your X win at the required level) or 4 years after your first placement in the series that leads to you qualifying?

[PeterGustafsson]

Hi!

Originally Posted by RoninX

"How many non-timedout results should affect fencer ratings?"

So, just to be clear: if I liked the current system I would say "1", correct?

If I thought you should have to "earn your A" twice for you to acheive that ranking I would say "2"?

Yes, and yes, that is what I intended it to mean.

Originally Posted by RoninX

But I could not acheive those results 5 years apart under the current system (4 year time out) because the first rating would have timed out by then?

If the above assumptions are correct then w