Previous Rating System

[Allen Evans]

I find the concept that we should want to add more extrinsic motivators to competition equally hysterical and horrifying.

....and in this particular case, quite confusing.

[tchwojko]

The current classification and points system is supporting two different goals. One is official, the other is unofficial.

Classification is used officially to seed, and to determine whether a fencer can participate in specific events. The points are used to determine the right to represent the US. It works just fine for these goals. You could argue it could be better, but it's not fundamentally broken.

Unofficially, some use it as a status symbol. That's a problem with attitude, not the system.

Also unofficially, many people use it as a measure of progress. The classifications are rather poor at doing that job, for various reasons.

An ELO system could work just fine for this in parallel to the classification system, as long as it is never used to make decisions about who can do what. If there are no consequences to a change in rating, then for most the motivation is intrinsic. The ELO can be used to measure progress, bragging rights, whatever anyone wants, but it doesn't have any real world effect, nor does the USFA need to be involved. The divisions could do this, or Peet could possibly be persuaded to add this feature (perhaps at a membership premium?) to AskFred. Or someone could develop something that took the data from AskFred and spit out ELO scores...damn, I've got another project I won't get to until it's too late...

[Jason]

The divisions could do this, or Peet could possibly be persuaded to add this feature (perhaps at a membership premium?) to AskFred. Or someone could develop something that took the data from AskFred and spit out ELO scores...damn, I've got another project I won't get to until it's too late...

And what happens after the thrill of Elo bragging rights wears off? Yet another system on top of that one, no doubt. Clearly, what fencing in this country needs is several classification systems with various degrees and types of nuance to distinguish the pretty good from the very slightly better from the almost but not quite exactly as good; different points systems for beginners, kids, adults in Div III events, people in Wyoming, transexuals, people who've fenced for a few years years and then stopped, people who fenced for a while but in a different weapon, people who've only ever fenced one weapon but wish they could try another; ratings to determine the best clubs, the best division, the best section, the best neighborhood, the best street corner; rolling lists, non-rolling lists, bouncing lists, thumping lists, lists for selection, lists for fun, lists for AskFred, lists for the USFA website, lists for home, lists for the club, lists for the internet; and, of course, the address of a good methadone clinic.

[K O'N]

Let me start by saying that I am sorry for this "wall of text" but I feel strongly about this subject and I think that Masin is right.
---

Personally, having competed for many years USCF chess competitions and having played the rating game there, I do like the Elo system (it is Elo for Arpad Elo BTW, it is not an acronym).

Arrgh, you're right. I don't know why I do that, I knew it was someone's name.

I agree that the fear of losing rating points might be detrimental in a few cases but overall my experience with the system was entirely the opposite. Mostly people didn't compete out of fear of gaining points. The sections of competition in a USCF event were often the following:

Under 1200
Under 1400
Under 1600
Under 1800
Under 2000
Under 2200
Open

Since US Chess competitions nearly always awarded cash prizes, something I wish USFA competitions did, and since it did this on a section by section basis, there was a strong incentive for folks to compete. Once a person reached the cusp of entering a new rating bracket however, that incentive was reduced since often such people would wait until a really large tournament was held with larger cash prizes.

Right, so you can see by this that the ratings do change the way people act in entering or not, they're not just for seeding. I don't think they'll have the same effect in fencing they do in chess, but at least this is some evidence that they're not just a pure measurement. It's sort of Heisenbergian, in measuring the players the system is changing what's being measured.

This later scenario was somewhat rare since most people liked the prestige of earning a higher rating enough that they would compete even when it wasn't in their best material interest.

In the paper by Masin he was not suggesting that the rating system ever be used for team selection, this is especially important. Since the main criteria for team selection should always be international results and since these events are going to be with opponents who are outside the USFA rating system, one should never use rating for selection. Instead the current point system, as Masin indicated, should be used for this. The Elo rating would be used for seeding in the event that a competitor had no points.

So, anyone with points would have top seeding? Or do we ignore points for Elo ratings in USFA events? You seem to be saying the former, that seems problematic to me.

While seeding is important and a person would want to optimize their initial seed in large tournament, it is unlikely that a person with a very high rating would move much, even with a poor result. This at least was true in chess.

Right, this is the crux of the issue. Fencing isn't chess. Even a small difference in skill in chess is a large predictor of victory. In fencing that's not true.

The reason for this is that as you reached the 2100+ rating level your K-factor became 24, and by 2400+ rating level your K-factor became 16. Effectively this meant that the movement of your rating for both wins and losses was cut in half for high rated players. If we followed a similar policy in US Fencing, one could reduce the K-Value to anything reasonable at whichever level made the most sense to encourage fencing.

K-value adjustments are a good idea, but they still don't fix the fundamental problem; fencers are a lot more variable than chess players. A fencer in a slump could lose several weeks in a row in the first or second DE, even if he's pretty good.

Since quite often fencers at DIV1 level will be fencing others of the same rating,

Wait wait wait, what? Part of the concern is that fencers who are what we now think of as As would not fence local events, since they might lose points. You can't dismiss that whole class of event as not important or not usual, if the local cadre of As stop coming to local events, that would be a disaster for us.

their chance of gaining or losing a large number of points is quite reduced. With a K-Factor of 16, two equally rated fencers would stand to gain or lose 8 points on any given bout. With a very bad showing a person might lose 10-12 bouts in the seeding pools. This would be an average rating loss of about 80-96 points if they lost every single bout, and this assumes too that we don't create an Elo system that accounts for points scored in the bouts, just the victory/defeat.

To put it simply, a person who is afraid that they will lose 80 points by losing every bout in the seeding rounds, is probably not a big concern for most fencers.

I... again, what? You're just talking about someone who's highly variable in his performance. You just don't care if such people fence anymore? Why are they so unvaluable?

I had a student who went from a U to an A in one event. He was hugely variable, when he was on he was a monster. When he was off he was a guppy. Someone like this who wanted to fence at JOs would reasonably fence himself up to a high rating and then hide. Now, maybe everyone wouldn't do that, this kid wouldn't have, he wouldn't have cared, but is it a good idea to have perverse incentives and then assume people will just ignore them?

If that guy decides to skip the tournament to hold onto his clearly inflated rating, it's not going to disappoint folks too much.

I don't want to cut out any class of fencer and say, eh, we're not too worried if you don't come. I don't care if they're variable, too focused on some future national event, whatever, I want them to fence. You have too many people in this list, you'll forgive me being blunt, whom you are not worried about losing.

The kinds of people we would concern ourselves with are ones who are actually good fencers who think they need to protect their rating. Those fencers would rarely if ever lose more than a few measily points in a tournament because of their actual skill.

Good fencers lose to scrubs all the time.

I think that the fencer who skips a tournament to preserve his "high" rating is a myth. Further I think that without cash prizes you will not see fencers skipping tournments to preserve low ratings either.

Huh. I don't know.

My feeling is, the variability of fencer performance really makes an Elo system problematic. As it is an A10 can come fence in a local event and have fun. If he's the highest rated fencer by a good bit, say only one B and a bunch of Cs and Ds, it's mostly an altruistic act on his part, or just socializing, or he wants to work on his foot touch or something. Ok. It has no possible effect on his rating. Under an Elo system the upside in points will be very minimal, but the downside will be pretty big. Ok. That's the same as in chess, right? But in chess the odds that an A-analog would lose to a D-analog is about zero. In fencing it's not. He could lose. Then he loses something near his entire K value, since he was expected to win that bout.

Why would he go fence? We've changed from a system that in essence doesn't care that he went to fence to one that actively discourages him. Great.

Anyway, interesting stuff. Thanks for the input, it's good to hear from someone who's played under an Elo system in real life.

K O'N

[oiuyt]

Out of curiosity, how do other one-on-one sports handle ranking/seeding? Things like tennis, ping-pong, other racquet sports, wrestling, etc.?

At least one, if not both, of the competing statistical systems in use for table tennis were on Evan's list of systems to examine.

-B

[tchwojko]

And what happens after the thrill of Elo bragging rights wears off? Yet another system on top of that one, no doubt. Clearly, what fencing in this country needs is several classification systems with various degrees and types of nuance to distinguish the pretty good from the very slightly better from the almost but not quite exactly as good; different points systems for beginners, kids, adults in Div III events, people in Wyoming, transexuals, people who've fenced for a few years years and then stopped, people who fenced for a while but in a different weapon, people who've only ever fenced one weapon but wish they could try another; ratings to determine the best clubs, the best division, the best section, the best neighborhood, the best street corner; rolling lists, non-rolling lists, bouncing lists, thumping lists, lists for selection, lists for fun, lists for AskFred, lists for the USFA website, lists for home, lists for the club, lists for the internet; and, of course, the address of a good methadone clinic.

That's exactly why I'm saying this should not be a USFA job. There's nothing evil about people following a meaningless point list, as long as it stays meaningless. (Look at the success of XBox Live achievements and many other video game scoring systems.) It's just a bunch of numbers to be crunched providing some data. Let people use that data however they want.

[gillaspy]

Mr Masin came up with the idea of Div II and III national events, so I'm permanently grateful to him, but ELO type numerical ratings for fencing are a bad idea. If you can lose points by fencing there will often be a reason to stay home. Bad effect, right off the bat.

... and meanwhile the kid who lucked into a 1250 at a weak A2 or whatever it's called in the brave new ELO world is sitting home nursing his rating, he wouldn't go fence before JOs or SNs for money. Oy. Terrible idea.

K O'N

If that kid got his/her A rating under the present system at a weak A2 and knew he/she was "not really an A", "not looking bad" would be an equally cogent reason to not fence at tough competitions in the future. As I've stated before on this forum, chess "A's" don't stay home for these kinds of reasons, that I know of. They play because they enjoy the game.

To me, one of the biggest problems with the 'A' rating is it is too broad a category. It doesn't distinguish between the before mentioned kid who may not deserve the rating and a national champion who happens to be fencing in a local tournament. If every 'A' got their rating in 2010 and the champion is named Zaphier, he would be seeded last among the 'A's, correct? How could that be right?

[Jason]

If every 'A' got their rating in 2010 and the champion is named Zaphier, he would be seeded last among the 'A's, correct? How could that be right?

No. National points trump rating.

If there's so much confusion as to which A is the best, maybe they should have a competition or something.

[Allen Evans]

If every 'A' got their rating in 2010 and the champion is named Zaphier, he would be seeded last among the 'A's, correct? How could that be right?

National points trump ratings, as Jason says, and among "A"'s of the same year with no national points, the seeding should be randomized, not alphabetical.

A

[fdad]

If every 'A' got their rating in 2010 and the champion is named Zaphier, he would be seeded last among the 'A's, correct? How could that be right?

Seeding within a Rating(Letter/Year) is supposed to be random, but as we know that random value is frequently assigned at birth.

[K O'N]

If that kid got his/her A rating under the present system at a weak A2 and knew he/she was "not really an A", "not looking bad" would be an equally cogent reason to not fence at tough competitions in the future. As I've stated before on this forum, chess "A's" don't stay home for these kinds of reasons, that I know of. They play because they enjoy the game.

The standard deviation of chess player performance is much smaller than the standard deviation of fencer performance.

As a technical note, Elo assumed all chess players had the same sigma. Fencers most assuredly do not. I have no idea what that would do to a rating system.

To me, one of the biggest problems with the 'A' rating is it is too broad a category. It doesn't distinguish between the before mentioned kid who may not deserve the rating and a national champion who happens to be fencing in a local tournament. If every 'A' got their rating in 2010 and the champion is named Zaphier, he would be seeded last among the 'A's, correct? How could that be right?

The lucky A2 event would have had to have been in 2010 to give an A10. Presumably he won't get lucky again in 2011, so in two months the national champ will win an A event and be an A11, and they won't be tied anymore. If this really bothers us we could use points for even local seeding, which presumably would put our champion Mr Zaphier above our lucky scrub Mr Aaphir all the time, instead of the half the time random seeding would.

The E-D-C-B-A-points system isn't really as coarse as it looks, it's 21 levels, U, E07, E08, ..., A09, A10, and then points within A10. Seriously, how much better will an Elo system do than this in seeding, do you think?

K O'N

[gillaspy]

...
Right, this is the crux of the issue. Fencing isn't chess. Even a small difference in skill in chess is a large predictor of victory. In fencing that's not true.

...

I had a student who went from a U to an A in one event. He was hugely variable, when he was on he was a monster. When he was off he was a guppy. Someone like this who wanted to fence at JOs would reasonably fence himself up to a high rating and then hide. Now, maybe everyone wouldn't do that, this kid wouldn't have, he wouldn't have cared, but is it a good idea to have perverse incentives and then assume people will just ignore them?
K O'N

Even if it is true of fencing, and I'm not so sure, what evidence to you have that it is not true of chess. A headache affects chess even more than fencing, I would think.

[Jason]

That's exactly why I'm saying this should not be a USFA job. There's nothing evil about people following a meaningless point list, as long as it stays meaningless. (Look at the success of XBox Live achievements and many other video game scoring systems.) It's just a bunch of numbers to be crunched providing some data. Let people use that data however they want.

I agree that the USFA is better off avoiding more lists, rankings, etc. My point is that, regardless of who creates these kinds of things, they are not going to be beneficial and will lead to more and more lists, rankings, etc.

[fencerX]

I agree that the USFA is better off avoiding more lists, rankings, etc. My point is that, regardless of who creates these kinds of things, they are not going to be beneficial and will lead to more and more lists, rankings, etc.

Pretty good post. I'll rate it a "7.5"

[leftweet]

I think the main problem with the current USFA system is that it looks at only ONE result. I'd prefer if there would be an average of the best three results in the past two years. This way, a fencer who happens to get a lucky 'A', but never fenced stronger than, say an 'E', would get a 'C' rating. If you give 5 points for an 'A', 4 for a 'B', down to 1 for an 'E' and take the best three results, fencers would be encouraged to fence more (using the points system for the fencer just mentioned 5+1+1=7 / 3 = 2.3 = C1, the number is the remainder). Say a fencer has two 'A' results and one 'C' result, they will keep wanting to fence more so to make that 'C' result into an 'A' (the fencer with two A's and a C would be an A1). If you really wanted to, you could also eliminate the lettering system, but that might have too much tradition in the world of US fencing. You could also go to four or five results, which would encourage even more fencing, as well as more levels of ranking.

Personally, having studied the ELO system in a number of sports, I find that it is a terrific system for judging true-strength. However, in terms of being an official rating system, I think it has no place in the USFA because fencers would want to sit home and nurse their ratings prior to major competitions. It would be a great indicator a fencer's progress if a third-party source ran the system.

[Allen Evans]

I'm in favor of anything that raises more questions than it answers and has more of us filling in spreadsheets and debating the resulting numbers. As an added bonus, a complicated numerical rating system would cause even more threads started by indignant parents who's kid dropped of 5.45 points at the last tournament they attended because some other kid withdrew and dropped the strength factor* of the Y12 tournament by ten percent and their kids pool gave them a terriable draw in the DEs

Yep, all for it.

A


*Ah....no one has even MENTIONED strength factors yet. This could turn into a multi-variable response surface in a hurry. Even better! :-)

[leftweet]

What I proposed is no more complicated than the system currently in use -- it just looks at more results, instead of the single one that the USFA currently uses.

[KD5MDK]

The E-D-C-B-A-points system isn't really as coarse as it looks, it's 21 levels, U, E07, E08, ..., A09, A10, and then points within A10. Seriously, how much better will an Elo system do than this in seeding, do you think?

K O'N

But in a situation where the vast majority of active competitors have a current year rating by midyear, it isn't a very helpful set of segments.

[K O'N]

But in a situation where the vast majority of active competitors have a current year rating by midyear, it isn't a very helpful set of segments.

If the vast majority of fencers have a current rating by midyear, they're re-earning their ratings every year. So they're not flashes in the pan(s), right?

That aside, I don't mean the question to be rhetorical. How much better would an Elo system be, do you think? How bad is the current system? What's the evidence for this supposed badness?

I suggested once that a regression of seeding to finishing place was a pretty good measure. Another way to think of it is to consider that the goal of seeding, really, is to produce even pools. So after an event look at the average finishing place for each pool. They should be pretty even. If a seeding scheme is working it should usually produce even pools. If it's not it should regularly produce uneven pools. How is our current scheme doing in producing pools? How much better could a numerical scheme do?

K O'N

[keith]

But in a situation where the vast majority of active competitors have a current year rating by midyear, it isn't a very helpful set of segments.

Or is that highlighting that more segments might not carry any additional information*?

Whether or not a more granular system would be useful for seeding can be tested by looking at Div1 national championship results vs the entrants initial seeding. Plenty of data that is not compromised (as the FIE data is) by the parachuting in of higher ranked fencers after the poules.



*It would of course allow the USFA to control entrant pools.