Forum Search:
RSD No Spam
rec.sport.disc without the spam


Home » RSD » RSD Posts » Nationals Bids Chaos (Serious flaws in the USAU bid distribution system and a possible solution)
Re: Nationals Bids Chaos [message #112058 is a reply to message #112054] Wed, 14 March 2012 12:52 Go to previous messageGo to next message
Jackson
Messages: 97
Registered: October 2008
Location: Stanford Bubble
Member
mgd.mitch wrote on Wed, 14 March 2012 15:37
sswang wrote
And, again, I'd argue that whoever comes on top of a brawl between the #21,#22,#27,#28, and #29 ranked teams has proven that they've probably been underranked, and this new information should cause us to re-evaluate their chances of being better than the team that was initially ranked at #19.
???? I can't agree with this at all. Not even close. Proven they are underranked? How? If #21 wins, how have they proven they are better than higher ranked teams not involved?
How is beating teams you are supposed to beat somehow make you better than teams better than all the teams involved? They could be underranked, but they also might be overranked. If the #29 team wins that brawl, they've shown they are the best of that bunch, not necessarily better than any team above #21. Again, I agree your approach has merit in the greater variety of outcomes for the brawl sends a "good" team to natties, but that sucks for a ~#17 team that could finish their regionals on seed and stay home because their neighbors, who also finished on seed at regionals didn't compare well with other teams *not going to natties*.


I very much agree with Stephen here. To counter your example of a #17 team missing out, consider the GL region last year. Michigan was ranked 8th at the end of the regular season. 8th!, surely they deserve a spot at nationals. But through no fault of their own, their region's teams didn't do well enough. Mich St was #20, Illinois was #25 and Ball St was #37. Then, Michigan lost to Illinois in the semis of regionals, and they're out.

In the top30 system, the GL region would have received a strength bid.
Re: Nationals Bids Chaos [message #112059 is a reply to message #112054] Wed, 14 March 2012 12:57 Go to previous messageGo to next message
pacemaker
Messages: 85
Registered: April 2011
Member
Mitch,

If you reran the formula *after* the regional, that team winning the 21,22,27,28,29 scrum would likely now be higher ranked than #19 at the other regional.

Does that mean they are better than #19? Nope. But I'd be willing to bet the system would have them higher up.

Somewhere a line needs to be drawn until we have the time and money for larger play in tournaments. As it is now, it is drawn by TEAM ranking. And maybe that should be drawn at $16-#17 and then regional strength bids. Maybe not.

sswang's suggestion is along the same lines as the suggestion (but certainly not implementation) by the OP -- allow regional strength to have some say in the bid process.
Re: Nationals Bids Chaos [message #112060 is a reply to message #112052] Wed, 14 March 2012 13:10 Go to previous messageGo to next message
Big_Jim
Messages: 111
Registered: August 2010
Location: Madison WI
Senior Member
Ryan Thompson wrote on Wed, 14 March 2012 11:47
In the end, I think the closest we can get to a "perfect" system is a
selection committee, or 4 superregions where we make teams earn it on
the field.

4 superregions would create more problems than it solves. Either you're adding another tier of competition, which means 3 series tournaments before Nationals, which means an extra weekend during finals week for college students and extra cash spent driving/flying to wherever these tournaments are held. If you simply replace the current 10 regions with 4 superregions, you're just pushing down the problem we're currently having to the conference level: how many teams from each conference earn a bid to superregions? How many teams in each superregion tournament? How many bids to Nationals does each superregion get?

The Selection Committee works really well for NCAA Basketball because every college basketball game is on film. The selection committee can watch every team they're considering play in many games. Footage like that doesn't exist in college Ultimate. Even the most covered tournaments, like the NexGen tournaments, only film a handful of games instead of every game in every round. And who has time to watch many games from 30-40 teams? For Open and Women? Dozens of people are paid full-time money to do this for the NCAA. The USAU doesn't have that kind of cash.
Re: Nationals Bids Chaos [message #112061 is a reply to message #112058] Wed, 14 March 2012 13:17 Go to previous messageGo to next message
pacemaker
Messages: 85
Registered: April 2011
Member
Jackson wrote on Wed, 14 March 2012 14:52

consider the GL region last year. Michigan was ranked 8th at the end of the regular season. 8th!, surely they deserve a spot at nationals. ...

In the top30 system, the GL region would have received a strength bid.


Whoa, Nelly!!

Maybe the GL could have earned a 2nd bid via regional strength, but maybe not even with ssawng's adjustment. Michigan would still have to *earn* its way to nationals.

They lost pretty bad to Illinois. Not the rankings fault and certainly not Illinois' fault.

Are you suggesting that had Michigan State and Illinois made nationals last year that Michigan would still have deserved to be there? Or are you assuming that Michigan would have beaten Michigan State for the 2nd bid?
Re: Nationals Bids Chaos [message #112062 is a reply to message #112058] Wed, 14 March 2012 13:20 Go to previous messageGo to next message
mgd.mitch
Messages: 1207
Registered: January 2009
Senior Member
Jackson wrote on Wed, 14 March 2012 15:52
I very much agree with Stephen here. To counter your example of a #17 team missing out, consider the GL region last year. Michigan was ranked 8th at the end of the regular season. 8th!, surely they deserve a spot at nationals.
And they did get their region a bid to nationals by merely being a region (autobid). Their designated bid to nationals went to a different team through 100% fault of their own...by losing their own games at regionals.

And since you are using actual post season results to "counter" (I disagree you have countered anything) the argument, you took that bid away from a team that made prequarters, thus, your system isn't better.
Re: Nationals Bids Chaos [message #112063 is a reply to message #112058] Wed, 14 March 2012 13:23 Go to previous messageGo to next message
Big_Jim
Messages: 111
Registered: August 2010
Location: Madison WI
Senior Member
Jackson wrote on Wed, 14 March 2012 12:52
Michigan was ranked 8th at the end of the regular season. 8th!, surely they deserve a spot at nationals. But through no fault of their own
...
Michigan lost to Illinois in the semis of regionals, and they're out.

It sure looks like it was their fault from here. If you can't perform in the playoffs (Michigan last year, Florida '09) then you clearly don't "deserve" to go to Nationals. It's really that simple.

This whole thread started based on the possibility of Michigan throwing a game to Michigan State to boost State's ranking and secure a second bid. To me, that still seems like a problem with the ranking algorithm and the moral character of teams potentially involved in such a scenario than the selection process itself.
Re: Nationals Bids Chaos [message #112067 is a reply to message #112063] Wed, 14 March 2012 14:22 Go to previous messageGo to next message
Ryan Thompson
Messages: 364
Registered: September 2008
Senior Member
Mitch, you said that if the ranking algorithm really suggests a lot of
1-pt differences in game scores, we'd see a lot of 17-16 games.

That's completely ignoring the spread of possible scores of games. It
means that in 10,000 games, one team has averaged one more point than
the other, and some games might be 15-2 and some might be 13-15. It's
the exact same lack of data problem that we're seeing. And this is
totally true empirically - Carleton finished 2nd at Nationals and lost
to Stanford 14-3 at Centex (after beating Stanford 13-1 at Vegas). The
finals rankings probably estimate the pt difference as something like
15-9, but anything from 15-0 to 0-15 is possible.

Krishna's point is a good one - we should absolutely switch to a
ranking system with probabilistic underpinnings so that we can more
accurately gauge "true strength" of regions by estimating the number
of top-20 teams in each region.
Re: Nationals Bids Chaos [message #112069 is a reply to message #112057] Wed, 14 March 2012 14:27 Go to previous messageGo to next message
Krishna
Messages: 98
Registered: October 2008
Member
Krishna wrote on Wed, 14 March 2012 15:50

That is why, what I would do is develop a ranking system that has a full probabilistic underpinnings (it is unclear to me if the current system does, but most modern ranking systems do). Then we could calculate the probability of every possible outcome at regionals based on the rankings. From there once we have all the autobids assigned, we can determine which region is mostly likely to have the highest ranked team left out and give them an extra bid. Continue till all bids are allocated.


I thought a little more about what I have in mind and realized that the current system is a special case of what I am proposing.

The current system implements what I described above, but under the assumption that a higher ranked team will always beat a lower ranked team, by exactly the score predicted by the rankings.

What I am proposing is to relax that assumption. A proper ranking system should allow us to calculate what the probability of various outcomes are. I want to use that information to allocate the bids based on which regions are most likely to have the strongest team left out after regionals.

It does seem a bit computationally intensive, but is basically what Jeff Sargin is doing when he simulates the NCAA tournament and says that Kentucky has a 14% chance of winning.
Re: Nationals Bids Chaos [message #112074 is a reply to message #112054] Wed, 14 March 2012 16:42 Go to previous messageGo to next message
sswang
Messages: 29
Registered: June 2010
Junior Member

Mitch, you still haven't answer my question:
Quote:
We can turn your question on its head: ... Let's say a region has the #5, #21, and #30 teams. They don't earn a second bid in the current system because #21 is too low. But if the #21 team had a slightly higher rating (or the #19/20 teams just slightly lower ones), they would. Why should the #30 team be rewarded with a better chance of going to nationals just because another team is stronger?

Isn't this question analogous to yours?

Quote:
I'm saying if you could somehow determine a 2nd+ team in a region is clearly in the top 20, they should earn their region a bid.

The problem is that even if the algorithm is improved, there is almost no chance of determining that the 20th-ranked team is CLEARLY one of the 20 best. It's essentially 50-50 that the 21st-ranked team is better. If we were to go through the standings and assign each team a probability that it is truly among the top 20 teams, it might look like this (of course, things would vary depending on the actual USAU ratings and how they are distributed):
1-10. 99%
11. 95%
12. 90%
13. 80%
14. 75%
15. 71%
16. 69%
17. 65%
18. 55%
19. 52%
20. 50%
21. 49%
22. 47%
23. 45%
etc.
If the 49% and the 47% and the 45% all hailed from the same region, it would be more likely that one of those three truly belongs to the top 20 than the 50% team.


Quote:
If we are going to say we really can't tell the difference between the top ~40 teams with any certainty whatsoever, then we should just punt and say two bids per region.

I'm not saying we can't tell anything. I'm just saying we have to take care not to read too much into it - we should view it probabilistically rather than deterministically.

Quote:

Quote:
There is virtually no difference between the #19 team (rating 1636) and the #21 team (rating 1602) - 34 ratings points is a 15-14.8 game, or something like that. You have to go all the way up to the #11 team to find one that is even a 15-14.0 favorite over #21.
But the reality is it isn't that close. If #11-#21 are a 1 point game at most, sunday regionals and most nationals games would be 17-16. But they aren't.


You're forgetting to treat the margins where underdogs win as negative numbers.

Quote:
Quote:
And, again, I'd argue that whoever comes on top of a brawl between the #21,#22,#27,#28, and #29 ranked teams has proven that they've probably been underranked, and this new information should cause us to re-evaluate their chances of being better than the team that was initially ranked at #19.
???? I can't agree with this at all. Not even close. Proven they are underranked? How? If #21 wins, how have they proven they are better than higher ranked teams not involved?


A parable:

There are four blocks which look pretty similar. You quickly pick up A, B, and C in sequence, and all three feel more or less the same to you, though you don't compare them directly. You also know that A, B, and C together weigh exactly 4800 grams. What's your best estimate for the weight of block A? 1600g, right?

Let's say you've weighed block D, and it comes out to 1605g. If you had to bet whether block A or block D were heavier, you'd bet on D.

However, let's do an experiment in the meantime. You give blocks A and B to 100 people and you ask them to try to decide which is heavier by holding them in their hands; 95 vote for A. You do the same with A and C and 99 vote for A. Between B and C, 90 vote for B.

Shouldn't your new estimate of A's weight increase by a not so insignificant amount? And, if so, wouldn't you now bet on A being heavier than D, even though D wasn't involved in any of those comparisons?

Adjusting expectations of a team's quality conditionally based on their winning is fairly standard. See, for instance, Nate Silver on how he modeled March Madness:
http://fivethirtyeight.blogs.nytimes.com/2011/03/14/how-we-m ade-our-n-c-a-a-picks/
It's down in the "Quality of Play During the Tournament" section. Not a perfect analogy for what we're doing, but the principle is the same.



PS. I'm pretty sure that under my proposal, GL in 2011 does NOT win a 2nd bid to nationals.
Re: Nationals Bids Chaos [message #112077 is a reply to message #112074] Wed, 14 March 2012 18:39 Go to previous messageGo to next message
Krishna
Messages: 98
Registered: October 2008
Member
sswang wrote on Wed, 14 March 2012 19:42
If we were to go through the standings and assign each team a probability that it is truly among the top 20 teams, it might look like this

...

17. 65%
18. 55%
19. 52%
20. 50%
21. 49%
22. 47%
23. 45%

...



In fact the rankings don't even say that it is more likely than not, that a team ranked 20 is in the top 20 of teams. Only that the probability is higher than any team ranked below it and lower than any team ranked above it.

Given how closely ranked team are to each other we probably live in a world where the probabilities look like this:

17. 40%
18. 39%
19. 37%
20. 36%
21. 35%
22. 34%
23. 32%

That is, the rankings system itself says that the team ranked number 20, probably isn't in the top 20 of teams (I know, probability is hard...). But the USAU system for allocating bids, treats them as if they look like this:

17. 100%
18. 100%
19. 100%
20. 100%
21. 0%
22. 0%
23. 0%

The advantage of the USAU system of moving from rankings to bids is that it is simple. The disadvantage is that it makes almost no sense (that is, meets no objective criteria for a good system I can think of).
Re: Nationals Bids Chaos [message #112082 is a reply to message #112077] Thu, 15 March 2012 04:36 Go to previous messageGo to next message
pacemaker
Messages: 85
Registered: April 2011
Member
Krishna wrote on Wed, 14 March 2012 20:39

That is, the rankings system itself says that the team ranked number 20, probably isn't in the top 20 of teams (I know, probability is hard...). But the USAU system for allocating bids, treats them as if they look like this:

17. 100%
18. 100%
19. 100%
20. 100%
21. 0%
22. 0%
23. 0%

The advantage of the USAU system of moving from rankings to bids is that it is simple. The disadvantage is that it makes almost no sense (that is, meets no objective criteria for a good system I can think of).


Krishna, I know probability is hard so...

What you wrote is wrong.

The USAU system treats the REGION in which those teams play as having another top 20 team based on the relative ranking of its highest ranked remaining team not already earning a bid. That's is the entire discussion here, right? How to allocate the non-automatic bids to regions.

Should it go to the region with the highest ranked remaining team or on a broader base?

I think what you are suggesting is giving EVERY team a probability of being in the top (# of remaining bids) of the unused teams. Add these probabilities up and start giving regions a bid based on the sum of those probabilities. Repeat.

Not bad. But I am not sure our information (games, tournaments, rating software) is quite robust enough for that.

Re: Nationals Bids Chaos [message #112083 is a reply to message #112082] Thu, 15 March 2012 04:46 Go to previous messageGo to next message
pacemaker
Messages: 85
Registered: April 2011
Member
Small adjustment.

Each region getting a team strength bid is treated as if it contains another team deserving of a bid to nationals. Whatever that means.
Re: Nationals Bids Chaos [message #112084 is a reply to message #111789] Thu, 15 March 2012 04:59 Go to previous messageGo to next message
Reggie Fanelli
Messages: 1958
Registered: September 2011
Senior Member
how about....8 regions....and the top 2 teams advance from each
region.
or....10 regions....and the top 2 teams advance from each.

regionals.....brackets to the champion....wrap around double
elimination for second place.

you'd get a great mix of teams from across the country....the top
teams would advance, or couldn't claim to be a top team.

simple.
no math
no figuring.
Re: Nationals Bids Chaos [message #112086 is a reply to message #112084] Thu, 15 March 2012 05:55 Go to previous messageGo to next message
pacemaker
Messages: 85
Registered: April 2011
Member
No Jam championship in Open?
Re: Nationals Bids Chaos [message #112088 is a reply to message #112084] Thu, 15 March 2012 06:35 Go to previous messageGo to next message
Big_Jim
Messages: 111
Registered: August 2010
Location: Madison WI
Senior Member
Reggie Fanelli wrote on Thu, 15 March 2012 04:59
simple.
no math
no figuring.

And also missing several of the best teams in the country. Every year.
Re: Nationals Bids Chaos [message #112089 is a reply to message #112086] Thu, 15 March 2012 06:38 Go to previous messageGo to next message
Ryan Thompson
Messages: 364
Registered: September 2008
Senior Member
"I think what you are suggesting is giving EVERY team a
probability of being in the top (# of remaining bids) of the
unused teams. Add these probabilities up and start giving
regions a bid based on the sum of those probabilities.
Repeat.

Not bad. But I am not sure our information (games,
tournaments, rating software) is quite robust enough for
that. "

Yes, basically. Each team's ranking is basically the median or mean of
a distribution of potential rankings, and so from that we can easily
calculate a probability that each team is truly in the top 20 teams
(or top whatever after autobids are assigned). And from there we can
combine probabilities (adding isn't really the right term) to either
assign each bid one at a time, removing the team assigned from the
calculation each time, or to calculate an expected value for "number
of teams in the 10 strength bid spots" and assign bids to regions
based on that expected value.

This is not very difficult mathematically or computationally - the
only question is whether our distributions are shaped correctly and if
our standard deviations on these distributions can be small enough to
not be weighted too heavily by regions with a lot of bad teams. We
might limit it to the top x number of teams in each region (or not,
we'd have to run some numbers to see if this is really a problem).
Re: Nationals Bids Chaos [message #112090 is a reply to message #112074] Thu, 15 March 2012 06:58 Go to previous messageGo to next message
mgd.mitch
Messages: 1207
Registered: January 2009
Senior Member
sswang wrote on Wed, 14 March 2012 19:42

Mitch, you still haven't answer my question:
Quote:
We can turn your question on its head: ... Let's say a region has the #5, #21, and #30 teams. They don't earn a second bid in the current system because #21 is too low. But if the #21 team had a slightly higher rating (or the #19/20 teams just slightly lower ones), they would. Why should the #30 team be rewarded with a better chance of going to nationals just because another team is stronger?
The top 20 team earned the bid. The #30 gets a chance to take it because their stronger neighbor earned it, but it will be harder for them to earn it than in a region with 3 teams in the low 20's. Someone get's screwed whether you do region strength or team strength. I choose to screw the lower ranked team. Yes I understand the rankings aren't anywhere near absolute. But at some point you have to take the results as is. You have to hand out the bids. You can't try to continue to gather more data, play what-if, etc.Quote:
Isn't this question analogous to yours?
To which question?Quote:
Quote:
I'm saying if you could somehow determine a 2nd+ team in a region is clearly in the top 20, they should earn their region a bid.

The problem is that even if the algorithm is improved, there is almost no chance of determining that the 20th-ranked team is CLEARLY one of the 20 best.
Of course you can't. I've only said that half a dozen times in this conversation. That sentence was not in the context of this is what we should do, it was to the straw man you put out there of the top seed in a 1 bid region losing at regioanls but still going to nationals. No one is proposing that whatsoever and you know that.Quote:
It's essentially 50-50 that the 21st-ranked team is better.
Yep. And again, at some point, you have to declare a winner. I don't like the idea of saying "well, yeah, you won out in the rankings, but they don't mean much and someone over there that you beat out might be better, so you have to break seed at regionals to go while someone lower than you doesn't. we base all this stuff on rankings, but we don't believe the rankings, so good luck." This is my opinion. As much as both of us would love a mathematical way to say one way is better, both ways have real disadvantages.
Quote:
If we were to go through the standings and assign each team a probability that it is truly among the top 20 teams, it might look like this (of course, things would vary depending on the actual USAU ratings and how they are distributed):
[If the 49% and the 47% and the 45% all hailed from the same region, it would be more likely that one of those three truly belongs to the top 20 than the 50% team.
I understand the math here and agree with it. I also don't like the idea of 2/3rds of 1 team's chance at earning a bid is based on other teams rather than their own performance. This will *always* be very problematic for me, to the point of favoring team strength bids over regional strength bid.

(note: you also seem not to be considering the probability of regional outcome. brawl of #21, 22, 23... the odds of the "true" best team winning are much lower than #17, #45, and #50).
Quote:


A parable:

There are four blocks which look pretty similar. You quickly pick up A, B, and C in sequence, and all three feel more or less the same to you, though you don't compare them directly. You also know that A, B, and C together weigh exactly 4800 grams.
the story can stop now since in our situation, we have a little information of their relative weight and no information about their total weight, and we are trying to determine relative weight.Quote:
PS. I'm pretty sure that under my proposal, GL in 2011 does NOT win a 2nd bid to nationals.
Never said it would.
Re: Nationals Bids Chaos [message #112093 is a reply to message #112089] Thu, 15 March 2012 07:54 Go to previous messageGo to next message
pacemaker
Messages: 85
Registered: April 2011
Member
In thought experiment mode:

Take the remaining top 32 teams and use their RRI or USAU ranking to run a simulation of a 32-team single elimination tournament 1,000,000 times. Yes 1v32 2v31 etc... With that many runs, the highest ranked team should come out on top the largest number of times. Use wins/1,000,000 as your "exact" percentage for each team. Add em up per region. Highest region gets bid.

Repeat for the next spot, removing the highest rank team from the region which just earned a bid.

Problem: reliable model to use rankings to simulate individual games.

------------------

At some point in any system a value judgement cut off will occur. I suspect you will always have overlap of probability distribution plots per region.

No system ever looks great under a microscope. Rough edges are always found and there will be disagreement on which pieces of those rough edges should count more than others.

---------------------

As it is now we have a slightly imperfect system for which we are arguing about the last few bids doled out to regions -- bids which will likely go to teams with little chance of winning the championship.

Again, what is the goal of handing out bids to college championship tournament?

Re: Nationals Bids Chaos [message #112101 is a reply to message #112093] Thu, 15 March 2012 10:37 Go to previous messageGo to next message
Jackson
Messages: 97
Registered: October 2008
Location: Stanford Bubble
Member
pacemaker wrote on Thu, 15 March 2012 10:54
Repeat for the next spot, removing the highest rank team from the region which just earned a bid.


I like the direction of expected values and probabilities, but we need to be careful about removing the highest rank team, no matter what the rest of the system looks like. Anytime you do this, you create an incentive for the region's 1st team to throw a game to the 2nd, or 2nd to 3rd.

Once you remove the top team, only teams 2,3,4... are remaining, so, after throwing a game, the gain in ranking (or win prob or PR...) for the lower teams is no longer balanced out by the loss in ranking of the upper teams.

Practically speaking, if a region has 2 or more bids locked up, I don't think a top team will worry too much about qualifying. However, for those regions with a top team and other teams in the 20-30 range and only 1 bid, that top team probably does want insurance. I don't think it is too far fetched to suggest that a top team would consider throwing a game in this situation.

Had Michigan and Mich St scheduled a sanctioned game between each other last year, just before the series, and Mich St won, the GL would have likely earned a 2nd bid and Michigan might have gone to nationals.
Re: Nationals Bids Chaos [message #112103 is a reply to message #112101] Thu, 15 March 2012 10:48 Go to previous messageGo to next message
pacemaker
Messages: 85
Registered: April 2011
Member
Jackson,

With my thought-experiment simulation model, I am pretty sure the throwing of games would not do as much for a region as how it can affect things now. In my model the top remaining 32 teams get thrown into 1,000,000 tournaments and the winner is recorded by region. The region with most wins gets the bid.

Not saying this is reasonable or doable, but the throwing of a single game becomes much less helpful.
Re: Nationals Bids Chaos [message #112104 is a reply to message #112103] Thu, 15 March 2012 11:10 Go to previous messageGo to next message
mgd.mitch
Messages: 1207
Registered: January 2009
Senior Member
I guess I'm a bit confused about the realistic difference in looking at things probabilistically, running 1,000,000 simulations, and taking the results. wouldn't the ranking algorithm be the ordered probabilities of each team being at a given rank? thus, running 1,000,000 simulations, wouldn't the expected outcome merely be the probabilities, thus, what would change versus just assigning them based on the probabilities, which would be the rankings?

or are you taking the outcomes, and rerunning the ranking algorithm based on those outcomes? (ie, you are assigning bids by simulated games versus actual games).
Re: Nationals Bids Chaos [message #112105 is a reply to message #112101] Thu, 15 March 2012 11:17 Go to previous messageGo to next message
pacemaker
Messages: 85
Registered: April 2011
Member
Jackson wrote on Thu, 15 March 2012 12:37
Had Michigan and Mich St scheduled a sanctioned game between each other last year, just before the series, and Mich St won, the GL would have likely earned a 2nd bid and Michigan might have gone to nationals.


...and everyone would have called BS on that game if it was not part of a more legitimate tournament.

At least the Harvard-Tufts game was part of a tournament. It was pretty late in the regular season, though, making it carry full weight.
Re: Nationals Bids Chaos [message #112108 is a reply to message #112088] Thu, 15 March 2012 11:18 Go to previous messageGo to next message
Reggie Fanelli
Messages: 1958
Registered: September 2011
Senior Member
> And also missing several of the best teams in the country.
> Every year.
~~~~~~~~~~~

---then ditch ALL the regions....and have the 'upsa bored' hand pick
the teams for nationals.

otherwise...have REGIONS with the goal of getting THE BEST
TEAMS.....FROM each REGION of the nation.

if "regions" don't work at getting you/anyone the teams that they WANT
at nationals....get rid of them!
if 'regionals' is just some sham to collect dues from every team
across the country......but 'regionals' AREN'T what you're using to
determine the teams to your 'championship'....ditch the regions!

if you want ONLY the 'best teams'.....don't fiddle around with the
teams that AREN'T.
Re: Nationals Bids Chaos [message #112109 is a reply to message #112103] Thu, 15 March 2012 11:11 Go to previous messageGo to next message
Ryan Thompson
Messages: 364
Registered: September 2008
Senior Member
Or we can assign the bids based on the expectation of teams in the top
20 (slightly adjusted for regions w no teams in top 20) for each
region based on Krishna's proposal, which means you never have to
discard any teams and therefore eliminates (or severely reduces) the
incentive to throw any games, as every team will want its ranking as
high as possible and not care about anyone else's.
Re: Nationals Bids Chaos [message #112110 is a reply to message #112104] Thu, 15 March 2012 11:25 Go to previous messageGo to next message
Ryan Thompson
Messages: 364
Registered: September 2008
Senior Member
pacemaker's suggestion is equivalent to:

Take teams "1"-"32". Let R("1") be the true ranking of "1".
P(R("1")=1) is the probability that "1"s true ranking is 1st.

We find y_x=P(R("x")=1) for all teams "x".

Sum together the y_x for every team in all 10 regions. Whichever
region has the greatest sum of y_x values gets 1 bid.

Now remove the top team from that region and repeat until all bids are
assigned.

My suggestion is to calculate y_x = P(R("x") <= 20) instead, and
summing by region should give an expected number of teams ranked in
the top 20.
Re: Nationals Bids Chaos [message #112113 is a reply to message #112109] Thu, 15 March 2012 11:38 Go to previous messageGo to next message
Reggie Fanelli
Messages: 1958
Registered: September 2011
Senior Member
eliminates (or severely reduces) the
> incentive to throw any games,
~~~~~~

---teams that did that.....should be OUT.
point shaving?
throwing games?
boom...out...and done with with for a couple years.
Re: Nationals Bids Chaos [message #112115 is a reply to message #112090] Thu, 15 March 2012 12:08 Go to previous messageGo to next message
sswang
Messages: 29
Registered: June 2010
Junior Member
mgd.mitch wrote on Thu, 15 March 2012 06:58
The top 20 team earned the bid. The #30 gets a chance to take it because their stronger neighbor earned it, but it will be harder for them to earn it than in a region with 3 teams in the low 20's.

I disagree here. It's easier to upset one team than it is to upset multiple teams that are pretty much the same strength as the one team. But this is a minor point.

Quote:
That sentence was not in the context of this is what we should do, it was to the straw man you put out there of the top seed in a 1 bid region losing at regioanls but still going to nationals. No one is proposing that whatsoever and you know that.
...
Yep. And again, at some point, you have to declare a winner. I don't like the idea of saying "well, yeah, you won out in the rankings, but they don't mean much and someone over there that you beat out might be better, so you have to break seed at regionals to go while someone lower than you doesn't. we base all this stuff on rankings, but we don't believe the rankings, so good luck." This is my opinion. As much as both of us would love a mathematical way to say one way is better, both ways have real disadvantages.


Sorry for the glib way in which I stated that. I was trying to flesh out what you meant when you said the perfect system was for the top 20 teams to earn a bid. By "bid" you don't mean berth at nationals, but "the right to not have to break seed at regionals in order to go to nationals". Is that an accurate (if stilted) way of phrasing it? I guess where I disagree is in not viewing such a right as particularly fundamental to the process (heck, the team that earns a team strength bid might well be seeded lower than a team that didn't do so at regionals), and in seeing bids as more for regions as a whole to fight over rather than as earmarked for particular teams which they then try to defend. But I suppose this is a place where we just need to agree to disagree.

Quote:
I understand the math here and agree with it. I also don't like the idea of 2/3rds of 1 team's chance at earning a bid is based on other teams rather than their own performance.

In my proposal every team still has a chance to earn a team strength bid for their region on their own merits. They just have to pass a somewhat higher threshold. It's only if they drop below that when they're no longer completely in control of their own destiny (although they still have some say).

Quote:

(note: you also seem not to be considering the probability of regional outcome. brawl of #21, 22, 23... the odds of the "true" best team winning are much lower than #17, #45, and #50).

Sure, but the drop in team quality should such an event occur would be much less than in the other region. It's a wash.

Quote:

the story can stop now since in our situation, we have a little information of their relative weight and no information about their total weight, and we are trying to determine relative weight.

OK, replace "together weight exactly 4800" with "according to a very inaccurate measurement, they each weigh about 1600". In any case, the point that you can sometimes anticipate that estimates might change conditionally based upon future results still stands.

Quote:
Yes I understand the rankings aren't anywhere near absolute. But at some point you have to take the results as is. You have to hand out the bids. You can't try to continue to gather more data, play what-if, etc.


The data includes things like the identity of teams that are essentially tied for the #20 spot. I'm just proposing that we not discard that information entirely, and not treat the ratings as absolute if we don't have to. But I guess this is another philosophical difference that won't get bridged.

Quote:

Quote:
PS. I'm pretty sure that under my proposal, GL in 2011 does NOT win a 2nd bid to nationals.
Never said it would.

Sorry, I got lazy and tried responding to others in the same post.


Re: Nationals Bids Chaos [message #112117 is a reply to message #112110] Thu, 15 March 2012 12:25 Go to previous messageGo to next message
pacemaker
Messages: 85
Registered: April 2011
Member
My suggestion is maybe doing that for the last few bids.

I still strongly support one autobid for each region and individual team strength bids for the first several at-large regional bids. Yes,..that means some region may get screwed out of the team strength bid when we draw a line somewhere if their regional strength is lacking.

And yes, a top 5 team could still throw a game under your SumProb(top20) method and not hurt their chances much. Playing and barely beating the 2 and 3 regional teams may lower their Prob(top20) very little, yet raise the in-region teams enough for it to matter. So yes, a fix could still be in. That becomes less likely when you take it one at a time.

Re: Nationals Bids Chaos [message #112118 is a reply to message #112115] Thu, 15 March 2012 12:54 Go to previous messageGo to next message
mgd.mitch
Messages: 1207
Registered: January 2009
Senior Member
sswang wrote on Thu, 15 March 2012 15:08
mgd.mitch wrote on Thu, 15 March 2012 06:58
The top 20 team earned the bid. The #30 gets a chance to take it because their stronger neighbor earned it, but it will be harder for them to earn it than in a region with 3 teams in the low 20's.
I disagree here. It's easier to upset one team than it is to upset multiple teams that are pretty much the same strength as the one team. But this is a minor point.
Fewer individual events have to happen, but the individual events are more lower probability, thus it could be more unlikely.Quote:
Sorry for the glib way in which I stated that. I was trying to flesh out what you meant when you said the perfect system was for the top 20 teams to earn a bid. By "bid" you don't mean berth at nationals, but "the right to not have to break seed at regionals in order to go to nationals". Is that an accurate (if stilted) way of phrasing it?
If we are talking perfect systems, I would hand out the 10 autobids, then hand out the next bids based on the next strongest teams. (ie, how it's currently done, knowing it's an imperfect system).Quote:
I guess where I disagree is in not viewing such a right as particularly fundamental to the process (heck, the team that earns a team strength bid might well be seeded lower than a team that didn't do so at regionals), and in seeing bids as more for regions as a whole to fight over rather than as earmarked for particular teams which they then try to defend. But I suppose this is a place where we just need to agree to disagree.
Exactly. I think we are both taking reasonable positions (at least I think yours is reasonable, I just prefer the other way to keep it more in your own control as a team).Quote:
In my proposal every team still has a chance to earn a team strength bid for their region on their own merits. They just have to pass a somewhat higher threshold. It's only if they drop below that when they're no longer completely in control of their own destiny (although they still have some say).
I definitely agree that's what your approach is doing. Based on some of the numbers you've thrown around (gap that means a 15-14 win), I'm not sure you could justify the higher standard as being good enough to base it on a team alone and should only look at regional strengthQuote:
Quote:

(note: you also seem not to be considering the probability of regional outcome. brawl of #21, 22, 23... the odds of the "true" best team winning are much lower than #17, #45, and #50).
Sure, but the drop in team quality should such an event occur would be much less than in the other region. It's a wash.
And I think if it's a wash, take the approach that puts the higher ranked team in should things go to seed.
Quote:
Quote:
the story can stop now since in our situation, we have a little information of their relative weight and no information about their total weight, and we are trying to determine relative weight.
OK, replace "together weight exactly 4800" with "according to a very inaccurate measurement, they each weigh about 1600". In any case, the point that you can sometimes anticipate that estimates might change conditionally based upon future results still stands.
Future results... I don't think we should be handing out bids based on future results. As I've said before, at some point, you have to take the results as is, not keep playing "what if". Team A... you outperformed team B, C, and D for the system we made, but one might be better under that same system if we kept playing, even though we agreed to a cutoff date, so you have a tougher road than if we just took things as they are now. That just doesn't sound right to me. Couldn't team A equally claim that if they played another set of regionals that they'd be back on top? the 2nd set of regionals would be just as theoretical as the first since we hand out bids before conference even starts. You have to stop and take the results at some point. Even if you play probabilities and such, you are taking those probabilities as absolute when they aren't. The relative rankings are estimates. The probabilities are estimates. At some point, they get changed to absolutes.
Re: Nationals Bids Chaos [message #112119 is a reply to message #112117] Thu, 15 March 2012 12:52 Go to previous messageGo to next message
Ryan Thompson
Messages: 364
Registered: September 2008
Senior Member
"And yes, a top 5 team could still throw a game under your
SumProb(top20) method and not hurt their chances much.
Playing and barely beating the 2 and 3 regional teams may
lower their Prob(top20) very little, yet raise the in-region
teams enough for it to matter. So yes, a fix could still be
in. That becomes less likely when you take it one at a
time."

I believe the exact opposite of this is true.
Re: Nationals Bids Chaos [message #112131 is a reply to message #112117] Thu, 15 March 2012 16:53 Go to previous messageGo to next message
Reggie Fanelli
Messages: 1958
Registered: September 2011
Senior Member
> And yes, a top 5 team could still throw a game under your
> SumProb(top20) method and not hurt their chances much.
~~~~~~~


---out for 2 years.
Re: Nationals Bids Chaos [message #112132 is a reply to message #112131] Thu, 15 March 2012 17:11 Go to previous messageGo to next message
Jackson
Messages: 97
Registered: October 2008
Location: Stanford Bubble
Member
Reggie Fanelli wrote on Thu, 15 March 2012 19:53
> And yes, a top 5 team could still throw a game under your
> SumProb(top20) method and not hurt their chances much.
~~~~~~~


---out for 2 years.


How do you prove that the better team threw the game rather than the underdog simply winning?
Re: Nationals Bids Chaos [message #112155 is a reply to message #111789] Fri, 16 March 2012 09:04 Go to previous messageGo to next message
Lance Marput
Messages: 989
Registered: June 2010
Location: Columbia Missouri
Senior Member
So "chaos" & "serious flaws" in the title/subject line is a bit of an exaggeration?

Peter Mc
MDSC
Re: Nationals Bids Chaos [message #112178 is a reply to message #112132] Fri, 16 March 2012 14:26 Go to previous message
Reggie Fanelli
Messages: 1958
Registered: September 2011
Senior Member
~~~~~~
>
> > ---out for 2 years.
>
> How do you prove that the better team threw the game rather
> than the underdog simply winning?
~~~~~~~~

---well all these folks are talkin' about it......
seems like there's proof somewhere.
Previous Topic:Huge Sky D by Wildcat #7
Next Topic:Yale Cup 2012: March 24-25
Goto Forum:
  


Current Time: Sun Sep 21 05:04:01 PDT 2014
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.0RC2.
Copyright ©2001-2009 FUDforum Bulletin Board Software