Register    Login    Forum    Search    FAQ Awesomenauts



Post new topic Reply to topic Go to page Previous  1, 2, 3, 4, 5, 6, 7, 8 ... 11  Next

Author Message
 Post subject: Re: Matchmaking Is Not Terrible
 Post Posted: Mon Jul 17, 2017 8:29 pm 
Offline
User avatar

Joined: Tue Jul 28, 2015 10:24 am
Posts: 1131
Location: In the fantastical realm of Fan-Made Creations—please do not tap on the glass: it startles us.
I mean, we do not need a full and documented analysis of all your matches and then certification of validity on top of seven independent checks of the data under penalty of death.

It just makes comparing cases easier.

_________________
Say no to SAM wrote:
Give this guy a duck for making such a good job with this thread

Nekomian wrote:
Give this guy a duck lololol

NekiCoule wrote:
...I'm not trying to seduce you or any kind of propaganda for your great wisdom I swear I'm not...


Top 
 Profile  
 
 Post subject: Re: Matchmaking Is Not Terrible
 Post Posted: Mon Jul 17, 2017 8:33 pm 
Offline
User avatar

Joined: Mon Nov 25, 2013 6:41 pm
Posts: 770
rankwise, DeezNaut's screenshot looks like a very even match, actually. everyone is L2 or L3, and the L3's are evenly distributed on each team.

but maybe his complaint belongs in tactics - his build order & stats looks like he built to clear droids and little else. if your teammates get in trouble, you can't do much to help them with that build - maybe you can send out a couple grenades...that'll show em! the enemy team comp is a bit better too IMO. and they have a smiles.

but yeah - matchmaking!


Top 
 Profile  
 
 Post subject: Re: Matchmaking Is Not Terrible
 Post Posted: Tue Jul 18, 2017 12:35 am 
Offline
User avatar

Joined: Fri Jul 08, 2016 3:51 pm
Posts: 206
robochase6000 wrote:
rankwise, DeezNaut's screenshot looks like a very even match, actually. everyone is L2 or L3, and the L3's are evenly distributed on each team.

but maybe his complaint belongs in tactics - his build order & stats looks like he built to clear droids and little else. if your teammates get in trouble, you can't do much to help them with that build - maybe you can send out a couple grenades...that'll show em! the enemy team comp is a bit better too IMO. and they have a smiles.

but yeah - matchmaking!

There is literally nothing I could have done in that match. They both kept suiciding for no reason. It felt like both of them never played their chars before. This happens way too often and could be prevented by having matchmaking that knows how good or bad you are with your char statistically. I think the design decision of picking the char after the matchmaking round was the wrong one because ultimately the benefits of better matchmaking far outweigh the ability to pick with your team. Most people don't even use that ability and just pick what they like to play, as they should (and you still can't draft against the enemy team). The matchmaker should do the rest to assure a good game, where everybody is on the same level. Because no matter how smart you think you pick with your team, if you even use that ability, you won't be able to outsmart someone who is playing their main while you are playing your 10th best char that match.

_________________
The game has been in a downward spiral since the XP-System and team levels. Even if the game was balanced, SoloQ is a torturous experience to submit yourself to anymore. Idk how blind the devs must be to not understand how these two things are related.


Top 
 Profile  
 
 Post subject: Re: Matchmaking Is Not Terrible
 Post Posted: Tue Jul 18, 2017 3:04 am 
Offline
User avatar

Joined: Wed Sep 12, 2012 4:39 am
Posts: 82
Location: Decatur Alabama, USA
I've seen a lot of emotion and hyperbole on the topic of matchmaking, so I thought I'd throw a little science at it and see if the data support the hypothesis that matchmaking is fine, or that matchmaking is terrible.

Keep in mind, in a perfectly balanced world, any given player should be winning about 1/2 their games; that is to say, if all matches are perfectly balanced, you should have an even 50/50 shot at winning any given game.

Obviously, that breaks down at the top and bottom ends of play because at the top, there are no longer enough players better than you to create balanced matches, and at the bottom, there are not longer enough players worse than you. We see these effects at the top, but at the bottom there aren't really enough active players for the effects to be obvious.

I did two analyses of leaderboard data; in the first, I looked at the win rate of all players with at least 30 games played this season. Students of statistics will know why I chose the number 30, and students of this game and its leaderboards will cry foul, knowing 30 games is no where near enough for a player's rating changes, and therefore position on the leaderboard to "settle."

Here is the result of the first analysis:

https://photos.app.goo.gl/NVmL6ij8s30buZHq2

As you can see from the distribution, the data skew slightly right, but are generally very normally distributed. In a very general sense, that means we're seeing exactly the kind of win/loss ratios we would expect in a well balanced system. There's a large number of people sitting right where they should be at 50/50, and tapering numbers moving to the left and right.

Now, it gets interesting when you consider players with more games played this season and, therefore, more "settled" leadboard positions. The results from that analysis are here:

https://photos.app.goo.gl/f4iRkyweF9DQFxmk2

Here, the rightwise skew is much more pronounced, meaning players with greater than 100 games played are winning more games than we would predict in a well balanced ecosystem. Here, we can probably see the effect that's driving all the complaints; for the most skilled players, the games are too easy because there aren't enough quality opponents to challenge them.

If Ronimo were to find a way to create perfectly balanced matches at all skill levels (an impossible ask, in my opinion) we would see win rates drop for all but the very top players, and this distribution would level out.

If you're interested in the full analysis including all the statistical measures I took on this dataset, let me know. I've left them off here because the histograms alone really tell the tale quite well. Cheers, love, and please be excellent to one another,

Shotgun Fetish


Top 
 Profile  
 
 Post subject: Re: Matchmaking Is Not Terrible
 Post Posted: Tue Jul 18, 2017 4:41 am 
Offline
User avatar

Joined: Wed Jun 28, 2017 7:53 pm
Posts: 36
Shotgun Fetish wrote:
I've seen a lot of emotion and hyperbole on the topic of matchmaking, so I thought I'd throw a little science at it and see if the data support the hypothesis that matchmaking is fine, or that matchmaking is terrible.

Keep in mind, in a perfectly balanced world, any given player should be winning about 1/2 their games; that is to say, if all matches are perfectly balanced, you should have an even 50/50 shot at winning any given game.

Obviously, that breaks down at the top and bottom ends of play because at the top, there are no longer enough players better than you to create balanced matches, and at the bottom, there are not longer enough players worse than you. We see these effects at the top, but at the bottom there aren't really enough active players for the effects to be obvious.

I did two analyses of leaderboard data; in the first, I looked at the win rate of all players with at least 30 games played this season. Students of statistics will know why I chose the number 30, and students of this game and its leaderboards will cry foul, knowing 30 games is no where near enough for a player's rating changes, and therefore position on the leaderboard to "settle."

Here is the result of the first analysis:

https://photos.app.goo.gl/NVmL6ij8s30buZHq2

As you can see from the distribution, the data skew slightly right, but are generally very normally distributed. In a very general sense, that means we're seeing exactly the kind of win/loss ratios we would expect in a well balanced system. There's a large number of people sitting right where they should be at 50/50, and tapering numbers moving to the left and right.

Now, it gets interesting when you consider players with more games played this season and, therefore, more "settled" leadboard positions. The results from that analysis are here:

https://photos.app.goo.gl/f4iRkyweF9DQFxmk2

Here, the rightwise skew is much more pronounced, meaning players with greater than 100 games played are winning more games than we would predict in a well balanced ecosystem. Here, we can probably see the effect that's driving all the complaints; for the most skilled players, the games are too easy because there aren't enough quality opponents to challenge them.

If Ronimo were to find a way to create perfectly balanced matches at all skill levels (an impossible ask, in my opinion) we would see win rates drop for all but the very top players, and this distribution would level out.

If you're interested in the full analysis including all the statistical measures I took on this dataset, let me know. I've left them off here because the histograms alone really tell the tale quite well. Cheers, love, and please be excellent to one another,

Shotgun Fetish


This should be stickied.
Players cry about poor matchmaking but they don't realise that there is a lack of good players. To conclude, you are too good. If you want a more even and measured experience then play a naut that you are not pro with. And voila, you will have a 50% win rate and matchmaking will be more balanced.

Please keep on sharing more data.


Top 
 Profile  
 
 Post subject: Re: Matchmaking Is Not Terrible
 Post Posted: Tue Jul 18, 2017 8:15 am 
Offline
User avatar

Joined: Wed Mar 06, 2013 12:40 pm
Posts: 1666
Hero wrote:
This should be stickied.
Players cry about poor matchmaking but they don't realise that there is a lack of good players. To conclude, you are too good. If you want a more even and measured experience then play a naut that you are not pro with. And voila, you will have a 50% win rate and matchmaking will be more balanced.

Please keep on sharing more data.


There's a huge difference between matchmaking being balanced and matchmaking being good. If you put a L9, a L5 and a L1 together against a L9, a L5 and a L1 (with identical winrates), then matchmaking will be balanced, the expected winrate for both teams will be 50%. This doesn't mean at all that matchmaking is good - in this example, it would be horrible.

These statistics tell you very little about the quality of the matchmade games, it ignores factors such as skill distribution and latency. I don't see a lot of players complaining about their winrates at all, what people are complaining about is the quality of their matches.

_________________
Whether it is sound or sight, true beauty resides in curiousity.

~ Embrace your dreams.


Top 
 Profile  
 
 Post subject: Re: Matchmaking Is Not Terrible
 Post Posted: Tue Jul 18, 2017 2:30 pm 
Offline
User avatar

Joined: Wed Sep 12, 2012 4:39 am
Posts: 82
Location: Decatur Alabama, USA
Zold wrote:
There's a huge difference between matchmaking being balanced and matchmaking being good. If you put a L9, a L5 and a L1 together against a L9, a L5 and a L1 (with identical winrates), then matchmaking will be balanced, the expected winrate for both teams will be 50%. This doesn't mean at all that matchmaking is good - in this example, it would be horrible.

These statistics tell you very little about the quality of the matchmade games, it ignores factors such as skill distribution and latency. I don't see a lot of players complaining about their winrates at all, what people are complaining about is the quality of their matches.


I don't think I understand this argument; by definition, balanced matchmaking is good matchmaking. The single goal of the matchmaking system is to get players in to matches with one another quickly where the outcome of any given match, for any given player, over time, is 50/50. The only place this breaks down is in the very top tier of players, a phenomenon that occurs for reasons stated above.

If you're not having fun in your matches, I don't know there's an opportunity to blame the matchmaker, here. Sometimes you're going to get roflstomped, and sometimes you're going to be doing the roflstomping, but I feel like the matchmaking is just a convenient scapegoat for a lot of the frustration competitive online gaming produces by its very nature.

Don't forget about confirmation bias; once you've (by which I mean the community at large, not you personally) made the decision the matchmaking is bad, you're much more likely seize on any data point that supports that idea than you are to consider that maybe your frustration is more complex than a single boogeyman.

The data indicate the matchmaker is doing its job, and as a community we may need to look deeper to figure out the real problem driving a lot of these complaints.


Top 
 Profile  
 
 Post subject: Re: Matchmaking Is Not Terrible
 Post Posted: Tue Jul 18, 2017 3:13 pm 
Offline
User avatar

Joined: Wed Mar 06, 2013 12:40 pm
Posts: 1666
I don't know how to phrase it any simpler than I already did. The goal of a matchmaker is to increase the overall fun and satisfaction gained from playing a game. Winrate is just one part of this recipe, other ingredients are a healthy latency, skill distributions that don't stray too far from eachother and so on...

In theory you could play 100 games where everybody has 400 ping, win 50 of them and you would call that 'good matchmaking'. That's outrageous.

_________________
Whether it is sound or sight, true beauty resides in curiousity.

~ Embrace your dreams.


Top 
 Profile  
 
 Post subject: Re: Matchmaking Is Not Terrible
 Post Posted: Tue Jul 18, 2017 3:25 pm 
Offline
User avatar

Joined: Wed Sep 12, 2012 4:39 am
Posts: 82
Location: Decatur Alabama, USA
Zold wrote:
The goal of a matchmaker is to increase the overall fun and satisfaction gained from playing a game.


Is it, though? How could anyone build a matchmaker that works this way? In order to accomplish an increase in overall fun and satisfaction from playing, the matchmaker would have to monitor and analyze the emotions of the players in each match and adjust the games it puts them in accordingly.

While progress is definitely being made in areas of human/computer interface and emotional recognition, we don't yet have a technology that Ronimo could leverage to detect your satisfaction with the game and dynamically change matchmaking to maintain your fun.

By their very nature, those are ephemeral, subjective, emotional concepts. Currently only another human (and dogs) are capable of reading these cues in another person, and even we struggle to get it right.

At the risk of becoming argumentative, I ask that you reflect on what the matchmaker is actually for and consider how the objective, robust data we have available reflect how well it's actually working. I am reminded of the words of H.L. Mencken: “Explanations exist; they have existed for all time; there is always a well-known solution to every human problem — neat, plausible, and wrong.”

That the matchmaker is broken and therefore the cause of your (again, I mean 'you' as in, the Aweseomnauts community in general, not 'you' personally) frustration in game feels to me, in the light of the available evidence, like the very epitome of the idea Mencken was trying to express.


Top 
 Profile  
 
 Post subject: Re: Matchmaking Is Not Terrible
 Post Posted: Tue Jul 18, 2017 3:38 pm 
Offline
User avatar

Joined: Wed Mar 06, 2013 12:40 pm
Posts: 1666
Well you know, I already clarified that fun and satisfaction can be split out into other elements which fall under the responsibility of a matchmaking system, such as winrates, latency and skill distribution - all of which are measurable.

Start with reading, then think, then post; basic rules.

_________________
Whether it is sound or sight, true beauty resides in curiousity.

~ Embrace your dreams.


Top 
 Profile  
 
Display posts from previous:  Sort by  
 
Post new topic Reply to topic Go to page Previous  1, 2, 3, 4, 5, 6, 7, 8 ... 11  Next