DEq - A Card Quality Metric#
Introduction#
DEq is a card quality metric for making decisions early in booster drafts. Unlike other available metrics, which are either brute win rate statistics or models of player behavior, DEq directly models the average win equity added by selecting the given card, and the formula is derived from that premise mathematically using simplifying assumptions that allow us to calculate the metric from the daily data available on 17Lands. In this series of articles, I will first give a qualitative description of the method and the reasoning behind it, then give some card quality philosophy and some reasons why you might want to consider using DEq to judge card quality. Next, I’ll go through my process for drafting guided by the metric and other data available on 17Lands, and finally I will give a full technical derivation of the metric for those interested in the math.
Why DEq#
Since developing DEq I have a 65% win rate in Premier Draft over about 1200 games and a number of top 250ish season finishes using the metric playing non-grinder amounts of Magic. I’m not good enough at Magic to do that without a significant leg up, which DEq has provided. Additionally, I’ve developed a unique analysis to judge the effectiveness of card quality metrics at making first picks, and set in, set out, DEq outperforms the other available metrics. The final reason to use DEq to judge card quality is that it was built from the ground up for exactly the purpose of picking the best cards, unlike any other metric. Read on to find out how.
The Formula#
DEq can be decomposed into four main components, which are summed together then multiplied by % GP.
Marginal Win Rate: Or MWR, this is effectively the GP WR excess to the mean of the entire data set. There is a small Bayesian adjustment towards the expected GP WR given the ATA of the card, which helps smooth out ratings for mythics with small sample sizes.
Pick Equity, the heart of DEq. Pick equity is a decreasing quadratic function of ATA, such that a card with ATA 1.0 will have 3.0% pick equity, and a card with ATA 14.0 will have 0.0% pick equity.
Bias Adjustment: For a late pick, a large portion of the MWR may be attributed to the cards likely to be already in the player’s pool, or likely to end up in the pool regardless of an insignificant pick, rather than the card itself. For an early pick, we must attribute that archetype choice to the card itself, since we are choosing our probable deck as much as we are choosing an individual card. The bias adjustment scales between the raw MWR for early picks and an archetype-adjusted MWR for late picks using pick equity as a scale factor.
Metagame Adjustment: Using assumptions about the rate of decay of metagame imbalance, we estimate the average decay towards equilibrium of each archetype between the average observation and the assumed draft date. Then the metagame adjustment attributes some of that estimated decay to the DEq of the individual card, since the value of a card includes assumptions about the quality of the cards you are able to surround it with.
Why GP WR#
DEq was born out of a pair of observations. The first is that for pick one, pack one, the average outcome of the entire draft after that point, not just the games where the card in question was drawn, gives the proper estimation of the value of the pick. In statistical terms, the mean of the “as-picked” match win rate averaged over all P1P1 observations gives the maximum likelihood estimation of the expected win rate from that point. GP WR, along with % GP, are the fields available on 17Lands that represent that information with the least distortion. The second observation was that GIH WR, the most commonly used card quality metrics, has significant issues that create systemic bias in the evaluation of certain cards. Bluntly, it is our job to win all of the games we register our deck for, not just the ones where we draw the card in question. I believe that GP WR records (most of) the necessary win/loss information to judge picks, and that a good metric can be developed by directly addressing its biases rather than by “artificially” boosting the card strength signal, as GIH WR does.
Why Pick Equity#
Pick equity is intended to account for the cost, as measured in win rate, of the “unopened” pick exchanged for the card in question. A card with a lower ATA value was exchanged for more valuable picks, leaving less valuable picks available to accrue to the measured win rate with the rest of the pool. By restoring that value, we can more accurate compare cards against each other in the context of the same pack.
If that explanation isn’t intuitive to you, think of another way of obtaining the same result. A more natural way to combine the metrics might be to modify the pick order suggested by ATA by the win rate data to obtain a new pick order. Pick underrated cards, cards with a high win rate, earlier, and overrated cards later. DEq is doing the same thing, but presenting it in win rate space.
It turns out that this simple modification transforms GP WR from being a rather poor card quality metric to an extremely good one. While I believe the below adjustments are useful and make the metric more robust especially in evolving metagames, there is scant evidence they lead to higher win rates.
Briefly on the shape of the curve – the quality available in each pick is reduced by the quality of the card removed by the previous player. Since the quality of that card on average is decreasing, we should expect a convex shape to the pick equity curve, and data indeed supports that. The quadratic curve was chosen to reflect that convexity with simple assumptions while fitting the constraint that the quality removed from the second-to-last pack is nearly zero. The value of the first pick, 3.0%, is frankly not supported by evidence and was chosen because it leads to the best results. My analysis continually suggests a number closer to 2%, but 3% performs better, I think because it captures something else that is going on with early picks besides the opportunity cost itself. My best hypothesis is that it’s related to the idea that some important cards tend to fall to players in the open lane and need to be further discounted as first picks.
The Bias Adjustment#
The most commonly cited issue with GP WR is that it presents the win rate of the deck, or even the archetype at large, and not so much the particular card. While pick equity goes a long way to fixing this, as the early pick cards get the additional boost they deserve for carrying the load of getting you into a winning archetype, it is still the case that late-pick cards get an undeserved boost from the quality of deck likely to be correlated with the choice. To measure the impact of a card marginal to the decks it is likely to appear in, you can take it’s participation rate in each archetype, and take a weighted average of the mean win rates for those archetypes to use as the baseline win rate rather than the cohort’s overall mean win rate. Then the GP WR marginal to that number would be more particular to the card than to the archetype.
For a 14th pick, I think this calculation makes the most sense. Does playing the card improve the decks it is added to or not? However, for a first pick, as mentioned, the GP WR proper is the optimal estimator because you are taking responsibility for the entire draft as you make that first pick. The bias adjustment uses pick equity as a sliding scale between ATA 1.0 and 14.0 to balance these two calculations. The earlier the pick is observed, the more responsibility it has for the archetype the observed players ended up in.
The Metagame Adjustment#
…