Are Average Estimates in Line Graphs Biased Toward Areas of Higher Variability?

Authors

Sheng Long (Northwestern University)

Abstract

When estimating the average value of a data series in line graphs, do viewers systematically overweight regions of higher variability? This is the question that Moritz et al. [14] set out to answer, and across two preregistered experiments they reported evidence for a bias they termed variability overweighting. Using their released data and code, I first reproduce their reported analysis, then ask whether their inference survives an equally valid analytic choice: measuring estimation error against the average of the displayed marks of each stimulus, rather than against a reference series that, for one mark type, is never shown. Under this alternative operationalization, the effect is substantially reduced and, for one mark type, reverses. Splitting the analysis by whether variability appears in the upper or lower region of the graph further reveals an asymmetry that the original direction-corrected analysis collapses away. I argue that the deeper issue is what verbal theories leave unspecified. When a theory is silent about which operationalization it requires, the metric ends up doing the theory’s arguing for it. I close with a discussion of what reproduction and reanalysis are for: not verdicts on truth, but instruments for making analytic contingency visible and for forcing theories to say more.