Show Your Work: A Plea For Better Research Methods

A new paper from the University of Maryland’s Robert H. Smith School of Business makes the argument that management researchers should show more data in their work and use a simple graphical tool to dramatically improve the quality and speed of both empirical and theoretical work.

“This paper is the plea for researchers to use better data visualization methods to write better papers,” says management professor Evan Starr, who wrote the research with fellow Smith management professor Brent Goldfarb. Their paper, forthcoming in Strategic Management Journal, proposes that using binned scatterplots will “lead to the identification of new and interesting phenomena, raise the credibility of empirical research, and help create richer theories.”

Starr and Goldfarb say binned scatterplots are the way to go to quickly determine relationships between variables, examine outliers and assess what drives the relationship between groups in research.

“This is a tool to visualize empirical relationships,” says Starr. “It’s so simple: When you have an X and a Y variable that are continuous, and all you do is you take your X variable and you break it up into bins, or categories. Then you plot the average of the Y variable within each bin. You can also do this conditional on many control variables, for different groups, and include confidence intervals and bands.”

Goldfarb adds, “for example, a researcher might discover that, say, on average, having a woman in the C-Suite is related to employee satisfaction. Using binned scatterplots can answer the question of whether having just token representation is sufficient to achieve this effect, or whether it only kicks in when there is more significant gender balance in the C-suite.”

Starr and Goldfarb lament that less than half of empirical papers in leading management journals show even a single graph — leaving out a lot of the data behind the research. He and Goldfarb hope their paper will help spur researchers to include more data in future work.

“It’s not groundbreaking, it’s not a new method, it’s not even something we came up with,” says Starr. “This is a diffusion piece to try to improve the common practices in the field.”

His message to fellow management researchers: “If you use this approach you will learn about what these relationships look like, you’ll learn what’s driving them, and you will write better empirical and theoretical papers. You will also ditch papers that make no sense earlier, because you can actually see what’s going on.”

Read the full paper, “Binned Scatterplots: A Simple Tool to Make Research Easier and Better,” in Strategic Management Journal.