Business schools must prioritise quality over quantity

Research that is neither rigorous nor insightful serves no one, say Stefan Stremersch, Nuno Camacho and Russell Winer

十一月 20, 2021
A row of wheelie bins, symbolising quality over quantity
Source: iStock

Is the research coming out of business schools rigorous, relevant and of high quality? In private, many scholars have their doubts.

Business schools routinely spend between 30 per cent and 50 per cent of their budgets on faculty research, but, according to the 14 associate deans we interviewed for a recent paper, there is too much focus on research quantity and too much of the research produced fails to “move the needle” in terms of affecting how industry operates or understands itself.

This view is echoed by the 234 marketing professors from 20 countries whom we surveyed. Instead of prizing creativity, practical importance and research quality, they told us that business schools mostly value the number of papers they churned out.

These findings, recently published in the Journal of Marketing, raise some important question for business schools – particularly because businesses themselves are much more interested in papers that can help them than in a higher volume of papers that are irrelevant to them. Unsurprisingly, the industry representatives whom we also spoke to want research that is useful to management practice and reliably produced. Without a stronger focus on quality, business schools risk disappointing those who often fund their work in the expectation that they can envision tomorrow’s solutions in today’s questions.

So how do business schools change their culture to prioritise quality over quantity?

Introducing clear external reference points to evaluate quality objectively would be a good start. In addition, leaders need to design incentive systems that take into account creativity, reliability and groundbreaking findings.

Making a call on what is merely a respectable piece of work and what is truly original and sector-leading is, of course, a tricky business. But it is not impossible; we propose several ways schools can benchmark and compare their research against valid external reference points.

One crucial change is for schools to regularly submit themselves to external research audits, conducted by a heterogeneous set of high-quality evaluators from both inside and outside academia. These mechanisms can help make the definition of “quality” more precise and are more informative than just counting papers.

A stronger focus on quality does not come without challenges, of course. For instance, peer review cartels – when groups of experts positively evaluate each other – can lead to the gaming of qualitative as well as quantitative metrics. Yet we see ways to combat this, such as broadening the pool of external evaluators or of those who write letters of support for promotion applications. Meanwhile, having both experts in the field of the candidate and top-notch scholars from outside the candidate’s field would eliminate much of the bias that social connections can sometimes encourage.

Our research also found that the over-reliance on quantity metrics is greater outside what one could call “elite research schools” – schools that are research-intensive, with low teaching loads and a tendency only to count top publications in their tenure requirements. This contributes to their difficulties in retaining top faculty, and may tempt them to game the system by allowing prior co-authors or even doctoral supervisors to write reference letters for candidates, which becomes very incestuous and unobjective.

We found that a higher research quantity does not have a significant effect on the teaching performance of business schools. In contrast, research that practitioners found useful for understanding their own organisations and situations is associated with higher teaching health. This is unsurprising because faculty members who produce such research typically immerse themselves in real-world managerial practice, such as consulting, case writing or executive education. This immersion increases their usage of concrete concepts, which are easier to understand than abstract concepts.

Healthy business schools are vital for the prosperity of modern universities, but the current flaws in measuring research quality are damaging their reputations for excellence. Unless this broken research model is properly addressed, the appeal of business schools to both students and companies will eventually diminish.

Stefan Stremersch is a professor at the University of Navarra’s IESE Business School and at Erasmus University Rotterdam, where Nuno Camacho is associate professor of marketing. Russell S. Winer is a professor at New York University’s Stern School of Business.

请先注册再继续

为何要注册?

  • 注册是免费的,而且十分便捷
  • 注册成功后,您每月可免费阅读3篇文章
  • 订阅我们的邮件
注册
Please 登录 or 注册 to read this article.

Reader's comments (9)

Business schools need new people, not new metrics.
What exactly is your definition of quality?
They refer to "rigorous research" like most “mainstream” marketing academics as those studies that follow the dominant research paradigm in that subject area. They usually adopt an unshakable but largely unfounded believe in an objective social reality with observable "facts" and law-like causal relations/regularities, founded on a mix of naive realism, reductionist tendencies and empiricist convictions. Especially US (or US oriented) marketing scholars (outside of CCT) are guilty of that naivety really. Anything that gets published in the Journal of Marketing is of the same "rigorous" standard. If you read most "mainstream" marketing stuff, it is as if the debates in the philosophy of (social) science and the developments in the sociology of knowledge over the last 70 years never happened.
"whom we surveyed. Instead of prizing creativity, practical importance and research quality, they told us that business schools mostly value the number of papers they churned out." So it appears that quality seems to some function of creativity, practical relevance and rigour. Are these components measurable? if so hoow and how will you combine these to come up with a measure of quality? Would this be a continous measure or a binary measure? Are we talking minimum quality thresholds i,e a minium score that needs to be achieved to certify a piece of work as one of quality or not? Quality is easy to talk about but difficult to put in to practice. On ther other hand Quantity is a count. It is 0 or positive. There is no ambiguity as to what a count it. So, if you really want to discuss "quality" show me exactly how you are going to measure this, so one can compare this over time or across ressearchers etc. If you cant do that then it is just waffle. I am by no means advocating merely churning out papers for the sake of doing so, but almost always "quality" is determined ex-post the paper having written and by others applying some criteria over it. If researchers always worried about this, nothing may ever get written!
"whom we surveyed. Instead of prizing creativity, practical importance and research quality, they told us that business schools mostly value the number of papers they churned out." So it appears that quality seems to some function of creativity, practical relevance and rigour. Are these components measurable? if so hoow and how will you combine these to come up with a measure of quality? Would this be a continous measure or a binary measure? Are we talking minimum quality thresholds i,e a minium score that needs to be achieved to certify a piece of work as one of quality or not? Quality is easy to talk about but difficult to put in to practice. On ther other hand Quantity is a count. It is 0 or positive. There is no ambiguity as to what a count it. So, if you really want to discuss "quality" show me exactly how you are going to measure this, so one can compare this over time or across ressearchers etc. If you cant do that then it is just waffle. I am by no means advocating merely churning out papers for the sake of doing so, but almost always "quality" is determined ex-post the paper having written and by others applying some criteria over it. If researchers always worried about this, nothing may ever get written!
"whom we surveyed. Instead of prizing creativity, practical importance and research quality, they told us that business schools mostly value the number of papers they churned out." So it appears that quality seems to some function of creativity, practical relevance and rigour. Are these components measurable? if so hoow and how will you combine these to come up with a measure of quality? Would this be a continous measure or a binary measure? Are we talking minimum quality thresholds i,e a minium score that needs to be achieved to certify a piece of work as one of quality or not? Quality is easy to talk about but difficult to put in to practice. On ther other hand Quantity is a count. It is 0 or positive. There is no ambiguity as to what a count it. So, if you really want to discuss "quality" show me exactly how you are going to measure this, so one can compare this over time or across ressearchers etc. If you cant do that then it is just waffle. I am by no means advocating merely churning out papers for the sake of doing so, but almost always "quality" is determined ex-post the paper having written and by others applying some criteria over it. If researchers always worried about this, nothing may ever get written!
"whom we surveyed. Instead of prizing creativity, practical importance and research quality, they told us that business schools mostly value the number of papers they churned out." So it appears that quality seems to some function of creativity, practical relevance and rigour. Are these components measurable? if so hoow and how will you combine these to come up with a measure of quality? Would this be a continous measure or a binary measure? Are we talking minimum quality thresholds i,e a minium score that needs to be achieved to certify a piece of work as one of quality or not? Quality is easy to talk about but difficult to put in to practice. On ther other hand Quantity is a count. It is 0 or positive. There is no ambiguity as to what a count it. So, if you really want to discuss "quality" show me exactly how you are going to measure this, so one can compare this over time or across ressearchers etc. If you cant do that then it is just waffle. I am by no means advocating merely churning out papers for the sake of doing so, but almost always "quality" is determined ex-post the paper having written and by others applying some criteria over it. If researchers always worried about this, nothing may ever get written!
You seem to have got my post wrong. I was not defending them, their claims or their paper, on the contrary.
Apologies if my post came across as dismissing your comment. That wasnt the intention at all. I get your point. The issue is that while this debate about what "quality is" etc. might appear to be just intellectual, there are real world consequences of ill thought out ideas and ambigous measures, particularly where these beliefs are held by people have influence over other peoples careers. Many careers and peoples mental health are being destroyed. Seems like only "top notch researchers" "top faculty" and "elite research schools" have any right to exist in academia - whatever these mean.