Suppose you want to buy a used car, but your knowledge of car maintenance is limited, and you need a car quickly. The dealership you visit has a range of cars, some better than others. Since you find it difficult to tell the difference between the good and bad cars, you might be inclined to lower your offer for any of the cars, to avoid overpaying for a bad car (a ‘Lemon’) by mistake. If the dealer knows you won’t pay enough for a higher quality car, but can’t tell the difference, they have then a stronger motivation to try and shift one of the lower quality motors. This means that the better cars remain unsold. This effect stems solely from a difference in the information about the product for the buyer and seller.
This thought experiment is based on George Ackerlof’s famous 1970 paper, ‘The market for Lemons’ (1), where he explored the concept and effects of asymmetric information in economics. In transactions where one party has differing information to the other, adverse effects can occur; economists refer to this as an imperfect market, where often both buyer and seller can lose out.
This idea has gained traction among economists, who have linked problems in (for example) social mobility, or Obamacare, to imperfect information. In the sciences, however, we rarely think it such stark, transactional terms. This may be to our detriment, however, since in the process of publishing science we are likely to encounter several points where different parties have varying levels of information about a given study which could potentially lead to a poorer communication of facts and data.
The audience for a scientific study has a range of information at their fingertips – author affiliations, potential conflicting financial interests, for example – that enable a judgement to be made about the content of the work. This kind of meta-information provides a useful link between author and reader that can help provide trust in the work at hand, but there are other ‘meta’ aspects of research that are trickier to communicate; why have the authors chosen to write up this specific set of data, rather than any other findings? What, if anything, changed during the review process? These are potential times where an asymmetry in the information available about a scientific study could limit the trust which a reader could place in the findings.
Competition could encourage the omission of details if it could be of financial benefit to an individual or a corporation. If an author cannot tell whether a study represents just the best results or not, then their trust in a research project could be limited; research shows that in many cases drug trials go unpublished or unfinished (2), so how seriously should we take those that are published? Without full information about unpublished studies, the value of published studies could be questionable, across the whole field in question.
Who knew what, and when?
In a general sense, we can think about the asymmetry of knowledge between different parties involved in publishing; what do authors know that the readers (and to a different extent editors) don’t?
We can assume that the author tends to have a greater grasp of the information involved in study than the reader; they make judgements about which data to include, and which aspects of their research should be written up completely. Few scientists would be able to claim they’d published every part of the trains of thought that had led them to where they currently are in their careers. In most cases, the decisions as to the selection of data can be which results are most interesting, or offer the best chance for success in a high tier journal, or in the simplest case, those data that pertain to the hypothesis in question (why mention data you don’t believe pertains to the question you’re asking?)
Even if the choices of experimental design and data to include in published papers are generally made in good faith, it can be difficult to explain to readers. The controversy surrounding the hacked emails of the Climate Research Unit at UEA highlights how the disparity between formal and informal communication in science can be misconstrued, at great cost to public trust in science in this case (3). Where readers may have a sense that a backstory could be omitted, the value they place in a given study may decline, regardless of the history of an article.
A culture that prioritises the publication of interesting research in higher tier journals leaves less room for academics to give weight to the work that lies between these topics. Perhaps we should give credit to scientists for keeping a public research diary of sorts, that could serve as an open archive of the direction in which they are working. This may be a harder sell where competition between differing research groups is a driving factor, but the flip-side could be to actually foster a more cooperative research environment.
An even slower publication process
During submission, review and publication of papers, there are a number of facets that may induce an asymmetry of information. The editor naturally asks for expert opinion as to the quality of an article through peer review – much like an antiques salesperson would seek a valuation of a supposedly priceless heirloom to avoid fraud. In this way, the editor seeks to increase their information about the article, and thus can value them more appropriately; but where the referee isn’t given sufficient evidence to make these judgements, the editor can be left blind. Thus we see the value in providing all data to allow complete review.
However, there are other, more opaque parts of the publication process that could limit what each party knows. An editor must make a subjective judgement about whether a submission is suitable for their audience; if readers or authors are unaware of the rationale for these decisions, it may affect their impression of the finally published articles.
Of course, publicising such details stands in contrast to the business models of many journals, and it almost need not be mentioned how much longer this would take overall. Should we advocate for a fully open publication process, at the expense of an even longer turn-around time for research papers?
Expediency or openness?
Where information is not evident to readers, it tends to be the result of processes to expedite the wheels of scientific advancement; the need for a reader to absorb all meta information about a study (history, outliers, rationale, even the train of thought) would markedly increase the time required to understand a research field.
Should we then be weighing up expediency against trust in science? In the present research enviroment, with questions about the trust placed in the scientific endeavour, this is a valid question to ask. It may even be the case that such a slowdown in research may not be the case; with fewer repeated trips down blind avenues of study, and the potential for greater communication and cooperation, there is potential for advancement to still occur swiftly, with a greater sense of trust from readers and governments that may be funding our studies.
(1) http://www.econ.yale.edu/~dirkb/teach/pdf/akerlof/themarketforlemons.pdf
(3) http://climatecommunication.yale.edu/publications/climategate-public-opinion-and-the-loss-of-trust/
Robert Emberson 
1 thought on “Information asymmetry in science & publishing”