Is Bad Science Undermining Big Science? A Conversation Between Guests and Audience

Theral Timpson

One can look at funding for science and break it up into Big and Small Science. Funders of Big Science would be large corporations, foundations, and the government, whereas Small Science relies on VC, startup funds, small businesses and entrepreneurs. Big Science comprises those big projects like the sequencing of the human genome and the war on cancer. But are such big projects yielding results that justify the massive spending?

Bill Frezza, a regular guest at Mendelspod, says not, the massive spending isn't justified. Bill pens a column over at Bio-IT World, The Skeptical Outsider, where he plays provocateur and questions some of the basic assumptions that we in the life science industry hold dear. He thinks NIH funds are badly mismanaged and talks about academic science as a feudal systems with lords (PI’s) and vassals (postdocs). His latest article asserts that “bad science” is threatening Big Science.

“Think about the modern business model of Big Science -- an interconnected set of interests whose tentacles extend into academia, foundations, and major corporations. Advocates of a variety of causes across numerous fields—from health care to agribusiness to energy and the environment—selectively promote scientific results produced by legions of scientists, some of whom are independent and others not. These pronouncements are generally aimed at attracting more public and private research funding, selling more goods and services, or impacting laws and regulations that control the selling of goods and services. Sounds science helps policymakers and consumers make wise choices. Bad science, not so much.” (The Skeptical Outsider, Jan, 2013)

For Frezza, “bad science” is that which is not reproducible. Recently we’ve seen an uptick in non-reproducible findings, and Bill says it’s because of corruption and moral hazard. Examples of corruption he cites are influences that are “buying a predetermined scientific outcome” such as cancer studies funded by the tobacco industry and anti-GMO studies funded by environmental activists. Moral hazard, he says, is a cousin of corruption that is more subtle, such as researchers cherry picking results, fooling not only others but themselves.

His solution: improve the quality of science by publishing in new peer review journals such as GigaScience and eLife where the data behind the findings can be comprehensively logged.


I sent Bill’s article to a couple industry friends for their take on his assertions. The first is from a scientist at Stanford who has worked in Ron Davis' lab for years.

“I think there is at least a grain of truth in most of the statements.  But it's pretty complex.  One could argue the so-called entitlement programs aren't a culprit for reduced R&D spending at all, that we all pay in our own share, and if politicians weren't trying to fix problems in the Middle East we'd have plenty to go around.

About the science, there is indeed a major problem with spending dropping.  Pharma used to fund billions in R&D but has all but stopped.  Everyone turns to NSF and NIH now, whose budgets have been flat for years meanwhile the lack of private money and the huge increase in biologists being trained, but a total lack of money to pay them or fund their work.  

So there is, in my opinion, greater pressure to get groundbreaking results or perish.  Unfortunately, the increased quantity of scientists being trained doesn't mean their skills are any better (an in fact they're probably on the whole less talented).  So there is a pull towards the "dark side" for some.

NIH is a huge problem.  What we need are breakthroughs and all they fund are conservative, incremental improvements.  So much of the lack of real innovation is based on our fear of trying.  Yet truly innovative science attracts only the very best and brightest, and often is our only hope at solving major problems.  

Much of it is also our desire for quick results.  Yet there is no quick fix to cure cancer or world hunger.  So we see Apple and Samsung in a technology battle that produces new products every few months.  We become fixated on this level or progress.  Curing cancer is a problem that is many orders of magnitude more complex than making a smartphone.  People need to realize that innovation in cancer treatments is a challenge of decades and not months.”<.em> (Keith Anderson, Scientist at Stanford Genome Technology Center)

The second responder, who can always amuse, is the CSO of a small life science reagent company in San Jose:

"To assume that we have loss of faith means that we had faith at the first place. Did we? Science is just like cars: cars get better over time and the more advanced they get, the less we understand or need to know how they function. The better they work, the less we worry about them or getting stranded on the freeway. If they don’t work, we have no idea how to fix them and we need to refer to a specialist that will charge us big bucks to do so. Science is the same: science does a pretty good job for most people so they don’t have to worry about it.

Science as a big lobby: nothing new in Bill’s article. When I did my PhD, the joke was that state scientists look for 5 pm and time to go home. However, if we want to put everybody to college, then one needs this legion of science instructors. Science at school is just a by-product of ‘education.’” (Anonymous Chemist in San Jose)

Recently I interviewed Lee Hood, founder of the Institute for Systems Biology, and used Bill’s skepticism in a question.

Mendelspod: Should we continue the current system of funding when so many tax payer dollars are going toward bad science that cannot be reproduced.

Lee Hood: I think a lot of the bad science that can’t be reproduced is really viewed in the wrong way. The essence of scientific experimentation is to try ideas, test them, and if they work, great. But what’s key is making sure they work in many other people’s hands. . . I don’t think there’s all that much bad science in the sense that it was done with the wrong motivations in mind. I think it’s just part of the scientific process that a lot of science doesn’t end up working out well in the future. How you distinguish between good and bad science--we attempt to do that with the peer review systems that we have now. And frankly I think that the peer review sessions that we have now do just about a job as anyone can do. And if the alternative were to say there was some politician who’s going to decide what’s good and what’s bad science, I think that would be terrible beyond belief. . . This is a draconian suggestion. . . I think we have pretty good systems now and I would hesitate to change that. . . . Big Science done well is absolutely fundamental to attacking big challenging scientific problems.

Dr. Hood set up the Institute for Systems Biology to go after the challenge he calls P4 Medicine that he says needs big funding. He actually calls NIH funding Small Science.

Bill’s response to Lee Hood:

As much as I respect Dr. Hood, he completely dodged your question on how to reduce the amount of bad science. His defense of the existing peer review system as the best of all possible worlds is pathetic. His call for more Big Science funding is entirely self serving. He's right that politicians can't be trusted to do the job, but that's a false alternative. Only transparent and efficient market-like mechanisms that “punish” scientists who continue to spit out unreproducible results will fix the problem. Claiming that the science is too complex to be easily reproduced is excuse making.

I laud Bill for taking on such topics and big interests. From what I’ve seen, there’s not enough of this in the life sciences. Perhaps the industry grew so fast that everyone got drunk on the same kool-aid. Bill questions the war on cancer, and rightly so. Frequently I hear from guests that we’ve hardly moved the needle after 30 years. However, I’d argue that the incredible high speed at which this industry has advanced is an indication of its success. Thirty years ago we were sequencing DNA manually with a gel and PCR was yet to be widespread. Most all of us have our stories how some medication or another developed in this time has made our lives much better.

Bill points to his former industry, the world of IT as an example of "good science." But the IT industry benefited greatly from Big Science projects, the biggest being perhaps the internet. Would Frezza say that the internet was not worth it? Would he say that the sequencing of the human genome wasn’t worth it? Indeed this project is proving to be the platform on which much of today’s research is built. With big fundamental projects, it can be difficult to imagine the impact they have on humanity's direction, let alone that of the country.

I'm happy with Dr. Hood's response. The system of science will always produce bad results. That doesn’t mean the system is broken. It means scientists are willing to try new hypotheses. Dr. Hood may be self serving, but that doesn't mean he's not community serving as well. Still, we must ask ourselves, why the increase in bad science. Bill’s suggestion about open publishing of all the data is a great start. As we’ve reported before, it’s also important that so-called bad or negative results are published as well. But does this answer the fundamental issues?

We’ll be pursuing this subject in several upcoming series of shows where we’ll have Bill on the program with someone who has not lost faith in Big Science. And we’ll try to understand better the causes for the increase in non-reproducible science.

What are your thoughts? Are we losing faith in Big Science? Why the increase in "bad science?"