Some of the chatter has been along the lines of "BUT...REPLICATION IS A PILLAR OF THE SCIENTIFIC METHOD. THERE IS A SERIOUS PROBLEM IF MOST STUDIES CAN'T BE REPLICATED. WASTE OF THE MONEYZ!!! GRUMBLE GRUMBLE..."
At the top of this post I'm hoping to add a slightly more nuanced opinion here, followed by some unpublished results at the bottom to serve as a cautionary tale. I don't really disagree with worries about the state of science. Replication is of the utmost importance for research, and if results aren't robust there must be a way to keep track. Perhaps post pub peer review and comments will fill this particular niche. Experiments now are built on a foundation of experiments and models pioneered over years and decades. If you are interested in getting involved in a new research direction, one of the most important things to do is actually see if you can replicate foundational results in your hands in your own lab. That being said, biology is hard. Replication of single experiments under well controlled conditions can easily be thrown off by Rumsfeldian unknown unknowns. I remember hearing from someone (I want to say it was Paco Moore and that there is a paper somewhere on this which I can't find with quick google searches) that measurement of fitness in the context of Rich Lenski's long term E. coli experiment can be slightly altered by University water quality. In grad school I remember Patrick Phillips describing an experiment with nematodes where the assay would only work for about two weeks a year because the stars and sun and temperature aligned to yield the perfect experimental environment. It turns out that physiology and behavior of living organisms can be extremely sensitive to just about everything if you measure closely enough.
This problem is compounded even more when you are dealing with multiple living organisms, for instance, when your research area is host-pathogen (host-symbiont, same diff) relationships. I can't speak for anyone that works with animal models, but I can definitely attest that plant immune responses are EXTREMELY sensitive to pretty much every stimulus you can think of. Since plant immune responses are dependent on cross-regulation across multiple hormonal pathways, even the slightest change in some environmental factors can completely shift the likelihood of infection. This is exacerbated by having to grow plants for multiple weeks before you can actually do the experiments, all the time worrying that some random lab malfunction (3am growth chamber overheating anyone?) will render batches of host plants unreliable. Different labs will have different water, soil, temperatures, humidity (low humidity in Tucson is the bane of my lab existence sometimes!), etc...When I started working on P. syringae and plants as a postdoc, I would get very frustrated at my inability to replicate other peoples published experiments. The more time I spent in the lab, the more I realized that that's just the way it is sometimes. Don't get me wrong, there are a variety of other reasons that replication may fail, but when you're crying into your lab notebook at 3am keep in mind that it's incredibly hard to control both the host and pathogen growth in the exact way that the published experiments were performed.
I'm guessing that every PI that works with phytopathogens and plants has a story where there was an interesting phenotype which couldn't be replicated when they moved to a different lab/University. As a postdoc I remember screening through 50 or so very closely related isolates of P. syringae pv. phaseolicola to look for subtle differences in virulence on Green (French) bean. The goal here was to minimize random genomic variability between strains, by choosing very closely related strains, so that I could hopefully quickly pin down genotypic differences underlying interesting phenotypic differences simply by looking at the genomes. Basically GWAS for microbes to use a looser term. This was one of the experimental directions I started as a postdoc and was hoping to continue as PI in my own lab. One of the most solid results I had was a subtle difference in growth between two strains on French bean cultivar Canadian wonder. Canadian wonder is the universal susceptible cultivar to P. syringae pv. phaseolicola, which basically means that this plant was thought to be highly susceptible to all flavors of this particular pathogen. I had actually found that one strain (Pph 2708) grew 10-fold less than a very closely related strain (Pph 1516) in this cultivar (Fig. 1).
When I did pod inoculations, although the response was somewhat variable, there did seem to be some immune recognition of Pph 2708 compared to other strains (Fig. 2).
You can tell that there is something different in this inoculation because the water soaked halo is smaller for Pph 2708 than other strains, except for the avirulent mutant that lacks a functioning type III secretion system (Pph 1448a hrcC-).
So there it is, I've got two very closely related strains of P. syringae that slightly differ in pathogenicity. I have genome sequences for these (will link when I've stored up the strength to navigate the Genbank submission). There aren't many differences between them, on the order of hundreds of SNPs and tens of gene presence/absence. I had everything set up and ready to go to finish off the story once I got to Tucson and set up shop.
Here's where the problem arises...even though the result is solidly replicated under North Carolina conditions there is no growth difference between Pph 1516 and Pph 2708 in Tucson. A lot of strains I've worked with behave differently here in the desert compared to the land of tobacco and barbecue, and my guess is that it's because there is literally no humidity in the air. Since plant immune responses are linked to abscisic acid I'm guessing that the lack of humidity really annoys them when I take plants out of the growth chamber to perform inoculations. Not necessarily the lack of humidity per se, but the necessary change in humidity that accompanies taking plants out of the growth chamber. Yes, there are ways to Rube-Goldberg my way around this problem, and I have thought about a walk in growth chamber, but truth is other things worked better and I've concentrated on them. On top of that I'm using slightly different soil (what I could get my hands on), it's a different growth chamber, etc...Point is, I have a result that I would not think twice about publishing if only I hadn't tried to replicate this experiment in a different place. This happens a lot.
No comments:
Post a Comment