Let me start with some links that prompted a coalescing of thoughts that had brewed for a long time. First up we have a study showing how everything causes cancer. Then we have an editorial on replication of findings in science.
Now, as a physicist, I avoid a lot of the problems common in social sciences and biology. Because we have math and models we are either guided to our result by theory or if something surprising happens we have to justify it theoretically as well. This means that in most cases you might question the power of published experimental results but not as often their veracity. To take a specific example from my field, it’s questionable whether a published paper actually did a quantum gate as well as they said, but nobody doubts that their procedure would in fact apply a quantum gate.
That said, I understand the frustration regarding how hard it is to replicate things. Published papers focus so much on their results that they mostly ignore how they managed to achieve them. Yet, the technical knowledge is often the most important part. Again as an example from my field, if you read the original paper on how to lock the frequency of a laser diode, you would have a hard time actual constructing a working prototype. In fact, the most valuable part of graduate school is acquiring all the technical knowledge that is written down nowhere. Instead, it is passed down orally from postdoc to graduate student for time immemorial.
Why is their no repository for circuit diagrams of a laser lockbox or a feedback circuit? How come code is not always posted for scrutiny and/or further use? There is nominally a culture of open discussion and sharing among scientists and yet we make every group reinvent the wheel. When scientists do reveal their code it can have huge ramifications. Cowan was a well-regarded atomic theorist and he released his code for atomic calculations. Decades later it still lives on, probably used by thousands of physicists at this point.
Yes, smart people can replicate your techniques given time, but that is time better spent on pushing the forefront of science.
Now, a lot of this comes from the limited space in journals. But we have a thing called the internet now which has essentially infinite space. Thus, the next topic I want to cover is how to get scientific publishing out of the vampiric jaws of the journal publishers. How a bunch of private corporations became able to profit mightily off the hard work of scientists often funded by the public I will never know. It seems antithetical to the entire scientific endeavor.
Instead the idea that has been taking shape in my head for a few years is similar to this. Most fields already have methods for distributing pre-prints of scientific papers; for instance, physics has Arxiv. We just need to expand these websites to verify authors and commentators like Twitter does and then allow discussions about papers posted on the website. In fact, I thought this was a feature of Arxiv for the longest time just because it seemed so obvious. After all, you would think you would want commentary before a paper is published. The next step is to allow people to vouch for the veracity of a paper; thus the importance of verifying contributors to the website. Not only is this far faster than the current system, but I guarantee that it has spillover creative benefits too. Talking to other people about physics is always a wellspring of ideas and giving a visible forum where authors must respond quickly to questions and criticism just encourages even more profitable interaction.
You may be skeptical whether this will work. However, the people currently refereeing will still be reading the pre-prints. Furthermore, we have enlisted the services of many more people, including grad students and postdocs. I foresee a future where your contributions to online refereeing are just as important as your papers when looking for a job. If that is the case, then you can bet specialists will set aside time to critique papers.
I guarantee that traditional scientific publishing is dying and the some kind of crowd-sourced refereeing is the future and it will be a vast and open improvement.