Showing posts from November, 2019

Computational reproducibility: Some challenges

There has been recent discussion about the reproducibility of scientific results, with some unflattering conclusions. One study suggests only a 62% reproducibility rate.

For some fields it's probably much worse. Any result that depends on computation is at great risk of continual change in its programming environment. A program written ten years ago has little chance of building today without change, let alone running or running correctly.

This concern is not widespread, but it is growing. One indicator is the creation of the Ten Years Reproducibility Challenge, which asks researchers to rerun their old code—ten years or more, very old by computing standards—and see if it still works.

Can you even find your old code? That's a challenge all on its own.

Any researchers out there with computational interests are encouraged to accept the challenge. The link above has details about how to proceed. The results are sure to be eye-opening. Even if you don't submit your results, th…