Science Casino Pitfalls
'"Knowledge for its own sake" that is the last snare laid by morality - we are thereby completely entangled in morals once more.'
-Nietzsche, Beyond Good and Evil
1. Degenerate gambling
The title, "science casinso" is somewhat misleading, as the intent is for reviewers to wager on the results of properly designed experiments, not to gamble on statistical variance as is done in most casino games. For example, one might be tempted to bet on the result of a single repetition of an experiment, which might require 100 repetitions in order to obtain a statisticaly significant results.
Solution: Use the same statistical rigor appropriate for a grant proposal or peer reviewed publication in order to distinguish knowledge generating uncertainty from experimental variance. For instance, instead of betting whether a single repetition of an experiment will yield a result greater or less than an expected value, bet on whether x repetitions will yield a p-value less than y, where x and y are determined by proper statistical planning.
2. Gaming the system
Scientists might be tempted to game the system by various means, for instance by:
a. Proposing an experiment for which they already already know the result.
b. Betting on their own experiment, or providing insider information to reviewers.
c. Bias in carrying out experiment in order to obtain the result they prefer.
d. Fabrication of data.
e. Other ideas?
Solution: Initially a point system should be used to minimize the motivatoin for this pitfal. In order for real money to be used as a crowdfunding mechanism, regulations should be applied such as that those used by the US Securities and Exchange Commission to protect investors and ensure fairness. Possibility for biased experiments should be considered in the review process and minimized by experimental design (e.g. using appropriate statistical planning / double blind studies / etc.) before betting begins. Policies for scientific misconduct such as fabrication of data should be followed.
3. Jargon
Different scientific disciplines have specialized terminology, often making them only understandable to others in the same field. An inability to clearly explain a scientific experiment to reviewers would make it less likely for reviewers to risk a wager on it, i.e. if they don't understand the proposed experiment.
Solution: All relelvant terms must be clearly defined. The scientific question and hypothesis must be phrased clearly and in a testable way with at least two possible outcomes . The challenge to the scientist proposing an experiment will be will be to make it understandable to as wide of an audience of reviewers as possible.
4. Unclear results
The results of some experiments may occur that are inconclusive. This is usually due to poor experimental design, but may also be caused by unexpected experimental error or artifacts.
Solution: A referee (equivalent to an editor of a journal) will be needed to decide whether a bet is won, lost, or canceled due to unclear results.
5. Secrets
Scientists will likely be hesitant to publish good ideas for experiment before they answers are known. This may be in order to avoid being scooped for a publication, as well as to avoid invalidating intellectual property that may result.
Solution: If used as a crowdfunding mechanism, experiments that can not be carried out using normal funding mechanisms need not be kept secret. Another possible solution may be more of a publication mechanism, where a scientist could carry out the entire experiment, but then publish it in steps - with the first step being explanation of the experiment so reviewers can bet on it, and then the second step being announcing the results.
6. Non-hypothesis driven science
Exploratory science and engineering has value for generating hypotheses and knowledge yet seems difficult to bet on.
Solution: bet on it anyways! e.g. if one is doing a high throughput screen one can bet on how many hits above a certain threshold will be obtained. Or if a scientist is exploring the deep ocean with a new technology one could bet on how many new species may be identified. The point is to immagine the possible results and guess what might happen.
Return to the Science Casino
Published online July 15, 2018