|
We assessed the 29 winning essays of the Bigelow Institute of Consciousness Studies (BICS) contest using an evidence hierarchy approach adopted by many scientific fields. Two independent judges rated the target essays for their quality of scientific evidence, reproducibility, and replicability using an evidence hierarchy adapted from several published models that accommodate both qualitative and quantitative evidence. According to our criteria, six essays (20.7%) were categorized as the highest level of scientific evidence, four essays (13.8%) were categorized at a medium level of scientific evidence, whereas the remaining 19 essays (65.5%) were considered a low level of scientific evidence. The overall agreement of the essay rankings between the present authors' classifications of evidence quality and the rating system used by the BICS judges was only 44.8%, with a nonsignificant Spearman's rho correlation of .03. This result indicates extremely little concordance (overlap) of the two evaluation systems, which corroborates prior research on the critical shortcomings of evidence hierarchies. The essays representing the highest level of scientific evidence per our criteria involved near-death experiences and mental mediumship. For other anomalies that ostensibly support the survival hypothesis (e.g., physical mediumship or electronic voice phenomena), more studies with refined experimental designs are needed to improve their quality of evidence as defined in current scientific terms. Important considerations and future research directions are likewise discussed. |
|