I think that Scott's essays The Control Group Is Out Of Control (particularly the idea of the Ouroboros of Scientific Evidence) and Why I Am Not Rene Descartes are relevant here. In difficult cases, assessing evidence properly is a matter for intricate discussion.
Comments
Andrea Gallagher
Thanks for those links. My view is partly inspired by the first post, and the second is new to me. (The bit about Descartes' dirty hands is pure Scott).
I think the value of structured discussion tools is driven by that need for an intricate discussion to asses evidence. Today, discussion platforms don't have methods of weaving arguments together, so it becomes too hard to follow a debate with detailed and nested evidence.
Instead, the crowd throws up it's collective hands in defeat and "trusts the scientists", assuming that any given study is the best and only way to get an answer. But within scientific literature, it's still very hard to trace the crux of a theory and the quality of evidence for it.
Scott is a master at the science literature tear-down, but he's not sufficiently scalable. I'm hoping that by baking some of the best logical structure and process into the tools, we can make the crowd just a bit smarter.