I’ve been experimenting with a small MVP that tries to structure online debates differently.
The idea is simple:
one claim/theory
users add individual supporting or opposing facts (with sources)
each fact is discussed and voted on independently
no final verdicts, no “truth score”, no authority layer
The goal is not to determine truth, but to observe how collective belief and disagreement form when arguments are forced to be atomic.
After sharing this experiment with skeptic-oriented communities, I ran into a set of strong critiques that seem to recur whenever projects like this appear:
Voting is inherently argumentum ad populum, even if applied to individual facts
There’s a strong asymmetry of effort: real evidence is costly, bad evidence is cheap
Coordinated actors, cranks, or propagandists are more motivated than average users
Non-experts struggle to distinguish relevance, quality, and weight of evidence
“Fact overload” and gish gallop can drown out meaningful signal
Moderation only works with subject-matter experts, which doesn’t scale
Similar platforms have failed when public voting elevated weak or misleading evidence over rigorous research
Many commenters argued that this model inevitably legitimizes misinformation rather than containing it.
Before taking this experiment any further, I’d really like input from people here who’ve seen similar systems succeed or fail.
My questions:
Is this kind of structure fundamentally doomed outside of peer review or expert-only contexts?
Are there known constraints or design patterns that prevent collapse into noise or popularity contests?
Does this only work in narrow, technical domains (e.g. software, math, engineering)?
Or is the failure mode intrinsic to letting non-experts evaluate evidence at all?
If it helps to see the concrete implementation, the MVP is here (no signup required): https://fact2check.com
I’m less interested in defending the project than in understanding where - structurally - this approach breaks.
That’s a fair point, and I think it’s related.
Fiction has a structural advantage: it’s coherent, emotionally satisfying, and low-effort to consume. Evidence-based reasoning is fragmented, probabilistic, and often unsatisfying - especially when you don’t get a clean narrative or conclusion.
One thing this experiment tries to test is whether forcing arguments to be atomic (individual facts instead of stories) helps or hurts. My suspicion is that it actually removes the narrative glue that makes ideas compelling - which may explain why such systems struggle to attract sustained engagement.
In other words, it may not just be about truth vs fiction, but about narrative vs non-narrative cognition.
If that’s correct, the failure mode is structural, not just social or political.
I’ve been experimenting with a small MVP that tries to structure online debates differently. The idea is simple: one claim/theory users add individual supporting or opposing facts (with sources) each fact is discussed and voted on independently no final verdicts, no “truth score”, no authority layer The goal is not to determine truth, but to observe how collective belief and disagreement form when arguments are forced to be atomic. After sharing this experiment with skeptic-oriented communities, I ran into a set of strong critiques that seem to recur whenever projects like this appear: Voting is inherently argumentum ad populum, even if applied to individual facts There’s a strong asymmetry of effort: real evidence is costly, bad evidence is cheap Coordinated actors, cranks, or propagandists are more motivated than average users Non-experts struggle to distinguish relevance, quality, and weight of evidence “Fact overload” and gish gallop can drown out meaningful signal Moderation only works with subject-matter experts, which doesn’t scale Similar platforms have failed when public voting elevated weak or misleading evidence over rigorous research Many commenters argued that this model inevitably legitimizes misinformation rather than containing it. Before taking this experiment any further, I’d really like input from people here who’ve seen similar systems succeed or fail. My questions: Is this kind of structure fundamentally doomed outside of peer review or expert-only contexts? Are there known constraints or design patterns that prevent collapse into noise or popularity contests? Does this only work in narrow, technical domains (e.g. software, math, engineering)? Or is the failure mode intrinsic to letting non-experts evaluate evidence at all? If it helps to see the concrete implementation, the MVP is here (no signup required): https://fact2check.com I’m less interested in defending the project than in understanding where - structurally - this approach breaks.
Maybe it has something to with the way that bestselling Fiction has always outsold non-Fiction?
That’s a fair point, and I think it’s related. Fiction has a structural advantage: it’s coherent, emotionally satisfying, and low-effort to consume. Evidence-based reasoning is fragmented, probabilistic, and often unsatisfying - especially when you don’t get a clean narrative or conclusion. One thing this experiment tries to test is whether forcing arguments to be atomic (individual facts instead of stories) helps or hurts. My suspicion is that it actually removes the narrative glue that makes ideas compelling - which may explain why such systems struggle to attract sustained engagement. In other words, it may not just be about truth vs fiction, but about narrative vs non-narrative cognition. If that’s correct, the failure mode is structural, not just social or political.