What's the plan to ensure that highly-ranked answers are free of non-obvious security flaws?
A good answer to a question should have two properties: it is helpful, and it is correct. Voting (the wisdom of crowds) is a great way to bubble up the most helpful answers (the ones that are constructive, clearly written, etc.). But when it comes to deeply technical subjects, like cryptography, it's not clear to me that voting will be effective at assuring that answers are correct (especially, free of subtle security flaws).
I'm concerned about the following scenario. Karl asks a question about crypto. Paulina the programmer posts a helpful answer: it is concise, clearly written, describes a simple solution, and even comes with some sample code. Lots of Stack Exchange members vote it up.
Sounds great, right? But suppose Paulina's answer is actually incorrect. It is simple, but wrong: it has a non-obvious security hole, which is only apparent to crypto wizards. Charlie the crypto wizard spots the problem in Paulina's answer. What should Charlie do?
Perhaps Charlie should just refrain from upvoting Paulina's answer, and move on? But this doesn't work. If 99% of the Stackexchange members are ordinary mortals, and 1% are crypto wizards, and the flaw is only apparent to crypto wizards, then instead of getting 100 upvotes, the answer will get 99 upvotes. Same outcome. That doesn't help.
Perhaps Charlie should post a comment after Paulina's answer, with an explanation of the flaw? But typically, a full explanation of the flaw won't fit in a one- or two-line comment. Comments aren't a very good mechanism for this.
Perhaps Charlie should just post a brief one-line comment, saying that Paulina's answer is insecure (without explaining why) and say that readers should just trust Charlie? That doesn't seem in line with the Stack Exchange philosophy of meritocracy, and it doesn't seem likely to cause Paulina's answer to be ranked lower.
Perhaps Charlie should write a separate answer that just explains why Paulina's answer is wrong? But that's not proper use of Stack Exchange: you're not supposed to use answers to comment on other answers.
Perhaps Charlie should write a separate answer explaining the proper way to do things, and give a detailed tutorial on why it is the right way to do it (without commenting on Paulina's answer)? That may be the least-bad solution, but there's not always a short and convincing way to convince non-experts why something is the right way, without first teaching them chapters of cryptographic theory. If Paulina offers a simple solution that sounds plausible to non-experts, and Charlie offers a complex solution that seems unnecessarily complicated to non-experts, guess which answer is going to get voted up? I'm guessing Paulina's gets voted up, even if Paulina's answer is insecure.
Part of the problem is that the Stack Exchange format doesn't seem designed to support detailed analysis of other people's answers.
Maybe a more important aspect of the problem is that crypto is subtle stuff. The details matter, and it's very common that plausible-seeming approaches actually contain subtle security flaws (which aren't always easy to explain to a non-expert). Moreover, in my experience, it's very natural and tempting for developers to want to invent their own solution and feel overly confident that they are up to the task. As Bruce Schneier has written, it's easy to come up with a scheme that seems secure to you. It's easy for Alice to come up with a scheme she herself can't break; the hard part is to come up with a scheme that others can't break. It's natural for readers to up-vote simple, plausible-sounding answers, but in cryptography, the simple answers aren't always secure, and the plausible-sounding solutions can have serious (but non-obvious) solutions. This seems like a special challenge for this Stack Exchange site.
So, is there any plan to deal with this problem? And what should Charlie the Cryptographer do in these kinds of situations?