9

What's the plan to ensure that highly-ranked answers are free of non-obvious security flaws?

A good answer to a question should have two properties: it is helpful, and it is correct. Voting (the wisdom of crowds) is a great way to bubble up the most helpful answers (the ones that are constructive, clearly written, etc.). But when it comes to deeply technical subjects, like cryptography, it's not clear to me that voting will be effective at assuring that answers are correct (especially, free of subtle security flaws).

I'm concerned about the following scenario. Karl asks a question about crypto. Paulina the programmer posts a helpful answer: it is concise, clearly written, describes a simple solution, and even comes with some sample code. Lots of Stack Exchange members vote it up.

Sounds great, right? But suppose Paulina's answer is actually incorrect. It is simple, but wrong: it has a non-obvious security hole, which is only apparent to crypto wizards. Charlie the crypto wizard spots the problem in Paulina's answer. What should Charlie do?

  • Perhaps Charlie should just refrain from upvoting Paulina's answer, and move on? But this doesn't work. If 99% of the Stackexchange members are ordinary mortals, and 1% are crypto wizards, and the flaw is only apparent to crypto wizards, then instead of getting 100 upvotes, the answer will get 99 upvotes. Same outcome. That doesn't help.

  • Perhaps Charlie should post a comment after Paulina's answer, with an explanation of the flaw? But typically, a full explanation of the flaw won't fit in a one- or two-line comment. Comments aren't a very good mechanism for this.

  • Perhaps Charlie should just post a brief one-line comment, saying that Paulina's answer is insecure (without explaining why) and say that readers should just trust Charlie? That doesn't seem in line with the Stack Exchange philosophy of meritocracy, and it doesn't seem likely to cause Paulina's answer to be ranked lower.

  • Perhaps Charlie should write a separate answer that just explains why Paulina's answer is wrong? But that's not proper use of Stack Exchange: you're not supposed to use answers to comment on other answers.

  • Perhaps Charlie should write a separate answer explaining the proper way to do things, and give a detailed tutorial on why it is the right way to do it (without commenting on Paulina's answer)? That may be the least-bad solution, but there's not always a short and convincing way to convince non-experts why something is the right way, without first teaching them chapters of cryptographic theory. If Paulina offers a simple solution that sounds plausible to non-experts, and Charlie offers a complex solution that seems unnecessarily complicated to non-experts, guess which answer is going to get voted up? I'm guessing Paulina's gets voted up, even if Paulina's answer is insecure.

Part of the problem is that the Stack Exchange format doesn't seem designed to support detailed analysis of other people's answers.

Maybe a more important aspect of the problem is that crypto is subtle stuff. The details matter, and it's very common that plausible-seeming approaches actually contain subtle security flaws (which aren't always easy to explain to a non-expert). Moreover, in my experience, it's very natural and tempting for developers to want to invent their own solution and feel overly confident that they are up to the task. As Bruce Schneier has written, it's easy to come up with a scheme that seems secure to you. It's easy for Alice to come up with a scheme she herself can't break; the hard part is to come up with a scheme that others can't break. It's natural for readers to up-vote simple, plausible-sounding answers, but in cryptography, the simple answers aren't always secure, and the plausible-sounding solutions can have serious (but non-obvious) solutions. This seems like a special challenge for this Stack Exchange site.

So, is there any plan to deal with this problem? And what should Charlie the Cryptographer do in these kinds of situations?

  • I suppose this is a weakness of democracy. I would hope that if 99 people up-voted the answer then it would be "good enough" to solve Karl's problem at the time. – Ventral Jan 8 '11 at 14:49
  • 6
    @Ventral, the problem with security is that a solution can seem OK to 99 people even though it is actually insecure. Presumably if it is actually insecure, it's not solving Karl's problem, even though it seems fine to the 99 up-voters. – D.W. Jan 9 '11 at 1:58
  • I agree with your question, and used to think the same of most security issues on SO. In fact, though there are questions where sound security experience and knowledge is drowned out in the noise, in most cases there are enough experienced people interested in the topic to shake things out right. I imagine, once there are enough users here, this will apply too. That said, I agree with the answers below. – AviD Jan 9 '11 at 23:12
  • Also, wrt specifically cryptography, note that there is a separate proposal: area51.stackexchange.com/proposals/15811/…. Here, it would mostly be applied cryptography, and less so the mathematics of it... Though even still, there is plenty to go wrong. Speaking as one of those that has made the subtle mistakes, and has also caught others' - the system works, if you show me my mistake I will happily correct it (and usually point credit towards you). – AviD Jan 9 '11 at 23:15
  • Absolutely right, hence the democracy comment – Ventral Jan 10 '11 at 18:48
  • 1
    This is the general problem of reputation-controlled hivemind - they look more on presentation that on contents, since most can evaluate first and much less - second. And going against the crowd can result in reputation problems. Enjoy your AIDS. – Smit Johnth Mar 28 '13 at 15:49
5

I think this is a problem that we may encounter during the youth of the site, while most users have relatively low reputation scores and fairly limited permissions. Eventually though, that should be a condition we can grow out of.

The ideal solution would be in a scenario where Charlie has enough reputation to edit posts. Charlie could edit Paulina's answer to reflect the more accurate details. However, this would only be appropriate if the correct solution was a relatively slight deviation from Paulina's proposal.

Charlie could post his own answer, and flag Paulina's for moderator attention. If Paulina's answer truly is flawed in a significant way, and it has received an indomitable number of up-votes, it may be worth having it deleted to allow the better solution to rise up. The moderators should be members who either have earned enough reputation through their work on the site, or who have been voted in because of their recognized experience in the field and demonstration of sound judgment to the community. Therefore, the moderators should be capable of identifying when deletion of an answer for this reason is reasonable, and taking the appropriate action.

Flagging the answer alone will not likely do the job, since the field in which you give a reason for flagging has character limits akin to those of the comment field.

Of course, Charlie always has the option of down-voting Paulina's answer as well. But, as stated earlier, one down-vote against the mass of up-votes is not likely to make much difference unless there's a lot of other crypto-wizards around to assist in this effort.

  • Do people lose reputation if their answer is deleted? – Ventral Jan 8 '11 at 14:52
  • @Ventral - I'm not sure. Perhaps that's another question to post to meta. However, if the answer was flawed in a significant enough way to warrant deletion, then isn't the loss of reputation also warranted? – Iszi Jan 9 '11 at 2:08
  • In trawling through stack exchange meta, it looks like people lose the rep gained from the answer when it is removed, but there is no punitive rep loss. – Rory Alsop Jan 9 '11 at 16:13
  • 1
    I'm not sure. Part of me feels it's wrong to deprive someone of their fairly earned reputation. The other part thinks the same way you do in that ultimately the answer is wrong and that's the main thing. – Ventral Jan 10 '11 at 18:34
  • @Ventral I agree with it being wrong to deprive someone of their fairly earned reputation. I'd submit that, if they gained that reputation by posting incorrect answers, that would make the site worse than if a poster lost a little bit of reputation by having a wrong answer removed. I think that losing a couple points here and there from wrong answers helps the reputation score have more of a value than "a lot of people liked this". – baldPrussian Nov 22 '17 at 2:35
  • Charlie could post his own answer -> And make a note on top pointing to Paulina's answer, saying that it has a security flaw and that he is writing an alternative that does not have that flaw. And leave a comment under Paulina's answer saying that it has a security flaw and pointing to his own answer. After that, if he is correct, future votes will start correcting the situation. – Jan Doggen Mar 15 '18 at 9:11
3

I think there are a couple of reasons the stack exchange model can work well in the information and IT security profession:

  1. There is actually quite a small pool of experienced practitioners, so it is very easy to get a feel for how much each individual should be trusted in different areas, for example I know a great deal about the most active posters here and what areas I would expect them to be a guru or thought leader. I also think I'm not bad at reading when they are providing an answer from less experience.
  2. Enough people will read Charlie's answer and go and find out more in order to confirm it that I think that is not a worry.

Downsides:

  1. If Charlie is new and we don't know who he is, he may get ignored.
  2. Inexperienced people coming here from a search engine to sort a quick problem will only look at the top answer or two.

The individuals highly skilled in the information security profession are well known through their actions (publications, talks, code etc) and this industry does very well in identifying those who publicise themselves as security gurus but who aren't.

Early days for this beta - give it time, and your support, and it can only improve.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .