Downsides of migrating questions from high-volume sites

Question

I've noticed a number of questions that have been migrated from, say, Stack Overflow.SE to Security.SE -- where the migration has occurred after the question already accumulated a number of erroneous answers with many upvotes.

This is a problem, because IT Security has many fewer voting members than Stack Overflow. This makes it infeasible to clean up all the errors in the answers that got migrated along with the question: even if every IT Security member downvoted the erroneous answers, we'd still be outnumbered by the Stack Overflow votes. The theory of a SE site like IT Security is that the users of IT Security.SE are likely to be, on aggregate, knowledgeable about security, so votes from these users are likely to be a good way to capture the best knowledge about security. Unfortunately, this theory falls apart when most of the votes on a given answer are not from IT Security users, but have been carried over from some other community as part of a migration.

This problem occurs because Stack Overflow has so many more users than IT Security. It's not unusual for answers on Stack Overflow to get 25-50 upvotes, sometimes even more. In contrast, it's very rare to see even fantastic, stellar answers on this site get more than 10-15 upvotes. This means that, when questions are migrated from Stack Overflow after they already contain some answers, Stack Overflow's voting community is going to vastly outweigh the views of IT Security. Unfortunately, when questions are about security, Stack Overflow voters don't always have the security expertise that IT Security has.

I've noticed that this seems to degrade the trustworthiness of information on the IT Security site, for questions that have been migrated from high-volume sites like Stack Overflow. This is unfortunate: it would be nice if the IT Security community could take ownership and responsibility for the quality of answers on its site, but the way migration currently works interferes with that.

Don't get me wrong. We've gotten some great questions this way, and some of the answers that are migrated are quite good, too. Also, there's a possibility that migrated questions might attract new folks to come participate at IT Security. So there are lots of positive benefits to migrating these kinds of questions, and on the whole I expect it is a positive thing. But there are also some issues that seem to recur.

Does anyone have any ideas for ways to mitigate this systematic issue with questions migrated from high-volume sites over to us?

Edit (9/18): I was asked for examples. Here are a few:

• Is it OK to tell your password to an admin? [closed] - Migrated from Superuser, already had incorrect answers with dozens of upvotes. See @AviD's answer for the full details on this one.

• How can I avoid my password being harvested by key loggers from internet cafes? - Many problematic answers here. The highest-ranked answer recommends insecure practice (one-time passwords). The second-highest ranked is wrong and insecure. The third highest-ranked is wrong and insecure. The best answer has only 4 votes, compared to 26 votes for the top-ranked answer. Many low-quality/incorrect/confused comments (no way to downvote them).

• How secure is a Windows password protected zip file? - The top two highest-ranked answers are flawed.

• Is Steganography a safe method to store secret data? - The highest-ranked answer is incorrect. Many low-quality/incorrect/confused comments (no way to downvote them).

• Why do people think that this is bad way to hash passwords? - The highest-two ranked answers are fine. Some of the other answers are not good. Many low-quality/incorrect/confused comments, even including some of the comments with many votes (no way to downvote them).

• Now that quantum computers have been out for a while, has RSA been cracked? (Crypto.SE) - At the moment, the two highest-ranked answers are pretty confused. The best answers have only 0 or 1 votes at the moment. This was migrated only yesterday, so may be in flux (but I can't imagine Crypto.SE having enough users to downvote the poor answers or upvote the good ones past the poor ones).

• Is something wrong with this algorithm in terms of security? (Crypto.SE) - This one is less problematic. Many answers are fine. However, it is cluttered with many low-quality/incorrect/confused comments (no way to downvote them).

(This list is undoubtedly non-exhaustive. I couldn't find a way to use the search tool to find all questions that have been migrated to Security.SE, and I haven't been keeping a list of these questions. But this is probably reasonably representative.)

However, there are also many examples where migration has worked out fine (which I've omitted), and I realize in the grand scheme of things a half a dozen problematic questions may not be a big deal. And, @AviD makes a reasonable point that this has been an issue even for some non-migrated questions: e.g., XKCD #936: Short complex password, or long dictionary passphrase? attracted a lot of non-standard IT Security users, and the top-ranked answer is flawed (sorry, @Jeff, but it is), and there are many other low-quality/incorrect/confused answers and comments, even some with many votes -- so as @AviD correctly argues, this is not limited to migrated questions.

Answer 1

The general guidance is to migrate early if a question is offtopic. Generally, once answers have built up migration is discouraged - partly due to the issue you have mentioned, but also as it causes disruption with users on the other site losing rep etc. There are a few meta.so questions on this topic.

In general we do tend to see them come over without too many answers (ideally zero) but I think there will always be the odd one that gets answers and votes before being flagged and migrated here - I wonder how big a problem it is in real terms.

From an SEI perspective, you do want people to come here to find the 'right' answer and they tend to believe the votes, so there should be mileage in adjusting where the votes tally is not from specialists in the field.

An option is to delete the answer - which could be fine if it is entirely wrong, but probably not ideal ifthe answer has good points in amongst the bad.

Another is to edit the answer to make it right - this should be encouraged. I don't think we do it enough here on sec.se.

Answer 2

If a popular answer is evidently wrong, and you can't edit or delete it, you can flag it for moderator attention. There are only three of us and we don't know everything, so if you're asserting that an answer is incorrect please provide references to back that up.

Another solution to the issue is to grow our community until security has more users than StackOverflow, making it easier to correct such voting anomalies from the original site.

Answer 3

One possibility would be, when a question is migrated, to zero out the vote counts on all associated answers. In other words, the public vote counts would only aggregate votes cast after the migration, by users of IT Security.

However, this seems like a pretty substantial change, and it could easily have unanticipated unintended side effects, so I don't know if it would be desirable.

Answer 4

I totally agree - in principle.

There was a recent example of this - Is it OK to tell your password to an admin? - it was migrated over from SuperUser (the smallest of the trilogy sites) after 3 hours, and it already had 3 answers with dozens of votes, even though they were very wrong.

Because of the problem you present, and the apparent incapability of our community to compensate for the incoming wrongvotes, I took the extreme action of closing the original question, and having the OP re-ask the question here.

My action was unpopular with some of the original SU community, to say the least... But in extreme situations, sometimes it needs to be done.

---

That said, I've come to realize that the above solution is kinda pointless.

It's simple for a popular question - such as the recent XKCD hoopla - to draw in large numbers of non-security folk, who could swamp the results by crowdvoting the wrong answer, even though it originated here together with correct answers.
Case in point, the highest-voted answer on the XKCD question...

My point is that it goes back to the validity of the SEI model of crowdsourcing, as we've discussed since back in the early days of the beta - there is simply no guarantee that the highest voted - or accepted - answers are correct.
In general, we hope that the majority of active users here are security-minded, but that's not always the case.
You'll just have to make up your own mind about it...

Answer 5

I know this is not "5 minute fix" and maybe even next to impossible.

How about adding "weight" to answer instead of numbers of votes? Voting system from voter POV would be the same. However, when the voter votes, system would use his reputation to add weight to the answer (let say 1%). This way, when question migrates from other site, 20 votes (by 101 voter reputation on security.SE) would be easly pushed down (AviD alone could kill all of them).