16

I've noticed a number of questions that have been migrated from, say, Stack Overflow.SE to Security.SE -- where the migration has occurred after the question already accumulated a number of erroneous answers with many upvotes.

This is a problem, because IT Security has many fewer voting members than Stack Overflow. This makes it infeasible to clean up all the errors in the answers that got migrated along with the question: even if every IT Security member downvoted the erroneous answers, we'd still be outnumbered by the Stack Overflow votes. The theory of a SE site like IT Security is that the users of IT Security.SE are likely to be, on aggregate, knowledgeable about security, so votes from these users are likely to be a good way to capture the best knowledge about security. Unfortunately, this theory falls apart when most of the votes on a given answer are not from IT Security users, but have been carried over from some other community as part of a migration.

This problem occurs because Stack Overflow has so many more users than IT Security. It's not unusual for answers on Stack Overflow to get 25-50 upvotes, sometimes even more. In contrast, it's very rare to see even fantastic, stellar answers on this site get more than 10-15 upvotes. This means that, when questions are migrated from Stack Overflow after they already contain some answers, Stack Overflow's voting community is going to vastly outweigh the views of IT Security. Unfortunately, when questions are about security, Stack Overflow voters don't always have the security expertise that IT Security has.

I've noticed that this seems to degrade the trustworthiness of information on the IT Security site, for questions that have been migrated from high-volume sites like Stack Overflow. This is unfortunate: it would be nice if the IT Security community could take ownership and responsibility for the quality of answers on its site, but the way migration currently works interferes with that.

Don't get me wrong. We've gotten some great questions this way, and some of the answers that are migrated are quite good, too. Also, there's a possibility that migrated questions might attract new folks to come participate at IT Security. So there are lots of positive benefits to migrating these kinds of questions, and on the whole I expect it is a positive thing. But there are also some issues that seem to recur.

Does anyone have any ideas for ways to mitigate this systematic issue with questions migrated from high-volume sites over to us?

Edit (9/18): I was asked for examples. Here are a few:

(This list is undoubtedly non-exhaustive. I couldn't find a way to use the search tool to find all questions that have been migrated to Security.SE, and I haven't been keeping a list of these questions. But this is probably reasonably representative.)

However, there are also many examples where migration has worked out fine (which I've omitted), and I realize in the grand scheme of things a half a dozen problematic questions may not be a big deal. And, @AviD makes a reasonable point that this has been an issue even for some non-migrated questions: e.g., XKCD #936: Short complex password, or long dictionary passphrase? attracted a lot of non-standard IT Security users, and the top-ranked answer is flawed (sorry, @Jeff, but it is), and there are many other low-quality/incorrect/confused answers and comments, even some with many votes -- so as @AviD correctly argues, this is not limited to migrated questions.

  • can you update this to provide specific examples after the first paragraph? – Jeff Atwood Aug 18 '11 at 12:23
  • @Jeff, good idea! Done. I've edited it to provide examples. – D.W. Aug 18 '11 at 23:00
  • This might be more of a bandaid solution, but is it possible to increase the relative weight of local votes based on the ratio of active users between the local and source site? – dexgecko Mar 26 '18 at 22:38
8

The general guidance is to migrate early if a question is offtopic. Generally, once answers have built up migration is discouraged - partly due to the issue you have mentioned, but also as it causes disruption with users on the other site losing rep etc. There are a few meta.so questions on this topic.

In general we do tend to see them come over without too many answers (ideally zero) but I think there will always be the odd one that gets answers and votes before being flagged and migrated here - I wonder how big a problem it is in real terms.

From an SEI perspective, you do want people to come here to find the 'right' answer and they tend to believe the votes, so there should be mileage in adjusting where the votes tally is not from specialists in the field.

An option is to delete the answer - which could be fine if it is entirely wrong, but probably not ideal ifthe answer has good points in amongst the bad.

Another is to edit the answer to make it right - this should be encouraged. I don't think we do it enough here on sec.se.

  • Thanks, that makes sense. However, my experience I feel like I've seen quite a few that come over with many answers and many votes. Perhaps I'm just overly sensitive to this phenomenom. – D.W. Aug 18 '11 at 9:06
  • Regarding editing answers: What is the etiquette? If an answer says "do X" but "do X" is bad advice, I've assumed that it would not be appropriate to edit it to say "don't do X"; am I wrong? I've normally viewed editing for correcting, e.g., typos and such -- but perhaps I've had the wrong view. Any favorite references to where I can read more about the recommended etiquette on editing? – D.W. Aug 18 '11 at 9:07
  • 1
    meta.so has a Q on this - meta.stackexchange.com/questions/5408/… and an asoociated one - meta.stackexchange.com/questions/21446/… – Rory Alsop Aug 18 '11 at 9:33
  • 2
    @D.W. also this question on our own meta - meta.security.stackexchange.com/questions/246/…. – AviD Aug 18 '11 at 14:28
  • @AviD - oh, yeah - that one too:-) – Rory Alsop Aug 18 '11 at 14:54
  • @AviD, thanks for the reference! Actually, that explains some of the limitations of editing. The problem is that often the poster doesn't realize the limitations of his/her knowledge; in my experience, adding a comment rarely helps and rarely leads the poster to delete their answer after it has accumulated many votes. Downvoting doesn't help (because one downvote is drowned out by many others from the other site). And it would violate etiquette to completely change the meaning of the post, so it may not be appropriate to fix a post that's wrong. – D.W. Aug 18 '11 at 22:29
  • @Rory, thanks for the excellent reference! Very helpful. In the future, maybe I'll try editing problematic answers to include a warning that it might be problematic, as suggested by the post you referenced. We'll see if that's helpful, or if people scream bloody murder. :-) – D.W. Aug 18 '11 at 22:54
4

If a popular answer is evidently wrong, and you can't edit or delete it, you can flag it for moderator attention. There are only three of us and we don't know everything, so if you're asserting that an answer is incorrect please provide references to back that up.

Another solution to the issue is to grow our community until security has more users than StackOverflow, making it easier to correct such voting anomalies from the original site.

  • 2
    riiiight.... +1 for crazy optimisim :) – AviD Aug 18 '11 at 14:29
1

One possibility would be, when a question is migrated, to zero out the vote counts on all associated answers. In other words, the public vote counts would only aggregate votes cast after the migration, by users of IT Security.

However, this seems like a pretty substantial change, and it could easily have unanticipated unintended side effects, so I don't know if it would be desirable.

1

I totally agree - in principle.

There was a recent example of this - Is it OK to tell your password to an admin? - it was migrated over from SuperUser (the smallest of the trilogy sites) after 3 hours, and it already had 3 answers with dozens of votes, even though they were very wrong.

Because of the problem you present, and the apparent incapability of our community to compensate for the incoming wrongvotes, I took the extreme action of closing the original question, and having the OP re-ask the question here.

My action was unpopular with some of the original SU community, to say the least... But in extreme situations, sometimes it needs to be done.


That said, I've come to realize that the above solution is kinda pointless.

It's simple for a popular question - such as the recent XKCD hoopla - to draw in large numbers of non-security folk, who could swamp the results by crowdvoting the wrong answer, even though it originated here together with correct answers.
Case in point, the highest-voted answer on the XKCD question...

My point is that it goes back to the validity of the SEI model of crowdsourcing, as we've discussed since back in the early days of the beta - there is simply no guarantee that the highest voted - or accepted - answers are correct.
In general, we hope that the majority of active users here are security-minded, but that's not always the case.
You'll just have to make up your own mind about it...

1

I know this is not "5 minute fix" and maybe even next to impossible.

How about adding "weight" to answer instead of numbers of votes? Voting system from voter POV would be the same. However, when the voter votes, system would use his reputation to add weight to the answer (let say 1%). This way, when question migrates from other site, 20 votes (by 101 voter reputation on security.SE) would be easly pushed down (AviD alone could kill all of them).

  • The problem that you have is this. Let's assume that the impossible happened and AviD made a mistake. Let's assume this impossible situation even more impossible, and pretend that I caught his mistake. What then? – Everett Dec 8 '15 at 18:50
  • @Everett And? It would be the very same situation as user made a mistake like on every other question - other (senior) users would push the answer down. This is three years old topic, from the time when site was very young and if I remember correctly, we had several questions migrated from other sites which were very popular (top of front page popular) with absolutely wrong top answer(s). After they came to this site, there was no way the correct answer could ever jump in front of wrong ones. – StupidOne Dec 8 '15 at 19:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .