Do we want regular security challenges?

Question

I recently suggested in chat that we could organize recurring security challenges, similar to this proposal on RE.SE (thanks, @SYS_V). The idea is to periodically publish a security problem / puzzle that has to be solved by the community. That's fun, educational and gives users an extra reason to regularly return to the site. What are your thoughts on this?

• First of all, do we want this and would this format work well within the SecSE community?

• What would be an appropriate cycle? Weekly/monthly/...?

• Who creates the challenges? (E.g., we could let users with their own challenge ideas specify in which week they want to submit it, on a first-come first-served basis.)

• How would the challenges be posted? As regular questions / purely on meta?

• What do we expect from a good challenge?

Possible challenge subjects could be to identify a vulnerability in a given code snippet, recover an encrypted message, reverse-engineer an unknown protocol, filter evasion, etc. In my opinion, a good challenge comes with a concise, not too far-fetched scenario and doesn't rely on external resources (like a target server that has to be maintained). A challenge wouldn't always have to be particularly difficult but should carry a lesson - like highlighting an often overlooked type of vulnerability or teaching an important security principle.

What do you think, do we want something like this?

Answer 1

I would personally love to have such feature but my experience with this kind of activities says that it wouldn't be worth for SE developers spending time on building challenges as, at some point in the near future of the creation, just a few would still be playing around.

Don't get me wrong, I'd do as much as possible to support such idea and I think it would bring a greater level of learning for the site and us, the users, but I've already seen how this kind of projects are, at the end, used just by a few.

True is that the past does not set the future, so let's hope this goes on.

Answer 2

What about unofficial, irregular, community-developed security challenges?

I do not believe that Arminius was implying or suggesting that SE developers be tasked with implementing SecSE security challenge events. They likely lack the field-specific expertise required to create appropriate and worthy challenges anyway. However, some of the users of this site do possess the knowledge and the ability to create such challenges.

There is a way to take advantage of the decentralized, democratic format of SE sites in general as well as the relative popularity and level of activity of SecSE in order to create the equivalent of challenge events:

• anyone can ask a question
• anyone can answer a question
• at 75 reputation one can place bounties on questions
• at 300 reputation one can create new tags

Here is what I am proposing: any user with at least 300 reputation can at any time create a security challenge, present it as a question, tag the question with something like "Security Challenge" plus the tags specifying the subject matter of the challenge (e.g. "buffer-overflow") and then place a bounty on it to attract attention. In other words, unofficial security challenges could be developed and posted at any time, all the while remaining within the current framework of what is allowed by the SE site format. All the tools already exist.

An Example

Allow me to give a hypothetical example of what this would look like. Let's say that there is a user with ~10,000 reputation. Let's call this user "Infosec_Pro". User Infosec_Pro writes exploits like Dr. Seuss wrote children's books and is a bit bored. He also thinks it would be amusing to watch hapless wannabe hackxors struggle solving an entry-level shellcoding challenge. He decides to craft a simple vulnerable C program, posts the source code and a description of the challenge in a question titled "Security Challenge: Exploit this vulnerable C program", tags it with "Security Challenge", "Entry Level","buffer-overflow", "exploit" and whatever else is appropriate, and places a bounty of 50 on the question since it is entry-level difficulty. Infosec_Pro then sits back and watches as his question (challenge) shows up in "Featured" and garners a bit of attention.

Now another hypothetical user, "XX H4ck1ng L0rd XX", who thinks he is just the best because he is good at guessing his grandma's passwords, sees the bounty and says to himself "Yes! I have a chance to increase my rep from 100 to 150!" and is thus motivated to learn about BoF etc.

People who support the idea of having security challenges can upvote the challenge (question) so that Infosec_Pro gets his 50 rep back and attract more attention to the challenge, and people who publicly claim to support security challenges but prefer to just sit around and pooh-pooh other people's ideas ignore it.

Eventually, a few write-ups are posted as anwers and Infosec_Pro, having much enjoyed himself watching the struggles of others, chooses the write-up (anwer) that pleases him the most. The scrub who wins the challenge (has his answer accepted) not only gets rep from the bounty but also from whatever upvotes his write-up received.

Features

• bounty amounts reflect difficulty level. Bounties of 50 indicate entry-level challenges at one extreme and bounties of 500 are awarded for real brain-burners at the other extreme.
• Tags are used to distinguish challenges from conventional questions
• Tags describe challenge difficulty level
• "Security Challenge" is in the question title to make it blindingly obvious

Pros and Cons

Pros:

• Total flexibility. Anyone can create any kind of challenge at any time. There could be crypto challenges, bof challenges, Windows challenges, Linux challenges, web app challenges etc. In other words, something like this could begin this moment while you are reading this line. Yes. Right now. Immediately.
• Unilateral action is possible. Even if every person who read this said to themselves "My word! Whoever proposed this is an utter dunce!", any individual user could exercise their free will to create and post a challenge question if they so chose (given that they abide by the rules defined in the help center).
• Tangible rewards for both the challenge developer and the challenge solver. The challenge developed receives upvotes based on the quality and popularity of the challenge, and the solver gets upvotes base on the quality of their write-up/answer

Cons:

• There is no way to prevent someone with high rep from solving entry level challenges and ruining it for everyone
• bounties expire after a week
• this scheme relies on interest. If no one is interested in developing and posting challenges, there won't be any challenges to solve. If no one is interested in solving challenges, it discourages people from posting challenges in the first place
• this scheme conflicts with the status-quo and may be poorly received by moderators and people with 100,000,000,000 reputation who have had an account on the site since 1983.
• it could just be a bad idea. But if I knew how to create a decent challenge, I would post it in the way I described simply to prove that it can be done