Musings on H1-2010: The Paranoids Bug Bounty World Championship

Musings on the Paranoids and HackerOne Delivery of H1-2010:
The Paranoids Bug Bounty World Championship

H1-2010 is in our rearview mirror, and we have spent time reflecting on what the experience taught us.  Opening up our bug bounty program to a global, virtual live hacking event was a journey that began for the Paranoids and HackerOne back in 2017/2018, as described in our pre-H1-2010 blogpost. In 2020, it became a story of inclusion, great research, connections, camaraderie, and endurance. We had an incredible time and lived to tell the tale!

H1-2010 comprised three events and brought our event total in just three years to thirteen. We do post-event analysis to improve our prediction models for planning, estimating, scoping and scaling future events. We have a lot of data to work with specific to our environment, our apps, our teams, and the hackers who participate. This event though, as soon as we opened the door to the idea of self-registration for the opening round, caused us to realize that the models we had developed probably wouldn’t work. We had no idea if we would get 80 people to sign up or if the event would draw all 800,000 hackers who are registered on the HackerOne platform. Without knowing how many hackers would sign up, we could not estimate the number of participants. That also meant we couldn’t predict how many hours they might spend hacking, the type of testing they would be skilled at performing, or how many bug reports we might see. We took the leap anyway and leaned on our experience from past events to make decisions that we believed would make for an exciting event for ourselves and the hackers.

We set out to stay ahead of COVID-19 back in January, as we eyed coming together for H1-415 in San Francisco.  At that event, we decided to keep our hackers, partners, and employees out of harm’s way for the remainder of the year and put a moratorium on travel. We pivoted to our first ever virtual live hacking event (in April) and immediately began planning for the penultimate hacking event - a global, inclusive, anyone-can-participate, epic event.  Together, the Paranoids and HackerOne sit here today excited and awestruck that we all came together for this industry first.

After announcing the event on our blog and on Twitter, we also leveraged some of our fantastic friendships to promote the event. @stokfredrik was excited to host Chris Holt, Principal Bug Bounty Operations Lead,  on his “Bounty Thursday” news series. @nahamsec, who has been a friend of our program for a very long time, brought the Paranoids bug bounty team onto his weekly streams to talk about the event and helped us host a “How to Get Started with Bug Bounties” half-day seminar session. Sean Poris, Director of Product Security and Assurance, also talked to the Absolute Appsec team before the event to share our thoughts.

Over 3400 hackers registered for the H1-2010-Open round, and when it came time to begin hacking more than 1200 hackers went to work on the event scope (a subset of our public scope).  When the dust settled a week later, 243 hackers submitted at least one report, and 91 hackers ultimately ended up with at least one bounty (including two who did not submit a report but were awarded as collaborators).  Here is where things got even more interesting for us.  32 of those 91 were completely new to the Verizon Media bug bounty program.  Before the event started, we were curious how the new format might drive new talent into our program, so we were really delighted to see so much talent experience success in the first round.  The question we then asked ourselves was, ”how would these new hackers fare as they moved deeper into the competition?”

The answer was...great!  We expanded the second round slightly beyond the originally planned 50 to 63 due to a huge tie at the bottom. Out of the 63 hackers in the round, 14 of the 32 new hackers once again showed off their research prowess and found a bug.  And in our final round, including only the top 25 performers overall, six of the new hackers earned a bounty.

The hacking community showed up, responded to the challenge of H1-2010’s three rounds, and new hackers on the Verizon Media bug bounty program enjoyed success in all three rounds.  The community embraced the new hackers, as chats and engagement activities throughout the event helped us all get to know each other a little better.  We continued doing our Hacker of the Day (HOTD) shout outs on Twitter, recognizing the best bug or research on each day of the event, and sometimes just to recognize fantastic partnership.  Of those tweets, three went to new hackers on Verizon Media, which was awesome to see and fun to recognize our hackers throughout the event on social media.

This was a grueling event.  The typical HackerOne and Verizon Media  live hacking event lasts  2-3 weeks and has a single set of scope and awards. It culminates with  two  days of in-person activities including networking opportunities, live hacking, and other engagement options.  The in-person interactions feed us energy throughout live hacking events.  Relationships are born and cultivated during face-to-face events.  We spend months leading up to each event planning it, and we spend weeks afterwards with post-event activities. We announced H1-2010 on September 1st and live hacking kicked off  on September 22nd with closing ceremonies on October 30th.  There was barely a week between each of the rounds, each of which was delivered as a full event. The hackers, Paranoids, and HackerOne triagers all had to dig deep to make it to and through the final round. We wrapped up with engaging closing ceremonies thanks to HackerOne and Luke Tucker. Now,  here we are, analyzing the mound of data we received over the course of the H1-2010 odyssey.

What did we learn?  Our efforts to continually challenge ourselves and uplevel our live events continue to be a worthwhile pursuit.  Hacker sentiment was generally strong that:

• the event was fun
• many learned something
• regardless of bounty outcomes, it was time well spent

We learned that we can build relationships with the community, even with the challenge of a purely virtual format and at the scale of a global event, and that the focus of bug bounty live hacking events drives greater program participation and results aligned with protecting our customer, consumer, and corporate data. A global event can indeed be done! 

We also learned that tweaking our daily office hours format would lead to deeper connections. Adding to office hours, we chose to do an AMA (Ask Me Anything) with our bug bounty team leads, Chris Holt (flyingtoasters) and Mark Litchfield (bugbountyhq), hosted by top hacker Tommy Devoss (dawgyg). That helped further connect our bug bounty program to the community in a meaningful way. Event participants asked questions real time with unfiltered answers and lively discussion. Aside from the AMA with Mark and Chris, the Paranoids hosted other events to bring everyone together as well:

• a pictionary competition,
• a multi-round trivia challenge pitting Hackers vs Paranoids that introduced both parties and tested knowledge  of Verizon Media, bug bounty and the history of the security industry,
• a final  AMA with Verizon Media’s CISO, Sean Zadig, hosted by our Director of Product Security & Assurance, Sean Poris, and
• other fun events hosted by HackerOne.

Jumping to awards and prizes, can we talk about the (coveted) rings for a moment? In October 2018, when we were finishing the analysis of H1-5411, we started designing a team-based tournament-style event. The resulting event, H1-212 (2018) was not actually a tournament, but we stuck with the idea that an event like this that pits the best of the best in teams against each other should have something like a Superbowl ring attached to it. Thus, the first Bug Bounty Showdown rings were born. A year later in November, we brought an updated design to Los Angeles for H1-213 (2019) for the second Bug Bounty Showdown and awarded rings to the top three teams.

Fast forward to August 2020. We were working on a global tournament that was shaping up to have thousands of participants. Can you think of a better place to crown a Paranoids Bug Bounty World Champion with a ring that shows off that unique title? This year, H1-2010 featured only 3 rings which we happily awarded to the hackers who finished the third round (h1-2010-final) of the event in 1st, 2nd and 3rd place on that last leaderboard. These folks went through the tournament with us and the rest of the hackers, fighting against top hackers around the world to take home the most bounties through three back-to-back full-scale virtual live hacking events. 

The Paranoids, along with HackerOne,  thoroughly enjoyed crowning the first ever Paranoids Bug Bounty World Champion.  We look forward to what 2021 has to offer and the journey as one big bug bounty community.  

P.S.
You might have noticed that we like to release new scope into our public bug bounty program after events; this event will follow that pattern.