Highlights from five years of StopBadware work

Posted on September 16, 2015 - 11:31 by ccondon

The Cambridge-based StopBadware team is signing off this week after more than five years of community building and collaboration with some of the best people in the security business. As we turn full operations over to Dr. Tyler Moore and his excellent team at the University of Tulsa, take a look at some of the highlights of our work these past five-plus years. 


What's next for StopBadware in Tulsa

Posted on August 20, 2015 - 16:27 by ccondon

We asked Tyler Moore, StopBadware's research advisor and the boffin who's taking over our core operations, to expand on his plans for the organization in Tulsa and to throw in some 90s references. He obliged. 

Dr. Tyler Moore on the new version of StopBadware

Recently we announced that StopBadware is transferring operations to the University of Tulsa. In today's blog post I will fill in some more details on this exciting new chapter of the organization. Some things will change as a result, but our non-profit mission to make the web safer will remain.

First, let me tell you a bit about myself and my history with StopBadware, which I hope will go a long way to help solve the mystery of how StopBadware has ended up in Tulsa. (Hint: it's not because of Hanson. And I promise the circumstances are happier than when Chandler was transferred there after sleeping in a meeting on Friends.)

I first began interacting with StopBadware in 2008 while I was a postdoctoral fellow at Harvard's Center for Research on Computation and Society. I wanted to engage with StopBadware due to my research interests in cybercrime measurement. We collaborated on several projects, one of which culminated in a 2012 paper describing an experiment that demonstrated the impact of transmitting detailed notices in cleaning up websites distributing malware. The paper was co-authored by Marie Vasek, who is now my Ph.D student and Research Scientist at StopBadware.

Since 2013, StopBadware has been closely collaborating with my research team under Marie's supervision. The website testing intern has regularly been an undergraduate student I have recruited from my courses. Last year, I became StopBadware's research advisor, further formalizing my long-term involvement with the organization.

When StopBadware's board of directors decided earlier this year to move away from being a stand-alone 501c3 non-profit organization, I volunteered to bring StopBadware back to its roots in academia. StopBadware will become a program operating within the Security Economics Laboratory at the Tandy School of Computer Science at the University of Tulsa, where I cut my teeth as an undergraduate security researcher and where I recently joined the faculty.

This change in organization will bring several benefits. One is that it should greatly reduce operating costs, as I will be serving as Director pro bono, and we can share other overheads with an existing institution. Another is that we will be able to continue to serve as a true non-profit—something that in the eyes of staff and community is both unique and essential in this space.

The new StopBadware will concentrate on the core competencies that we offer. First, we will continue the testing and review program, in which anyone can request independent review of URLs blacklisted for malware by StopBadware's data providers. Second, we will continue the Data Sharing Program (DSP), in which StopBadware serves as a trusted broker for community-contributed feeds of security datasets. Third, StopBadware's research mission will be expanded. We plan to more extensively mine the data contributed to the DSP and other sources. Finally, we intend to greatly expand the publication of data related to web-based badware. Our aim is to provide even greater transparency into the fight against web-based malware, so that we might more accurately track progress, highlight accomplishments and encourage improvements on part of the community.

We still need your help, in terms of contributing data, services and financial assistance. Donations will still be required in order for StopBadware to continue thriving in the years ahead. If you are interested in supporting StopBadware as we move onto the next chapter, please get in touch by emailing me at

- Tyler

Visualizing eight years of independent reviews

Posted on August 14, 2015 - 12:24 by ccondon

StopBadware has been performing independent reviews of websites blacklisted by our data providers for more than eight years. As we've explained in the past, a manual review done by our staff is not always necessary: if a webmaster requests a StopBadware review of a site on Google's Safe Browsing blacklist, the first step in our review process is an automated request for Google to rescan the site in search of malicious code. If Google's automated systems don't find anything suspicious, that site will come off Google's blacklist without our ever having to touch it. When Google still finds malware, or when one of our other data providers is the blacklisting party, one of our website testing team uses a variety of tools to scour the site for malicious code and other bad behavior.

As our home page proclaims in red, we've helped de-blacklist more than 171,000 websites since 2007. Before we shutter operations as an independent nonprofit next month, we want to give our community a better idea of what goes into that number. 

Since we started collaborating with Google, and later ThreatTrack Security and NSFocus, we've performed 53,167 manual reviews. We've also processed an additional 188,149 review requests that were resolved automatically thanks to our automated integration with Google. Those aren't all unique requests, so combining them doesn't yield an accurate number. Here's what all those review requests look like over time:

Why the decline? 

You'll undoubtedly notice that we received many more review requests early on than we do today. Better security awareness, wide availability of relatively low-cost security tools, and default use of things like Webmaster Tools all contribute to the decline we've experienced in review requests. We also have better ways of detecting and weeding out abusive requests than we used to. 

Unfortunately, something else that's contributed to the decline in review requests is malware distributors' widescale use of stealthier, more targeted methods like malvertising. When a resource is compromised only very briefly (e.g., through an infected ad network), even when blacklist operators are able to detect the infection and warn users away, the compromise is often resolved too quickly for StopBadware's Clearinghouse to reflect that the resource was ever blacklisted. Generally speaking, if something is blacklisted for fewer than six hours, we won't have a record of it in our Clearinghouse. On the one hand, this is good news, in that we want blacklists to operate as narrowly as possible to maximize user protection while minimizing penalty to site owners; on the other hand, this is bad news, in that malicious actors are able to effectively utilize powerful technologies to spread malware in ways that are difficult to detect and counter. 

What's not included in this data? 

What you don't see in this chart is the tens of thousands of URLs we've reviewed in bulk for web hosting providers, AS operators, and other network providers over the years. We've worked with everyone from dynamic DNS companies and bulk subdomain providers to small resellers and abuse departments at big companies to clean up malicious resources on their networks and help remove them from blacklists. The majority of this process is manual, and because it's initiated based on trust and human communication instead of by clicking a button, bulk review data isn't reflected in our public review data. 

StopBadware's review process will continue to operate normally during and after our operations transfer to our research team at the University of Tulsa. Thanks to our research scientist, Marie Vasek, for putting this data together!