Blog

Building a better Clearinghouse

This month, StopBadware started a pilot project to explore what a new, expanded Badware Website Clearinghouse might look like. Our idea is to create a collaborative platform that aggregates and makes available extensive data and metadata about badware URLs and domains. That might include information from malicious URL feeds, reports from our community, results of scans against some of our partners' analysis tools, DNS and AS data drawn from public sources, and so on. The platform would power tools, services, and data reports designed to benefit our partners, website owners, and the broader Web ecosystem.

We're in the early stages of what we expect to be a three month pilot. So far, there are a lot of unanswered questions. Here are a few of the big ones:

  • What will the inaugural set of tools/services look like? So far, we're thinking of a data exchange API and a basic Web interface for searching the data.
  • Who will have access to the data? Those with the best data often have valid (and occasionally not-so-valid) reasons for not wanting to share their data openly. We want to offer flexibility that encourages broad sharing but allows more limited sharing where appropriate. So, we're imagining some sort of tiered permissions model.
  • What incentives will there be to contribute data? Two models I've seen used before are quid pro quo—you earn access equivalent to what you contribute—and "minimum threshold," in which you must contribute a certain amount, after which you get full access. Both of these could have value, but it would be nice to provide access to a broader audience than just those who have substantial data to contribute.
  • Which database platform should we use? Right now, our developer, Matthew, is experimenting with MongoDB (using Java for the middleware layer that will manage the data).

We'll do our best to blog periodically throughout the pilot as we refine our answers to these and many other questions. Meanwhile, we'd love to hear your suggestions and other feedback in the comments or via email (contact <at> ourdomain).