Blog

tl;dr Don’t write your own web application security scanner, it is too hard. Contribute to an existing project instead.

Every now and then I receive an email with this format: “I’ve used web scanning tool X for a while and it doesn’t work the way I want. I’m writing my own tool and would like you to help me with some ideas/pointers”. Usually I answer privately, but that doesn’t seem to be scaling, so here’s my public response to all of you.

Writing and maintaining a web application security scanner is a really hard task. The web is complex to crawl, vulnerabilities are difficult to find with low false positive rates, the applications are very different from each other, web frameworks do crazy stuff that will trigger ugly edge cases and you (just like I did seven years ago when I started w3af) don’t know enough about development to write a decent scanner.

The usual timeline is simple and I’ve seen it many times:

1. User doesn’t like the way scanner X implements a feature
2. User has free time
3. User starts writing his own scanner and usually succeeds in implementing the one feature he really needed
4. The new web application scanner only works on a small subset of sites, since it doesn’t know how to extract links other than the ones in a tags, or can’t handle broken HTML, or is too slow to be used on any site with more than a few hundred pages.
5. The creator of the new tool maintains it for six months
6. The project dies when the project lead finds more interesting things to do, finds a tool that did what he needed, changes jobs, etc.

There are positive things around this model, most notably the new project lead improved his development skills and potentially encouraged others to learn as well. Maybe he even wrote something interesting enough to be re-used in other tools.

On the other side, there are many negative things:

• The project will die, nobody will maintain it, your efforts will be forgotten and only a few will use your cool new feature
• Your project will be only usable against a small subset of the internet.
• You forgot to implement many features, which you don’t even cared about or knew existed!

And the saddest thing is that if you invest the same amount of time in implementing this new feature into one of the pre-existing open source tools, you’ll still improve your development skills, encourage others to learn but won’t end up with all those negative things I’ve mentioned.

By contributing to an existing project you’ll have the cool feature you needed and can rely on others implementing the rest of the complex and unknown features you don’t care about.

There are a few interesting open source projects doing web application scanning, in different programming languages and with different objectives, so please find the one that better fits your needs and contribute. Only start your own thing if you really know what you’re doing!