malerisch.net

specialk is a very simple tool. It has been created because standard web crawlers are not able to find "all" directories/pages in a web site.

wget -r is an example of web crawler that is not able to pick up hidden/not linked/misconfigured directories/pages.

specialk executes a list of entries (contained in a separate dictionary file) and for each attempt reports different HTTP headers such as HTTP status code and message, Page Title, Content Type.

This information is useful as some web servers are configured to return a 200 - OK code even if a particular directory and/or page does not physically exist.

This will help the user to better understand the reactions of the web server, the structure of the web site and if that particular directory and/or page really exists.

The scan result can be easily redirected to a text file.

Usage:

specialk -url http://www.sitetotest.com -dic dictionary_file.txt

If you want to redirect the scanning result to a file then:

specialk -url http://www.sitetotest.com -dic dictionary_file.txt > scan_result &

Requirements:

In order to run specialk, some perl libraries are needed:

HTTP::Status
WWW::Mechanize
HTML::TreeBuilder 3.0
HTTP::Request::Form
Getopt::Std
Getopt::Long

You can download them from CPAN.

Dictionary file

A dictionary file is available with a list of common directories and pages (regardless of web servers and web technologies). Please feel free to contribute with other entries and send it back to me.

Download: specialk.pl [ here ] + Folder Dictionary [ here ]

- permalink -