Anyone know where to find a program that scans for anything on a website?

Ex: http://www.testing.com if that website got a subdomain called http://test.testing.com it would be listed...

Or anything reminding of such program.. Something that just scans the site for all information about the site.. html, htm, asp files aso...

As a matter of fact there is a tool that gives out as much information as possible. http://serialcoders.sytes.net/Download/DNSLookup/index.html to get DNS Lookup, coded by our very own SilentRage. Someone asked about subdomains (test.testing.com) and SR informed us that there is nothing in the DNS protocol to allow for discovery of subdomains. So I think you're outta luck there. But as far as getting lots of info on a site, that tool will do the trick.

[Insert by SilentRage: I was wrong earlier, you CAN find subdomains for a given domain by doing DNS Transfers. If you want help on that, create a new topic asking how.]

Its impossible for a program to list hostnames from a url (as per your example) unless they were actually listed on the webpage...

The best thing that you can use is SR's DNS Lookup as pergesu listed, BUT it does not add the feature to list all hostnames associated with the domain since its ultimately impossible.

He could code or download a web spider to craw the site for all links. Then alphbatise them, and delete dulitaces, then delete any that do not start with the first 3 characters of the domain you are intrested in.


Of course it would require some work, but I think it can be done. Anything can be done if a coder puts thier mind to it. Know any languages?

I believe he is asking for a web spider. Try searching for one that may or may not have the funcitionality you are asking for at http://www.download.com

If you can't find one that has that functionality, use SR's little tool.