Keyword Crawler: Online website optimization tools: explore and see your site like webengines. Count and display broken and internals links. Get Google pagerank for each file and generate a SiteMaps file in XML format. Keyword Crawler could also calculate word density in pages and display the metas and observes the directives of the robots.txt and 'robot' meta tags.
What is new in version 1.1:
Support now robots.txt files, Content-Type HTTP header, META robot tag and some minor bugs corrections
Soft411.com periodically updates pricing and product information from third-party sources, so some information may be slightly out-of-date. You should confirm all information before relying on it.