MiggiBot Crawler Robot in A1 Software

MiggiBot is the website crawler robot powering a range of tools including our A1 Crawler software.

Features in MiggiBot Robot Crawler Engine

Can extract vast amounts of meta data about all pages in a website. File sizes, download times, server response codes etc.
Has various tools for analyzing keyword density. Can use stop words and similar filters.
Can generate sitemaps of websites with the crawler supporting advanced filters.
Can check, validate and store links, redirects and even source from data (e.g. images and frames).
Can calculate the internal importance of all pages within a site.
Can generate CSV and XML files for all crawled and derived data.
Has website download and links conversion functionality.

Future MiggiBot and Crawler Software Development

Continue to develop and polish the engine.
Some very exotic and secret ideas.

Products Where MiggiBot Crawler Engine is Used

Name	Acronym	Link	Description
A1 Sitemap Generator	A1SG	link	HTML/XML/Image/Video sitemapper
A1 Keyword Research	A1KR	link	Keyword SEO tools and analyzer
A1 Website Analyzer	A1WA	link	Technical site, SEO and link checker
A1 Website Download	A1WD	link	Website downloader for offline browsing
A1 Website Scraper	A1WS	link	Extract and scrape website data
A1 Website Search Engine	A1WSE	link	Add a search solution to your website
TechSEO360	TS360	link	All-in-one solution merging functionality from multiple A1 tools.

For Developers and Programmers

Having easy-to-parse XML data can be useful for companies building custom research tools.

We are also currently considering if and how to expose our crawler engine for developers. Feel free to contact us if you are interested.

Fun facts About Miggi / MiggiBot

The name is a contraction of the following words Micro-Sys, Microsys, microbot, microbotic, robot, bot, big etc.

Microsys | help | previous | next

Microsys develops desktop software, mobile apps and websites.This page is part of the shared help documentation across many of our tools.

This help page is maintained by Thomas Schulz

As one of the lead developers, his hands have touched most of the code in the software from Microsys. If you email any questions, chances are that he will be the one answering.