|
|
Download and Archive Ask and Answers Websites
If a website full of important questions and answers is about to close, you may want a local copy for your own use.
Do You Have Permission to Download Questions and Answers
As we are not lawyers, we can not give legal advice, and hence we do
not know if you are allowed under e.g. fair use to download such content, even if just for your own usage.
You may want to either:
You may want to either:
- Ask for explicit permission from the website with the questions and answers.
- Contact a lawyer and seek legal advice before you proceed.
How to Configure A1 Website Download
-
Scan website > Paths
- Set website domain address and/or directory path to the same root domain address the content. (In this example questions and answers are located at http://answers.example.org.)
- In Beyond website root path, initiate scanning from paths
add some paths that lead to he content you want, e.g. search phrases http://answers.example.org/search-results?search-for-questions=example
Note: The file download path of this URL is also the best starting point to view and surf the downloaded content offline.
-
Scan website > Analysis filters
- In limit analysis of internal URLs to which match as "relative path" OR "text" OR "regex" in list
add the following limit-to filter to limit which page URLs get and analyzed for URLs.
In this case: ::search-results\?search-for-questions=example(&page=[0-9]{1,4})?
Using the above limit filter A1 Website Download will crawl all search result pages that up till and including result page 9999.
- In limit analysis of internal URLs to which match as "relative path" OR "text" OR "regex" in list
add the following limit-to filter to limit which page URLs get and analyzed for URLs.
In this case: ::search-results\?search-for-questions=example(&page=[0-9]{1,4})?
