Add an Entire Website (or Sitemap)

One of our most powerful features, Add entire website crawls all of the pages on your website and allows you to scrape content from hundreds of pages at once, the fastest way to build your AskAI

Site crawl is a feature that allows you to add an entire website at once just by adding a single page.

Upload one "home" page and we have a bot “crawl” through your website to find all the pages underneath it and add them to your AskAI.

Or follow our step-by-step instructions below

1. If it is an existing AskAI, open the Dashboard: myaskai.com/ask-ai-dashboard, otherwise skip to step 3

2. Click "Edit content" on the AskAI you'd like to a website to

Screenshot of: Click "Edit content" on the AskAI you'd like to a website to

3. Click "Add entire website"

Screenshot of: Click "Add entire website"

4. Click the "https://myaskai.com" field.

Screenshot of: Click the "https://myaskai.com" field.

5. Paste in the website you want to crawl.

6. Click "Search website". We'll then start looking for webpages, you'll get an email when we've finished finding them so you don't need to wait around.

Screenshot of: Click "Search website". We'll then start looking for webpages, you'll get an email when we've finished finding them so you don't need to wait around.

You can also exclude pages from your site search if there are certain subpages you don't want to include (and eat up your content limits).

7. Optional: If there are any pages you don't want to add to your AskAI then just click the Trash can icon next to them.

Screenshot of: Optional: If there are any pages you don't want to add to your AskAI then just click the Trash can icon next to them.

8. Click "Add webpages to AskAI" and they'll start adding right away. You'll get an email notification when they have all been added (just in case there are a lot).

Screenshot of: Click "Add webpages to AskAI" and they'll start adding right away. You'll get an email notification when they have all been added (just in case there are a lot).