Scheduled crawling
- What it does
- Each supplier site is crawled automatically on its own interval; the default is 4 hours.
- How it works
- The scheduler starts due sites every minute. An interrupted crawl resumes where it stopped when the service restarts.
- Problem solved
- Price or stock changes reaching the feed late.
- Technical detail
- Interval, concurrency and request delay are configurable per site; every crawl is listed with duration and changed product count.