Linear algorithm (FIFO)

All URLs will be processed in FIFO order. This means, link 2 will be processed after link 1.

Distributed by domains

Download threads will be distributed by domains. This can help avoid ban for extreme network activity.

Maximum download threads per domain

Set maximum download threads per domain.

Domain detection (level)

Set level for domain name to group all subdomains as one domain.

Skip direct links to search engines to avoid ban on extreme activity

If processing document contains direct link to search engine (Google for example), then EmEx 3 will not add this link to download queue. This can help to avoid ban your IP from search engine

Ignore 404 HTTP errors when download

If web server return 404 error then ignore error and parse received result.

Timeout for download

Set timeout in seconds.

Attempts to download document on timeout issue

Maximum download attempts. This setting affect on timeout error only.

Custom limit for parsing threads

By default EmEx 3 will launch up to 50 parsing threads. This can affect high memory/CPU usage on high speed bandwidth internet connection. Use this settings to avoid this problem.

Replace user-agent

By default EmEx 3 works as Internet Explorer 6. You can change this value.

Random, but remember for domain/host

EmEx 3 will use random user-agent string for every domain or host.

From list

Predefined value.

Custom (user defined)

User defined User-agent string