You should take into account the factor of protection from high network activity from one (unique) IP address (Flood). when you are collecting the information from public sources. The most servers have such limitation.

What does it mean?

If so many queries turn into specified web-sire from one IP-address in short time, this web-site starts to block such queries for a specified tine. Error page or empty page will be shown in this case - it is a kind of protection from possible DDoS-attack from initial IP-address.

Why it is so important?

EmEx 3 is able to download up to 50 documents in one time. It can initiate the protective reaction from the web-site with such protection. The process of scanning can be reached as potential DoS attack and you can not to receive any information including search results from this server.

How to avoid this?

●Use mechanism of allocation by domain, when download flows will be distributed on different domains evenly

●Use Anti-Flood Filter

●Use the list of anonymous proxy-servers Every new query will be initiated from different IP-address in this case. Notice, the speed of download will depend on proxy-server's speed.