That is why you put it on a VM behind a QoS router. so you can manage both bandwidth and CPU.
Besides, you never run that sort of thing on your desktop... Bad idea.
Thu Feb 18 2021 09:57:53 EST from darknetuser
Also, I have been trying Yacy. I want to like it, but it sucks. It is a distributed search engine. The idea is awesome, but it rapes the CPU, gets frozen when you least expect it, and sucks for privacy because it does all the crawling from your computer. So, if you search for gnome cartoon gay porn, chances are it will start fetching pages related to the subject to index them, and everybody in the world will know you like gay gnomes.
Meant to include a screenshot. and a 2nd statement : "or you just limit the bandwidth at the vm level"
2021-02-20 21:14 from IGnatius T FoobarA true, traditional search engine with spiders and crawlers, under
Apache license. Surprisingly complete and bullshit free.
This of course leads to the obvious next step: a decentralized search
engine?
Gigablast's engine comes with clustering capabilities and a load balancer proxy.
If you want a truly distributed search engine, there is Yacy, but it sucks, as mentioned above.