Spam blocking in htaccess

[Rickf1985]

They added the bot to my htaccess for me. And cloudflare caused me more problems than it solved once I added it before. I actually think it is designed to add issues to your system just so you have to buy all the paid extras to solve those problems. When I mentioned this to the Kualo people they did not disagree and said since I had had issues with it in the past then do not worry about using it. That sounds to me like a veiled version of "yea, we know all about that" without actually saying it and ruffling feathers. So I will not be using cloudflare. I want to stick with something I can get help with and learn how to use without have to also keep learning new tech at the same time.

[Rickf1985]

Well, I just went into my file manager and looked at the htaccess file and that bot was NOT added in so I added it.
The other processes that they identified as problems were as follows, Are these something you recognize and can advise me on?

Upon further checking, we can see the following processes, which are causing this high CPU usage:

lsphp:/home/g83800/public_html/viewtopic.php

lsphp:/home/g83800/public_html/memberlist.php

[Administrator]

Its got to be bots masquerading as real users, or as the majority seem to be from phones a rogue app that they have persuaded someone to install that does the indexing remotely. As far as I know there is nothing that can be done about them. From time to time I get them (as no doubt all websites do)

At this precise moment I have 48 guests on my main board, for example:-

screenshot_206.jpg

All from different ip's which have been hidden to comply with GDPR

[Rickf1985]

Starting to make some progress I think. Not so much in winning the war but winning some of the battles. It appears most of my problems are bots. New ones every hour it seems. Tint-bots this afternoon and Babbar-tech/crawler for the last few days. Kualo did not mention the Babbar but when I checked it it came up as a spam site out of France. They did mention some of the things that you had suggested a while ago about capcha and setting up better bot detection through php. Here is the e-mail they sent me. What are your thoughts on it? I have not even had time to do the update yet.

Here are also some steps you can take to protect your phpBB forum from bad bots: 1. Implement CAPTCHA challenges on registration and login pages to distinguish between human users and bots. 2. Explore and install phpBB extensions that are designed to combat spam and bots. For example, the "Anti-Spam ACP" extension provides various anti-bot features. 3. Regularly update your extensions to ensure they are effective against the latest bot threats. 4. Keep your phpBB forum software, extensions, and themes up to date to ensure you have the latest security features and patches. 5. Regularly monitor forum activity and access logs for suspicious behavior. Set up alerts for unusual patterns.
I'd like also to confirm that the bots we previously blocked from this email thread have not resurfaced. Nevertheless, please continue to keep an eye on the situation, and do let us know if you encounter any website latency or other issues.
We are always here to help!

[Administrator]

The first thing to accept is that you will never win the war. I have been fighting them for over 20 years and I am still learning new techniques.

All you can do is make your board as secure from BOT signups as you can. There is nothing you can do about human spanmmers signing up of course, but needing first post approval filters 99% of them out (assuming that the answer to the anti spam question isnt a modern motor car manufacturer its a great one, and dont post the correct answer here as it may get indexed)

The bigger problem is the crawlers (which is of course the problem you have been having) IMHO the only ones you 'need' to index your board are the usual suspects, Google, Bing, DuckDuck and so on. Many of the others are just indexing sites to collect data which they then sell on.

... and then there are the ones that just crawl crawl and keep on crawling, and those are the ones we are more interested in as they generally ignore any robots.txt rules too

So if/when you discover an aggressive bot you need to find its useragent (in the case of babbar its Barkrowler) and add it to the list of bad bots in your htaccess files (do it to everyone you are using) Yes its shutting the stable door after the horse has bolted, but other than ban all bots there is little else you can do.

I know its a lot of work but no one said running a website was easy and just be grateful its a hobby site not an international corporation)

Regarding the message from Kualo, that is more about spam signups than bots indexing your site as that is two totally different issues, and at the moment I feel that we have addressed them both as best we can.

FWIW this is how I do basic housekeeping. I always update phpBB within a day or two of its release (dependant on real world commitments) and check for any updated extensions once a week as well as subscribe to the relevant topic. I'll add any aggressive bot to my list as and when I get them, but quite often they will come, index like crazy and will never be seen again so its not always necessary.

I hope that helps

[Rickf1985]

What you say is educational, but one thing does not make sense. you say Babbar is Barkrawler? Barkrawler is, and has been, on the useragent ban list for a long time. And yet I have been getting slammed by Babbar with at least three different ip's. I added it to the list but I have not had a chance to check anything yet today. I just now got on.

[Administrator]

Its strange that its ignoring the htaccess rule as this is from the source

screenshot_208.jpg

You could try adding it to your robots.txt as well.

[Rickf1985]

I did and it appears to have stopped those visits. I have fallen way behind in my trip preparations to pick up my antique truck so I have been putting things aside for a bit. I am leaving at 04:00 tomorrow morning to go 400 miles to load up a 1953 military M37 truck and then get to my motel room and relax. Then the 400 miles home the next day. I guess that is a true sign of getting old, I used to drive 800 miles a day without even thinking twice about it, even enjoyed it. Not anymore. Especially with today's cell phone drivers!!!!

[Administrator]

Glad you have finally blocked it. Now prepare for the next ...

[Rickf1985]

How can I block an entire country? All of a sudden China is giving he a lot of hits and I suspect also downloads from my manuals. I have no members in China nor do I have any business there nor do I have any desire to do business with them. I would suspect they are downloading the manuals to resell.