Author Topic: Resource Limit Reached...  (Read 31916 times)

0 Members and 2 Guests are viewing this topic.

Offline gamesurf

  • 4/4/2023
  • Director
  • *******
  • Posts: 1665
  • Crane Game
    • Golden-Gallery.net
Re: Resource Limit Reached...
« Reply #30 on: May 11, 2026, 02:01:36 AM »
That proved effective to discourage the scraping. In 30 minutes we went down from 8,000 guests browsing the forum to under 200 (and most of those I would bet are clicking links they've already cached and are staring at an error screen and not the actual forum).

Obviously turning off guest access is not a long-term solution. We want actual human guests to be able to browse the forum and we don't want to turn them away. It should help get us through the night.
Quote from: Bill Todman
"The sign of a good game, is when you don't have to explain it every day. The key is not simplicity, but apparent simplicity. Password looks like any idiot could have made it up, but we have 14 of our people working on that show. There is a great complexity behind the screen. It requires great work to keep it simple."

https://golden-gallery.net

Offline gamesurf

  • 4/4/2023
  • Director
  • *******
  • Posts: 1665
  • Crane Game
    • Golden-Gallery.net
Re: Resource Limit Reached...
« Reply #31 on: May 11, 2026, 02:46:51 AM »
We appear to have weathered the storm. We've been holding steady at around 200 guests for the past 45 minutes.

I've turned guest access back on and haven't seen any serious increase in activity in the short term since then.
Quote from: Bill Todman
"The sign of a good game, is when you don't have to explain it every day. The key is not simplicity, but apparent simplicity. Password looks like any idiot could have made it up, but we have 14 of our people working on that show. There is a great complexity behind the screen. It requires great work to keep it simple."

https://golden-gallery.net

Offline gamesurf

  • 4/4/2023
  • Director
  • *******
  • Posts: 1665
  • Crane Game
    • Golden-Gallery.net
Re: Resource Limit Reached...
« Reply #32 on: May 21, 2026, 05:46:17 PM »
We had another bot swarm today peaking at around 6,000. As an experiment I turned on the "Restrict guest access when over X guests are online" mod during the swarm and set the cap to 1,000 to see if it would be effective at limiting the bots.

There is also a mod available that “caps” guests access to a certain number of guests at any one time. I do not think it would be a good idea to turn it on based on the results of the “15 page cap” experiment.

If we have 1,000 bots attempting to scrape the site, and we set a “cap” at 50 guests, what will actually happen is 50 bots will scrape the site, and 950 bots will attempt to read a page and get an error message. We’d still be dealing with 1,000 bots of traffic, and we probably wouldn’t see any reduction in traffic. But this would frustrate *actual* human guests trying to read the site in a much more obnoxious way than the “15 page cap” does. Downside with no upside.

As predicted, this was not effective at preventing bot traffic. It was effective at preventing the site's info from being scraped, but the bots did not leave. The bots continued to try to click through all of their links even though they were getting prompts to log in rather than the pages they were trying to access. This mod has the side effect of preventing actual human guests from accessing the forum, so it will probably remain turned off or with a limit that is so high it will only kick in once we are close to getting a Resource Error. It is a band-aid fix at best.

Marc has been contacted regarding CloudFlare about two months ago and we still have yet to hear back from him. As far as I can tell only he has the admin keys that could get that kind of bot protection, so we're hoping he gets back to us.

As a plan B and C, we're looking at other mods that could be installed that might intercept known bots before they even see the website. When installing mods there is always the danger that they could break the site if done improperly so I am asking people smarter than I am and with more IT experience to look at them before exploring that option.

With the help of AI some of the more aggressive bot clusters have been identified and labeled. If you go to the "who's online" page at the bottom of the screen you should be able to see which suspected bots are online now and where they come from.
Quote from: Bill Todman
"The sign of a good game, is when you don't have to explain it every day. The key is not simplicity, but apparent simplicity. Password looks like any idiot could have made it up, but we have 14 of our people working on that show. There is a great complexity behind the screen. It requires great work to keep it simple."

https://golden-gallery.net

Offline AvsFan

  • In the Audience
  • **
  • Posts: 51
Re: Resource Limit Reached...
« Reply #33 on: May 21, 2026, 06:22:15 PM »
Thank you for your work on this!

Offline gamesurf

  • 4/4/2023
  • Director
  • *******
  • Posts: 1665
  • Crane Game
    • Golden-Gallery.net
Re: Resource Limit Reached...
« Reply #34 on: May 22, 2026, 06:50:26 PM »
Hit another record with a five-figure guest count last night.

Needless to say, that “restrict guest access when 1000 guests are online” has proved completely ineffective and has been uninstalled. I don’t think there is any reason to think the two are related, but not interested in taking the chance.
Quote from: Bill Todman
"The sign of a good game, is when you don't have to explain it every day. The key is not simplicity, but apparent simplicity. Password looks like any idiot could have made it up, but we have 14 of our people working on that show. There is a great complexity behind the screen. It requires great work to keep it simple."

https://golden-gallery.net

Offline tpirfan28

  • Double Showcase Winner
  • ******
  • Posts: 3528
Re: Resource Limit Reached...
« Reply #35 on: May 23, 2026, 10:31:42 AM »
Got the 508 it just now (10:30 5/23)

Zenlayer (20), US Mobile (13), Luogelang (174), India/SE Asia (613), Google (other) (11), Google, China Mobile (2), ByteDance (76), Brazil/Latam (1255), AWS Singapore (3) were the offenders if that helps any.
They can be close at the top, too.
#42SP

Offline htmlcc92

  • Double Showcase Winner
  • ******
  • Posts: 1272
  • "Mrs. Peacock was a man?!"
Re: Resource Limit Reached...
« Reply #36 on: May 23, 2026, 10:50:09 AM »
Marc has been contacted regarding CloudFlare about two months ago and we still have yet to hear back from him. As far as I can tell only he has the admin keys that could get that kind of bot protection, so we're hoping he gets back to us.

What happens if Marc never gets back to you guys?
There is a fifth dimension beyond that which is known to man. It is a dimension as vast as space and as timeless as infinity. It is the middle ground between light and shadow, between science and superstition, and it lies between the pit of man's fears and the summit of his knowledge. This is the dimension of imagination. It is an area which we call the Twilight Zone.

Offline gamesurf

  • 4/4/2023
  • Director
  • *******
  • Posts: 1665
  • Crane Game
    • Golden-Gallery.net
Re: Resource Limit Reached...
« Reply #37 on: May 23, 2026, 04:27:42 PM »
Status quo, basically.

Marc has the domain set to autopay and that seems to have been the case for five years now, so there is no immediate danger of losing the golden-road.net domain. But aside from Marc, none of the current administration has access to edit the DNS, which would need to happen to install CloudFlare.

There are third-party mods called Bad Behavior and CleanTalk that we do have the capability to install. It is possible they could have some effect at stopping known abusive IPs from crawling the site. It’s also possible to upgrade the forum software.

However, doing some research into other Simple Machines forums (including those protected by CleanTalk, and those running on more recent software than ours) has shown most of them have had the same problem we have with bot surges in 2026. This is not a unique problem to our site, any sufficiently large forum is going to fall victim to crawlers.

The nature of the Internet has changed drastically over the past few years. The biggest annoyance on forums used to be spam messages, and most of the protections Simple Machines developed over the past 20 years were based around stopping bots from posting. We’ve done a great job at stopping that over the years. Now the biggest annoyance is AI crawlers, but we don’t have as many tools to stop that.

What we’ve tried has basically been playing whack-a-mole with the bot clusters we do have ability to detect. We’ve restricted some regions, including the four IP clusters listed earlier, from crawling the site. But a huge number of bots don’t come from known bot farms and their IP addresses change every time. It isn’t practical to block thousands of new IP addresses every week and it would likely have a detrimental effect on actual users.

So that leaves “find the right third party software who can handle this”. The band-aid fixes have given some temporary respites, but they are just band-aids. We’re not in any immediate danger, but we’re also not much closer to solving the problem.
Quote from: Bill Todman
"The sign of a good game, is when you don't have to explain it every day. The key is not simplicity, but apparent simplicity. Password looks like any idiot could have made it up, but we have 14 of our people working on that show. There is a great complexity behind the screen. It requires great work to keep it simple."

https://golden-gallery.net

Offline htmlcc92

  • Double Showcase Winner
  • ******
  • Posts: 1272
  • "Mrs. Peacock was a man?!"
Re: Resource Limit Reached...
« Reply #38 on: May 23, 2026, 07:06:57 PM »
Makes sense. I use the same whack-a-mole analogy at work when people want me to block phone numbers - the spammer is never going to call from the same phone number, so I'd be spending my whole eight-hour work day just doing that.

Good luck though, hopefully something sticks!
There is a fifth dimension beyond that which is known to man. It is a dimension as vast as space and as timeless as infinity. It is the middle ground between light and shadow, between science and superstition, and it lies between the pit of man's fears and the summit of his knowledge. This is the dimension of imagination. It is an area which we call the Twilight Zone.

Offline Chelsea

  • Executive Producer
  • **********
  • Posts: 708
Re: Resource Limit Reached...
« Reply #39 on: May 26, 2026, 04:49:38 PM »
All guests have had access to all boards except Front Office, Talk is Right, and current season Recaps revoked temporarily. This is a pilot test through May 31st.

I have also made a change to the theme. Part of this is to keep the site from loading a 20kb image on any session that loads the forum stylesheet, part is to match the look of the upgraded Golden Gallery, and part is tied into an announcement coming soon. (No big deal, just something overdue).

Can't install Cloudflare for now, so doing some testing and some triage.

Offline Chelsea

  • Executive Producer
  • **********
  • Posts: 708
Re: Resource Limit Reached...
« Reply #40 on: May 27, 2026, 07:25:53 PM »
Thanks to some help from Marc (and thanks to gamesurf for keeping this on my radar), Cloudflare has been installed as of today.

Note that you may receive a 'challenge screen' that may or may not include a CAPTCHA-type image puzzle. Visitors outside the US and Canada are more likely to face stricter puzzle challenges or be prompted more often.

Also, I've issued a temporary traffic block to seven countries making up the bulk of our suspicious 'guest' traffic (Brazil, China, Thailand (sorry Rod), Singapore, Dominican Republic, Vietnam, India).  Within the next 48 hours, the site will become unavailable in those seven countries temporarily. I hope to move them over to the challenge screen security level by early June, but the block is an attempt to shutdown the major botnets currently slamming the server, most of whom are from those seven.

If the bots move to other countries, these rules may change. Another announcement coming soon. 

Anti-AI and anti-Crawler options have also been enabled.

Offline TPIR98665

  • 2/5/2026
  • TPiR Alumnus
  • *
  • Posts: 928
Re: Resource Limit Reached...
« Reply #41 on: May 27, 2026, 08:37:00 PM »
Box of bendy donuts for the tech crew...Site is much improved with the new measures in place...
My taping history:

1 Bob taping somewhere around 1986
30 Drew tapings starting February 5th 2024 ending October 9, 2025
My 'Come on Down' moment occurred on October 8th 2025 (1284L).  Air date of February 5th 2026. Two years exactly from my first taping.

https://www.golden-road.net/index.php/topic,37211.msg523152.html#msg523152

Online mechamind

  • Double Showcase Winner
  • ******
  • Posts: 2410
  • Left is up, now.
    • Buy a Vowel Boards
Re: Resource Limit Reached...
« Reply #42 on: May 27, 2026, 09:17:08 PM »
Don't know if we'll make it to the 11:59:59 PM on the 31st...but guilty as charged, it's a breath of fresh air seeing only 82 guests on the board right now. :P
Quote
Telephone Game was retired because it was lame. (And no, we’re not making that up – that really is the official reason.)
________

We. are. not. the. casual. viewer.

Fan of The Price is Right since the 1990's, my early childhood...and really happy that summer break was a thing.

Retro recapper for The Price is Right and Wheel of Fortune episodes. (Recaps on hiatus as of May 27, 2024 due to life changes.)

Sticky Keys champion as of October 22, 2023.

Offline Flerbert419

  • Double Showcase Winner
  • ******
  • Posts: 2892
Re: Resource Limit Reached...
« Reply #43 on: May 27, 2026, 09:56:15 PM »
Alas, I was hoping that there were 12,000 people interested in the show at once.

Thanks to those who kept fighting to get a sustainable solution in place!
"When I come back in my next life I need to be a man. A mediocre, fat, ugly man that just gets away with everything."
~ Claudia Jordan on Phil Wayne Rossi, Dirty Rotten Scandals, March 2026

Online alansh42

  • Walking the Golden Road
  • ****
  • Posts: 265
Re: Resource Limit Reached...
« Reply #44 on: May 27, 2026, 10:15:17 PM »
Glad you're able to get Cloudflare set up. Like I mentioned upthread I had to install it on my little hobby site, the bots have gotten that bad.

I did briefly see the bot screening page but it auto approved, I didn't have to do anything.