Site Slowness

beardie

BD.org Sicko
Staff member
Administrator
Moderator
Founder
Beardie name(s)
Cailyth, Pinky, & Brain
Hey all,

I noticed today that the site became pretty slow to respond. Something seems to have happened a few days ago that slowly depleted our processing reserve (CPU/compute allowance on AWS). It needs to recoup a little spare processing power before I can further diagnose. I'll take a closer look tonight to see what has caused this increased CPU usage and address it.

For now, your patience is appreciated.

Thanks,
-Alex
 

beardie

BD.org Sicko
Staff member
Administrator
Moderator
Founder
Original Poster
Beardie name(s)
Cailyth, Pinky, & Brain
I'm still working on this, but will enable the site again for now. It will be slow. It seems that there was a massive increase in traffic, but it looks suspiciously like something crawling the site and copying content in mass... which is loading down the database. And unfortunately, it looks like it is coming from many IP addresses at the same time. I wouldn't call it a distributed denial of service, as an attempt at that would certainly take us down (we're not on a super powerful server). But it is distributed. That makes it harder to block.
 

CooperDragon

BD.org Sicko
Staff member
Moderator
Valkyrie47":ch14tj7b said:
That's scary o_O why would someone attack us ?
We just need help with our beardies!

Attacks like that are often automated. Not fun at all to deal with, but not a personal or targeted attack necessarily.



Alex, please let me know if you need any help and I'll do what I can.
 

beardie

BD.org Sicko
Staff member
Administrator
Moderator
Founder
Original Poster
Beardie name(s)
Cailyth, Pinky, & Brain
Valkyrie47":2w0r7lir said:
That's scary o_O why would someone attack us ?
We just need help with our beardies!
I don't think it's truly an "attack", as much as it is someone trying to mirror content or index it (like Google), but that is doing so in an underhanded manner... not sure why, except to avoid being blocked easily.

I have noticed that the user agent (the data that identifies the version of the browser, plugin info, etc.) is VERY similar for all their requests and doesn't seem to overlap with regular users. I've just implemented a block on that and am waiting to see if that works. They can easily get around it, but I can adjust. Hopefully they are just using tools and ignorant as to how to be more sneaky. I think they're likely just using tools with little technical understanding of them.
 

beardie

BD.org Sicko
Staff member
Administrator
Moderator
Founder
Original Poster
Beardie name(s)
Cailyth, Pinky, & Brain
It looks like the blocks are effective.. we shall see. This graph shows that the forbidden response code started being sent for their requests (orange on the right).

3-8322391947.jpg

And this shows just how noisy they were. You can also see they probed a day or two before they went nuts.

Unfriendly crawler bots

And this AWS graph shows that our CPU cycles are recuperating. So, overall, looking better. I'll feel better, though, when that CPU credit is up into the 500's (our normal levels).

3-8563355826.jpg

Hopefully, they'll just go away.
 

CooperDragon

BD.org Sicko
Staff member
Moderator
I'm not super familiar with hosting on AWS. Do they have anything similar to pfBlocker that can maintain blacklists via anti-spam databases and/or geoblock databases? I figure geo blocking would be sticky with a site like this, but perhaps the IPs that were accessing the site are in one of the anti spam databases and could be filtered out.
 

beardie

BD.org Sicko
Staff member
Administrator
Moderator
Founder
Original Poster
Beardie name(s)
Cailyth, Pinky, & Brain
CooperDragon":1wecp9x4 said:
I'm not super familiar with hosting on AWS. Do they have anything similar to pfBlocker that can maintain blacklists via anti-spam databases and/or geoblock databases? I figure geo blocking would be sticky with a site like this, but perhaps the IPs that were accessing the site are in one of the anti spam databases and could be filtered out.
They don't offer much for such things with how I have the server setup. We have an EC2 instance, which is like a virtual server that you do things for yourself with. I prefer that in most ways.

Even if they had black lists, the traffic we were getting was too broad. I was rather surprised at how spread out it was, but they were clearly coordinated. I do have geo-location blocking in place for the site, and other measure to protect the site from spam on the forums. But general traffic... that's a harder thing to guard. I think there were some 60 class C subnets (up to 255 IPs per subnet) involved, with anywhere from 10 to 150 IPs actually being used per subnet. So, it was probably nearly 1000 IP addresses seen in the 2-3 days.

I am glad that I noticed the consistency of their user-agent. That made it easy to block. It may have to be adjusted again (before we get to a depletion of CPU credits; which did take a few days, but I wasn't looking that closely). I doubt they'll put in the effort to diversify the user-agents for all their nodes. It did some variation, but minimal, and easy enough to write a rule to block that.
 

Drache613

BD.org Sicko
Staff member
Moderator
Hello Alex,

I am not real familiar with a lot of that computer language, but understand enough on how
to block certain areas, addresses, etc.
I think that the entire internet & banking systems are being affected due to political moves
going on right now.
The site is doing well though! I can't imagine the behind the scenes work that it takes to keep
it all smooth.

Tracie
 

beardie

BD.org Sicko
Staff member
Administrator
Moderator
Founder
Original Poster
Beardie name(s)
Cailyth, Pinky, & Brain
Thanks Tracie.

The "attacks" stopped about 8 hours after I blocked it. I guess they realized they were caught.

I did get an email from a regular user that was blocked by this method. Upon review, I think in the last few days there were maybe 4 people affected that weren't part of the attack. That's not too bad, given how many visitors we have per day.

I've removed the block now though, as the attack stopped, and there's no need to block regular people who might be inadvertently impacted.
 

Members online

No members online now.

Still Needs Help

Latest resources

Latest profile posts

I miss you so much, Amaris 💔
What is a quick way to warm up a cold beardie? His heating element went out overnight and now he's very cold.
Pearl Girl wrote on moorelori1966's profile.
i feel so sad reading your about me 😢
Clapton is acclimating okay I think. He's quick as lightning so I'm not sure how much I should bring him out of his house yet. He's not at all interested in his salad though. I wonder if I should change what I'm giving him. Least he's eating his crickets.

Things to do:
Buy calcium powder
Material to raise surface for basking spot
Scenery decals for back of tank

Forum statistics

Threads
155,899
Messages
1,255,678
Members
75,965
Latest member
williamyoung
Top Bottom