"Cannot use search" error?

[Skipper7]

For the past few days Ive been getting this message about 75% of the time when I click on "view your posts" or "view new posts." I also got it about 5 times when trying to post this thread.

"Sorry but you cannot use search at this time. Please try again in a few minutes."

Is anyone else having this issue?

 

[Varalidaine]

Not sure if the error said "search" specifically, but I've gotten a few saying the board was down or something along those lines. I also couldn't post a topic earlier.

 

[CooperDragon]

I've been seeing both of those messages intermittently as well. More so this evening.

 

[Gormagon]

Ya, I have been getting it all night, hell I'm not sure if this post will even go through.

 

[beardie]

Thanks everyone. Something has been increasing the load on the server for the last few days. I hadn't noticed it until now. I restarted a couple processes and will keep an eye on things. I'm not sure why it started or is happening, but hopefully I'll find out soon.

Thanks again for the info/heads up. Sorry about the inconvenience.

 

[beardie]

So, I'm still not sure what the root cause was. Restarting processes wasn't helping. I ended up rebooting the server. That seems to have calmed things down.

Thank you again for letting me know the impact you were experiencing.
-Alex

 

[Varalidaine]

Definitely wasn't a big deal, but glad it helped Thank you for all the work you do in keeping this place up!

 

[Skipper7]

Thank you! And thank you for running the sight! I haven't had any issues with it at all this evening.

 

[beardie]

I'm still working on resolving the issue. I think it actually is more complex than I thought. It's a combination of how the CPU allowance works on the Amazon cloud (EC2). I think I had actually had things slowly working down the "surplus" of CPU credit. And when it hit zero, we get limited to the minimum CPU for the service paid for. But if you have surplus, you can have spikes of usage and it will respond quickly.

End result, I think I need to turn off some non-essential services until I can build up more "surplus" and then tune things... and then set a monitor to watch the Amazon CPU allowance balance. I should be able to see this sort of thing coming in advance if I do.

Again. Thanks.

 

[Skipper7]

So... we're like bad kids who used all of the rollover data? :lol:

Good luck, and thanks again. This site is a HUGE help.

 

[beardie]

Actually... good analogy. LOL Yeah... kinda. I think it's a combination of some of the other things I run on the server (including backup software, and data analysis tools), the fact that spam for email has dramatically increased in the last couple weeks, AND for the past 5-6 days, we've had higher traffic levels (more page loads) on the website than normal.

Here's a screen shot of the Amazon CPU credit balance.
View media item 29045
Normally, the peaks should be in the 200-300 range. So, this has been diminishing for the past month or so, I'm guessing. It's normal to have it go up and down. That's how Amazon manages to provide the power of higher powered systems, but giving you a limit for how long you can use that higher power before you hit your slower speed of the service level (I think we're guaranteed 30% of a full powered processor minimum).

View media item 29038
And in the above image, you can see more recently, when the website would have less traffic (evenings), it would slowly rebuild. Some nighttime processes would take some of that, such as backups and the like. And then when morning picked up, we drained the CPU credits again. The gap near the right side is the server reboots I did today. The fluctuation after that was me trying to figure out where the issue was. And the spike near the end is where I turned off some more intensive processes that I would like to have, but will need to rune better.

The nice thing about Amazon is that I can upgrade the service and get more power, without having to rebuild a new server.

Anyway... I like to share these kind of details. I know a few people on here like to hear the technical stuff.

Cheers,
-Alex

 

[Skipper7]

Interesting, I like the graphics. Much easier to see what you're saying.

Can you see logs on posts per hour/day, numbers of new accounts, or actual numbers for site visits views? You mentioned more page loads, but is there any quantitative data on that? (I'd assume stuff like this would be through phpBB.) I'd be interested in seeing all of the logs... but I'm also a nerd :roll:

Do automated notification emails eat at the credits? If so, is there any way to give us an option to disable them? I personally find having to mark and unwatch all of my threads very annoying. LOL


I'm sorry if I'm delving to deep into private information. I guess that's what nerds like me do

 

[beardie]

I have a lot of data. I primarily use Google Analytics for general traffic reporting. But it's actually the advertising reports that I was looking at when I'd noticed that site traffic had been going up. I could dig into granular data for other things, but usually not important to do so unless I have a specific need/goal in mind.

The email notifications are very minimal in their load on the server, so changes for that wouldn't help much. I have a lot of things in place on the system, such as an application firewall to help detect and block hack attempts.