« back to discussion

Satisfying server story

For one of my clients, they drive an enormous amount of traffic via advertising to their product funnels. My contact who is there told me they're in fact buying a second dedicated server because their first one is slowing down way too much.

I had to double check with him that was indeed their plan and that I wasn't hearing incorrectly. Currently their team is paying upwards of $400+ a month for a good dedicated server, and there would be no reason it couldn't handle their current traffic. They have no sql, no dynamic code, simply static pages.. I was somewhat baffled how their server could be slowing down.

So, I told them before they buy a new server they should have me take a look. They agree, and I log into their server not expecting what to find.

After only a minute, I figure out the problem. I was in disbelief at first, I thought there must be an error or that I was reading it wrong...

I ran top and first on the list I see a process that has been running TWO YEARS... that is utilizing 99% CPU nonstop. I was astounded.. furthermore, I go to investigate the process. It turns out to be a rogue wordpress plugin on a blog that hasn't been updated since 2009.

After getting confirmation, I killed the process and their load average goes from 3.5 steady to < .1...

How it could ever get to that point, I have no idea, but it was probably the most satisfying thing I've done recently to kill that cancerous process :D

about 4 years ago, by pineapple

5 Replies

wsg

You should have told them that you made extensive tweaks to their server settings to squeeze out more performance and made yourself look like superman or something lol.

I actually had a similar thing about 10 years ago with manchester.com - their site was going at a crawl and they couldn't understand why. Took a look at top and they had mysqld hammering the CPU, like REALLY hammering it. Someone had written a script to "backup" their db by reading every row, and then executing a query to save it into another DB on the same mysql instance. The thing was taking so long that it was starting again the next day before the previous instance had finished, it built and built until it brought the server to it's knees. I built them a proper backup option that just dumped the DB and saved it offsite, server troubles gone!

wsg, about 4 years ago

pineapple
posted by wsg on Fri, Oct 26 at 07:18AM

You should have told them that you made extensive tweaks to their server settings to squeeze out more performance and made yourself look like superman or something lol.

I actually had a similar thing about 10 years ago with manchester.com - their site was going at a crawl and they couldn't understand why. Took a look at top and they had mysqld hammering the CPU, like REALLY hammering it. Someone had written a script to "backup" their db by reading every row, and then executing a query to save it into another DB on the same mysql instance. The thing was taking so long that it was starting again the next day before the previous instance had finished, it built and built until it brought the server to it's knees. I built them a proper backup option that just dumped the DB and saved it offsite, server troubles gone!

Bahaha, that made me rofl out loud.

pineapple, about 4 years ago

jamesbarnett
posted by pineapple on Fri, Oct 26 at 12:17PM

I ran top and first on the list I see a process that has been running TWO YEARS... that is utilizing 99% CPU nonstop.

That's crazy town, 2 years and they never wondered what was up.

jamesbarnett, almost 4 years ago

pineapple
posted by jamesbarnett on Mon, Jan 14 at 05:08PM
posted by pineapple on Fri, Oct 26 at 12:17PM

I ran top and first on the list I see a process that has been running TWO YEARS... that is utilizing 99% CPU nonstop.

That's crazy town, 2 years and they never wondered what was up.

Yeah tell me about it.. The best part is, which I forgot to mention. Top also shows the time the process has been running. Most of them were a few digits like 252 or 62 (if i remember right, cant remember if it's in seconds or minutes). Then the rogue process was like 16826239179248572 the number was so long it was breaking the chart.

pineapple, almost 4 years ago

SkinnyGeek1010

Wow, talk about job security! I bet you're on there A list now =)

SkinnyGeek1010, almost 4 years ago


Login or to comment.

« back to discussion

Tutorials are any resources you learn from.

Examples: an intro to html5 screencast, a pdf about git, photoshop effects tutorials, meta-programming in ruby, lambda calculus, higher-order fixed-point combinators.

Tools are websites, apps or services used -on- your project (indirectly), to aid the process.

Examples: A color scheme generator, email marketing software, usability heat maps, css3 code generators, a downloadable png compressor.

Assets are downloadable files used -in- your projects, usually as code, textures, or images.

Examples: a jquery sticky menu, photoshop brushes, background textures, mvc frameworks, twitter bootstrap, 960 grid system.