Site issues update
As many of you have noticed, the site is periodically (as in, multiple times a day!) becoming unavailable with an “account suspended” message. So here’s the status:
A while back, apparently, my host put a tighter CPU limit on me because the site was getting so popular it was affecting other sites on the same server. You would not be alone in thinking this an odd way to handle the situation: “Hmm, this site is pushing at the limits of its allotment. So let’s decrease it.” They of course neglected to tell me that they did this. In fact, they neglected to even note it in my account, so all subsequent support folks have been baffled. Kudos to support guy Jamie for working to figure out the problem. Unfortunately, it’s still not fixed.
There’s now a support ticket in asking for the normal CPU limit to be reinstated, plus a request to move to a high capacity server (which is a hosting plan they offer). But they may not have a higher capacity slot available right now. In which case, we’ll be moving to new hosting anyhow. I’m waiting for them to get back to me — they are now a day overdue on the answer, so I will be calling them tomorrow. Many thanks to all you readers who offered hosting advice or space.
Lots of folks were wondering what exactly the traffic and site looks like, so here’s an infodump:
- The issue is CPU load caused by WordPress SQL queries. Not fancy ones, either — the standard ones.
- January cumulative stats were 84000 sites, 240000 visits, 540000 pages served, 2.2m files, and 2.7m hits.
- January average daily stats were 86000 hits a day, 70000 files, 17000 pages, and 7700 visits a day.
- It’s worth pointing out that at the start of January we were at around 6-7000 visits a day, and 65-90k hits. At the end of January, it’s more like 8-10000 visits a day, and the hits were trending in the 80k-160k range.
- January was the busiest month ever, and represented a big increase in sites and visits over December (21% more hits, 46% more sites, 19% more visits)
- It probably would have been more had the last 1/4 of the month not been stunted by the site issues.
- Bandwidth usage in January was 43 gig — I have a 100gig allotment per month with my provider.
- The blog database is around 4 meg all told, which isn’t too bad at all. There’s 13,000 records in the comments table though!
- That said, the rest of the site including all the presentations, artwork, music, blah blah blah is around a gig.
This being the only site I’ve ever run other than the book’s website, I have no good context for these stats. 🙂 So I don’t know whether the blog traffic is indeed so high that it’s to be expected that I would be maxing out my shared server situation, or whether something seems clearly misconfigured somewhere.
On a related note — I did install WP-Cache2, but I notice that when I check what files it has cached, it only ever seem to have the pages ending in /feed cached, or the front page. All the other pages (like, every post!) do not seem to be caching. I have no idea why, and would love an answer since that should help the load enormously. (It would also help if the rest of the site, which gets hit quite a lot, were in the WordPress CMS, but alas that would break everyone’s links and it’s a huge task to move everything over!).
One positive note: a bunch of the people reading on feeds that seemed to drop off a few months back seem to be back. I assume that somehow I had broken the stats tracking through FeedBurner, because post-upgrade to 2.1, the numberleapt back up by several hundred.
Lastly, I think we will be redoing the visuals and trying to reduce clutter soon. Anyone who’s got ideas, go ahead and submit ’em. 😉
17 Responses to “Site issues update”
Sorry, the comment form is closed at this time.
Your stats aren’t all that crazy. To compare, my site (Wanderhomies) had about 35k hits today, and that’s a slow day. I usually take up around 10-20gig bandwidth a month out of my 100gig. My database however is pretty damn big (uncompressed it was around 100-200MB in December). Before I switched hosts in December, the old version of the site was much more resource intensive (since it ran two different heavily used programs and DBs). Still I’ve never gotten any CPU usage errors that were my site’s fault (it’s usually some other person chewing it up, since I’m on a shared server hosting program). There really shouldn’t be any problem. I’ve gone through quite a few hosts though, since usually one host ends up getting some crazy idea to disable a key feature I need (my first host ever decided using Gallery was a security threat). My suggestion is to research a hosting company at FindMyHosting and actually read the ratings and reviews that actual customers gave them. This will give you a better insight as to what you’re getting yourself into. (for the record, bluehost has a 90% rating, but only after 26 reviews
Now, I’m technically done with my 3 year web-design hobby craze, but just a suggestion on clutter-removal: start by taking out the Recent Comments/Trackbacks feature. I’m not quite sure that’s actually necessary, especially now that you get so many. Furthermore, the links for all the different devs, academics, comics, etc take up a lot of space and sort of make the site lopsided. Perhaps you can use dynamic tree menus (the ones that fold down when you click on the title to reveal the content) to save up some space. It may not give as much exposure to those links, but it will definitely save up space. Oh, and one more thing, your copyright is a bit repetitive. Not sure if you need it both on the sidebar AND the bottom of your page. That’s just my 2 cents, hope it helps!
I would add more community features, so that the website isn’t entirely “self”-centered. At the end of day, sure, this is RaphKoster.com, but that doesn’t mean there can’t be a RaphKoster.com community…
Subscriber registration would be complemented by a browsable directory of subscriber profiles. Subscriber registration would also allow at the time of registration, and via account configuration, the ability to subscribe to Amazon, Areae, and CafePress updates. Comments would show small subscriber avatars, and comment creators would have their names linked to their profiles. Comments would be customizable to be displayed as either flat or threaded. The feature that allows discussion participants to subscribe to the comments feeds would be more prominently displayed. A feed for each user would also be interesting.
A rating system for links and posts would be provided. Posts with ratings would shift positions on a “top 10” list of popular posts on the sidebar. Links with ratings would shift positions on the sidebar too. More popular links would be closer to the top of their respective categories. Less popular links would be closer to the bottom of their respective categories. I guess this rating system could be applied to pages too, for use in your library of writings. And hey, even a rating system for your music would be cool.
Users would be able to easily engage the Digg submission process as well as the del.icio.us bookmarking service, per post.
I’d also add a printer-friendly link to every post and page, and I would create a version of the website exclusively for mobile phones.
Personally, I don’t like the three-column layout. This layout is appropriate for portals, which are aggregators of content. I think some bloggers, who are content creators, are better off with a two-column layout. The portal layout tends to be confusing, and encourages operators to place more unfocused content to fill the white space.
These are all the things (in some form) that will be part of the all-new IGDA San Diego website that I’m planning.
Without the recent comments feature, I don’t know how easy it would be for anyone to keep up with the comments. I get email notifications, of course, but I don’t know what blog readers do. You tell me!
I’ve already pretty much come to the conclusion that the links list is a bit of a waste — I never use it… does anyone here?
Yikes, Morgan, all that sounds like a) way MORE load on the CPU and b) maybe not stuff that exists for WordPress — and I am definitely not switching off of WordPress at this point, that’d be a huge huge import task to switch platforms… it was bad enough when we moved TO WordPress, and that was 500+ posts ago.
I think there is a post rating plugin. I don’t know if there’s one that gives a top ten chart. Totally unsure there’s a subscriber profile page plugin. The pages aren’t in Worpdress at all, and therefore not in the CMS at all, and therefore couldn’t be covered by the ratings system unless we put them all in the CMS< which would break the thousands of permalinks to them -- unless we added redirects, and there's hundreds of pages to do that for... Ironically, adding the Digg snd del.icio.us thing is trivial -- it's actually in the blog already, but it is turned off the site, and is only active on the feed. There already is a version of the site for mobile phones. 🙂 It only gives access to the blog tho, not the rest of the content. Printer-friendly... hurm, another one I have no idea how we'd do. As far as one or two column... you'd have to axe quite a lot of stuff to get it into 2 cols again, I think. What would you axe?
Requiem from Uberguilds.org offered you free hosting on your other thread. http://www.uberguilds.org/
Not going to get a much better deal than that.
ahem. um. you know… you could add in a hypefighter submission icon too. how cool would it be to have an alt tag that says “my bookmark can beat up your bookmark!”
not that i’m, you know, pimpin’ my own stuff or anything. but, while we’re on the subject….
m3mnoch.
I’m assuming, presuming… hoping… that you’ll launch a 2007 version of the website after transitioning to a new host.
That’s where you can either create the plugins yourself or tap your readership for plugin support. I’m surprised that you haven’t tapped your readership as a resource for maintaining your blog yet. Guy Kawasaki recently had his blog redesigned. I’m guessing that the folks that redesigned his blog were members of his readership and volunteers.
I’ve looked for alternatives. I’ve tested alternatives. I haven’t found any alternatives that can spar with WordPress.
A good directory to check is the WordPress Plugin Database. I can sometimes find a plugin that does what I basically want and then develop that plugin to do exactly what I want.
There probably is… but I would just use some metadata plugins to help output the existing user profiles.
Ouch. I’d get those into WordPress. When I created the IGDA San Diego website, all the links (including the business directory) were listed in regular ol’ HTML. With the help of Mike Acton, we were able to import all that data into WordPress fairly quickly. I also started using the WordPress link manager for banners and buttons.
I’d use .htaccess redirects. Export a list of URLs for the pages, import the pages into WordPress, and then add the appropriate redirects to the .htaccess file. That method is much less work than editing each page and using meta redirects.
I accessed this blog and the Areae website on my buddy’s high-tech cell phone, which has a slide-out QWERTY keyboard and uses Internet Explorer 7, and those websites didn’t appear correctly. I also tried my phone, which uses the default text browser.
WordPress Printable Post View Plugin 🙂
As far as the comments, you can still check them via your WP admin (from the dashboard, go to Manage > Comments) where you can view 20 comments at a time instead of the last 10.
The links, I’ve actually used before, but bottom line is that most people will only click on a link if the name rings a bell. If you want, you can even put the links on an entirely separate page.
Can’t get much better than free but like others have eluded too, what do you have to comply with to get it free. Your site doesn’t use a heap load of bandwidth but on the other side, its just beginning to get to the point where its becoming noticable. You’re eating nearly 50% of your allotted bandwidth for the month. Eventually someone has to pay for the bandwidth, power and space to serve your site. Careful that your site’s growth doesn’t become a strain on someone’s network that’s providing you the service for free. No matter what you decide to do, be sure to get the hosting contract in writing so that both parties have an understanding of what happens when/if your site triples in size. (ie as Areae gets closer to release…)
I know I can do that. I’m asking about you. Discussions on threads here frequently go way past the posts on the front page — at the rate I post, something stays on the front page for only three days, whereas discussions can and have rages for as long as three weeks.
I suppose I could move that to a separate page as well. Opinions?
I definitely don’t want to be in the position of imposing on someone for free. I used to do that with LegendMUD’s hosting, and eventually I was causing issues for the mud.
What I meant was, the more plugins, the more you impact performance. I don’t want to make performance worse, certainly.
We’ve done the plugin mod thing several times, actually. (And yeah, we use the various plugin sites regularly… the Database, the Codex, etc).
Redesigning the whole site… well, it’s just not trivial. It’s time-consuming. The site is kinda large, with a lot of files. I agree it should be done, but each time we do it (and we’ve never done it to the scale you’re proposing) it takes several days, at least a week. I’d be hesitant to ask anyone else to do that for me for nothing. 🙂
Heh. That’s TINY compared to what I am talking about. Just the writing section here has 36 files, and it’s one of the SMALLEST sections on the site. The gaming folder… well… yikes. 🙂
The CPU limit has been removed, so today we test to see if we exceed the higher limits… if we do, then we see about moving!
I dont see how the traffic should have been shutting you down, those are some damn good numbers though 🙂
“At the end of day, sure, this is RaphKoster.com, but that doesn’t mean there can’t be a RaphKoster.com community…”
The RK Brand?
RE: WordPress, eh, you know the folks over at WP are really pretty helpful (actually vey nice people) and well they’re avid gamers also so they’d likey be very helpful on getting the blog up to speed and optimized…
http://automattic.com/contact
I’m sure your pretty busy on the other project, but as to redesign whatever you choose make sure you stick with those categories you already have on the columns. There are overly complex and verbose ways of saying it and many would argue against it but people like the “familiar”.
Just my 2cp
It’s surprisingly easy – you just need to set up a second printer-friendly CSS stylesheet that hides all the elements you don’t need and formats everything appropriately.
Then you just add it with a <style> tag like so:
<link rel=”stylesheet” type=”text/css” media=”print” href=”print.css” />
I did this for some documentation at work ages ago and it worked like a charm.
BTW, Morgan, I am posting this from my phone… the blog looks correct on it…
To be honest the site could use a fresh new look 🙂
I’d probably kill most of the links or else put them on a separate page. Also, as previously mentioned, a two column format would be better.
Try and get your most important stuff ‘above the fold’…stuff like recent posts and categories. Right now it’s information overload. I’m thinking about something like: http://veerle.duoh.com/
There’s lots of content but it’s done in a very organized way.
Heh, Veerle’s design is three column too. Just has a double sidebar on the right.