So, with the release of Safari 5 and it’s new extensions system, I thought I’d knock together something that does the same thing for Safari. It uses exactly the same CSS-based method that the Google Chrome extension does.

Want the extension? Download it here. Let me know if you have any problems with it.

Well, the short answer is I’ve just been busy!

Media Wow

Since I started at Media Wow, I’ve been busy there with a few projects. The biggest of which, our first “proper” web site build, is nearing completion and we’re excited to be launching it next Thursday. It’s one we’ve built from the ground up using WordPress, a custom design, some neat integration and we’re going to be hosting it on our in-house hosting platform we’ve set up with VPS.NET.

As soon as that’s done, we’re starting on eight web site builds for our biggest client. Whilst they’re certainly not sexy brands, they’re going to be fun to build, and it’s going to be nice to bring some brand consistency to the group.

Invision Power Services

Outside of that, I’ve also officially joined the Invision Power Services team, to help out on a part time basis. A lot of you will know that I’ve been working with IPS on and off for several years, with everything ranging from support, to skin design, to SEO. I’m very excited to be helping them in a more official role, and have already been pretty busy. Some of what I have been doing includes:

Search Engine Optimisation

I started out a few months ago helping to improve the search-friendliness of the IP.Board product, and we made some huge improvements for version 3.1, which was released last week.

I’m also now helping with optimising the Invision Power Services company web site. We’re aiming initially and primarily to target the term “forum software” (shameless linking, I know) and it is going remarkably well. In less than two months we’ve gone from not ranking at all in the first 150 positions to, as of the time of this post, position fifteen! We’ve even had brief stints in position five. Obviously this is a work in progress, and we’d naturally love to see ourselves hit position one.

Mobile Skin

Rikki did a great job designing the first iteration of the mobile skin for IP.Board 3.1. It was a vast improvement from previous light skins used for this purpose, but we decided that we wanted to target modern smart phones and give them a truly mobile experience. So Josh set to work on a native iPhone application, and I started to build upon Rikki’s mobile skin.

We’ve made it feel like a more native app, by replacing most of the links with touch-friendly buttons, making entire rows clickable instead of just the links within them, adding a loading indicator to give the impression it’s not simply a web page. These simple things have been very well received by regular users, and when coupled with the iPhone application, it looks and feels great.

Affiliate Banners

I noticed some time ago that a lot of the people in the affiliate pilot program are either using text links, or banners they have designed themselves. We’d never offered affiliates an official set of graphics to use on their sites and as a result, I felt we were missing the mark. So, I set out to create some official ones and they seem to have been very well received.

Support

Whilst this is not part of my role, since the release of IP.Board 3.1, I’ve even been helping out with support whenever I can. The sheer number of queries has been phenomenal, but the support team have been doing a truly amazing job at keeping up!

Outside Work

Guess what? My car broke again! This time, however, it needs an entirely new engine. I went through quite a bit of stress with Evans Halshaw trying to arrange for this work to be done on warranty, but thanks once more go to Ian McCandlish at Pendragon PLC for ensuring that the full cost of a new engine (£2,800~) was covered by my warranty, even if that did mean voiding the warranty going forward.

I also bought a red Rover 220 SDI as a temporary replacement car, which lasted me a week before an oil leak brought it to its knees. That car is now stuck at work until I’m brave enough to try and move it home, or I call the RAC to collect it. I’m going to try and sell it for spares or something, otherwise it’ll be a scrap job. It’s a bit of a shame, because the car itself is in good shape, it just seems a bit pointless trying to repair it now.

So, much to Charles Warner’s amusement, I’m spending a huge amount of time on trains back and forth to Durham, Manchester, and today, Denbigh in Wales. I’m almost starting to enjoy it, as it gives me a chance to think about things, write blog posts, make banners, etc.

When I started looking at it, I realised that my own blog site not only ranks for a lot of curious terms, but it gets a reasonable number of impressions even when it ranks poorly (page 3+, for example.) Here are a couple to demonstrate:

1,300 impressions for my site on page 3+ for the term ‘blackpool tower’? What does that imply? I’d hazard a guess that it could be people seeking out a cheap rate for getting into the tower, a freebie perhaps… otherwise some people must be really desperate to read a lot about places they visit.

We’re hoping to use this new dataset give us a better insight into how changes to page titles and meta descriptions affect click through rate, with the ability to ensure that fluctuations in search volume are not the actual cuase. I’d be intrigued to know how other people are using this and whether they’re finding it useful.

One thing I’ve thought about before is turning off the “nofollow” attribute in my comments to allow people to get a link that counts for something when they post on my blog. Obviously I’d need to monitor my comments more closely, but I can only see that as being a good thing. It could also spark “shallow” commenting that is designed purely to get a link out from the blog, rather than to promote actual discussion, which is definitely not what I’m looking for.

What sparked this post is a video Matt Cutts just posted, here’s what he has to say about the idea:

I’d really appreciate some feedback on this one, what do you think about the idea? Is it a bad way to encourage commenting? Would it work? Have you done the same or done it and changed back?

However, as soon as I started to read it, I realised that I’m probably not actually the target audience for this book and thus not the best person to be asked to review it. It appears to be more of a “Getting Started with MODx” style walkthrough, as opposed to a reference book – for developers or otherwise. So whilst I have set out to give this book a fair review, I will be doing so from a developer’s perspective and this may not be representative of it’s usefulness to the average webmaster.

Chapter Outline

Chapter 1: What is MODx? – This chapter does a relatively good job of explaining what MODx is, including definitions of what a content management system is, what a template system is, and the general architecture of a database-driven web site. There are a few misunderstandings on the part of the author that make it feel a bit patronising and it comes across in some ways as a bit of a buzzword party. These include describing use of friendly URLs as ReST – a notion that can bring out intense rage from actual ReST fans, such as Ben – or the description of Web 2.0 as being sites in which the content is generated by the user, and the suggestion that a CMS makes creating Web 2.0 sites easily. Some content management systems can do this, but on the most part, that is not their purpose.

Reading the final page of this chapter as an experienced web developer does not inspire much confidence in the rest of the book, however. The author describes the administration system in MODx as being AJAX driven, and thus “really user friendly”. Now – I understand that AJAX extensions can indeed make your applications *more* user friendly, when implemented well, but this is not as a result of the technology itself, but of the usability skills of the developer that implemented it. Secondly, the author suggests that MODx is search friendly, and provides everything you need to get your site listed in Google. Leaving aside the obvious requirement for off-site optimisation for search, the content management system is generally not even what matters for on-site optimisation, it’s the templates you give it. If your templates are rubbish, there’s nothing a content management system can do to help you.

Chapter 2: Getting Started - This chapter starts with the prerequisites for MODx installation, along with how to set up XAMPP (Windows/Linux) or MAMP (Mac) to test it on. This is a few pages I feel would have been best left as an address of an online guide, or similar, but leaving that aside, the instructions seem simple enough. There’s a small niggle with the section that doesn’t suggest the best editorial oversight on this book – which is that brand names for operating systems are not adhered to. Windows does not carry a trademark symbol, and Apple’s Mac platform is referred to as MAC throughout – Mac is a brand name, not an acronym. The same applies later in the chapter in the form of several typing mistakes. Following on from that, the instructions for installing MODx itself seem comprehensive and appear to cover most obvious questions as you go through.

The chapter closes with an introduction to what documentation and support is available for MODx on the internet. This is great. The section explains clearly what each resource is and what it’s useful for, along with general etiquette expected by the MODx community when using the provided resources.

Chapter 3: MODx Basics – This chapter, as the name suggests, will teach you the basics of MODx. The first thing it taught me, however, is a bit about the author of the book – including his GMail address, del.icio.us username, and his interest in the subject of Cross Site Scripting. This goes back to my earlier mention of editorial oversight, the screen shots in this chapter were obviously all taken by the author, using his own browser, whilst browsing other sites – a bit of a no-no if you ask me. The chapter does a good job of introducing you to some of the core concepts in MODx, such as “documents” being the key element. It also provides a good top level overview of various sections of the management panel, some of the things they do, and how they should be used.

Chapter 4: Templating - This is the first section I thought I could really get my teeth into, even if it’s not development per-se, it’s a technical subject. Unfortunately, it continues at the same slow (for me) pace that the preceding chapters started and I think the most interesting thing I learned is that MODx uses an absolutely crazy variable syntax, or should I say multiple? There’s [*#content*], [!Snippet? &variable=`value`] and my personal favourite [~[*id*]~], amongst others. It strikes me that this is as a result of a poorly written template parser, but I truly hope I am wrong.

Chapter 5: Authentication and Authorization - This section starts out by, fairly clearly, explaining what these two things are, followed by how to get started with adding a log in box to your site. It does throw in some un-necessary guides to changing your site layout from one to two columns, which confused me a little, but otherwise it starts off well. It then goes on to explain how you can restrict access to certain content and functionality to users, or groups of users on your site. Finally it covers how to allow users to register on your site, following very much the same format as the rest of the book.

By this point, you’d have a simple blog site that allows members to sign up and post comments, along with allowing you to allocate people to your “friends” group, which in turn will allow them to post entries on your blog.

Chapter 6: Content Aggregation – This chapter primarily describes how to use one MODx snippet, “Ditto.” Ditto allows you to pull lists of content from other areas of the site into your pages and/or your templates, allowing you to do things like latest posts listings, or latest comments. It also goes on to introduce how to use Ditto to output RSS feeds, JSON and XML documents. It then briefly introduces the concept of extending snippets to add extra functionality without duplicating existing code.

Chapter 7: Creating Lists - Much like the Content Aggregation chapter above, this one focusses on the use of one snippet. This time, it’s how to use the WayFinder snippet to create hierarchical lists of content on your site, for use as menus. It then discusses how to style them to better suit their job as a menu. The chapter is mostly HTML and CSS sample code, along with screenshots, and doesn’t really go into a great deal of detail.

Chapter 8: Snippets - In a bit of an odd twist, the book begins to introduce snippets themselves here. We’ve used them several times already, but this chapter explains what they are, how they work and how you use them. It does so by explaining how to set up and configure a user-commenting snippet called Jot. Once again, this chapter is incredibly example-code heavy, which may be frustrating if you’re not reading the book to create the sample site it covers.

After having done this, you’ll be shown how to download and install a snippet, which again feels a bit late, but not as late as the wrap-up, which tells you how to find snippets online.

Chapter 9: PHx - Another puzzling chapter ensues, this time discussing a feature called PHx. It’s a way of doing conditional logic in your MODx templates, something I assumed was part of the template engine by default, but apparently it is not. PHx is actually a plug-in that is not included by default. After a brief guide to installing the plugin, the reader is treated to another long and unrelated example, before being shown how to actually use the PHx syntax, which is just as amusing as the template variables were.

Chapter 10: Simple Recipes – When I first read the title for this chapter, I assumed it was going to be a summarisation of what you’d been taught already, by means of building a simple recipes site. I was wrong. The chapter was instead a guide to installing, configuring and using a few of the author’s favourite MODx snippets. Some of them useful, one of them was entirely separate software package for running a forum (SMF) and how to integrate it.

Chapter 11: Creating Snippets - We’ve been learning about what snippets are, and how to use them, for the majority of this book. This chapter gives us an outline of how to create them. You’re taught the obligatory “Hello World” example in detail, initially using the simplest method possible, and then later using more complex MODx-like methods. The chapter is a good guide, giving examples that are useful, as well as diagrams explaining how MODx processes snippets. From a developer’s perspective, this is actually very useful.

The end of the chapter gives you a brief introduction to the MODx API, and it’s database abstraction layer. As with an earlier chapter in the book, this is well written – but as a result serves to highlight some curiosities in MODx itself. Their database layer provides an intriguing method for building queries, in which you pass all of the usual elements of a query in as parameters.  It’s like a poor man’s SQL, as it doesn’t implement any of the features as well as SQL itself, but enough that it isn’t going to work properly across database platforms. Not the fault of the book, but an interesting thing to note none the less.

Chapter 12: SEO, Deployment and Security - When I first received this book, one of the first bits I read was this chapter. It irritated me immediately. This book serves as a good guide for new web masters trying to set up their first MODx site, and that’s where it should have stopped, in my opinion. The SEO section is full of faults, inaccuracies and doesn’t go into enough detail on any one section to actually be useful. There are a few small nuggets of useful information, but to call this section SEO guidance is wrong. If I were editing this book, I’d rather have seen a list of external resources that would be useful for this topic, than a non-expert trying to cover it.

The deployment section is much the same, whilst it’s more factually accurate, it doesn’t cover anything in enough detail to be useful. I’d like to have seen web addresses for a selection of popular FTP clients and their documentation. The reader does not need to be confused with guides to FTP, SFTP and WebDAV.

Finally, the security section. This is a lot better, it gives a brief and concise primer to securing your MODx installation, which I’d imagine will help keep new web masters a little safer.

Chapter 13: Plug-ins and Modules - This section of the book gives you a detailed guide to installing, using and eventually creating plug-ins and modules for MODx. Plugins seem to be collections of hooks (or “events” in MODx lingo) that execute at certain stages of the execution flow, and as with other development focussed areas of this book, the chapter gives you a good introduction to how to create them. You’re also given a couple of useful real world examples to get you on your way.

Summary

So, to the summary. All in all, as much as my review has not been amazingly positive, this is not a bad book. I’m sure it’s a great guide for a beginner web master, setting up their first MODx site or for a new MODx modification author, looking to get started. There are a number of flaws and inaccuracies along the way, but it covers most things someone wanting to get into MODx would want to find.

It’s definitely not a reference book, however. It’s also definitely not for professional web developers. As someone getting started, you’ll work through this book once or twice and by the time you’re done you’ll have an understanding of how to work with the system, but that’s where it’s usefulness comes to an end. For a typical web developer like myself, a reference book would be a much more useful publication, one that can be picked up and dipped into over and over again that details every part of the system with no frills or commentary.

However, I decided I’d also give them some advice to improve the product out of the box in a few areas I saw potential. I’ve been working with them for the past few weeks, and the results are starting to be implemented in the product. Here are a couple of the changes that are coming in IP.Board 3.1:

• Using appropriate HTTP status codes: When you encounter an error in IP.Board, it will now default to sending a “500 Internal Server Error” status. If the error is due to missing/deleted content, you’ll get a 404 error, and if it’s due to an authorization problem, you’ll get a 403 error. This should help the search engines better understand these pages and drop things from their indexes as appropriate.

• Improved alt attributes on images: As you may know, the HTML standard requires that all images have an alt tag, which means that a lot of web applications are littered with alt tags such as “icon”, or “image” – which do nothing to help for accessibility or SEO. IP.Board was no different. In 3.1, however, a lot of these alt tags have been updated to use useful descriptions that either describe what they represent, or the action they may be associated with.

• Social networking sharing: Whilst strictly not an SEO feature, another cool addition coming in IP.Board 3.1 that will help with both search and promotion, is the addition of “Share This” buttons in topics. The buttons will allow your members to share content from your forum on social networks such as Twitter and Facebook, making it more easy to find for users and search engines alike.

… And there’s more to come. I for one am delighted to see IPS taking these important steps to cover some of the details that forum vendors typically forget. For full details, see the official blog post at the IPS Community Forums.

• Renamed to Page Profiler – makes more sense, as it only checks individual pages
• Speed improved, also no longer an AJAX loaded page (result pages can also be linked to)
• Presentation improved
• Added data size (Plus GZIP compression checking and GZIP data size)
• Added HTTP response header
• Added title and meta description
• Added internal link reporting
• Added number of redirects encountered when loading URL
• Added request time
• Improved whois checking
• Added warnings for key SEOMoz Web Developer SEO Cheat Sheet Recommendations.

The user agent you may see in your access logs for this tool is now:
Mozilla/5.0 (compatible; PageProfiler/2.0; +http://www.dancryer.com/page-profiler)

If you’re using the tool, or have any suggestions for improvements, please let me know in the comments below!

If you want to try it out and you don’t see the “Latest” link, give this link a try.

I’ve been playing around watching a few queries for a while now, and I’m wondering, can a little blog like mine pop up in the stream? If so, how quickly? This post is my test.

I’d like to start out with some key points:

• It’s all about the content.
• Changing your Meta keywords and description tags alone is not SEO, merely a contributing change.
• Friendly URLs alone are not SEO, nor are they “SEO URLs”. Even a perfect friendly URL won’t help you at all, unless you get the rest right.
• The PageRank score you see in Google’s toolbar does not correlate with your ranking for any given keyword, so don’t worry about it.
• Forums are great for search by their very nature, you shouldn’t need to sweat the details.

I’ve split the remainder of this article into two distinct parts. “On your site”, and “elsewhere”. On your site will help you with what you can do to set up the site itself. Elsewhere will talk about what else you could or should do.

What you should do on your site

The great thing about forums is that they’re naturally filled with content. Infact, they’re all about the content – everything else is secondary. If you’re using your forum for any other purpose (such as primarily using it as a revenue stream, or a “social network”), you may want to think about why it is that you’re using a forum. When you set up a forum, it should be about something you (or your company) know and love, as you’ll be key to getting the initial content in place.

This is great, as search engines love content. Another great side effect to having a lot of content related to your chosen subject is that the search engines will begin to see you as an authority on that subject, especially as people start to link to you. Additionally, they’ll pick up all of the different keywords you and your members have included in your posts, and you may start to rank for them – so if you have a forum about farming, for example, you may start to see it appearing in searches for ‘cattle’, ‘tractor’, or ‘sheep shearing’, just because your members have mentioned them. Whilst it’s not a forum, my blog is a great example – I’m currently ranking very well for searches related to the X Factor and the UK’s Christmas Number One, along with the Evans Halshaw car company. I didn’t do this by worrying about URLs, link exchanges, or my PageRank – just about writing (arguably) good content.

Some changes you can make, however:

• Friendly URLs: Whether you’re setting up a new forum, or you’ve upgraded your existing forum to IP.Board 3, i’d definitely recommend enabling the built in friendly URLs feature. Make sure you have all the redirection settings enabled, and use “301″ (permanent) redirection, and make sure you’re using the Apache mod_rewrite option. Whilst this isn’t going to affect your rankings much, it does help users recognise what they’re going to see, and the search engines will start finding relevant keywords in your URLs.
• Skins: If you’re going to use a custom skin for IP.Board (which I’d definitely recommend), make sure the HTML for it is well written. Your forum and topic titles should be in the <h1> tag, you should include the meta description tag that’s present in the default skin. If you want to take it a step further, you or your designer could look at making sure that the forum content (The topic you’re viewing, the forum listing, etc.) appears above the header and navigation in the markup – this ensures that it’s the first thing search engines find in your page, and can be achieved with some simple CSS changes.
• Sitemap: Get yourself a sitemap plugin. There are a bunch of them available, some likely much better than others. This will generate an XML file that you can submit to search engines to help them to crawl your content more easily.
• Robots.txt: IP.Board ships with a simple robots file, named robotstxt.txt. As soon as you upload the board, rename this to robots.txt and it’ll help to exclude some pages on the forums that search engine spiders should not index. This includes things such as your admin directory, cache directory, user control panel and redirect pages.
• Categorisation: Whilst this is arguably not SEO, make sure that you have only the number of forums that you require, don’t create categories and forums and sub-forums for every potential discussion grouping. Strong categorisation of content will make your user’s lives easier, and will, in the long run, help the search engines to understand the structure your site better. It’ll make things such as Google’s new breadcrumb trail recognition better, along with increasing potential for getting site-links generated by the engine.
• External integration: Depending on the type of content your forum will contain, consider adding “Tweet This”, “Digg This”, “Bookmark this”, type social links to the first post of every topic. This is a great way to get some attention for your content. Additionally, as long as it is appropriate, make sure you enable OpenID and Facebook connect log ins. Whilst also arguably not SEO, this does lower the barrier to entry for new users.

One thing I wouldn’t recommend, however, is to use third-party “SEO” add-ons such as Minerva SEO or Community SEO, they’re really not necessary. A lot of users migrating from vBulletin see the lack of the vBSEO plugin as being a big loss when they switch to IP.Board, but realistically it isn’t. It’s rare that these users can actually give you a valid reason that vBSEO was useful or necessary for them, nor does anyone have any metrics to prove its value. IPS themselves have a very well indexed forum, and, obviously, have never used any of these. The built in functionality is more than enough, and installing these third party hacks merely prevents them from working correctly, and makes it more difficult for you to take advantage of improved features that will come in future releases.

To reiterate and add to my initial point. Your forum should be all about the content. That means that if you’re just starting out, you need to be posting good content, regularly. Once your site starts to become popular, you’ll be getting a lot of content from your members, but your input shouldn’t stop there. The best thing you can do to aid in good rankings, and to attract more users to your forum, is to be writing great content on a regular basis. That means if you know a lot about the subject you’re covering, you should be writing detailed articles about it. If it’s something that changes often, make sure that you’re keeping existing content up to date, and posting news about changes and related events. This content should be unique to your site, don’t just copy and paste from elsewhere. It’s fine to be inspired by outside sources, but write your own articles.

The final thing I’d recommend in this section is that you make sure that any advice you take is backed up by more than one person, or comes from a reputable source. There’s been advice from people who sound like they know exactly what they’re talking about, which is just plain wrong. This includes things such as removing the board name from the <title> tag on every page in the name of SEO, this isn’t going to help, it’s likely it could even hinder you. Unique titles are important, but having common keywords in there, especially your site name, is not going to hurt.

What you should do elsewhere

The previous section of my post was all about what you should do on your own site, from optimisation tips to writing great content regularly. This section will cover the key things to do elsewhere, and in a lot of ways, it’s more of the same. Arguably, if you’ve followed the key point of the last section, you’re writing great content – and that’s going naturally get people linking to you, so there’s no need for you to go on a link building / link exchange campaign.

However, to get your site going, one of the best things you can do is to participate on other people’s sites as well as your own. This might sound a bit odd, but bear with me. If you’ve got a link in your signature on a forum where people of similar interests can be found, you’re doing a great deal to promote your own web site, if you post quality, authoritative content on other people’s web sites. You’ll be respected by the site owners, so they won’t remove your link, and you’ll attract the interest of other members on those forums, who’ll start looking at your site too. The same advice applies to related blogs.

With that being said, do not spam other sites. Only post on other sites if you’ve got something worthwhile to contribute. Posting on another site merely to advertise your own is not useful, and will likely only serve to lose respect for your site – it might even get you banned from that site, or your own site banned from the search engines.

You might also wish to consider setting up a Twitter account and Facebook page for your forum, and make sure that you push only your best content to it. If you’ve written a great article, tweet about it – but don’t tweet every time someone posts, you’ll only be ignored. If you set up a Facebook page, make sure that members can’t post much content to it, have it so that your content is pushed to it as another method of distributing it to users and potential users.

With promotion set aside, a lot of off site SEO work is about statistics. You need to get yourselves accounts at all of the following, and look at the data they provide you on a regular basis:

• Google Webmaster Tools – This is Google’s outlet for telling you how your site is doing in the search engine. It’ll tell you what terms your site is appearing for and which terms are actually generating clicks. It’ll tell you if it’s struggling to index your content, and why, along with a whole host of other information. If you’re appearing for a lot of terms but not getting any click throughs, you might want to consider writing an article specifically about that topic. If you took my earlier advice of getting a sitemap plugin, submit the resulting sitemap here. It’ll help Google to index your content more efficiently, and learn about what content you consider to be worth ranking (hint: that’s not everything.)
• Google Analytics – This is another Google outlet, more specifically focussed on how users find your site, who they are and what they do when they get there. It’s truly invaluable.
• Bing Webmaster Central – This is the Microsoft Bing equivalent to Google Webmaster Tools. It’s not as good, and not as helpful, but can still provide a useful insight into how they view your site.
• Yahoo Site Explorer – As above, but for Yahoo! search.

Summary

All in all, I’d argue that as long as you and your users are creating great content, as a forum owner, you don’t need to worry too much about search engine optimisation. If you write great things, people will come and those with sites will link to it. If you have great content and people are linking to it, the search engines will like your forum too. This is what they’ve all been saying all along, and frankly, it’s true.

I hope that you’ve found this article useful. Whilst it’s not a comprehensive guide, it should get most people started. If you’ve got anything to add, or any feedback, please send me it. I’ll very gladly update this article or post follow ups with other great tips and tricks. Additionally, if you have any specific questions or would like some advice, post it below and I or someone else, will try to answer it for you.

a[rel~=nofollow]
{
background-color: #FCC !important;
border: 1px dashed #F55 !important;
color: #600 !important;
}

I should really have thought of that in the first place! Want the extension? Download it here.

It’s worth noting that the tool crawls roughly five pages on the target site each time it is run, so you may notice the following user agent in server logs:

Mozilla/5.0 (compatible; SiteProfiler/1.0; +http://www.dancryer.com/site-profiler)

I thought I’d make it available on this site, so others can use it. If you like it, or you’ve got any feedback, please let me know.

• Anti-Gravity Mice
• Dutch Penis
• Penis Chandelier

His next attempt is for Swedish Sperm Bank Lesbians, but I want to know if I can make him rank for “croatian camel herding scallywag“, just by linking to him. If not, I should at least pop up amongst the other 303 results.

var elements = document.getElementsByTagName('a');
var length = elements.length - 1;

for(var i = length; i >= 0; i--)
{
if(elements[i].getAttribute('rel') && elements[i].getAttribute('rel').indexOf('nofollow') >= 0)
{
elements[i].className += ' nofollow_has_no_follow';
}
}

Google Chrome Dev Channel users can Download and Install it now.

This all started on Wednesday, we’d been looking at the queue and decided that it was far too large. We’d hit 12.5M pages waiting to be checked, and were processing about 200,000 a day, tops. The initial reaction, as this is a distributed system, was to scale it horizontally. We rented six more VPSs (to add to the existing 5) and planned to run them at capacity until the queue was empty. After setting all the boxes up, we left them over night, in hope of completing 750,000 or more pages. Boy, were we wrong. On Thursday morning, we came in to discover it was running at a peak of 5,000 pages an hour (144,000 a day). We’d slowed it down, and not only that, we’d slowed everything else down too.

This is when the optimisation work started. We looked at the queries we were running, and noticed that an obvious bottleneck appeared to be the queue itself. Each VPS ran 15 instances of our application at any given time, and each one did a query for 250 pages to check when it was starting up, updated them as in progress, and deleting each from the queue on completion. The first change we made was to change the indexing on the table, which merely served to slow us down further. The second, more interesting change, was to change the entire table from innodb to the memory/heap table engine. This meant changing both max_heap_table_size and tmp_table_size to over 2.5GB to accomodate the 1GB table at the same time as allowing temporary tables to be created. Getting this change replicated onto our two slaves was a delight.

The memory table “solution” worked well after turning all of our VPSs back on, for about 10 minutes. At which point, they were up to capacity and the queue table started locking, a lot. As did all of the other tables, inexplicably. In addition to inducing panic, this highlighted to us that another table was starting to cause problems, our anchor text table. This is responsible for storing only the words that are used to link from one site to another. This table was seeing anything up to 2 million inserts an hour, and the same number of selects. After several failed attempts to remedy this, we finally settled on selecting all of the anchor texts we’d just collected, any that came back from the database were then dropped from our insert, thus only inserting those that were not present. Those inserts were done with batches of 10 per query. This cut the number of queries down significantly. Whilst this has improved the locking situation, it has, unfortunately, led to hanging selects. These selects are using potentially gigantic ‘IN(one, two, three)’ conditions, and are taking anything from 0.01 to 120 seconds to complete. Completely unpredictably.

Next priority was fixing the table locking on our queue. We’ve done this by marking all deletes from the queue as LOW_PRIORITY. In doing that, we’re allowing MySQL to queue those deletes for when no other clients are accessing the table, preventing unnecessary locking.

At the time of this writing, 113 instances of our application are running without significant conflicts on either of the tables we’ve optimised. However, we’re still processing just 5,000 pages an hour. I’m sure I’ll be working on this problem again on Monday, and will update here with anything we do discover.

So to summarise this somewhat rambley and unhelpful post a little, don’t assume that because you’ve built your system to run across multiple servers, that it can infact run across multiple servers. Ours scaled up to 4, and then the scaling started working against us. If you found it interesting, have any MySQL questions, want to offer advice, or are doing similar work yourself… drop me a comment below. I’d love to know that people are reading this!

Naturally, I posted about this on the Google Webmaster Help forums, and the general guidance I got was that the homepage doesn’t feature my name enough. I’ve now added a short “about me” bit on the homepage, and also changed my author name in WordPress, but I can’t think what else I can do to improve it. I’m also stuck for what to do to correct the problem, I’m assuming I haven’t been penalised, as I’m not doing anything dodgy.

Hopefully it’ll come back, but if anyone has any ideas, I’d very much appreciate them.