Posts tagged: PHP Conference 2010

MySQL Load-Balanced Cluster Guide – Part 3

Posted January 28 by Dan Cryer

This article is the last in my MySQL Load-Balanced Cluster Guide series, following on from my earlier post today on setting up MySQL Monitoring on the cluster nodes, this article is a guide to setting up the load balancer with HAProxy, using the monitoring scripts we’ve created to check the node status.
HAProxy is a well [...]

Read More »

MySQL Load-Balanced Cluster Guide – Part 2

Posted January 28 by Dan Cryer

Following on from yesterday’s guide to setting up MySQL master-master replication, the first of today’s guides is how to set up a script to monitor the status of your MySQL cluster nodes, which we’ll use in the next guide to set up our proxy.
On every one of your MySQL cluster nodes, follow the steps below [...]

Read More »

MySQL Load-Balanced Cluster Guide – Part 1

Posted January 27 by Dan Cryer

I’m currently working on setting up a load-balanced many-master MySQL cluster at work, without using MySQL Cluster as it’s incompatible with our needs. Finding good guides on how to do this has proven incredibly difficult, so I thought I would document the process here.
This first post is going to cover setting up the servers themselves [...]

Read More »

Looking forward: Crawler version 2.0 and 3.0

Posted September 17 by Dan Cryer

Alongside my work on our current crawler, Wade has been rewriting the system to make better use of what we’ve learned so far, and make a number of changes:

The current codebase is a mess, many cron jobs running terribly written classes, across two separate crawler systems. The new system is neatly organised with models and [...]

Read More »

PHP/MySQL Scaling Followup

Posted September 14 by Dan Cryer

I posted last week about our work on scaling our crawler application and it’s corresponding MySQL database, but left it with very little conclusion, apart from that Memcached is good. I wanted to follow up with some of the changes we made on Thursday and Friday.
Having run the code with the changes we made previously [...]

Read More »