VPS.NET Auto Recovery Monitor

Posted June 29 by Dan Cryer

I’ve been using VPS.NET for months now, both for myself and for Media Wow. Whilst their platform is incredibly stable, and their automated failover is great, there are still occasionally times when a server will go down and not come back up by itself.

The nicely integrated Server Density monitoring does a great job of informing you if a server is down, but it can’t do anything about it. So I’ve created a script that I run on an external server to check all of the servers are up (via HTTP) and reboot those that are not. It’s fairly simple, the process is as follows:

  • Get a list of VMs from VPS.NET.
  • For each VM, send a GET request to http://{hostname}.{domainname}/ and give it (by default) 5 seconds to respond.
  • If the server does not respond, it will check if it is marked as “running” on VPS.NET – by default it will then assume that there is simply a routing issue.
  • If it is not, it will check if the server is already in a power change state (i.e. startup, reboot).
  • Finally, if neither of the above are true, it will send a power on request to VPS.NET.
  • After all of that, it’ll (optionally) send you an email to let you know.

The code is below. You can also download it here. Let me know if you find it useful!

Notes:

  • This script needs setting up as a cron job, on an off-site server / computer. I recommend running it every 5 minutes or so.
  • I don’t use the VPS.NET API PHP classes because they seem to be out of date.
  • This script *will* spam you to oblivion if there is a cloud issue and you set it up to email in all cases.
< ?php

//----
// Set up the monitoring script:
//----

define('MONITOR_MAIL_TO', '');     // Email address to send alerts to.
define('MONITOR_API_EMAIL', ''); // Email address associated with VPS.NET API account.
define('MONITOR_API_KEY', '');     // VPS.NET API key.
define('MONITOR_SEND_EMAILS', true);               // Whether or not to send emails if a server is flagged as down.
define('MONITOR_SEND_EMAILS_ON_NO_ACTION', true);  // Whether or not to send emails if no action is taken.
define('MONITOR_REBOOT_IF_RUNNING', true);         // If HTTP check fails, but VPS.NET says the server is up, reboot?
define('MONITOR_HTTP_TIMEOUT', 5);                 // Number of seconds to allow for a server response. 

//----
// Initialise API Class and get VMs:
//----
$api = new SimpleVpsApi();
$api->setApiEmail(MONITOR_API_EMAIL);
$api->setApiKey(MONITOR_API_KEY);
$vms = $api->getVMs();

//----
// Loop through all the VMs and check their status:
//----
$out     = '';
foreach($vms as $vm)
{
	$server = $vm['virtual_machine']['hostname'] . '.' . $vm['virtual_machine']['domain_name'];
	$id     = $vm['virtual_machine']['id'];

	$get = false;
	$get = @file_get_contents('http://'.$server.'/', false, stream_context_create(array('http' => array('timeout' => MONITOR_HTTP_TIMEOUT))));

	if(!$get)
	{
		$_out .= 'Server ' . $server . ' appears to be DOWN!'.PHP_EOL;

		$vm = $api->getVM($id);

		if($vm['power_action_pending'])
		{
			$_out .= '- VPS.NET is reporting that the server is rebooting already. Taking no action.'.PHP_EOL;

			if(!MONITOR_SEND_EMAILS_ON_NO_ACTION)
			{
				$_out = '';
			}
		}
		elseif($vm['running'] && !MONITOR_REBOOT_IF_RUNNING)
		{
			$_out .= '- VPS.NET is reporting the server as running. Taking no action.'.PHP_EOL;

			if(!MONITOR_SEND_EMAILS_ON_NO_ACTION)
			{
				$_out = '';
			}
		}
		elseif($vm['running'] && MONITOR_REBOOT_IF_RUNNING)
		{
			$api->rebootVM($id);
			$_out .= '- VPS.NET is reporting the server as running. Rebooting.'.PHP_EOL;
		}
		else
		{
			$api->bootVM($id);
			$_out .= '- Submitted an automated power on request to VPS.NET.'.PHP_EOL;
		}

		// If we've logged anything for this VM, append it to the log:
		if($_out)
		{
			if($out != '')
			{
				$out .= PHP_EOL;
			}

			$out .= $_out;
			$_out = '';
		}
	}
}

//----
// Emailing functionality, for notification of server downtime:
//----

$email = '
	
Server Monitoring Notice
'; if($out != '' && MONITOR_SEND_EMAILS) { $email = str_replace('', nl2br($out), $email); mail(MONITOR_MAIL_TO, 'Server Monitoring Notice', $email, 'Content-Type: text/html'."\r\n"); } //---- // Custom class for interacting with the VPS.NET API: //---- class SimpleVpsApi { const TYPE_GET = 1; const TYPE_POST = 2; protected $apiUrl = 'api.vps.net/'; protected $apiEmail = null; protected $apiKey = null; /** * Set the email address to use in requests: */ public function setApiEmail($email) { $this->apiEmail = $email; } /** * Set the API key to use in requests: */ public function setApiKey($key) { $this->apiKey = $key; } /** * Get a list of all VMs: */ public function getVMs() { $vm = $this->sendApiRequest('virtual_machines', 'GET'); if($vm) { return $vm; } else { return array(); } } /** * Sends a request to VPS.NET to load details about a VM: */ public function getVM($id) { $vm = $this->sendApiRequest('virtual_machines/'.$id, 'GET'); if($vm) { return $vm['virtual_machine']; } else { return null; } } /** * Sends a request to VPS.NET to reboot a VM: */ public function rebootVM($id) { return $this->sendApiRequest('virtual_machines/'.$id.'/reboot', 'POST'); } /** * Sends a request to VPS.NET to boot a VM: */ public function bootVM($id) { return $this->sendApiRequest('virtual_machines/'.$id.'/power_on', 'POST'); } /** * Send an API request to VPS.NET */ protected function sendApiRequest($uri, $type = 'GET') { if(!$this->apiEmail || !$this->apiKey) { throw new Exception('You must set an API key and email address first.'); } $opts = array( 'http'=>array( 'method' => $type, 'header' => 'Accept: application/json' . "\r\n" . 'Content-Type: application/json' . "\r\n" . 'Authorization: Basic ' . base64_encode($this->apiEmail.':'.$this->apiKey) . "\r\n", 'timeout' => 3, ) ); $url = 'https://'.$this->apiUrl.$uri.'.api10json'; $context = stream_context_create($opts); $response = @file_get_contents($url, false, $context); return json_decode($response, true); } }

edc17388fb0743a4910e77ba4c1b8171

2 Responses to “VPS.NET Auto Recovery Monitor”

  1. Harry Warner says:

    Sounds good… the only thing that concerns me is that the VPS.NET console sometimes reports the server as turned off, when it’s actually “Running a task”, and if you tell it to power on it’ll get stuck, and be impossible to turn on without a ticket to support.

  2. Mikey says:

    I’ve had a less than favorable experience with VPS.net – I get downtimes every 2 weeks (like clockwork) and wouldn’t recommend them to anyone.

Leave a Reply


Notice: Undefined index: HTTPS in /home/dan/public_html/wp/wp-content/plugins/stats/stats.php on line 111