As any readers of the iContact blog may have learned, MogileFS has become an integral part of our infrastructure at iContact. Rather than store the bodies of messages in our database, we moved them to a quick&dirty storage method in our infrastructure long ago. This method was essentially a cheap WebDAV server and on each STORE command it would write to two backend servers and issue a GET from only one. About a month ago, we migrated most of our messages away from this older, less scalable method to our newer MogileFS backend.
Our MogileFS setup allows the disk space on each web server (normally unutilized) to form a cheap storage node, and make use of space that would otherwise go entirely unused.
On Monday 1/21 the database servers behind MogileFS paged with too many connections, which leads to Mogile going very slowly for a while, and sometimes requiring a restart of some of the nodes.
This database issue cascaded into us asking our Mogile client for item A, but receiving item B in response…
(more…)
Related posts
To run the class I posted yesterday, I typically use a class called DaemonRunner. This class sets up for proper signal handling, and ‘executes’ the class extended from Daemon.
declare(ticks=1);
class DaemonRunner {
public static function exec($className) {
$argv = $_SERVER['argv'];
$daemon = new $className($argv);
$daemon->init();
while (!$daemon->isDone()) {
$daemon->run();
}
$daemon->shutdown();
}
}
The first statement is a PHPism. This allows PHP to check for signals every 1 tick. A tick is simply a low-level PHP interpreter step. The way I’ve setup my daemon and signal handling by default, the PHP will finish its current iteration and then quit when receiving a signal. This ensures that the database, and whatever background stuff aren’t left in inconsistent states. As with anything, you’re free to modify, extend, or otherwise alter this behavior.
Related posts
PHP (or any language for that matter) can just as easily be used as a daemon as on the web. This can be especially useful when solving problems that can’t “complete” in less than 500 ms which one shoots for on the web. As you write an application to handle things such as those mentioned below, be sensitive to the processing, disk space, and time needed to process these requests in planning how to handle these. As an example jobs of a small size might be handled inline by the code that handles the form submission, but for larger jobs it queues them for background processing.
great for background/async processing of:
- photos
- movies
- reports
- imports of data
advantages:
- use the same codebase as your app
- use the same expertise on your team of programmers
challenges:
- signals
- one thread
- memory (this will have to be a whole separate article)
- starting and stopping
A typical Daemon class I use:
abstract class Daemon {
protected $done = false;
public function __construct($argv) {
}
public function init() {
pnctl_signal(SIGTERM, array($this, "onSignal"));
pnctl_signal(SIGINT, array($this, "onSignal"));
}
abstract public function run();
public function shutdown() {}
public function isDone() {
return $this->done;
}
public onSignal($signal){
switch ($signal) {
case SIGTERM:
case SIGINT:
$this->done = true;
break;
default:
// handle all other signals
}
}
}
As you can see, we’ve started to tackle some of the challenges here. The idea is to handle signals, specifically I usually care about SIGTERM (default when using ‘kill’ on *nix) and SIGINT (from pressing Ctrl-C). Both will cause the program to exit gracefully.
I’ll be posting the class I use to run Daemon’s tomorrow.
Related posts
