david rasch - making stuff work » web

FuelFrog is out!

drasch — Sun, 04 May 2008 01:49:35 +0000

The answer is, FuelFrog. Over the past 9 weeks, we’ve conceived an idea, branded, implemented, tested, launched, and started to market a brand new web application. I’ll be talking more over the next few weeks about the technology behind FuelFrog and what’s coming up for new features!

We know that you care about how your car affects the environment. What better way to understand and manage this than to start tracking it! FuelFrog will help you keep an eye on your gas consumption, mileage, and also help you understand how gas prices affect your budget.

Through integration with Twitter, it’s easy to keep FuelFrog up to date. Each time you fill up, simply send @fuelfrog miles price gallons for example, my last fillup was @fuelfrog 320 3.539 10.293. We’ll do all the rest!

Log on to FuelFrog now and try it out!

online storage

drasch — Sat, 16 Feb 2008 14:45:31 +0000

More in the “bigger isn’t always better” thread alluded to in my offsite backup post. Apparently, Amazon S3 was down for over 35 minutes.

MogileFS and race condition

drasch — Wed, 30 Jan 2008 02:14:47 +0000

As any readers of the iContact blog may have learned, MogileFS has become an integral part of our infrastructure at iContact. Rather than store the bodies of messages in our database, we moved them to a quick&dirty storage method in our infrastructure long ago. This method was essentially a cheap WebDAV server and on each STORE command it would write to two backend servers and issue a GET from only one. About a month ago, we migrated most of our messages away from this older, less scalable method to our newer MogileFS backend.

Our MogileFS setup allows the disk space on each web server (normally unutilized) to form a cheap storage node, and make use of space that would otherwise go entirely unused.

On Monday 1/21 the database servers behind MogileFS paged with too many connections, which leads to Mogile going very slowly for a while, and sometimes requiring a restart of some of the nodes.

This database issue cascaded into us asking our Mogile client for item A, but receiving item B in response…

The problem turned out to be at the lowest level, that of the socket communication with Mogile. The request chain asked for item A, but since the database was responding very slowly, the “fread” waiting for the response timed out. Unfortunately, the timeout simply passes back up the chain as a “not found”. In the background, the data for the response to the request was buffered by TCP. We handled the error, told our daemon to check later for that item, and proceeded to ask for item B. When doing another “fread” for the response, we received the data from the original request. Since all was well, we continued requesting items and receiving a non-matching item in response.

In order to fix this on our end, we simply made the connection terminate when any of these timeouts occurs. Additionally, we’ll be adding application-level matching for these requests and responses (the data we store in Mogile will likely include the key so when we pull it back out, it can be check-summed against the requested key).

However, I wonder if it’d make sense for the MogileFS protocol itself to support this matching. It seems key that this type of problem not affect others, so I encourage you to check your client for such a problem, and I plan to file bugs against those who have the bug we found. We were using an old PEAR module as we sought a client with a non-GPL license for inclusion in our codebase. However, a cursory examination of the PHP extension we were coveting seems it might be subject to the same problem.

Other than this issue (and the large headache it caused), which mostly lies on the client side (unless I get any pickup on the protocol-level support above), MogileFS has proved to be a great asset to our infrastructure and I look forward to seeing it scale to much more data.

Gallery 2.2–Multisite and Multiroot

drasch — Tue, 25 Sep 2007 02:48:54 +0000

Recently I tried to use the Gallery Multiroot module/plugin to setup a new site based upon an existing Album in our gallery. Since we started sorting albums chronologically, we wanted to sort out the Dachshund photos separately. To do this, I used the Multiroot feature. Unfortunately, I couldn’t get this to work as I’m already using the Multisite plugin to host several Galleries off of my web server. The magic touch was to use the following file, the first line is key and had to be added to make things work in the Multisite configuration.

define('GALLERY_CONFIG_DIR', "/var/virtualwww/gallery.raschnet.com");  //I had to add this line

require('/usr/local/share/gallery2/embed.php');
$ret = GalleryEmbed::init(
    array('embedUri' => '/',
          'g2Uri' => 'http://gallery.raschnet.com/',
          'apiVersion' => array(1, 2)
    ));

if ($ret) {
    print '' . $ret->getAsHtml() . '';
    return;
}

$gallery->setConfig('login', true);
$gallery->setConfig('defaultAlbumId', 28426);
$gallery->setConfig('breadcrumbRootId', 28426);

GalleryMain();

Now live: DRNA Gallery

The Future of Web Apps

drasch — Sun, 27 Aug 2006 03:59:20 +0000

I’m looking forward to attending the Future of Web Apps in San Francisco in a few weeks.

prior art–web application installers

david — Sun, 12 Mar 2006 01:39:47 +0000

Every widely distributed PHP application has its home grown installer that looks for depedencies, sets up a database, and checks permissions. Since PHP is interpretted, has no language support for namespaces, and no stored application state–simply defining how a PHP application gets installed is a non-trivial function.
There have been a attempts to create a common PHP application installer. They’ve faced lack of adoption, the same loose coupling and installation as the problem it attempts to solve,

I propose an extension to PHP, or even to Apache which allows the adding, upgrading, and removing of web applications in a fashion similar to the maintenance of WAR files for Tomcat. This will allow applications to ensure their needs are met, depdendencies installed, and allow the distribution of component libraries for supporting installations. I only hope this can enable the easy distribution of open-source applications like NSIS (Nullsoft’s Installer) has done for desktop open-source applications in Windows.

[tags]installer,opensource, web[/tags]