October 29, 2006

CouchDbX - CouchDb for the Mac

Jan Lehnardt has created CouchDbX, pre-built, universal binaries for CouchDb 0.5.0 alpha on Mac OS X. At 29 meg it's a big-un, but it's everything you need to get CouchDb running with no hassle. (but it is alpha level software, so no guarantees!)



Project Home

Jan also wrote up a great post detailing his adventure porting CouchDb to Unix.


October 26, 2006

Y Combinator

A month or so ago I filled out the rather short and simple Y Combinator funding application form for CouchDb. I heard back a couple of days ago and I'm interviewing with them in Cambridge on November 5.

This is really cool because while I'd love to have funding for CouchDb I've decided against pursuing venture capital funding (for a number of reasons). But Y Combinator's investment approach seems far more rational than the traditional VC way. Although they really do seem to be targeting college kids, everything else about them is a good match for the project.

So we'll interview and hopefully good things will happen. I've already asked them about blogging this stuff (a very big no-no with VCs), and they didn't immediately tell me to go hell, so stay tuned.

Anyone have a place in the Boston area for me to crash for a night or two? (I'm planning Nov. 4 and 5) Taken care of!
Big thanks to Ned Batchelder and Bruce Perry and their respective generous families for giving me beds to sleep in.


October 23, 2006

Living with a Ghost

Your house has a ghost. Most of the time as you go about your business, nothing eventful happens, but occasionally you notice weird things. Usually it's harmless stuff, you turn off a light, then it turns back on. Sometimes you hear the toilet flushing randomly or the radio turn on. And every once in a while, a sharp knife just floats out of the kitchen drawer and comes at your face.


It's quite terrifying, stuff just moves around on its own!

But then one day you come across an Ouija board and you decide to communicate with the ghost. Turns out the ghost is a nice fellow and isn't trying to harm you. Not at all, in fact, he's similarly jeopardized by your activities (he said you've almost stabbed him 4 times!). He says he's not leaving, he needs to live in the house the same as you. But he's a reasonable ghost and you work out a system where you can both live in the same house, share the same stuff and not stab each other in the face.

You quickly figure out for many things in the house there is only one of you who ever use it, so those things will need no coordination. And for some things it doesn't matter if you both try use it. Like the couch, it's not a problem if you both sit on it, it still works just fine as a couch. That's what couches do, they allow many people to REST simultaneously.

Ghost reading Chicken Soup for the Soul

Other things require some coordination, like the shower and stove. So you devise ways to signal to the other side when you are going to use it, like putting a piece of tape on the stove to show its being used, and if there is tape already on there you wait until its gone before you use it. This mostly works, but sometimes you suspect it's not as efficient as you'd like. Most of the time when the ghost has his tape on it he's only using one burner and sometimes the stove isn't even on. What the hell is he doing over there?

Occasionally the ghost or you get a little confused about the coordination procedures and who's allowed to use what and then accidents happen. Stuff gets spilled and things get lost. It sucks, and sometimes causes a big giant mess for everyone, but for the really important things you guys are pretty careful and bad stuff doesn't happen. Much.

You'd like to move somewhere without ghosts, but you find out pretty much every house in town is this way. Some houses have more than one ghost. Some of the really big houses have dozens, hundreds, even thousands of ghosts. You've been warned to avoid those houses, most people quickly go insane trying to live there. One ghost is bad enough.

But there is one neighborhood you've heard of where there are no ghosts, things always behave consistently and rationally, nothing ever magically moves or changes before your eyes, knives never suddenly fly at your face. It's rumored to be pretty expensive to live there, you hear only the richest, most vitally important people live there. While you don't have the exact figures you know you probably can't afford it, not with your meager resources. No way.

So you go on living with ghosts.


Twist ending: The other side wasn't a ghost, it was M. Night Shyamalan and this whole story was really about the problems of shared state threading (try to act surprised). Take that Hollywood!


October 21, 2006

CouchDb 0.5.0, Linux Port and More

Thanks to the hard work of Jan Lehnardt, CouchDb now builds, installs and runs on Linux. That's a huge step for the project!

And on Windows the project can now also build a full server kit from a single command. The build setup process is still a bit harder than it should be on both platforms, just a warning. Coming soon will be the Mac OSX port, Jan already has it running on both PPC and Intel Macs but it needs a little more work yet. Correction: This version does build and run on Mac Intel using most of the same build instructions for Linux. Updated build instructions and precompiled universal binaries for the Mac are coming.

Also in this build is the new built-in CouchPeek admin utility created by William Beh.

To use it, just visit a special url on the CouchDb server:

It uses the Dojo framwork and is quite slick. I have a zillion features I already want to add to it ;)

Get Stuff

CouchDb for Windows (unzip and launch startCouchDb.cmd):

CouchDb GPL Project Source (Builds on Windows and Linux):

You can get the latest source from the SVN repository here:

If you have problems, questions, suggestions, etc the best place for them is the CouchDb project forum.

One last thing, check out this really cool project badge Ken Tango created:

That is just so awesome!


October 16, 2006

Bad CaRMa

Randy proposed building, from scratch, a completely data-driven application. Most conventional database applications are written in code using programming languages that access and manipulate data. Code is very difficult to change once it is written, but code controls data.

Randy's idea was that all of the forms, all of the reports, all of the functionality of the system would be stored as metadata, or data about data, and the only thing coded in a programming language would be an engine to process both metadata and data. For example, instead of defining a table just to store order-line information, why not also store the basic business logic about inserting, updating, and deleting order-lines in the table, as well as details about the structure of an order-line. Once an engine was built, then that would be the end of all coding, forever and ever, amen. After that, the engine would process data about data then the data itself, and in this fashion any form, report, or program could be built. When new functionality or modifications were needed to the system, you only had to change the metadata, not the code.

Bad CaRMa

This great story reminds my why I cringe every time I hear about a new project creating some revolutionary rules engine. Nowadays developers tend to do this stuff in XML files instead of databases, but the basic motivation is the same: to avoid boring coding.

The general thinking is instead of programming a boring old business application, you can build an amazing new meta-application framework that allow your users to build their very own customized business application. You get to build something cool, and your users get just what they want, everyone wins! But since users can't code (that's why they are the user and you are the programmer), you develop new declarative structures your users must now learn how to manipulate. In the end, to make it powerful enough to meet user application needs, you build something too complicated for them to use. So you wind up being the only who who can program the system. Been there done that.


October 13, 2006

CouchDb new look

projectsitecap.PNGThe CouchDb project site has gotten an awesome new design by Ken Tango. I worked with Ken at Iris Associates and he does first rate graphics and design work. If you want something to look nice you should hire this guy, he's a real pro. Here's a link to Ken's site.

And thanks again to William Beh, the project site webmaster for implementing Ken's new design.

Also, I've got a new version of CouchDb coming, probably Sunday. Thanks to the efforts of Jan Lehnardt this new version builds and runs on Linux and Mac OSX!


October 8, 2006

CouchDb, PHP and the offline Web

For a while I’ve been kicking around an idea of how to integrate CouchDb with PHP. PHP is a great fit for CouchDb, both the development toolsets and runtime model should integrate nicely, and CouchDb can teach PHP a new trick: web apps that can work in a offline mode.


The Integration Model

The way PHP works, applications are typically just a directory of PHP/HTML files that interact with a database, usually MySQL. Also in the directory are various resource files needed by the application (.css, .jpg, .js, etc). These application files can be zipped up - code, HTML, images and all - into a single archive file and then redeployed on another server.

To integrate CouchDb with PHP, the same model is employed, with code in PHP files interacting with a CouchDb instance instead of a MySQL instance. The Couch/PHP application and resource files can be then zipped up into a single file, just like before.

With CouchDb, the PHP application zip file can be stored in the application database as an attachment to a design element. The design element can also contain the computed table formulas and other resources necessary for the app at runtime. A whole PHP/CouchDb application can be stored in a single design element.

These design elements are just a special documents that will replicate around just like normal application documents. Then, a small bit of code on the CouchDb/PHP server will monitor when CouchDb design elements are updated. When they are, it retrieves the compressed files and unzips the PHP application into a temporary directory on the PHP server. These files are served up like a regular PHP application.

The unzipped PHP application code will then interact with CouchDb normally: viewing, adding, updating documents and records, etc.

Replicate Applications, Not Just Data

Combine this application and distribution model with a locally installable, browser integrated PHP/CouchDb application host, then end users can easily take full PHP web applications offline with them.

These offline applications run the same PHP application code and access the same data (or a subset of the data) as the online hosts. They can browse, edit, and update the data in CouchDb just like they were connected, using the same browser interface.

Then when they reconnect to the network, they click a button and replicate all their changes back to the server and everyone else’s changes back down locally. When the PHP application design is updated, the user will automatically get the updated PHP files during replication, where they again will be unzipped and deployed on the local machine.

The work to build the infrastructure for this should be relatively small in scope. A PHP server side package manager is necessary to unpack application files to the proper places and configure them to be served up. Much bigger work items are thing like a local installer and browser integration necessary for end user deployment.

Exploiting PHP

The beauty of this system is it is possible to use the whole PHP development stack and extend it to a new storage/distribution model. Development of PHP/CouchDb applications should be almost completely compatible tool-wise with regular PHP tools.

So for example the development/debug process shouldn’t have to change, just the production deployment. During the development and update-debug coding cycle it wouldn’t be necessary to deploy the zipped application files to the Couch server. Instead you keep all files wherever is handy during normal development and testing while updating the design document’s computed table formulas, etc as necessary in the test database. When ready the PHP application files can then be packaged and stored in the Couch database, where they are automatically deployed by the application server.

One of the most interesting things about using PHP is the possibility of reusing popular free PHP/MySQL applications and porting them over to CouchDb. This can give new life and demand to many PHP application businesses already use but cannot access offline.


A couch being jumped on by actor Tom Cruise

One downside to using PHP is it supposedly has configuration problems, where apparently much of the core PHP behavior is tweakable in a server wide ini-file, make compatibility issues from server to server a problem for applications. I’m not experienced enough in PHP to know how bad this problem really is, but I’ve seen more than one complaint about it.

This application model could work with Ruby or Python or whatever instead of PHP. But the existing PHP application architecture is such a great fit I’m not sure another language and framework can be as easily and completely integrated. PHP may not have the meta-programming sex appeal of Ruby and Python, but it’s damn productive for web development and its tools are already compatible.

However, the development of this PHP system wouldn’t preclude other languages. It would just be the first language/framework integrated. It’s perfectly possible to simultaneously integrate with any number of web hosts and languages in the same CouchDb database. Like MySQL, CouchDb is language agnostic at its core.

Ok, I hope I've explained this clearly for the tech heads out there. I need to make some pretty diagrams of a complete system and web application to help get the ideas across better. Any feedback on this idea is welcome, especially if you know why it's fatally flawed, technically or otherwise.