October 31, 2005

SBC CEO on crazy pills

CEO Edward Whitacre talks about the AT&T Wireless acquisition and how he's moving to keep abreast of cable competitors
How concerned are you about Internet upstarts like Google (GOOG ), MSN, Vonage, and others?
How do you think they're going to get to customers? Through a broadband pipe. Cable companies have them. We have them. Now what they would like to do is use my pipes free, but I ain't going to let them do that because we have spent this capital and we have to have a return on it. So there's going to have to be some mechanism for these people who use these pipes to pay for the portion they're using. Why should they be allowed to use my pipes?

The Internet can't be free in that sense, because we and the cable companies have made an investment and for a Google or Yahoo! (YHOO ) or Vonage or anybody to expect to use these pipes [for] free is nuts!

Online Extra: At SBC, It's All About "Scale and Scope"


October 29, 2005


We are giving away our tags - and losing control of them at the same time. It is time to take back the tags. A solution is comming. Real soon now.


CouchDb updates are slow

I'm doing some very early performance analysis of CouchDb, and it looks like my design is going to be a little slow for updates. Compared to how quickly Lotus Notes/Domino NSF databases can commit new documents, CouchDb will take about twice as long, maybe more. It thought my design would be significantly slower, now I know for sure.

The reason is NSF uses a single flush for atomic updates, while CouchDb uses double flush. By "flush", I mean the database code forces any memory buffered data that it wrote to the database file to be finally written to disk. Flushes are expensive operations and generally something that should be avoided if possible.

The single flush scheme NSF uses is pretty fast but it has a hidden cost. I'll explain how it works from a high level.

When NSF opens a database for writing it sets an "Open" flag in the header and writes it back to disk. Then as documents gets added to the database, immediately after it writes the document to disk, but before returning to the caller, it flushes the data to disk, it also updates the in-memory copy of the file header and other cached data structures, but they are not written to disk. This happens for any number of updates while the database is still open. Once the caller is done with the database and closes it, NSF clears the "Open" flag and writes the up-to-date header and other structures to the database file and then flushes to disk. At this point everything on-disk is in a completely consistent state. This a efficient way to achieve atomic updates, but there is a hidden cost: The fixup.

Fixups are necessary any time a database is opened and it already has the "Open" flag set in the header. Usually this is the result of a crash or a power failure the last time the database was opened, and it means certain data structures may not be up-to-date and consistent. When NSF encounters a database like this, it must read significant portions of the database file and examine the data structures in question, checking for invariants in the data structures and making sure they agree with other related structures. When it encounters inconsistencies it corrects them. With a large database, this can be very time consuming and very disk I/O intensive.

Now this a simplification of the real NSF design, but conceptually that's how it works. The fixup approach is often a good compromise, it offers good performance in the average case and provided you don't crash too often or you don't have huge databases, the actual fixups aren't that big a deal. However, when either of those is true, then the fixup becomes big availability problem, you might have big problems getting to your data when you really need it.

To alleviate this, Lotus Domino NSF also offers transaction journaling. I won't go into the details of that, but provided you have a disk dedicated for, the journal file, it's offers fastest performance for update. And since it eliminates fixup it's also a great high availability option too. However, it also increases code complexity and has performance problems if used without the dedicated disk.

Instead of the fixup approach, CouchDb uses a simple double flush design that always keeps the database file in a consistent state. When writing a document into the database, CouchDb first flushes the actual document data and meta-data structures to disk, and like NSF it doesn't write the actual header during the first flush. Then once that data is definitely on disk, it then writes the file header and flushes that to disk, only then does it return to the caller. Since CouchDb has zero-overwrite design, which means nothing in the database file is ever overwritten or modified after the initial write (except for the file header, maybe it should be called a one-overwrite design since? meh). That means that no crash will ever cause the database to be in an inconsistent state (assuming the underlying file system is robust and doesn't corrupt already written data), incomplete writes are simply data/metadata the header can't see, and all the old data is exactly as it was before. This means a fixup is never necessary, so CouchDb should be highly crash tolerant.

But there is the write performance cost: CouchDb will be much slower for adding documents than Notes, and that's a bad thing…right? Under certain circumstances it is, but generally speaking, most real world Notes and Domino NSF performance problems stem from indexing of already written documents to create "views", not the updating of documents into the database.

However, CouchDb also offers a way batch a large number of updates into a single commit if high performance updates are desired. When doing updates this way, CouchDb performance increases dramatically (like 100 fold). This can make the copying or modification of large number of documents much quicker. But if a crash happens in the middle of these updates, some or all of the new documents might not be available the next time the database file is opened. NSF actually has a feature like this too, but I don't know how big a performance boost it gets from it.

So CouchDb updates are slow, but hopefully I've chosen a design that is more optimal for "indexing", the creation and incremental update of computed tables (CouchDb's close analog to Notes views). I'm hoping some related decisions I'm making pay off. But I haven't yet got that to the point where I can validate those performance characteristics, but if I was right about updates being slower, maybe I'll be right about indexing being faster. (crosses fingers)



The Caps Lock key must die. It serves absolutely no purpose other than to annoy, I never, and I mean never, intentionally use it. There is nothing quite like the joy of seeing what you just typed in reverse case, with the each letter the opposite case of what it should be. It's like an eternal practical joke.

Then there is the Insert key. Like the Caps Lock key, it's a key I never actually use. Fortunately it's a little harder to accidentally hit, but once you do oh the joy as your previously composed text gets obliterated. And even better, there is no keyboard indicator and few applications (like a web browser) give you any indication when you are in overwrite mode. You just have to notice that your new text is eating the old. Hi-larious.

With so many keyboard manufacturers eagerly adding new keys, is it too much to ask they remove some old useless ones? I'm looking at you Microsoft.


October 25, 2005


Ronnie rails against uniforms, or rather what they represent in science.


Scott Adams has a blog!

The Dilbert Blog

I'm most pleased. Here he explains why a cop was firing a donut (I remember wondering about that strip).


October 24, 2005

Visual Studio: Replacing spaces with tabs and vice versa

To replace spaces with tabs:
  1. Select the text that contains the spaces you want to replace with tabs, or choose Select All from the Edit menu to select the entire file.
  2. From the Edit menu choose Advanced, then click Tabify Selection.

Found on MSDN. I've needed this about zillion times, but never knew it was built into Visual Studio.


October 23, 2005

If You're a Christian, Muslim or Jew - You are Wrong

Cenk Uygur: If You're a Christian, Muslim or Jew - You are Wrong


October 16, 2005

Somebody spoke and I went into a dream

Ahhhhhhhhhhhh ahhh ahhh ahhh


October 12, 2005

Who moved my Hasselhoff?

Recently I wrote about Ice-T producing a new rap album with David Hasselhoff. Naturally I was a little derisive. I mean, I love David Hasselhoff as much as the next guy, but a rap album? Then I glanced again at the Hasselhoffian recursion. Well not so much glanced as stared. Stared for 16 hours straight actually.

And I'm glad I did, because it really clarified my thinking, and allowed me to figure out what's causing the problems our country has been having: 9/11, Iraq, Hurricanes, Bennifer.

It's all caused by a lack of David Hasselhoff.

I know this may be stunning to you – I didn't want to believe it at first either. But the evidence is overwhelming that when David Hasselhoff's million-watt smile isn't part of our cultural fabric, things turn to shit in a hurry.

Pre-Hasselhoff America
ironeyescody.jpgIn the seventies and early eighties, everything sucked. The energy crisis, stagflation, indians weeping incessantly from pollution, it was terrible. The office of President was in disarray, repeatedly rocked by scandal, incompetence, big teeth and even assassination attempts. Failures in Vietnam, Tehran and Beruit made our military look weak and incapable. Gas prices skyrocketed, as did inflation, unemployment, and crime. Disco flourished.

But there was hope, a brilliant young actor named David Hasselhoff was rising through the ranks.

Knight Rider's America
hasselhoff_coleman_small.jpgKnight Rider debuted in 1982 and starred one David Hasselhoff as Michael Knight, a man who owns an indestructible talking car. He took us on a shadowy flight into the dangerous world of a man who does not exist, and along the way taught us how to love again. During this time the US experienced a broad resurgence, unemployment fell, as did inflation, gas prices and taxes. Even crime and murder rates dropped. America was alive, as it flirted with a new kind of music called "New Wave". It was a time when anything was possible.

Knight Rider Cancelled, Nation Plunged into Darkness
swaggert.jpgIn the '85-86 TV season, America's love affair with Hasselhoff was ending. People just stopped watching, hubris causing us to think we can live without Hasselhoff, we'll be just fine. But it was an illusion, and one that cost America dearly. The Challenger disaster happened, followed shortly by the crack epidemic, the AIDS crisis and Bill Buckner. Crime and murder rates began to climb faster than we'd ever seen. The "Black Monday" crash occurred, the stock market having suffered the largest one day loss in history.

Hasselhoff Throws Us a Life Preserver
Hasselhoff_baywatch_small.jpgKnowing the damage done in previous years had to be quickly repaired, Hasselhoff had to do something bold, his charisma and smile could only do so much to help America out of its jam. Then in a flash of inspiration while shaving, he gazed upon his tanned physique and knew it was the answer. So in 1989, a bare-chested Hasselhoff starred in Baywatch, and we once again became alive as a people. And our enthusiasm shot around the globe and knocked down the Berlin wall, and Germans began their own love affair with Hasselhoff. Within months, Baywatch mania spread around the world, and even the whole of the Soviet empire and the great evil of Communism was no match, the Hasselhoff gap too great for their totalitarian regime to counter. The empire crumbled before our very eyes, the communists knew they had been beaten. Beaten by Hasselhoff.

Cancelled After One Season?
millivanilli.jpgBut then, in stunning move, NBC cancelled the show after a single season, stating it was a fad that could no longer justify its production costs. America was once again rudderless, its hopes dashed. Some suspect that NBC's parent company G.E., a large military contractor, was protecting its Cold War profits. Others think the KGB planted mind control chips in NBC upper management in a last ditch effort to revive their empire. Whatever the reason, the effects were profound. Shortly after the cancellation, Iraq invaded Kuwait, gas prices skyrocketed, unemployment and inflation began to rise. Vanilla Ice released his first album. Murder rates reached an all time high.

Hasselhoff to Network Execs, "Bite me"
Hasselhoff_baywatch2_small.jpgThen, in 1991 on a mission to save humanity from itself, Hasselhoff took matters into his own hands and got Baywatch back onto the air, bypassing the networks and going straight to syndication. And shortly thereafter America entered a 10 year golden period, with unprecedented drops in crime, murder, unemployment, inflation, interest rates, even teen pregnancy. The World Wide Web took off, spurring an expansion in investment and commerce the likes of which we've never seen. Gas prices dropped, as did taxes, all while the government ran a surplus. The human genome was mapped and Viagra was invented. And not just America felt the effects; Baywatch was syndicated around the globe and became the most popular show on Earth, blazing a trail for democracy and implants to flourish worldwide.

And then like that, it was gone.

Hasselhoff, Why Have You Forsaken Us?
In May of 2001, the last episode of Baywatch aired. Hasselhoff, having given mankind his all for ten years, ended the show. Shortly thereafter, everything else turned to shit.

The worst attack on America occurred just months later, followed by another stock market crash. Unemployment rose and for the first time in 10 years murders did too. The space shuttle crashed again. We got entrenched in war in Iraq, again, but this time we have no end in sight. Two hurricanes caused unprecedented devastation. The national debt is at an all-time high, as is personal debt while personal savings are at an all-time low. Gas prices are rising again, so are interest rates and inflation.


It looks bleak. Please David Hasselhoff, produce your rap album. Make it good. Make it Hasselhoff good. But don't take too long, we need you. You're our only hope.


October 11, 2005




A method of styling hair to cover partial baldness using only the hair on a person's head. The hair styling requires dividing a person's hair into three sections and carefully folding one section over another.

United States Patent 4,022,227 - Method of concealing partial baldness

I've heard about this patent for years, but never actually seen it. Good stuff.

Bonus link - Combover: The Movie


October 5, 2005

Disposable Scrabble Game

here's a scrabble board that conveniently prints on an A4 or lettter sized sheet of paper. if you are ever stuck on a plane or in a pub with no scrabble board you can use this sheet to get your fix.

go anywhere disposable scrabble game.

Cool idea. Just print and cut out the letters. How long before the lawyers descend?


The Noguchi Filing System

This system relies on Noguchi's idea that it is more likely that the user will know or remember about when a document was created, than he/she will be able to remember where it has been stored in a conventional system.

The Noguchi Filing System

This is for actual physical documents and folders but the concepts are applicable to software too. Interesting.


The blog in my head

"The blog in my head is way more interesting than the one on the web."
- Ned Batchelder (from an IM chat this morning)

Heh. So true. In my head I'm so articulate and witty, you wouldn't even believe it. Take this post for instance, my thoughts on the matter were way more interesting until I started to type them out. I blame my keyboard.