blags of matt
or, From the Future-Past: You Spilled Your CouchDB in My Riak
We all remember the great key-value wars of the early 21st century. People, in their tiny dawn of the Internet-era minds, conflated data models, distribution, replication, and stability requirements into a mish-mash of ideologies and technological philosophies.

"Give us replication or... give us an acceptable alternative!"

"If I have to write another schema migration, I swear to Monty, I'm going to become a hardware store day labourer."

"Serialized JSON is the answer to everything!"
"But what if I want to search by a property of the JSON object and not just the id?"
"You just write a map-reduce function and re-reduce until you get your answer."
"Did you just tell me to go computationally fuck myself?"
"I believe I did, Bob. I believe I did." [q]

The core of their debate came down to representation versus distribution. The squealers claimed to have perfect representation, but hacked together distribution methods. The kvalers claimed to have perfect distribution, but limited representation models.

Trees sitting next to tables
What was the problem with representation? Why is perfect representation diametrically opposed to perfect distribution? Perfect representation came down to one issue: asking for ranges of things. Who is between 18 and 29? Which orders completed between last Thursday and today?

The squealers worshiped at the altar of their B+ tree, offering up mathematically perfect schemas in return for logarithmic access times to their data. Squealers ignored the one downside of their Lord of All Data Structures: B+ trees were manipulated in-place on disk, requiring huge amounts of logically contiguous disk space for large datasets. Distributing and replicating a B+ tree across multiple machines was impossible. Squealers took to chopping up their data into smaller and smaller collections until entire copies of B+ trees could be kept on multiple machines. Squealers claimed they never needed needed cross-table joins in the first place and they actually enjoyed the additional administration of dealing with statement based replication [a].

The kvalers worshiped at the altar of their hash table, offering up non-colliding keys in return for separation of data enabling perfect distribution and replication. The kvalers ignored the one downside of their Lord of All Data Structures: hash tables provide no data locality. Kvalers happily maintained hand-crafted indexes if range queries or search-within-the-dataset queries were required. Kvalers pretended to never have heard of ad-hoc queries or business intelligence requirements.

Who put my tree on the table?
Trees are made of nodes. Nodes must have names to be found. Traditionally, B+ trees were stored in files with nodes being referenced by positions within the B+ tree file. Arbitrarily chopping a B+ tree in half and distributing it to another node was not practical. Even with a B+ tree designed to reference outside its own file [1], the entire collection of mini-B+ trees needed to be present for operations to complete, resulting in no advantage in distribution or replication.

The B+ tree scalability problem came down to one issue: how can you name a node and later reference it independent of its storage location?

One mild summer's day in Goolgonia, historically California, someone stumbled upon the answer: a new way of naming nodes [2]. What if, instead of offsets within a file, a Central Naming Service could provide a globally unique node identifier serving as a recall key for a node's location? Instead of storing nodes within a sequentially allocated data file, B+ tree nodes could be stored on anything accepting the name of the node and storing its contents.

The kvalers squealed.

The squealers begged the kvalers, "Please, may we have access to your infinite-storage-capacity, automatically-redundant-with-self-managing-failover, and incorruptible storage system?" The response was delivered with a wry smile: "We've been waiting for you."

In a sigh of ecstasy, the squealers and kvalers realized what was born. The squealer's toes curled at the thought of being free from routine sharding, free from identifying hot spots and spinning off copies, free from arbitrarily chopping tables in half, and free from single points of failure. The kvaler's eyes rolled upwards while synthesizing thoughts of sequential data access, retrieving records by ranges, and iterating over database snapshots just by holding on to a root node.

After an exhaustive journey, the kvalers and squealers drifted off to a sound sleep that night. The story of the squealers versus kvalers came to a close, but the journey of the sqvalers had just begun.

wtfbbq? (aka Back to Life, Back to Reality)
Over the past few years I've been looking for a stable, low maintenance, zero up front decision making, scalable sorted data platform. I looked high and low to no avail. I even went as far as writing a single-purpose BigTable clone called TinyTable for my own use, but it still had annoying edge cases when I overflowed a single disk or RAID array.

A potential solution dawned on me after looking at an R-tree implementation using CouchDB's append-only storage format. To store a node, you pass in the data and get back its position. To retrieve a node, you pass in a position and get back the data. It's that simple.

All file writing and reading is completely encapsulated within the couch_file module. The two storage calls are straight forward. No seeking, block management, or any other file-level details are leaked through the interface.

{ok, Position} = couch_file:append_term(Fd, Node).
{ok, Node} = couch_file:pread_term(Fd, Position).


Important point: Position is an opaque type. The code doesn't care if Position is an integer or a tuple or a binary. Position is passed through to the storage layer on lookups and returned to the caller on appends.

Let's do this
To make CouchDB store documents remotely, we only have to replace the implementation of the two functions listed above. For our remote storage let's use Riak as our Key-Value store (because it's awesome). CouchDB persists Erlang terms to disk and Riak persists Erlang terms to disk. We get to remove redundant code from CouchDB since Riak is converting terms for us. Riak also automatically replicates everything we store, easily handles adding more machines to increase capacity, and deals with failures transparently.

append_term's implementation becomes riak_write(Bucket, Key, Value).
pread_term's implementation becomes riak_read(Bucket, Key).
We replaced Fd with a Bucket name here. Bucket is the database filename as if it were stored locally (e.g. <<"_users.couch">>).

We also have to generate unique keys for our nodes. The CouchDB file format is append-only, so we never have to worry about updating a value once it is stored [e]. Where does Key come from? Key is equivalent to Position in the original couch_file calls. Key is now simply {node(), now()}. That's the Erlang way of making a globally unique value within your cluster. node() is the name of your VM (original in a cluster) and now() returns the current time (guaranteed to be monotonically increasing) [k].

We turn our nice {node(), now()} into a binary and use it as a key: term_to_binary({node(), now()}), et voila we have a globally unique key to store values in Riak [m].

All the tricky couch_file page alignment code is now gone. The only files written to the local file system are couch.uri and couch.log.

We did this
Does it work? Sure, it works. Check out my CouchDB Riak Integration for yourself [p]. Modify your bin/couchdb script to include Riak client libraries (riak_kv) and connect to a local Riak cluster:
-pa ../riak/deps/riak_kv/ebin -pa ../riak/deps/riak_core/ebin -setcookie riak -name 'couchdb@127.0.0.1'
Insert those after $ERL_START_OPTIONS and before -env ERL_LIBS

What did it take to convert CouchDB to use completely remote storage for documents? It took: 7 files changed, 87 insertions(+), 502 deletions(-)

See the commit log for quirks, caveats, and how to use the admin interface properly in my proof-of-concept implementation.

Patches welcome? Maybe? To make this mergeable upstream, we need a way to have both local-disk and Riak storage. The original couch_file needs to be restored with flags about when to use remote storage versus local storage per-DB. A dozen other quirks and deficiencies would have to be fixed and accounted for as well. It makes more sense to use the CouchDB B+ tree code (and/or the R-tree code) with a modified couch_file to create an independent data platform [L].


-Matt
@mattsta

You shouldn't follow @mattsta on twitter.
He only writes about annoying problems, trivialities of life, The Bay Area, and programming.
Pretty boring stuff.


Footnotes:

[q]: With apologies.

[a]: Yes, this is the shitty MySQL way of doing things. Let's not tell them about PostgreSQL's log shipping replication.

[1]: e.g. Replacing your node reference from {Offset} to {Filename, Offset} still means all referenced files must be replicated.

[2]: ANWONN: The most advanced, world-changing, node naming scheme to ever be conceived by an almost god-like being. Do you question it? Here, read a 600 page autobiography about how amazing, unique, and intelligent I am.

[3]: This provides more location independence than the the Sequential Append-Only Write-Once-Read-Many Just-Try-To-Corrupt-Me-I-Dare-You B+ tree file format. [[Yes, this footnote isn't referenced in the article above. I'm not sure when the source to the footnote got cut, but I like the footnote anyway.]]

[k]: Yeah, I just re-wrote Twitter's Snowflake in one line of Erlang. And we can deal with unsigned longs too because we're Not Java. Let's ignore the fact that {node(), now()} is about 300 bits instead of 64 bits.

[m]: Yes, you can (and should) make it much more efficient storage-wise. I'm going for concise and readable here.

[p]: The CouchDB code tree is very messy. It really needs to be cleaned up. I wish they would discover Rebar already. I don't give a shit about the bulk of it, still, I keep it professional.

[L]: I'm not a fan of storing documents as JSON.

[e]: Except for the header storing root node information.
from the future, couchdb, riak, erlang, json, go computationally fuck yourself
code herein is released under the OMGTwitterz! license
Simple node client to import the public twitter timeline into couchdb running at http://127.0.0.1:5984/ (create the database at /twitter/ first).

Everybody can run 150 public API requests per hour which nets you around 3000 tweets per hour. The client polls the public API every 0.5 seconds.

Runs against 2010.07.16 node-v0.1.101.


#!/usr/bin/env node

var http = require('http');
var couch = http.createClient(5984, '127.0.0.1');
var twitter = http.createClient(80, 'api.twitter.com');

function post_all_json(twitter_json_text) {
var creq = couch.request('POST', '/twitter/_bulk_docs',
{'content-type': 'application/json'});
creq.write('{"docs":' + twitter_json_text + '}');
creq.end();

}

function runTwitterRequest() {
var treq = twitter.request('GET', '/1/statuses/public_timeline.json',
{'host': 'api.twitter.com'});
treq.end();
treq.on('response', function(response) {
var twitter_response = "";
response.setEncoding('utf8');
response.on('data', function(chunk) {
twitter_response += chunk;
});
response.on('end', function() {
var tjson = JSON.parse(twitter_response);
console.log("Posting retrieved tweets which number " + tjson.length);
post_all_json(twitter_response);
});
});
}

setInterval(runTwitterRequest, 500);

nodejs, twitter, couchdb, omgtwitterz
(75 days ago)redis meetup notes
more redis, plz
Quick notes (brought to you by iPad while laying I'm bed):

libcluster
Redis is getting native clustering one way or another. What if we make a generic libcluster to handle the hash ring, membership, networking, failover and other clustery things? It could be helpful in other projects if libcluster is as well thought out and as well written as Redis.

zeromq
How about adding zeromq as a connection option in addition to TCP and (soon) unix domain sockets? Zeromq can give us sane udp out of the box (maybe) and possibly a more efficient tcp layer since zeromq coalesces as many outgoing messages together as possible (nb: would break a central NIH rule of antirez).

command response tags
Someone brought up the issue of multiple threads using one redis connection. What if each redis command could accept an optional tag that got echoed back in the response to the user? Client libraries can make a tag (something unique within a short time period of the app), send commands to redis as needed, then use the tags in responses to match up redis responds with who should receive them.

Sounds good to me.

-Matt
redis meet up, sf, June, cold
in glenn beck's america startup works you
MOUNTAIN VIEW, California, June 17 -- On a shady block of Pioneer Way a revolution is brewing. One hundred pre-screened applicants answered the call of a white-on-orange sigil, which has become synonymous with budding enterprise and nouveau riche social influence in Silicon Valley. The applicants traveled from as far away as Boston simply to hear Y Combinator-sponsored startups give a 90 second pitch about their wants, needs, desires, and generous meal plans.

One after another in sternly administered choreography, the startups gave brief pitches to the attendees. Pitches consisted of projected slides, one or more of the startup founders presenting an employee-focused view of their company, and endless fidgeting with a shared lavalier. Every few pitches, a person comfortable with public speaking became slightly more engaging by including a developer-friendly joke.

Notable presentations
Drew Houston of Dropbox received a vocal "Woo!" when he introduced himself. Everybody loves The DropBox.

The pas de deux between a seemingly nervous Sam Altman talking quietly to his slides and Paul Graham in the back holding up his trademark "LOUDER!" sign entertained the crowd for a full 60 seconds. It turns out you can't see Paul Graham holding up a sign, no matter how bright his shirt, if you are looking at a wall. (Don't worry Sam, I still love you.)

The presenter for airbnb ("The Abees") was unquestionably the most direct, forceful, and well rehearsed speaker of the event. He knows what airbnb wants, how they are going to get there, how awesome they are, how much the world suffers, and why they are worth more than you can possibly imagine.

Paul S joined ex-Twitter employee Chad to talk about notifo (pronounced note-ee-foe, but which reminds one of Jeff's notify.io). Paul, being with notifo for about two days, talks as if he worked on the project for months. You can't out plan, out hipster, or out self-promote Paul S.

Reception
After the second "this is the last presentation" presentation, the attendees were invited out to the parking-lot-turned-pizza-buffet for a few hours of mingling, scheming, catching up, and getting to know one another. Startup founders darted through the crowd as little entrepreneurial sharks swimming through the chum of potential employees. Pairs of platonic speed dates broke off. Some pairs chatted for half an hour while others lasted barely thirty seconds. Some people greeted with a firm handshake while others only yielded a sideways glance of yes-i-see-you-but-don't-talk-to-me.

More interesting than talking with the intended audience of startup representatives was talking with fellow attendees. You can see in their eyes when they are good at what they do. You can see in their eyes when they are uncertain what tomorrow brings. You can see the crazy ones, the dreamers, and the failures. At least one attendee did not know what Y Combinator actually did. Other attendees ranged from Lockheed Martin employees, to in-between-semester undergrads, to i'm-taking-this-loop-off people taking a break from being employees, to people just scouting for a slightly more interesting life.

Notable reception happenings
One person was carrying a copy of On Lisp. I assume he obtained a useful signature.

The venerable Sam Altman, the chosen one, the golden child of Paul Graham, said he likes my HN comments. He then proceeded to tell me the story of how he almost killed someone then escaped death at the hands of mother nature. I've stories for you too, Sam. Give me a call.

[[This brings me to another point: name badges are for names. We all have multiple names. I was the only one I saw with my Human, HN and Twitter names on my badge. Come on people, use a blank canvas more creatively. We want to know you.]]

I thought Trevor said the telepresence bots would be active, but they never emerged. A row of bots lined up obediently at the Anybots entrance waiting for their chance to be of service. Their chance never came.

Paul Graham attracted his large conversational following as usual. I didn't notice Trevor emerge outside. Kate gave me a nod but didn't want to talk. Jessica was doing what she's known for and kept everything running smoothly.

When trying to convince me of the merits of San Francisco instead of San Jose, Dan from Weebly attempted to interject a point about dating being more prolific in San Francisco. His approach was a bit off. Dan tried to ask, "Do you have a girlfriend?" To which I replied, "No, I don't have a boyfriend." One additional round got the point across: "No, a girlfriend," he clarified to himself. A final recitation of "No, I don't have a boyfriend," cleared up any confusion. If you are a good Analytics person who likes SF and wants to work at Weebly, give Dan a call. He may even be able to hook you up with a girlfriend.

I monopolized Emmett's time for probably 30 minutes while having Sam Altman, Build Enginner Dude, Weebly Dan, and one or two others try to break into the conversation. Some break-ins were more successful than others.

I missed talking to Gnip Dude. He kept vanishing.
I missed talking to Paul S. I've a bone to pick with him.
I forgot to talk to the DailyBooth guys.
I'm not interested in moving to San Francisco, which writes off probably 90% of the startups in attendance.

Conclusions
Was it useful for the startups giving presentations? It's difficult to say. We'll have to wait a month or two until startups can report back how much usable talent appeared due to the event.

Was it useful for the attendees? Yes. An emphatic Yes. It's empowering to sit through stories of early-stage (and some not-so-early-stage), on-track startups. Thoughts of running a startup flow through so many of us, but not nearly enough people act on it. How about an Express-YC for more experimental ideas not needing a 30 question application?

Was it the right format? As I told someone before the event, and as Paul Graham reiterated during his opening talk: What matters are the people with whom you work. Go with amazing people and you'll be fine. A format of presenting the company's projects may be misguided. We should be looking for people to join. Have every employee in attendance speak for 30 seconds at the front of the room. Yes, some people are horrible with public speaking, but that's a data point in and of itself. Let's get the people to present themselves followed by one 20 second overview of their products.

Final conclusion? I need to get off my ass and get my own startup in gear. Time to redefine some paradigms.


-Matt
650-450-8826
work at a startup, ycombinator, paul graham, sam altman, lolgay, paradigms, that's no parking lot
your comics have never been so slippery
intro
I hide books from myself. The more dangerous the book, the more cleverly it gets buried. Recently I uncovered "xkcd: volume 0" again and lost an hour of my life re-reading everything on the spot.

It dawned on me: there must be a way to extend the productivity killing capacity of fluid comics to a mass audience.

This is around the time an HTML5 presentation was making the rounds. A slippy HTML5 presentation. A fluid HTML5 presentation. "Let's replace the damn slides with comics," screamed the productivity devil on my shoulder.

A few weeks later, we have slippycomics.

creation
slippycomics is a re-written version of the HTML5 presentation above. Most of the CSS is stolen, but the javascript has been completely rewritten.

Since the xkcd book inspired slippycomics, it was only natural to include all of Randy's comics. Next up are two comics which I always mean to read but constantly forget about: bunny and smbc.

All comics, alt text, and alt comics were scraped from their respective sites. The comics and alt comics for slippycomics are hosted on the Akamai CDN[1] as to not overwhelm the originating sites. In no way am I trying to steal or claim these comics. Each comic has a link back to the original site along with a link to the store for each comic every few frames.

[1]: If the CDN gets too expensive, I'll host directly from my servers.


tech stuff
front end site: html5, css3, coffeescript (-> javascript -> yuicompressor), jquery (1.4.2 + jquery.history + jquery.cookie -> closure compiler)
back end site: nginx -> yaws -> erlang -> redis (via my in-progress redis client er); akamai -> nginx -> disk
back end scraping: python with urllib2 and lxml.html for finding comic URLs, titles, and alt text/comics. python stores everything using redis-py. python downloader for comic graphics after comic data is stored in redis.


controls
Up. Down. Left. Right. r. Spacebar. [numbers then enter]. w.a.s.d. Click on stuff. Mousewheel. Touch sliding left/right for iPad/iPhone. Forward/back history. Remembers your last position if you close your window and come back later.


future
If you want your comic added, email me. I can add as many comics as people would like. Accounts may even happen one day for personalization.

It will be trivial to make an automated comic submission process if enough authors want their content here and don't want me scraping their sites.

If you want your comic removed, let me know and you'll be removed with extreme prejudice and a frowny face.

There are some UI bugs. The backend needs to be able to accept automated updates from authors directly. The iPad/iPhone interface needs fleshing out.


contact
questions? praise? love? hate? hugs? rage? Direct it all at matt or at @mattsta.

Go forth and slip: http://slippycomics.com/

xkcd, bunny, smbc, sudo make slippycomics, we like romans THIS much, crazypope, orange made me do it
(88 days ago)about
why are you here?
slippycomics is http://slippycomics.com/
about
(155 days ago)Going Nowhere
the tears are filling up their glasses
Canadian bloke with a great voice, great audio, and great visuals. Great.

Shankill Butchers




Nick has a great voice but a less-great audio recording setup. Still worth a listen or two or twelve.

Mad World


On the Bus Mall



mad butcher mall world, covertastic, hoodie
is that nutella in the background?
(230 days ago)Wadeing
his kulele


tiny tiny guitar
you should see a doctor about that leak
background
I've spent the last month in isolation meditating on The Device Who Comes from Apple. Based on sensory deprivation induced hallucinations and stalking Apple coworkers, I've come up with this outline for The Device Who Comes. Also, it helps I found a leaked draft copy of the January event script.

The leaked script had the actual product name redacted, so I call it i[REDACTED] throughout the script. The script below leaves out details relying on actually seeing the device (demo parts of the script), industry recaps, and the scripts of guests invited on-stage.

summary
The device will not be called iSlate. What is a 'slate?' It reminds me of a misshapen piece of granite chipped out of the earth with shards flying everywhere.

There is a good chance The Device Who Comes will not be prefixed with i.

The Device Who Comes will use a Natal-like control interface thereby pissing off Microsoft to no end.

Basic Description: 13" 300 ppi multi-touch screen with free-space gesture recognition. "Giant iPhone" mode (for App Store apps) and full OS X mode. First client device to fully rely on Apple's new personal content hosting service. Perfectly designed reader functionality supporting books from iTunes (they may rename iTunes at this point) and supporting all open data formats for books (pdf, mobi, epub). Also, a new custom Apple eMedia format will be released for richer content consumption experiences.

Wireless Partner: Verizon Wireless.
Initial Content Partners: NYT and Conde Nast.

Prices: $1199 with free 2-year Verizon Wireless access. $699 with $30/month two-year Verizon Wireless contract. $899 with no cellular radio included (wifi/bluetooth only).

draft keynote
Thank you for coming. We're going to make some history together again today. Welcome to the future.

[industry specific recap]

This is a day I've been looking forward to for six years. Every once in a while, a revolutionary product comes along that changes everything. And Apple has been -- well, first of all, one's very fortunate if you get to work on just one of these in your career. Apple's been very fortunate. It's been able to introduce a few of these into the world. In 2007, we introduced the first iPhone, and it didn't just change the way we all use phones, it changed the entire mobile landscape. Well, today we're introducing three more revolutionary products of this class. The first one is a widescreen electronic reader with a full color, highest quality, multitouch screen. The second is the most portable Mac ever developed. And the third is a breakthrough service for managing your digital life. Electronic reader, extremely portable Mac, managing your digital life. Reader, portable Mac, digital life. Reader, Mac, Life... are you getting it? These are not three seprate devices. This is one device. And we are calling it i[REDACTED]. Today, Apple is going to reinvent the tablet computer, and here it is.

So, before we get into it, let me talk about a category of things. Current eBook readers on the market use something called eInk displays. These displays require no power while showing a page, but they only work in black and white and they are slow to turn pages. Slow to turn pages. In a book. Why do people use these? Some eBook readers also have Internet access, except everything is black and white and slow to change pages. Did you see tablets and readers from CES two weeks ago? There must have been over 100 tablet PCs and three dozen eBook reader models released. The "tablets" are all tiny and they all run Windows without regarding what form-factor the devices use. Desktop? Windows. Tablet? Same Windows. Same hair-tearing experience. The eBook readers all use the same underlying technology. Their only difference is what plastic case manufacturers use. Well, we don't want to just re-brand existing technology. What we want to do is make a leapfrog product that is way smarter than any mobile computer has ever been, and super-easy to use. This is what i[REDACTED] is. Okay?

So, we're going to reinvent the mobile computer. Now, we're going to start with a revolutionary user interface. It is the result of years of research and development. Now, why do we need a revolutionary user interface? Other tablets out there use a stylus -- a stylus -- or half-broken multitouch implementations. We can do better. i[REDACTED] is a full multitouch device, but it also has a camera built in. i[REDACTED] will also respond to just motions of your hand. See the camera there at the top? It constantly watches for gestures to be performed. In Reader Mode and want to go to the next page? Gesture from right to left and Boom. There's your page. Want to go back? Gesture from left to right. Want to scroll? Gesture in the direction you want to scroll. Boom. Isn't that amazing?

Let's talk about design. We've designed something wonderful for you to hold, just wonderful. This is what it looks like. It's got a thirteen inch screen on it. It's really big. And, it's not only the highest resolution screen we've ever shipped, it's also the highest resolution screen of its size available anywhere. It's 300 pixels per inch. Highest available anywhere. It's gorgeous. And on the front, there's only one button. It's the home button everybody is faimilar with from iPhone. Let's take a loook at the side. It's really thin. It's thinner than any tablet out there, at 8 mm. Thinner than an iPod touch. It's really nice. And we've got some touch spaces on the sides for quick navigation, along with volume controls. Let's look at the front. We've got a five megapixel camera and microphone built in to the top. Video capable. We've got a headset jack for listening to music. We've got a quick push-to-sleep button just like iPhone. And on the bottom we've got our 30-pin dock connector for charging.

Now, we've also got some stuff you can't see. We've got an extra-long-lasting battery. 8 hours of full use and up to 30 hours during light use. We've got an accelerometer so we can automatically switch from portrait to landscape. We've got an ambient light sensor for automatic brightness adjustment. We've got wifi and bluetooth built in. You can pair i[REDACTED] with your iPhone to see incoming calls and send text messages directly from the i[REDACTED] through iPhone.

So, now, let's take a look at Reader as part of i[REDACTED]. We've never offered books, magazines, or news for sale before. We're proud to announce a partnership with Conde Nast and the New York Times who will be creating content exclusively for i[REDACTED]. What other electronic reader gives you an insanely great mobile reading experience with full color videos and all the benefits of the web built right in? None that we've found. How do you get real-time, live updating content on i[REDACTED] no matter where you are? We'll get to that in a minute, but first let's show you hour Reader works. [DEMO]

Isn't that great? Now how do you get real-time articles on i[READER]? We've partnered with Verizon in the US and our iPhone partners internationally. You have real Internet access anywhere with i[READER]. [DEMO]

All right, now I want to show you something incredible. I want to show you how you get all your music, videos, books, photos, documents, email and more on to i[REDACTED]. This is a new way to manage your digital life. Over the past few years we've spent over $800M building data centers. $800M buys a lot of hard drives. When you have a desktop, laptop, iPhone, and i[REDACTED] we wanted to make it as simple as possible to keep everything up to date. Your home computer has an always-on internet connection. iPhone has an Internet connection anywhere. i[REDACTED] has an Internet connection anywhere. See a pattern? We're going to store everything for you. We'll stream music, stream videos, synchronize yours last read book positions, sync photos, documets, email, and everything else. On i[REDACTED], you have access to your entire library without having to download anything. Let's see how it works. [DEMO]

Let's talk about apps. On OS X, anything goes. On iPhone OS, we approve every application to maintain the integrity of the network and OS. You can't have people destroing their phones by downloading a malicious application. People live and die by their phones. But i[REDACTED] is something new. Do we open it up completely or do we approve everything? We've thought long and hard about the pros and cons each way. We decided: we'll let you choose. i[REDACTED] can run a full desktop OS X or i[REDACTED] can run in an iPhone-like mode where it only runs App Store apps. All i[REDACTED]s are in "giant iPhone mode" by default. If you go into the configuration menu, you can enable full OS X. We've satisfied the needs of every day users and advanced users alike. Simple mode gives you a giant, visually stunning, iPhone-type interface. Advanced mode gives you full desktop OS X in its most portable form ever.

So what's under the hood? i[REDACTED] has a super powerful dual-core 2GHz ARM CPU developed by Apple along with 2 GB of RAM, 802.11n wifi, and a cellular Internet connection. We talked about our plans for an always-on streaming device with all the US mobile carriers. Verizon is the best suited to meet the needs of our users now and in the future. They are building out the fastest nation wide data network and we've negotiated unlimited Apple streaming access on i[REDACTED]. All other Internet use through i[REDACTED] is capped at 10GB of celluar data per month. Data anywhere. Data everywhere. The bulit in five megapixel camera can be used for video chat anywhere. We have a gorgeous 13 inch 300 pixel per inch screen developed specifically for i[REDACTED]. It shows text sharper than a laser printer and images sharper than high def.

Battery life. Tablets out there have pretty low battery lives. We've managed to get eight hours of battery, and that's for watching videos and a lot of Internet browsing. That's eight hours of streaming video and audio. With light usage of just Reader and browsing a few websites, the battery life goes up to 30 hours.

So what should we price it at? Well, what do these things normally cost? First, what does i[REDACTED] replace? It can replace a MacBook you carry around when you don't really need the full power of a MacBook. That's at least $999. It replaces an ebook reader. A comprable ebook reader costs $500. We're including two years complete access to all Conde Nast and NYT content -- that's $1200. We're also including two years of our revolutionary media and document streaming service. i[REDACTED] replaces almost $2700 in other devices and services. What should i[REDACTED] cost? We've come up with three configurations. The first includes two years of Verizon Wireless unlimited data transfer between your streaming media and i[REDACTED]. i[REDACTED] + always on Verizon Wireless = $1199. The second lets you add i[REDACTED] to a Verizon Wireless account for $30/month. Two year contract required with Verizon Wireless. $699. The third option is i[REDACTED] with no cellular connection -- wifi only network access. That's $899. Now, when's it going to be available? We're going to be shipping these in May. We're announcing it today because with products like this we've got to go ahead and get FCC approval which takes a few months, and we thought it would be better if we introduced this rather than ask the FCC to introduce it for us. So here we are, and we're going to be shipping it in May in the U.S. We're going to Europe by the fourth calendar quarter of this year. Then Asia after that.

The Mac in 1984 is an experience that those of us that were there will never forget. And I don't think the world will forget it either. The iPod in 2001 changed everything about music. The iPhone in 2007 changed everything about mobile communications, and we're going to do it again with i[REDACTED] in 2010. We're very excited about this. There's an old Wayne Gretzky quote that I love. "I state to where the puck is going to be, not where it has been." And we've always tried to do that at Apple. Since the very very beginning. And we always will. So thank you very very much for being a part of this.

Questions?
i[REDACTED], The Device Who Comes, iworkforapple, youworkforapple, weallworkforapple, verizon, conde nast, NYT, ebook reader, live syncing, free space gestures