introduction |
current events |
projects |
stuff
For my current web log entries, click on the "current events" hyperlink above.
For other archives:
All hyperlinks which lead to a different web site are in this pretty green color
I ought to be packing, but I've decided to play in Type 1 Magic Cards tournaments, so I'm updating mtgspoiler.py to parse the older spoiler lists.
I'm excited to visit my family tomorrow!
Here's an Irby picture for Grandmaxo. (Other people are also welcome to look at it.) It's blurry because he's trying to get my camera. Click on it for the full size picture.
Amber took Irby for a walk and I'm playing with Magic Cards. It's nice to relax. Irby has learned to produce both plural and possessive in the last couple of days. In both cases he pronounces the "s" or "sh" sound separately. "Daddy. s. adi." ("Adi" means "coffee" in this context.) "Rahrah. sh." ("Rahrah" means "dinosaur". It's from the sound they make.) "T. sh." ("T" meant "nut" in this case. There are a lot of words of which he pronounces only the final hard consonant.)
The state of the art in emergent network design goes something like this: step 1, treat everyone you talk to over the Internet identically. Whether the other person is your best friend or the one millionth anonymous stranger from a university network half-way around the world, you offer your resources to them and request services from them just the same. step 2, design a wonderful, infinitely scalable, efficient, elegant emergent network on top of this substrate. step 3, observe that if all of the players aren't perfectly well-behaved and altruistic, your wonderful design doesn't work, and start trying to figure out how to salvage your beautiful design from being destroyed by the ugly fact of malicious and/or selfish agents.
Now, solutions proposed by people in step 3 include very clever game-theoretic tricks and cryptography, the addition of ubiquitous micropayments (as pioneered by Mojo Nation), the addition of sophisticated redundancy so that as long as the malicious/selfish nodes are not a sufficiently large subset their misbehavior is drowned out by the majority, the addition of reputations so that only people who have already done given something of value are allowed to take something of value, and more.
But it seems to me that the first thing we should do is go back and reconsider step 1: the part where you forget everything you know about your friends and family and treat messages received over the Internet from strangers half-way around the world the same as messages received over the Internet from your best friend. I think that the emergent network designer should focus on the human context, both because the human context is where our ultimate goals and values are defined, and also because the human context is the best source of a uniquely valuable network resource: trust.
(Raph Levien and Mark Miller are both thinking along these lines, although they're thinking of dramatically different designs.)
This is the core consideration of what Lucas Gonze calls "friendnet". "Friendnet" is similar to what business people use under the name "VPN" -- Virtual Private Network. A VPN is an "overlay network", in that it runs on top of the Internet, but it acts like a private local network, in that all of the computers and all of the human users on a VPN belong to the same company, obey the same rules, and are loyal and altruistic toward one another. (Ahem.)
Now obviously there are some things we would like to do differently with friendnet. For starters, I strongly prefer the emergent and human topology of "I have some friends, and they have some friends, and some but not all of their friends are also my friends.", over the centralized and, well, inhuman topology of "Every user on this network is an employee/member of X organization, and they are not allowed to have any network connections to other people who are not also employees/members of X organization.".
For seconders, we could investigate the intriguing possibility of automated transitive operations. This could be automated transitive proxying, where I request something of my friend, and since he can't give it to me, but his friend can, his computer automatically requests it of his friend and then gives it over to me. It could also be automated transitive introduction, where my friend's computer automatically introduces his friend to me (so that I can then make direct requests of my friend's friend).
Raph's research is all about doing this automated transitive stuff in a systemically constrained way. That is to say doing it safely -- without letting it run out of control such that our computers offer all of our resources and all of our privacy to a friend of a friend of a friend of a friend, who turns out to be an enemy. I think that his research is promising and will probably lead to very important techniques someday.
But when doing pragmatic design on the Mnet network, I would rather try the variant without any automated transitive operations. This is simpler to design, and almost certainly safer.
My current big issue in the design and evolution of Mnet is that this notion of friendnet (with or without systemically contrained transitive operations) is at odds with a fundamental architectural feature that it inherited from Mojo Nation, as originally designed by Jim McCoy and Doug Barnes. This architecture offers the storage and transfer of bulk data as a global, automatically transitively managed resource, while the encryption keys necessary to download and decrypt the data can be private (and can be shared by actual human friends via telephone or e-mail). I always liked that idea, and when I initially launched the Mnet project and named it a "universal filestore", my goal was to focus the project on implementing that simple abstraction (universal public data store, private keys).
Nowadays I'm less keen on that abstraction, since the global part of it will eventually require some "step 3" answer, and I'm doubting that layering step 3 on top of step 2 is the right approach, compared to the approach of revisiting step 1 and building a unified and elegant emergent network from step 1 up. There are also technical problems with the abstraction which I'll save for a later day.
Now, a lot (all?) of my fellow Mnet Hackers are very keen on micropayments, and even if I were to actively oppose the micropayment notion, they would go ahead and implement it and give it another go. So that's one future of Mnet (or a branch of Mnet): another try at Mojo Nation's architecture wherein step 3 (integrated automatic ubiquitous micropayments) is layered on top and provides attack resistance and resource management for step 2 (universal data store and transport). A second future of Mnet (or a branch thereof), is to break the universal filestore abstraction and return to step 1, building a friendnet-Mnet in which any two computers are allowed to have a relationship if and only if their human users already have a similar human relationship. A third future of Mnet, which is almost certainly going to happen in the near future, is just a good implementation of step 2 without any step 3 solution. The basic public-storage, private-key architecture is already better than most other emergent networks in current theory or practice, and will form an excellent base for more experiments.
Intriguingly, all three of these possible future Mnets can in principle interoperate with one another...
(Thanks to Lucas Gonze and Tschechow for the discussion that prompted this and thanks to Artimage Nelson for the comparison between friendnet and VPNs.)
Darius Bacon is selling off his personal book collection to pay rent. It makes me sad that he has to do this. I've put in an order for several of the books that I want to read, and I've told him I'll take care of them and keep them available in case he wants to buy them back someday.
The recent Irby anecdotes have been popular with a select audience of grandmas and uncles.
Here's an Irby picture, weighing 1.5 MB (Grandmaxo: I estimate less than 10 minutes download time).
Irby received Newtonmas gifts from Grandmaxo and Bee today. He keeps asking me to open them. Finally he started opening one himself, so I put them up on a shelf. We need a Newtonmas tree to put them under.
I'm a bit sad to realize that while Irby uses ASL signs for words that he doesn't know (for example, he uses the ASL for "open"), he does not pay attention and understand when we sign complex phrases to him in ASL. For example, I can offer him a drink of milk in in ASL, and he won't really pay attention, but then if I immediately offer the same thing in English he'll attend and respond. I think we have only a couple of months to get him (and us) involved with real, fluent ASL speakers, or else we'll all stop using it.
Irby ran back and forth from room to room to compare the two digital clocks and the analog clock. Then he sat for five whole minutes watching the big hand of the analog clock move from "1" to "2". I explained to him all about the constant angular velocity, the division of hours into 60 minutes, days into 24 hours, and analog clocks into 12 sections and so forth, but I doubt he understood any of it.
I've decided to post ideas/questions that I come up with (or come up against) while reading crypto papers. The first paper to get this treatment will be Authenticated Encryption: Relations among notions and analysis of the generic composition paradigm (references from citeseer).
I've been having fun with Irby, who talks now. He uses emphasis to communicate to us. Recently he asked for "apple" and I started talking about the apple juice on the counter. He said "DAT apple. DAT apple.". I looked and he was pointing at the apple on top of the fridge. In the bathtub today he was naming the bath-toy numbers: "MUM" (one), "WHEE" (three), "FO" (four). After each one I would congratulate him "Yeah! There's a number FOUR there!". He said "NINE", and I said "Yeah! Number nine!", and then he pointed at another number nine -- a yellow one -- laying underneath the first number nine and said "NINE", and I said "Yep! That's number nine!". He thought that I was still talking about the first one, so he made eye contact with me, pointed again at the underneath one, and said "LELLOW nine".
I quit my job. I had been continuously unhappy about it -- feeling like I wasn't putting in enough hours to really fulfill my responsibilities but not wanting to cut any deeper into family time and weekly Mnet hack day. I feel much better now. I think that we can manage our finances with the money Amber brings in (scholarships and stipend), the occasional consulting gig that I might get, and a very careful program of frugality.
My boss was quite disappointed that I quit, which made me feel bad. He was a very good boss and the product he is working on has great potential. This is the first time I've left a boss or customer disappointed.
Now I have time to update this web diary.
Irby has started preferring to do some things by himself. It's convenient because sometimes there are things that we want him to do, but that he doesn't want to do, such as sit in his stroller for a fast ride home instead of exploring the neighborhood. In such cases, it helps to ask him if he wants Daddy to do it (put him in the stroller) or if he wants to do it by himself (climb up into the stroller). Normally I approve of exploring the neighborhood unless (a) Irby is tired and needs a nap or (b) it is -10 degrees Celsius outside.
One year ago today I had submitted a paper to IPTPS02. The paper was accepted and I got to travel to MIT to present it to a roomful of great scientists. I really loved the experience and I hope to do things like that again someday.
Hello world! Zooko.com is back. There will be nice pictures and movies of Irby soon. Icepick sent me a webcam, so we might even get LIVE O'WhielacronxCam soon.
One year ago today I was recovering from a zooko.com outage and a bug in the Python interpeter. Same as this year.
Packing for wedding trip. Did I think I was going to be able to do some work tomorrow before leaving for the airport? Ha!
Irby's language skills advance faster than I can update my web log. He has used possessive a few times: "IRBY carcar!".
Here's an Irby movie in which I show him how to make a sign ("yummy") and he does so. It doesn't really show his sophisticated communication skills such as making requests or answering questions, but it's pretty cute.
It's been a hard week. Irby's been sick since last Thursday night (just a cold), and he has to stay home from preschool. Amber is working hard to give me time to do my work (she does grad school all day, then takes care of Irby and the house in the evening while I work), but she has two assignments and a test next week, and we are travelling to my father's wedding this weekend, leaving Friday afternoon and returning on Sunday. (I've noticed that graduate student homework is qualitatively different from undergrad homework.) I realized last night that I haven't even told my boss about this weekend's wedding travel. Argh!
Update: I had an hour to work, by combining work with dinner. Unfortunately I spilled stew all over my keyboard, lap, desk, and carpet. This stopped the keyboard (at least temporarily), which gave me the incentive to fix the wiring in my old Microsoft Natural keyboard. So now I have a working Microsoft Natural keyboard again. But yet another hour is lost with no progress on my work
Yesterday I was delighted to see that Bill Gates is donating $100 million U.S. to fight AIDS in India. The fact that the donation came as part of a coordinated effort to dissuade the Indian government from its plan to adopt Linux gave me some qualms, but I was still happy to see that much money directed to fight the plague that threatens to cripple India.
Today I was extremely disappointed to learn that it wasn't Bill Gates who donated the money, nor was it Microsoft, but it was the Bill and Melinda Gates Foundation. I had thought that the B&MG Foundation was a charity whose Board of Directors would decide how to spend the Foundation's money in the interests of global health and education, not an extra, tax-free, $24 billion war-chest for Microsoft to use in strategic maneuvers.
The Bill and Melinda Gates Foundation is the world's richest 501(c)3 charity by a very large margin, holding assets worth $24 billion. I wonder how much tax would be due if it was decided that instead of being an independent non-profit charity, it was actually a subsidiary of Microsoft, Inc.?
I hasten to add that I do not wish that this would happen! The B&MG Foundation is uniquely empowered to save billions of people from disease and ignorance in the coming decades. It has already donated over 5 billion dollars for valuable philanthropic projects, focussing primarily on global health and education. It would be a terrible tragedy if the impact of their work were diminished by the perception that they serve two masters: the improvement of humankind's health and education, and the continued monopoly domination of humankind's computation.
My comments yesterday apply only to "chunking", in which you divide a file into pieces and then manage each piece in isolation, without applying any erasure code or other technique to ensure that the loss of a subset of the pieces doesn't mean the loss of a subset of the file. The original Mojo Nation, and the current version of Mnet, first "chunk" a file into separate 256 KB chunks, and then perform erasure coding separately on each chunk. That first step is a lose. Mnet will soon have a completely redesigned distributed file store that has no chunking, but still has erasure coding (over the entire file).
Raph mentioned on IRC last night that he will use a 2:4 erasure code for gznork. The reason that Mnet can't do that is that the "no chunking" rule means that block sizes will be equal to L/K for file size L and number-of-shares-needed K.
Re: JDarcy's blog, the new version of the Mnet distributed filestore will use blocks instead of files, but Brandon has some good points. One good point is that breaking files up into blocks is more complicated. The second is that if a file is broken up into non-overlapping "chunks", e.g., the first 1 MB of the file, the next 1 MB of the file, and so on, and then chunk failures are uncorrelated, then the chance of retrieving a complete file is P^K, where P is the chance of retrieving a chunk and K is the number of chunks. That's bad. Another way of looking at it is if you have this (chunks with uncorrelated chunk failure), you will tend to lose one chunk out of 100 different files, when you want to lose 100 chunks out of a single file. (Assuming that you have "normal" files where the whole file is way more valuable than 99% of the file.) So the next Mnet distributed filestore will not do any "chunking".
People who helped me realize this important fact include Donwulff, Bram Cohen, and Justin Chapweske.
We're all sick here. I am somewhat awed by this list of books that Darius Bacon has read. I wonder if he is related to Francis Bacon.
I've started posting weekly summaries of the progress of the Mnet project.
one year ago today I wrote about Irby's +2 Helmet of Upstanding.
Irby has had an incredible burst of development of language skills in the last two weeks. His vocabulary hasn't increased much, but his communication skill has increased dramatically. He now consistently listens to questions and answers them, including yes/no, where, what, who, and "do you want" questions. He follows directions, although complicated directions require extra explanation, such as the time I asked him to put the jaybottoms in the laundry and the jaytops on the bed. He listens to conversations between Amber and me and then acts on what we are talking about. He has become very definite about communicating with us, whether he is doing it by word, ASL-sign, or gesture. It's really amazing to see this development!
He is very willing to try new ASL-signs that I show him, although he has a lot of trouble getting handshapes right.
He has also taken to "reading" books by himself. Most books come out as a long series of baby babble talk, interrupted by page-flipping. Last night, however, he read his favorite book, "Go, dog. Go!", and his rendition of it was all English words, and all on-topic (although not really related to the written words): "Doggie! Carcar go! Doggie up! Doggie carcar! ...".
Irby is very good at "Unkaspell", a game where words are displayed at the top of the screen, and whenever you hit a key, the corresponding letter appears, and if you hit a key that matches a letter at the top of the screen, then the letter that appeared will move "up" to join the word. movie (large: 11 MB).
Irby word update: he says "back" to mean the back or bottom of an object or person. He says "adee" to mean "right here", always while pointing with his finger touching the place in question, to show (a) where he landed when he fell and got hurt, (b) where he found a thing, or (c) where he wants his parent to sit in order to play with him.
"cheese", "fork", "cook", "away", "beans", "water", "dog"-ASLIrby says the spoken words: "apple", "onion", and "Mumby", which is how he pronounces "Lamby". He makes the sign for "finished", when prompted to do it. He makes the sign for "hold" (as in "hold me" or "carry me") even when not prompted. He makes the sign for "up" (it's the natural sign). He'll sometimes make the ASL sign for "apple" when shown how to do it.
Also very exciting is how he combines words, like "Daddy up". Actually "Daddy up" and "Mama up" are the only combination I can remember right now. Probably "Lamby up". He also sings songs: "blabalumblee, Mama! Daddy! Irby! yumivbelum...".
I updated my Zooko's projects page, as well as the Mnet subprojects page.
I'd like to talk about the differences between Chord and Kademlia, but I need to work on my job now.
Mnet Hack Day on the 27th was a big success. I intend to send an e-mail containing a summary of Mnet Hack Day on the 27th along with an invitation to the next Mnet Hack Day on the 3rd of November. That should be done as soon as possible since the 3rd is this coming Sunday, but of course I'm too busy! I'm also too busy to be updating this web diary ("bliary"?). Coming soon: videos and pics of Irby helping his parents carve jack-o-lanterns.
Irby can count "one, two, three, four, five" carcars, given sufficient prompting. movie 1, movie 2.
I'm excited because by using Windows in vmware I can view the ASL sign dictionary with video demonstrations. Irby finds these a lot more interesting than the illustrations in our printed sign dictionaries.
Irby has long said the spoken word "baby", and he used to make the ASL sign for it, too (it is one of the "natural" ASL signs), but he hasn't made the ASL sign recently.
Gimmer wrote and asked if Irby still thinks that "me" refers to Zooko. I'm not sure. He said "me" a couple times recently while pointing at me (or more precisely, at my leg), but perhaps he was talking about something else. It's harder to pick out words nowadays because he fluently babble-talks. I think preschool has accelerated his language development, with its two adults and several other children, all between 1 and 12 months older than he. I also think that preschool is why he suddenly learned to say his own name. I was reminded of this when we arrived at preschool today and two-year old Macy said "Irby!" to him.
If you have a Quicktime-capable web browser, you can see the ASL sign for "baby". (Explore the rest of the site for other ASL signs.) Sadly, I cannot view this on my Linux system. It makes me angry that natural monopolies resulting from the network externality thus interfere with Irby's education, but that's irrational. The rational thing to do is to just install a system that can view Quicktime plugins. It's against my principles to do harm to my own values on principle. (I have vmware, so this is actually fairly easy to do.)
Words: "babo" (nurse), "doggie", "Mama", "Daddy", "up", "no". Signs: "more", "down", "beans", "milk", "lie down", "work".
There are probably a lot more that I'm forgetting. I'll try to update this list as I think of them.
Oh yeah! How could I forget words "carcar" (frequent) and "Irby" (recent).
The emergent network brainstorming of yesterday is confusing because there are two different things I could mean by my proposal. The first is that Archer Daniels Midland should run a normal standard MIT Chord network, with the added factor that only nodes that are members in good standing of the organization are allowed to join. The Toronto Python Users Group could do likewise. Now my observation is that if any node happens to be a member of both Chord rings, then consistent hashing automatically gives us the best possible outcome with regard to resource sharing and file-location: the more nodes the two rings have in common, the more likely it will be that data published on one ring is available on the other.
The second idea is: suppose that Archer Daniels Midland nodes are not allowed to learn the IP addresses or identities even of other Archer Daniels Midland nodes! (Excepting the small number of nodes to which they are explicitly connected by dint of being family members or other close friends in Real Life.) Now the Chord protocol doesn't work as designed, because it is predicated on the ability of the nodes to automatically introduce one another. This latter idea is more interesting to me. The former just shows that the magic of "coordination without communication", which all scalable DHTs have, and which is implemented in Chord with consistent hashing, gives us some help in our attempt to make such a thinly connected and constrained network useful.
The idea in one sentence: Suppose you had a Chord-like ring in which all links were manually controlled -- every link is formed by original introduction (human-to-human), and no links are ever formed by transitive introduction (automatic computer-to-computer). This implies that there are multiple, overlapping Chord-like rings -- the Greater Toronto Area Python Users Group would have a ring, and Archer Daniels Midland would have another, and those two rings may or may not have nodes in common. By forswearing all transitive introduction, our Chord-like rings lose the correctness guarantee that Chord provides -- that all data which is stored in a Chord ring can be found in a lookup -- as well as its performance guarantee -- that all lookups take at most log(N) hops.
What do we gain? Well, we've eliminated the problem of "systemically attack-resistant transitive introduction" (this is the problem that Paul Baranowski was telling me about with regard to Peekabooty the other night), and perhaps with the addition of some kind of flow-bounding we will also gain systemic attack resistance with regard to resource usage. It also seems to me that in general the technique of forbidding automated transitive introduction provides the strongest possible systemic defense against malicious nodes attacking the network at its layer. (I say "at its layer", because it doesn't help at all against attacks such as traffic analysis, which I consider to be an attack at a "lower layer" of the system. Indeed, anonymity systems such as digital mixes need the exact opposite requirement: you never link to nodes which are your real friends -- you only link to automatically introduced strangers. Waitasec -- that isn't true! Consider a mafia which routes its illegal business transactions through legal family and friend relationships.)
All of this is quite fresh in my brain and little-considered. I no longer have time for the luxury of working out the implications and fixing the goofs before I post. :- | advogato.org/person/raph /
If you think of a spectrum from Mnet (massive automated transitive introduction -- ideally every node has been introduced to every other node with zero user involvement) to Chord and other DHTs (transitive introduction is limited for scalability reasons), to Achord and Freenet (transitive introduction is strictly limited for privacy/safety/security reasons), then the idea I've proposed above falls even further along that spectrum, at its extremum: automated transitive introduction (without the human user's direct control) is completely forbidden for privacy/safety/security reasons. I don't know if the details of the above will really stand up, such as Chord-like rings (which I choose over Pastry/Kademlia-like structures simply because they can be more quickly explained to others), and flow-bounding of traffic or of requests, and the distributed algorithmic mechanism that I've thought of for routing. In fact I don't know for sure that routing can scale up enough to be useful in this system.
I'll bet the Freenet people have thought a lot about that last question. (The system I've proposed looks a lot like Freenet in its structure, if you ignore the difference between manual and automatic links). It's too bad I don't hear from them more.
This idea is only one application of the more general notion that one should distinguish manually configured links from automatically generated links in emergent networks. All of the DHT research that I have read mentions the former (manually configured links) only in passing as a necessary requirement which is immediately discarded once a nice set of automatic links have been bootstrapped from it. One noteworthy exception to this is Raph Levien's research. Raph's thesis (incomplete) is all about the fact that your system can use the information contained in the existence of manually-configured links to bootstrap systemic attack resistance. The opposite view is written in "The Sybil Attack", which explicitly assumes away such information in its model, and then aptly proves that no system can have systemic attack resistance without a central controller.
Maybe we need a new buzzword to serve as the memetic vanguard of this idea. f2f -- the friend-to-friend network. It's the biggest, most powerful self-organizing network ever, and every day more of its nodes acquire computers and computer networks.
As an aside, I'm disappointed to see via citeseer's reference search that Michael Freedman and Robert Morris have bought the Sybil conclusion that a central authority is required. They write: "There are no known techniques to prevent individual entities from presenting multiple identities to the system, without some out of band authentication procedure and trusted centralized authority.". The part that isn't true is the "and trusted centralized authority" part. Actually, maybe the sentence is true, depending on what they meant by "known". ;-) This reminds me that Michael Freedman asked me for feedback on that paper (Tarzan), and I have yet to give it to him. :-(
Here is an incomplete list of relevant attributions: Mark Miller who taught me the concepts and the terminology of Original vs. Transitive Introduction (as well as the related design of the Vat Location Service), and Ross Anderson who suggested at the IPTPS'02 that I should look into "clubs and cliques" for Mnet, and Lucas Gonze who told me about -- whoops maybe I'm not supposed to mention that part. Oh yeah, and I should mention that two other projects moving into Mnet's idea space have got me thinking harder about alternatives: hivecache, which uses Mnet-like technology (they are related by ancestry, so its hard to tell how similar they are in their current forms) for the enterprise backup market, and OceanStore pond, which is the first actual implementation of an extremely ambitious and high-brow system that shares Mnet's basic metaphor of a "universal file space".
Paul and Joey came over and after dinner, wine and coffee we invented peer to peer networks on the whiteboard in the Attic Workshop.
Wow. The one year ago today page says that I have been slashdot-free for over a year now. (I have viewed slashdot a handful of times since then, either because someone told me that there was a relevant story there, or on a couple of occasions because I was trying to avoid thinking about something I should be doing.)
Irby is doing great. Amber is doing well, but she hasn't have enough time to keep up with her studies. I'm doing pretty well. I have just enough time to do my job, plus one day every other week or so to hack on Mnet.
Aaron Swartz tells a story about visiting Washington D.C.. I loved the part about The Bookmobile. I hope that people do that more often. It needs a better name, to distinguish it from traditional bookmobiles (which I also love). In order to emphasize the political point that they are making, they could name it The Public Domain Bookmobile, but the part of it that I like isn't the political point-making. (I rather dislike political point-making as a rule.) Hopefully long after its inventors and others have stopped making that particular political point, there will be rafts of high-quality mobile book factories of that sort.
In response to my diary entry about Lamby Irby's UnkaNathan, the original owner of Lamby, wrote in with this heartfelt e-mail:
Date: Thu, 12 Sep 2002 18:54:09 MDT To: zooko@zooko.com From: Nathan.Wilcox@Colorado.EDU Subject: Getting things straight. <sigh> Whenever loved ones have children, and you come to love the children, there's always a possibility of tension if you see them raising the little ones in a manner you disagree with... And now the tension is mounting. Lamby is a LAMB, not a bunny! I remember when I was about 4 or 5 going to a party at our parents' friends' house. I had brought Lamby the lamb along and as we were leaving I was cradling Lamby (the lamb) and some arrogant adult said: "Oh what a cute bunny you have there." I felt fire rise to my cheeks and stated resolutely: "It's a lamb!". The adults laughed patronizingly and then one proceeded to tell a story to the others about how when they were young they had a stuffed cat and thought it was a dog (as if I couldn't hear and didn't feel patronized). Oooh, those arrogant adults. How long had they seen Lamby and already decided what Lamby was. Well *I* had been in Lamby's direct vicinity for several years and knew quite well that Lamby is a lamb. Insistently Yours, Nathan PS: "Lamby" is pronounced like "lamb" followed by "bee". PPS: When Irby's old enough I'm going to tell him the truth!
I'm not updating this blog during the work week, but this is a special edition for Mom. Irby can feed himself beans. Click on each picture for larger images or click here for the movie.
I'm not updating this web log during the week, but I want you to know that we released a new version of Mnet over the weekend.
Yep. I'm going to start a policy of not updating this web log during the work week. Hopefully I will update it each Saturday.
As JDarcy mentions, I've been posting to my web log less regularly. I have little time to spare after Irbysitting, working (20 hours a week), and hacking on Mnet. Perhaps I'll start posting to my web log only on weekends.
I love this Irby movie. It was made just now. Irby the little hacker (10 megabyte AVI).
We're going to release v0.5.1-STABLE of Mnet today. See mnet.sf.net for details.
After a long period of almost no socialization, we suddenly have a passle of parties to attend. Last night we went to a UToronto Comp. Sci. department barbecue, then to a small party for Ian Goldberg. We got home at about 1:00 AM, and then I stayed up until 4:30 AM hacking on Mnet. When I woke this morning, I realized that I had stayed up until one hour before my normal wake-up time. (I normally rise at 5:30 every morning.).
Tonight we're going to a birthday party for Paul Baranowski, and hopefully on Tuesday I'll go to the inaugural meeting of the Toronto Python club.
Hm. I also need to work on my paying job this weekend...
JDarcy discussed lines of code ("LOC") production. I really don't like using LOC as a measurement. (Nobody does, but some people seem to use it anyway, for lack of anything better. I don't like using even in this case.) My main objection is that removing lines of code (a negative LOC-per-day value) is often an improvement! I really don't know how to map LOC measurements to anything that I actually value.
But, I can't resist, so here are the counts of lines of code I've committed to the Mnet CVS repository since 2002-02-11:
Total lines I committed: 5448
Run through `sort -u' to remove duplicates: 3578
Here are the same stats for my lieutenant coder, Myers (also since 2002-02-11):
Total lines Myers committed: 4380
Run through `sort -u' to remove duplicates: 3253
Note that this is only measuring lines that have survived until the current (unstable) version of Mnet. So if Myers and I each changed a certain line, then whoever changed it most recently gets counted in these stats.
Also note that this is Python, so the numbers should probably be multiplied by a constant factor to compare to C or C++ code.
This breaks down to about 15 lines a day for me, and about 14 for Myers. Pretty good, especially considering that Myers had a Real Job during this period. :-)
Dave McCusker posted an e-mail I wrote to him about handling overflow in C. At the end I wrote something that isn't true at all. I wrote buf[3] is semantically equivalent to *(buf+3), but the compiler can implement the latter more efficiently in some cases. This particular case is never implemented more efficiently on any real compiler. However, the example I gave immediately afterward involving changing the value of the pointer itself can be more efficient, and you would naturally use the "*buf" idiom instead of "buf[0]" in that case, as shown in the example.
Irby has grown quite fond of a little pink stuffed bunny that was Irby's UnkaNathan's bunny twenty-one years ago. His name is "Lamby". "Lamby"? Maybe he is a lamb instead of a bunny. Anyway, Lamby is a very useful bunny to have around, as Irby identifies with him, and likes to do whatever Lamby does. Lamby is a very good role model. Lamby is always willing to lie down and get his diaper changed, and he likes to give Daddy hugs. Lamby is happy to hop up out of bed in the morning, and always lies down right away when it is time for naps or bed-time. We'll see what other good behaviors Lamby is able to demonstrate.
We didn't make it to the splash pad yesterday. Hopefully today!
One year ago today, zooko.com moved to the servers of HavenCo, running on the artificial island of Sealand. I'm very grateful to HavenCo CTO Ryan Lackey for giving me this free hosting on the coolest co-lo on Earth.
The first day of school, September 3, was also the first day that the wading pool at the nearby park was turned off. Irby and I were very disappointed. Although he seemed to get over his disappointment almost immediately, I remained disappointed each day when I was reminded that wading pool season was over the year.
So when we went on a walk this afternoon, and gazing down from a hill into a large park saw children laughing and splashing in a splash pad, I was very pleased. Irby absolutely loved it, and I regretted not having the digital camera with us. Hopefully we'll get pictures tomorrow. A "splash pad" is a shallow concrete pool with a half a dozen artsy sprinklers stationed around it. One or two of the sprinklers turns on at a time, spraying a bunch of shrieking children. Unlike wading pools, splash pads are still open even though all of the high school students have quit their life guard jobs and returned to school because splash pads don't require a life guard, since all of the water is supposed to drain out immediately. Some enterprising child had flattened a plastic bag over the drain, though, so there was a pool of cold, sparkling water about 6 inches deep at the deepest.
Words that Irby uses, in roughly descending order of frequency and meaningfulness: "Mama", "Daddy", "carcar", "up", "no no no", "oooh!", "go", "two", "baby". There are others that he has used in the past but not recently.
American Sign Language signs that Irby uses, in roughly descending order of frequency and meaningfulness: "help", "yummy yummy", "breast milk", "more", "down", "yes", "baby". There are others that he has used in the past but not recently.
Letters that Irby recognizes and names, in roughly descending order of frequency and meaningfulness: "O", "S", "B", "D", "E", "A", "N", "M", "T".
He also says the spoken English word "me", but he thinks that it means, uh, me. He uses it as a synonym for "Daddy". I think this is because I sing songs (e.g. "next time won't you sing with me"), along with signs where the sign for "me" is an index finger pointing at myself.
Several alert readers wrote in to point out that Irby appears to be hairless. They are perceiving correctly: we shaved his head! This was his first haircut ever.
I was reading Aaron Swartz's web log, and was astounded at what a reasonable proposal Aaron had quoted Dave Winer as making for RSS 3.0. No XML, no namespaces, no HTML. Such simplicity and reasonableness were really unexpected. Then I realized that those weren't Dave Winer quotes -- those were Aaron's own suggestions. This isn't to knock Dave Winer in particular -- he's no more complexifying and unreasonable than the rest of the Web industry.
Yesterday I took Irby to a farm in very large park near our house. Here is my favorite picture (click for large picture). Thanks to havoc, icepick, and Taral for advising me how to shrink images.
One year ago today I asked If anyone out there knows of a patent-free linear-decoding-time erasure code that works for arbitrarily large numbers of shares, please share it!. An old friend of mine from Amsterdam, Niels Ferguson, wrote in with a mathematical argument that such a beast is impossible. I believed him, and since then I've been thinking about how to achieve robustness and efficiency in a large-scale emergent file store with normal old super-linear-decoding-time codes. (Based on ideas which were told to me by Donwulff, Bram Cohen and Justin Chapweske.)
It's nice to have breakfast, coffee, and e-mail in the morning while the sky lightens outside the windows of my attic workshop.
Hello world! Things are fine! Also busy!
As "filler" for my web log, I'll mention that one year ago today, I first wrote about "proxying and introduction", and I had just started reading Ross Anderson's Security Engineering.
Large photo (not a movie): dscf0127-2002-09-Irby_drives_Maxos_carcar.jpg
Tomorrow I begin my new job and Amber begins hers. We have not arranged any "day-care" or other Irbysitting help yet. Amber's sister-in-law, who is a professional day-care worker, says that children Irby's age usually cry unconsollably for the first week that their parents leave them there. Now I don't mind if Irby cries for 20 minutes, but I am not leaving him to cry unconsollably for four hours straight. Not even one day.
I'm hoping to find a day-care organization in which the parents are expected to contribute their time as well as (or instead of) their money.
One consequence of all this is that progress on my non-paying-work, non-Irby-sitting projects is going to be slower from now on.
Irby loves picking up the phone and listening to the modem noises.
Hello, world! Well, we had a very nice visit to Nova Scotia, but unfortunately I lost access to my home machine "pion" so I couldn't update my web pages.
Tomorrow we fly to Nova Scotia to attend a wedding. I will have limited, but hopefully useful, net connectivity for the next 10 days. We return to home base Toronto on the 30th.
There is a new stable version of Mnet available from the Mnet download page.
The complete version of dscf0118-Irby_sits_in_his_little_chair_and_kickety_kickety_kicks_his_ball.avi is now available.
The Twisted Project put out a release announcement for Twisted v0.99 that mentions my Gnutella implementation, Zoot. But Zoot isn't complete, and doesn't do much. I think this is a sneaky tactic on the part of the Twisted folks to get me to finish it.
I'm working on EIO today. I've fixed several bugs in the README, thanks in part to Wes Felter. I've refactored the package layout so that users are faced just with the "API relevant" classes and not with any internal-use-only classes, and I've started fixing up the JavaDoc (thanks to bondolo for the suggestion). Those latter two improvements will be posted to the EIO homepage (and to this web log) later today.
There is a new Irby video: dscf0118-Irby_sits_in_his_little_chair_and_kickety_kickety_kicks_his_ball-INCOMPLETE.avi. It shows the living room of our new, and as yet nameless, home in Toronto. (It is incomplete because I am about to turn off my modem so that I can talk on the phone, and the video file is only 50% uploaded to the web server at this moment...)
one year ago today both of Irby's monkey unkies (on his father's side) came to visit.
Okay, I uploaded "v0.8.1" of EIO, which has the changes as described above. The JavaDoc documentation is incomplete, but the most important classes have been documented.
Hello world! Everything is okay. Irby and Amber are fine and happy in our new home. That was the longest "zooko current events" outage ever.
I've written an event-based I/O library for Java programmers. Here is the EIO home page.
Sorry for missing days. It's pretty busy packing for the move. Tomorrow we load the truck, the next day we drive away. Haven't finished packing yet!
Yesterday I relaxed barefoot in a hammock in the afternoon in the forest and read Security Engineering by Ross Anderson while listening to my in-laws playing guitars and other musical instruments and singing around the fire. (They like Neil Young and Pink Floyd and folk songs that sound like "bluegrass" music to me.) Amber brought me food (freshly baked on the fire) and drink, and a piggle baby who wanted to be up in the hammock with his daddy. Later the piggle (Irby) and I played rowdy games on the grass -- he squealed and giggled as I chased and tickled him, swung him around-and-around, and held his hands so that he could "walk" up to my chest.
Unfortunately I forgot my camera and there are no digital pictures from this event.
Mnet release has been delayed because the Microsoft Windows version of "wxWindows", the GUI toolkit, crashes too much.
A new release of Mnet is imminent. We are looking for volunteers to test it. To volunteer, send e-mail to <mnet-devel@lists.sourceforge.net>, or join IRC server irc.openprojects.net and go to channel #mnet.
Today in the grocery store, Irby (age 16 months, you know) was starting to climb up on the magazine rack while I was paying at the cash register. This wasn't the magazine rack right there in the checkout line, it was a big magazine rack adjacent to the checkout area, so I couldn't physically interfere without leaving the register and delaying the people behind me in line. I called out to him and asked him not to climb on the magazine rack. He hesitated and looked at me. I told him that they were "not for climbing on, just for looking at". He stopped climbing and began walking around "just looking" at them. "You're a good boy!" I told him. The old woman behind me in line spoke up, with a touch of amazement in her voice: "He sure as heck is!". I was proud.
We have some sweet new Irby movies today: Irby helps his mama pack for the move and Irby carries a packing box all by himself. As you can see from these videos, Amber and Irby are packing all of our belongings for the move while I play around on our computer all day.
I worked on Mnet and Gnutella today. Also I spent some time contributing improvements to the Twisted project.
There seems to be a lot of activity in Mnet world. A total of seven people, including me, contributed something to Mnet today.
My Gnutella implementation, which I've named "Zoot", is now able to do ping-pongs on the Gnutella network. (This is the way that Gnutella nodes meet new nodes -- they ping-pong each other.) You can ping the first ever running Zoot node by directing your Gnutella client to connect to 140.184.83.47 port 3653.
What's that? You don't have a Gnutella client handy? Okay then, connect to 140.184.83.47 port 3653 and send "GNUTELLA CONNECT/0.4\n\n". Zoot will write back to you saying "GNUTELLA OK\n\n". Then you send a 16-byte string of your choice, a byte containing 0 to denote that this is a ping packet, a "time to live" byte which should be at least 1 (Zoot is not going to forward your ping packet on to anyone else, so it doesn't matter if your time-to-live byte is greater than 1), a "hops" byte which Zoot will ignore, four zero bytes to denote that the length of the rest of your message is zero. Zoot will write back, echoing your chosen 16-byte string, plus giving you some other useful information.
The new Irby movies are here! Irby eats ham and gives each of his parents a kiss (1m10s, 10 MB), and Irby can say "Uh-oh" and "Goo-goo". (0m20s, 3 MB).
The Mnet project web page got 1400 visitors as a result of HiveCache being slashdotted.
Ping Yee's proposal for research on handheld user interfaces features a funny movie starring Ping. Ping is exceedingly cool. Check out the clever way that he encoded his e-mail address on his project web page. (Mom: his e-mail address looks entirely normal as far as you can tell, but programs that automatically scan the Web for e-mail addresses that they can spam won't see it as an e-mail address. That's what's clever about it.)
I've promised my mom a new Irby video as soon as we find the digital camera.
I wanted to practice my rusty Perl skills, so the list of URLs at the top of this page is now automatically updated by my new blogroll.pl Perl script.
Drue has released the first version of his Kademlia-like DHT in Python, Khashmir.
YESSS! I scored ICEPICK'S SEAL OF APPROVAL:
A cousin of the Mnet project, HiveCache, is featured on slashdot.
Bram Cohen wrote a diary entry.
It reminds me of one of the problems with TCP in a mutually untrusting world: individual users can hack their local TCP stack to be more aggressive about retrying and elbow past better-behaved TCP connections. In theory, this problem could be combatted by routers monitoring the retrying behavior of each endpoint, but it would be tricky and expensive.
As soon as there is a mature replacement for TCP ready, I'm going to pseudonymously publish a utility to turbocharge your local TCP stack. ;-)
Last night I made a small bit of progress on Twisted Gnutella and I fixed a very important bug in Mnet.
Icepick's web log and Aggle's web log have fresh tidbits.
Amber is sick -- probably kidney stones. I'm trying to balance Irbysitting and working at the same time.
By way of AaronSw's web log I found an essay by Jaron Lanier about "Minority Report". It is a touching essay, and I recommend it. What he says about "Minority Report" is perhaps what I was struggling to articulate two days ago when I complained that Spielberg doesn't believe in the reality of science fiction.
I stayed up late last night hacking. I contributed an incomplete implementation of Gnutella to the Twisted project.
Looks like I'll stay up late tonight too. Mnet or Twisted/Gnutella? Maybe both!
Amber and I went on a movie date last night (Irby was very happy to be at his grandmother's house again, and he didn't mind at all that we left him there for a while). We saw "Minority Report". It was fun, but I was slightly disappointed with the "rough edges" in the storytelling. I guess that I had been unconsciously assuming that Spielberg would put as much effort into perfecting the details of this movie as he did for "Schindler's List" and "Saving Private Ryan". In retrospect, this was a pretty silly thing to think. Spielberg doesn't believe in the reality of science fiction, the way he believes in the reality of World War II.
My mom is requesting new Irby videos. We'll make some soon.
I want to see an video of Aaron Swartz doing the Zooko dance.
Aggle did indeed rescue all of my e-mail. Thank you, Aggle!
I wonder why Cynthia Dwork doesn't have a home page?
I'm writing a tutorial on how to use the EGTP API.
We have returned to the Aerie. The Toronto Apartment Quest was difficult, but we did very well. Irby was very patient about being in the car day after day (we spent three nights on the road, each way). We visited Ian and Kat in Montreal. We allocated an apartment in Toronto, in the "Danforth" neighborhood, which is full of old Greek people. I love how multicultural Toronto is -- during our two weeks of house-hunting, we overheard Arabic, Greek, Spanish and some Asian languages, not to mention French.
My mail server fell off of the Net on the Fourth of July. I think that Aggle has backed up my incoming mail, but I haven't seen it yet.
Twisted is an ambitious project to provide lots of high-level, sophisticated networking features in Python. I've been aware of it for a long time, but haven't seriously considered using it for Mnet because it seemed too featureful for what Mnet needs. Today the Twisted developers contacted me and gave me their newest pitch for why Mnet should use Twisted. The best motivating reason that I've seen so far is that then the Twisted developers will maintain the lower-level networking code, leaving Mnet developers more time to work on the unique features of Mnet. The second reason is that Twisted would provide several more underlying protocols. Mnet already uses TCP and Relay. By using Twisted, we would also be able to transmit messages over HTTP, UDP, and whatever other wacky things the Twisted developers come up with. The third reason is that Twisted has already implemented a clever (perhaps overly clever) feature that I have long wanted for EGTP: protocol-specific compression.
I've offered to explore optimizing some of their networking code for them, and one of them has offered to replace pyutilasync.py with the Twisted equivalent. If this kind of talk continues, I'll add Twisted to the Mnet related projects page.
Aaron Swartz had a dream (this link has been fixed to point to Aaron's dream) in which I appear. I think it would be fun to make daily notes about my dreams, but it would be too personal to post to the Web.
It seems that these two technical concepts are garnering interest among the hacker community: privilege separation, courtesy of Niels Provos, and alternatives to pre-emptive multithreading, as in SEDA, and web log notes from McCusker, Raph, and JDarcy. A related trend is the way that Microsoft's recent publicity maneuvers seem to be predicated on the belief that consumers will be willing to buy a "Big Brother Inside" product which prevents its owner from misbehaving, because those consumers believe that submitting their computer to Microsoft's direct control is their only salvation from virusses.
If I'm perceiving these trends correctly, then the E language might receive a wave of interest in the near future. But maybe I'm not perceiving the real trends in the world -- maybe my pattern-recognizing human brain has just picked out those few news items that happen to match my own preferences. What do you think? Yeah you -- the one reading this web page right now. Please e-mail zooko@zooko.com and tell me if privilege separation is becoming the new focus of researchers or is still obscured by the fog of identification and so-called "trust". And tell me if pre-emptive multithreading is losing its reputation for being the modern, sophisticated way to do concurrent computation. And most importantly, tell me if mainstream end users are really coming to make decisions based on security issues, or if they continue to ignore those issues out of confusion and apathy.
This is something that I care increasingly about, due to recent events. If The Man (comprising the major computer industry companies and the major national governments) has his way, then Irby will not be allowed to buy a computer that does what he, Irby, tells it to do. Instead, the only computers that he will be allowed to own will have "Big Brother Inside" chips which monitor Irby's choices and intervene if he tries to do something that The Man hasn't approved. For those of my readers who are not following the news, this is not in any way speculative on my part -- this is the explicit and public goal of several recent initiatives on the part of Microsoft, Intel, IBM, certain U.S. Senators, and others.
Well, at least Irby will always have pion. I'd better buy a few more IDE drives to swap in once this one gives out.
Speaking of which, here is a picture (very large) of Irby getting to know pion a couple of days ago.
Sorry I haven't been posting frequently. I've finally gotten my new machine all configured so that it is comfortable to use. We're departing tomorrow to invade Toronto and look for an apartment to rent. If you know of any apartments for rent in Toronto please e-mail me.
The version of Valgrind that gave me the error regarding multithreading was an old snapshot. The new release candidate handles that situation fine.
Whenever I'm going to leave town, I feel a compulsive urge to commit small bugfixes and cleanups to Mnet before I go. Other people I know feel a similar urge, but theirs is about doing all the dishes. I have to decide on reading material to take on the road trip.
Valgrind is an extremely valuable tool, but it is incomplete. I was using it to look for memory bugs in EGADS, and it found some, but then EGADS tried to do something with pthreads that valgrind isn't prepared to emulate.
valgrind's libpthread.so: UNIMPLEMENTED FUNCTION: pthread_sigmask valgrind's libpthread.so: Please report this bug to me at: jseward@acm.org
Hello, world!
Irby is doing well. I'm working on Mnet, and related projects such as Crypto++.
I got USB working on the new computer today. Here is the first Irby image transferred via pion. Amber says his face looks different to her in this picture. I guess she's seeing a non-baby Irby face.
I got the sound card working on my new computer today. We've spent a lot of time over at Amber's mama's mama's house. Irby babbles a lot verbally, but still no real words. He does, however, use a couple of ASL signs consistently. He consistently uses the signs for "yummy", "breastmilk", and "down" in order to communicate with us, and he'll emulate us if we do the signs for "food" or "more".
It turns out that the best crypto idea I've had to date (forced latency for Chess-Grandmaster prevention) has also been previously published. It is gratifying that these brilliant minds use their time machines to retroactively steal all of my ideas.
Our new computer, named "pion", is almost ready. The hardware is operational, and the operating system (Debian) is installed, but is currently experiencing a snag in the boot process.
Irby went to community play-time today and got some practice sharing (or not-sharing, as the case may be) with lots of neighborhood kids.
Irby is doing great. He's been playing with Witbee more, and he is increasingly babble-y. His cousin Jacob is a few days older than he, and Jacob already talks. One difference is that Jacob has spent a lot more time around a lot more other children than Irby has. I want to be sure to give Irby lots of play time with other children from now one.
We're trying to figure out how to find an apartment in a city where we don't live (Toronto). We're shopping for airplane tickets to take us there to house-hunt ASAP, where ASAP means >= 21 days in order to get the cheap tickets.
I played in a Magic Cards tournament at a local shop Friday night. There were ten competitors and I won 3rd place. Perhaps the most gratifying part was that the other people tended to boggle when they saw the cards I used in my deck. I like using cards that most serious players consider to be too weak or just too weird for competitive play. (It's also easier to acquire those cards than the powerful ones that everyone wants.)
Hal Finney wrote that the crypto primitive that I thought of a couple of days ago is in fact (one kind of) Oblivious Transfer. He said that "They say that all of cryptography can be built on top of Oblivious Transfer". What an intriguing suggestion! I have so much to learn.
AaronSw asked why Amber doesn't just lie about the position of the Centaur. In the simple model, she chooses bit b at her discretion, and commits to it, so because she has committed to it she can't later change it. In the actual Centaurs versus Carrion Crawlers game, she sends a commitment to each move every time she moves a piece, and at the end of the game she has to open all those commitments. Therefore Bryce can see where all of her pieces had been throughout the game.
Old friends of mine from high school are having a picnic reunion this weekend in Colorado Springs. I can't make it, and I had started composing a letter to them to let them know that I wouldn't be there because of circumstances, not because of lack of interest. Sadly, I do not have any of their e-mail addresses after the hard drive crash, so I don't know how to communicate with them. I was listed as being responsible for "snacks/dessert". I hope someone else brings snacks/dessert.
I've had a catastrophic IDE hard drive failure. Nothing was backed-up, and so now, unless a "data recovery" service can pry my hard drive open and scrape together all the spilled bits, I have lost two years' worth of e-mail, all of my Irby videos (except for those few that were posted to zooko.com), many incomplete writings, and so forth. I have successfully struggled to avoid becoming depressed over this.
Thanks to Octal for reconnecting me to zooko.com right away so that I can post this announcement.
One thing in particular I have lost is several hundred e-mails that I have been saving with the intent to reply to them (or at least read them more closely). If you are one of the several hundred people waiting for a reply from me, I humbly ask you to send me a note. I would love to hear from you, and I'll write back, at least briefly.
Amber I started putting together the new computer today. It's all together except that it didn't power up at all in the first power-up test. I think we have the power-toggle switch wired backward. We're too sleepy to debug it right now.
Irby is doing great. He has become more talk-y recently, possibly due to spending a week with his loquacious three-year-old cousin Witbee. He says "Mama" reliably to mean Amber (or sometimes me), and "Nanananana!" to reliably mean Amber or that he wants to nurse. He says "Oh" for the letter "O", but not reliably. He reliably points to the capital O letters in his books if I ask him to find the letter "O". He has also made a couple of sign language gestures, but not reliably. His speech comprehension is very good, and he is a very good-natured and helpful boy. For example, he recently dropped a book into the toy box and turned away to play. I asked him to please take the book out of the toy box and put it on the bookshelf (on the other side of the room) and he did so. He also brings us our shoes when we are getting ready to go for a walk.
I came up with a neat crypto idea which I haven't heard before (which doesn't mean that it isn't already a standard part of the modern crypto literature). I'll tell you what it accomplishes, but I'll save how to do it for later. (Raph Levien came up with the best implementation -- my first implementation was of uncertain security.) The goal is that in Phase One, player A (let's call her "Amber") is going to choose a bit b and commit to it ("committing" means that she can't change her mind later). If b is 1, then she will receive some information from player B (let's call him "Bryce"). If b is 0, then she will not receive this information. The catch is that Bryce mustn't know which bit she has chosen, until later. He will find out what she chose, and verify that she didn't cheat, in Phase Two.
So in Phase One, Amber chooses b, and only if she chose 1, she receives some information from Bryce. In Phase Two, she reveals her choice and Bryce verifies that she didn't cheat. Now several things must be true: First, she must receive the information from Bryce if and only if she chose b == 1. Second, Bryce must not be able to figure out what b she chose until Phase Two. Third, once they enter Phase Two, she reveals her choice and if it was b == 0 she must be able to prove that she never received the information. It isn't good enough that she received the information but declined to look at it -- this would allow her to cheat, and Bryce would have no way to be sure that she didn't cheat.
The motivation for this idea is that I am figuring out how to implement a fair two-player limited-information strategy game. "Limited information" means that the players don't know everything about the state of the game, such as in Stratego where you don't know the type of all of your opponent's pieces, or card games where you don't know what cards are in your opponents' hands. "Fair" means that neither side can cheat. Furthermore, I intend to do this without recourse to any "judge" or other third party that both players would have to trust. (Because after all, then he could cheat.) "Phase One" of the crypto protocol happens when the two players are playing the game with each other, and "Phase Two" happens at the end of the game when their computers reveal all of their pieces and go back through the history of the game and verify that the other player didn't cheat.
In the game I was envisioning, Amber might have a Centaur piece which could see far and wide. If the Centaur piece was a few spaces away from Bryce's Carrion Crawler piece, then the Centaur would be able to see the Carrion Crawler, but the Carrion Crawler would not be able to see the Centaur. (Carrion Crawlers do not have good eyes, and rely on their slimy tentacles to detect what is right in front of them.)
For the game to be fair (i.e., neither player can cheat and get away with it), then Amber would have to learn about the location of Bryce's Carrion Crawler, but only if she had a Centaur piece in a location that it could spot the Carrion Crawler. At the same time, Bryce must not be able to detect whether or not there is any piece spying on the Carrion Crawler. (Carrion Crawlers do not have spidey-sense.)
If there are any real cryptographers in the audience, I think that this technique is related to oblivious transfer, but is not identical to it.
An elevator door at the drugstore ran against Irby's little big toe and hurt it. He cried and cried but everything is okay now. I hope that he's learned to be more careful about doors.
Wes Felter pointed out several things about my log entry named Open Problems In Emergent Routing. First, it's called "Byzantine Fault Tolerance". Second, "proxying" and "emergent routing" are not identical, but related. To "proxy" is to accept a message for someone else and pass it along. "Routing" is transitively passing such messages. I've retroactively fixed these two issues in the essay.
Wes also points out that Napster and Gnutella do not use the real human social network. This is a very good point. I have two comments to make on it: first Napster and Gnutella are not attack-resistant. (This has been demonstrated on Napster, but not yet on Gnutella.) Second, although they do not use the human social network, they do rely on the fact that the users get satisfaction out of altruistically sharing their files with others. That's because of humankind's social nature.
Wes also made the intriguing point that emergent lookup and emergent routing are not necessarily identical.
In a related story, Myers Carpenter has a web log. Myers is a major hacker of Mnet (I have trouble trying to summarize his accomplishments in one line for the CREDITS file). In his web log he says that he wants to implement emergent routing (his favorite is currently Achord) for Mnet, so maybe Mnet will gain emergent routing soon after all.
The Vole of Justice asked me what I meant by "emergent" routing, exactly. I thought about it a bit and said that I mean that there is no node which has global knowledge of the network. I'm not satisfied with that definition. This property is definitely necessary to be an emergent routing system, but it isn't sufficient. Here's my next attempt:
A network is an emergent network if its salient global qualities (e.g. efficient global routing, system-wide robustness, etc.) arise from purely local state and purely local computation (where "local" here also allows shared state and computation between two peers).
This explicitly excludes systems in which there is some overseer that knows something about the whole network (or even a large part of the network), but it doesn't exclude systems in which there are different kinds of nodes with different roles, as long as none of them require non-local knowledge.
Of course, this leads me to wonder about the flip side of knowledge: authority. If you have a system in which no single point has non-local knowledge, then you necessarily have a system in which no single point has non-local authority. Maybe what I really care about is the topology of authority.
Systems researchers are interested in emergent networking because the fact that no node needs non-local knowledge enables designs that are more efficient, robust, and scalable. A different reason to care is because the fact that no node has non-local knowledge means that there is no node that can tell other nodes what to do.
In any case, I'm not at all satisfied with my first two attempts at a definition of emergent networking. I think I'll ask Nelson Minar for his opinion.
This is the month that we begin active house-hunting in Toronto.
Raph wrote in his web log that whether a p2p project is working on implementing XOR routing or not can be used as a test of its vitality. I'd like to take a moment to explain why I'm not actively working on adding emergent routing to Mnet but I still consider Mnet to be a living and growing project.
The first reason is that we don't need it yet, and I'll go into that in a minute. The second reason is that the science of emergent routing isn't mature yet, and the open questions are big enough that it might require re-engineering the entire design once they are solved.
Why doesn't Mnet need emergent routing now? Because the Mnet network is too small. (In fact, it is quite tiny right now, consisting of less than a dozen hackers and experimenters. When we make the next package release, I expect between 100 and 1000 new people to try it out, depending on how far the release announcement gets propagated. It's a point of honor with me not to hide this kind of information, and in fact we are working on increasing the project's transparency by means of a script that regularly updates a web page showing current Mnet statistics.)
In September of 2001 I posted an article to the p2p-hackers list estimating that Mojo Nation wouldn't need "proxying" (which is the basic operation in routing) until it reached 2^18 total nodes. Recently I reconsidered that position, when I realized (thanks in part to Raph) that it isn't the total size of the network that becomes too big first, but the churn rate. However, now I have re-reconsidered, because I don't want to support fast churn rates. I would rather have an introduction system which is too slow and limited to handle 10,000 new nodes appearing every hour, because the upper layer (the Mnet distributed file store) will break badly if it is asked to deal with that much churn (assuming that 10,000 new nodes appearing in one hour means approximately 10,000 nodes disappearing in the next, taking most of my precious file data with them. See Lessons From A Large-Scale Failure for details.)
Of course, I need to be careful that it fails safe -- for example, a satisfactory result would be that when a flash crowd of 10,000 new nodes show up, 9,900 of them are unable to connect to the network at all, but the current nodes which were connected before the flash crowd are able to remain connected and operational while the flash crowd passes.
Number one: trust. All emergent routing schemes that I've seen so far make no a priori distinction between nodes. The algorithm in question (be it routing, file storage, or whatever), is given N nodes, and it treats them all the same. Now the problem arises: what if some of them are not following the rules, but are instead acting selfishly or maliciously? The well-explored scientific answers all begin with "assume that no more than K (< N) of the nodes are dishonest...". That body of scientific knowledge (which I think is called "Byzantine Fault Tolerance") is valuable for both theoretical and practical reasons, but it always fails when you ask "Okay, so what if K+1 of the nodes are dishonest?".
(See The Sybil Attack. I disagree with John Doceur's conclusion that a centralized master control program is necessary, but his attack on Byzantine schemes (which boils down to "Okay, so what if K+1 of the nodes are dishonest?") should really be taken seriously by Byzantine schemers.)
(See Dynamically Fault-Tolerant Content Addressable Networks for the best Byzantine-style scheme for emergent networking that I've seen yet.)
The "What if K+1 are dishonest?" challenge is the number one unsolved problem in emergent routing. Ironically, it is a problem that was largely created in the first place by the academic habit of abstracting the computer network away from the human network in which it is embedded. The most interesting networking opportunities that we are facing in real life are not of the form "An arbitrarily large number of strangers are connecting to me and trying (apparently) to cooperate with me." but "Two dozen of my friends (and transitively, hundreds of friends-of-friends) are connecting to me and trying to cooperate with me.".
Raph's incomplete thesis is, as far as I know, the first approach to emergent networking which combines mathematical rigor (such as that practiced among emergent networking and Byzantine Fault Tolerance academics) and the essential and inescapable human trust network in which the computer network is embedded (such as a large number of start-ups and independent hacker projects are doing).
Note: the E project and Mark Miller in particular are the ones who originally taught me the necessity and usefulness of the pre-existing human trust network. I do not count E along with Raph's work in the above declaration because E is not itself an emergent network -- it is a more fundamental point-to-point network (as well as being a capability-secure runtime and an elegant language) upon which an emergent network could be built.
There are other unanswered questions in today's emergent routing schemes, but they pale in comparison to this one, and besides I'm tired of typing after having furiously pounded this rant into my long-suffering keyboard.
I want to write more about emergent routing systems (e.g. the XOR routing scheme), but it's too late tonight. Irby has had a good week, playing with his cousin Witbee and playing outside in the back yard.
It's somewhat amazing to see so many different people (six today, including me) collaborating remotely on the Mnet project. Soon this collaboration will yield a new stable release for you to try out.
Due to discussion with Ted Anderson, Bram Cohen, Oskar Sandberg, and Amber O'Whielacronx, my understanding of XOR routing is deepening. One especially intriguing thing to me is: can we combine XOR routing for efficient global routing with trust links (real trust links, derived from the real human social network) for attack resistance? Here is a recent message I posted to the bluesky mailing list.
It was a nice warm summer day yesterday, and Irby and I spent an hour in the afternoon bouncing a ball on the lawn of Glimmerglass, which we are house-sitting on behalf of its world-trotting owners. http://zooko.com/dscf0017.avi-2002-05-29-14_Irby+plays+ball+on+the+lawn.avi
I've been pretty busy with child-care and professional work recently, so progress on Mnet has been slow. On the other hand, there are now three active Mnet hackers other than me (plus several sporadically active hackers), so we have robustness that keeps the project creeping along even when half of the participants are distracted by Real Life. :-)
Here is an inspiring rant by a businessman about "The New Economy" that propelled the stock market into the stratosphere. It says that The New Economy is real, and it is here, but that it doesn't increase corporate profits, it decreases them. This is bad for managers and investors but good for employees and consumers. I've always believed that this would be the effect of the Net on business.
The Economist has an excellent article on the dangers of neuroscience. One especially troubling aspect, which they do not mention in this article, is that cognitive science is already revealing techniques for manipulating behavior without requiring any physical access to the subject. See this earlier Economist article for some frightening early results in how to reliably influence people's behavior simply by showing them the right kinds of images, words, and music.
When you read this latter article, you will probably think "Oh, good thing these techniques don't work on me.". That's one of the especially disturbing parts: scientists have demonstrated both that most people believe that it doesn't work on them, and that it does work on them.
In a related story, sciencedaily.com carried a press release about a lightweight, portable brain-imaging device that is worn like a cap. Maybe someday you'll be required to wear one of these in order to have certain kinds of jobs.
I'm fixing Mnet's HTML-based GUI to work with Mozilla while retaining compatibility with non-JavaScript, text-only browsers. Of course, it is probably better to use the new wxWindows-based GUI if you can. Once the Mnet bug list contains no more "v0.5" bugs, and someone builds win32 packages, we'll publish a new release of Mnet.
MarkM wrote to say that I should give Li Gong "more benefit of the doubt". I hadn't realized that my joke about a possible reason for his absence looks like a criticism. I have nothing but respect for the man. I do have a degree of disdain for corporations as such, including Sun, but I have nothing but admiration for Li Gong, whom as I mentioned is a pioneering scientist in some of my favorite topics of security research.
Whoa -- it a mega-log-entry. In today's episode we have "Amber has a Bachelor's degree... in science!", "Patents Covering Open Source Software", "names names names", and "Li Gong! Where Are You?".
Amber is an official alumna (sp?) of Saint Marys University. I'm very proud of her.
Today I read that Red Hat is getting patents on techniques that are (? or will be?) implemented in Linux. Yesterday I read that IBM gets 1000 software patents a year and made $1 billion in patent licensing revenue in the year 2000.
This kind of thing should be of intense interest to free software lovers. One interesting detail is that IBM's Common Public License, v0.5, explicitly mandates that each contributor grant to each recipient a license for any patents that are necessary in order to use the licensed software. On the face of it, this would belie my suspicion that IBM intends to take over Linux and other open source software with its overwhelming patent advantage once the field is ripe. But when I look closer, there is still something funny about it.
This detail is the "termination upon patent litigation" clause. That patent license that I mentioned in the previous paragraph is not irrevocable. The termination clause reads like this:
"If Recipient institutes patent litigation against a Contributor with respect to a patent applicable to software (including a cross-claim or counterclaim in a lawsuit), then any patent licenses granted by that Contributor to such Recipient under this Agreement shall terminate as of the date such litigation is filed. In addition, If Recipient institutes patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Program itself (excluding combinations of the Program with other software or hardware) infringes such Recipient's patent(s), then such Recipient's rights granted under Section 2(b) shall terminate as of the date such litigation is filed."
So what this means is that if a company contributes to a CPL'ed project, and it has a patent that could apply to that project (note: it is not required to disclose this fact), then anyone who uses that CPL'ed code is now constrained from initiating any patent litigation against the contributor, at the cost of instantly losing their right to use the CPL'ed code. If IBM creates a CPL'ed app like Eclipse, and if IBM has a patent that covers use of Eclipse, and if you are another company that uses Eclipse, like, say, Red Hat, and if you (Red Hat) were to sue IBM for patent infringement over one of your software patents (even if it were unrelated to Eclipse, such as Red Hat's recently discovered patents) then you would instantly lose the right to sell your Eclipse-based product. Note that there is nothing to stop IBM from suing you for patent infringement in general, although they cannot do so for patents that are covered by the CPL that comes with Eclipse (i.e., patents that are required to use Eclipse).
I'm not saying that IBM's lawyers and strategists planned it this way, but it does appear to me that the CPL's termination clause gives hidden power to the company with the most software patents and the least dependence on using CPL'ed software.
Clay Shirky's aforementioned essay about naming has stimulated me to think more carefully about the subject. (By the way, Clay apologized to me for omitting mention of my essay when he published his, and promised to give a reference to mine in the next Shirky Newsletter.)
I was wrong to assert that Clay's "non-political" is a translation of my "secure" and that his "global" is a translation of my "decentralized". They are not identical at all, although they are interrelated.
There is a new perspective on the naming issue that I am chewing on now. Earlier I wrote, "What names am I allowed to use when I talk to someone else?". But a more specific way to look at it is "What is the context of a name?". When I first read Shirky's "global", I thought he meant "unlimited reach", as in: I can talk to anyone anywhere in the world and give them a name and they will be able to use it. But I've subsequently figured out that what he, and a lot of other people, really care about is context freeness, as in: I can give you a name (e.g. "Amber"), and you can use the name without having to use any other information about the context in which you received it.
So I have a new appreciation for one of the ideas pioneered by SDSI and Pet Names -- that idea is simply that whenever a person names something in conversation with you, you do have context -- you know that the name came from that person. If I tell you "Please check out the new 'Amber' page.", this means something different to you than if a jewelry merchant contacts you and says "Please check out the new 'Amber' page.".
I think that big picture pundits like Clay rightly fear that engineers like me are trying to force humans to adopt computer behavior (like using public keys instead of names), which is a futile and disruptive endeavor. But my current perspective suggests to me that what the Pet Names idea is really doing is trying to force our computer systems to reflect our normal human behavior -- including the social context in the meaning of a name.
Sadly, the printed program that I received when I arrived at the Workshop on Economics and Security contained no mention of Li Gong or his talk Non-Technical Influences on the Design of the Java Security Architecture. I had been very eager to compare Gong's version of events with MarkM's incisive summary of same. (By the way, Li Gong is a pioneer, along with Paul Syverson, in a security topic that I personally follow closely: protocols generally secure against replay and all other active attacks: [1, 2].) I wonder why he didn't show? Perhaps for the next conference, he will present a talk entitled "Non-Technical Influences on the Presentation of Talks on the Security Architectures of Powerful But Dying Megacorps".
Graduation convocation.
Things are fine here. We're doing some graduation-celebration things. We went and spent the night at The Lake and I got to play Titan with Luke and Doug. Irby seems to have become a lot more expressive in the last few days. He now laughs in order to indicate his approval or pleasure (e.g. "Irby: do you want to go outside?"; "Eh-eh-eh-eh!") and cries in order to indicate the opposite (e.g. [his parents are not giving him their full attention]; "AAAAH!").
It seems that Irby is happy to see me. He was smiley and energetic all day yesterday after he and Amber picked me up from the airport. He laughed a lot in response to things he observed. Amber reports that this was a dramatic change in his temperament from earlier that day and from previous days.
I believe the job market for software engineering is picking up. Many of my friends have reported that they have received inquiries and job offers recently, after many months (in some cases up to 12 months) of fruitless searching. I, too, am making contact with more potential clients, including a nice inquiry I received stemming from my old work at DigiCash.
The O'Reilly Emerging Technology Conference was a good show. The common theme that I heard is that exponentially increasing complexity is now the defining problem. This was repeated in keynotes by Bob Morris of IBM, Rick Rashid of Microsoft, and Bruce Schneier, as well as in the talk given by Peter Neumann, as well as in private conversation that I had with other captains of industry. Presumably the same theme was expounded in Steven Johnson's keynote on "emergence", but I'm afraid I missed it. I will read the summaries by various bloggers.
I want to write some notes about the Workshop on Economics and Information Security. It was quite intellectually stimulating. One especially exciting bit is the new research on the incentive structure of "distributed algorithmic mechanism design". This is essentially the same topic that Mark Miller and Eric Drexler described in their 1988 papers on agoric open systems.
Intriguingly, three of my favorite fields of research are now showing some "author overlap" (there are several scientists who have published in more than one of these fields now):
Packety, pack, pack, pack! Yesterday, aside from furiously debugging Mnet (I hit the bug I was aiming at), I thought about naming. I don't fully grok the relation between Clay Shirky's "non-political" and my "secure". (See yesterday's log entry for details.) It is intuitive to me that "if you have secure names, then there is no possible political issue", but when I examine that idea more deeply I see there are two different issues.
The first issue is "Does the that name I give to you mean to you what I intended it to mean, even if a powerful interfering attacker tries to make it mean something else?". For example, when I ask you to please view a file called "Irby_likes_to_be_sprayed_with_water.avi", if we are guaranteed that you will get the file that I mean and not any other, then the process of you using the name to lookup the file was "secure".
(My distnames.html essay needs to be updated to reflect this simpler and better definition of "secure", as well as the other updates that I have planned since reading Clay's essay.)
The second issue is "What names am I allowed to use?", while retaining security?
If you use a self-authenticating name system like the Self-certifying File System, then you are only allowed to use names that begin with your public key. For example, that hyperlink to "Self-certifying File System" has this URL: http://www.fs.net/sfs/new-york.lcs.mit.edu:85xq6pznt4mgfvj4mb23x6b8adak55ue/pub/sfswww/index.html. The part about "85xq6pznt..." is a public key. If you and I both had software that used keys in that format, then when you clicked on that link it would be a secure name lookup.
So one possible answer to "What names am I allowed to use and keep security?" is "only names that include cryptographic authenticating data", such as the SFS link above. Clay Shirky rightly points out that trying to persuade all the humans in the world to do this is futile.
But look at the alternatives...
In the hypothetical "PKI/Identity" future espoused by corporations like VeriSign, Sun, Microsoft, IBM, etc., you are only allowed to use names which live in "your" part of the global namespace. For example, (still assuming that I want secure lookup) I would not be able to say "Please see 'Irby_likes_to_be_sprayed_with_water.avi'.", but would instead have to say "Please see 'nsi://Earth/Canada/Nova Scotia/Wilcox-O'Hearn000/Zooko/Irby_likes_to_be_sprayed_with_water.avi'.". This is exactly like the way that I currently have to say "Please see 'http://zooko.com/Irby_likes_to_be_sprayed_with_water.avi'.", but I'm assuming that I don't get a top-level name in their hypothetical future.
Okay I'm going to stop now, but I think I'm on the verge of gaining a clearer understanding of this important subject now.
Oh geez am I going to have a tough decision to make on Thursday. Should I be in Santa Clara with hundreds of p2p hackers, pundits, and businessmen who might offer me consulting work? Then I could see these talks: Clay Shirky on "the Semantic Wipeout", the Creative Commons coming-out (of which Mnet will be a part), Lawrence Lessig et al. on "The Future of Ideas", a presentation from Sun about "Federated Identity" (which in Choose-Two terms is "meaningful", "secure", and you get to be vulnerable to some subset of the servers).
Or I could be in Berkeley with dozens of economists and security researchers making presentations about their ideas. Then I could see Ross Anderson: "Maybe we spend too much?", Bruce Schneier: "No, we don't spend enough!", Andrew Odlyzko on the history of security economics and on "Privacy, Economics, and Price Discrimination on the Internet", Li Gong on "Non-Technical Influences on the Design of the Java Security Architecture" (I wonder how it will compare to MarkM's version.).
Okay, I'm definitely leaning toward the Security and Economics Workshop. The only drawback is economic -- those scientists are less likely to offer me lucrative consulting contracts or employment.
Irby is a very happy toddler. He thinks Amber's name is "DA-DA". For example, if I ask him "Irby! Where's DADDY?", he'll point to Amber and smile proudly for having known the answer.
I'm debugging Mnet so that when I'm at the conference I can proudly say that it actually works right now, rather than being able to brag only about future plans.
Clay Shirky, influential pundit and consultant, has written an essay about "Names: Memorable, Global, Non-political: Choose Two". From reading it briefly, it appears to be a translation of my Names: Decentralized, Secure, Human-Memorizable: Choose Two into terms that are more relevant to today's policy makers, plus Clay's own (good) point that "names are inevitable". (Which is to say: we can't just pick "decentralized" and "secure" and ask everyone to use meaningless names.)
Hm. This makes me realize that my essay should really talk about "meaningful" instead of about "human-memorizable".
I'm glad that the idea is getting wider exposure! Clay's version is probably more accessible than mine, and Clay's audience is probably more powerful.
He doesn't mention the notion of names which are secure ("non-political") and meaningful and can also be decentralized ("global") by means computer-assisted person-to-person sharing of names. That is: Pet Names. Perhaps I'll have a chance to talk to Clay at the imminent Emerging Technologies Conference and suggest that idea to him.
(Clay: if you read this, check out "distnames.html" again and note the part about how I use amazon.com URLs without memorizing them. That's not the whole story (it doesn't have human-meaningfulness other than the "amazon.com" part), but it is a good demonstration of the fact that computer-assistance is natural and already happens, even with basic tools like text editors and cut-n-paste.)
Today I optimized my Monitor class because my new AppendIO class uses it and because I had realized that Monitor was adding a big and unnecessary latency in the case of lock contention.
Mnet and Mixminion, see links from projects.html for details. Went to the park with Irby -- he looked the slide and the swings, then he just played with the gravel for more than hour. It was great!
It's 7:30 and the sun is a kernel of fire wreathed in a sphere of fog.
I feel a bit stressed about what to do with today's morning-of-work. I want to integrate the new BlockWrangler into Mnet so that actual users can use it. I want to review a paper about anonymous remailer design by the MixMinion team. I have a lot of e-mail to read and a teleconference. It's almost time to pack for ETCON2002! Panic!
later.... Hm. I didn't get much done today, it seems. I did manage to commit a new utility class to my pyutil library: AppendIO. The weather report says it is going to nice and warm today, so hopefully Irby and I can go to a park and play.
It's 13:00 ADT and I'm just now posting today's log entry. Sunah correctly guessed that what I am doing with my day is hacking. We had a network outage in the morning. Thanks to aggle for running a backup server that collected the bouncing mail and stored it until our network came back up.
I committed some bugfixes and improvements to the BlockWrangler code, and more to the unit tests. Turning up the number of bogus peers on the simulated network from 500 to 32,000 reveals that the "Greedy, Simple, Randomized" strategy is not scalable -- not even a little bit.
I hope tomorrow to work on integrating BlockWrangler into the actual app so that people can use it on the real live network. (The current network has fewer than 500 real nodes, so GSR will work fine for now.)
The rest of today is Irby and family time!
One of the unit tests fails, not due to an error, but due to the block wrangler taking too long to perform its computations. So right now I'm using the Python profiler to identify which computations are the culprits.
... later: I indentified the culprit, sped it up, added unit tests to find memory leaks, found a memory leak in the accelerated C code, fixed it, and committed. Now off to play with Irby while the weather is warm!
I have unit tests which simulate an entire Mnet that behaves strangely. Now I'm using these tests to identify bugs in the BlockWrangler and the Greedy, Simple, Randomized block wrangling strategy.
Whoo-HOO! BlockWrangler passes the randomized unit test!
I've been checking my web server stats every day or so. I typically get 50 visitors a day to `http://zooko.com/log', 15 of whom have never visited before (from that IP address). I think this means there are at least 35 people who visit regularly! Wow.
We went to Irby's mama's daddy's house last night for one of their frequent parties. My brother-in-law Luke the Clown built a huge bonfire which was truly spectacular -- it was topped with wet green pines which gave rise to a giant column of steam, lit from within by fire. He also juggled burning branches. His daughter Witbee was eager to roast marshmallows. She doesn't eat sugar, so she just threw them all into the fire, one at a time, and admired the way they puffed up and burned. I managed to snag one of them with my repurposed coat hanger just in time -- it was perfectly crispy and it had been resting on thin branches instead of down on the burning coals so it had only the merest hint of ash flavor.
The Kademlia home page has been updated to suggest that a release will be available next week.
Dan Moniz says I should stop with the one-year-ago-today ("OYAT") bits, since he's already seen them. Perhaps OYAT is just for Amber and me: a nostalgia trip. OYAT we were moving into this apartment, "Irby's Aerie", from the previous one "Waystation". This one is named the Aerie because it is high up -- 15th floor, with many large windows looking east over the harbor. That one was named Waystation because it was originally intended to be used only for those occasions when I would visit Halifax.
Actually, I do read my e-mail each morning, when I come to the Aerie to work.
It's snowing outside. I'm still almost done with the download logic. That is: there are no more spots in the code with comments that say "XXX come back and implement this part", but it isn't really done until it has complete unit tests, and is integrated with Mnet, and works on the live network. Until then, it's entirely possible I'll discover more work that needs to be done before it can possibly work.
You can see the current versions of BlockWrangler, ChunkObj, and FileObj on the web.
Actually the most interesting starting point is the "Greedy, Simple, Randomized" strategy implemented in BlockWranglingStrategy. That shows one possible strategy for when to start searching for blocks, which blocks to download first, etc. I haven't spent much time thinking of more clever strategies yet, so there might be significant improvements to be made by a new strategy.
One year ago today Amber got a paper accepted into the Journal of Theoretical Computer Science.
We're going to housesit for Irby's Grandmaxo -- no e-mail.
Hackety hack hack hack. Yesterday I decided to rewrite the download logic in Mnet. I'm almost done already! I think.
One year ago today was the (day after the) first time I mentioned the Self-certifying File System.
SFS was invented by David Mazières. Coincidentally it was yesterday, one year later, that I stole an idea from another of his inventions: the new download logic in Mnet uses XOR for its distance metric as suggested in David's Kademlia design. (Note: Mnet doesn't do routing -- this is only for server selection -- so it doesn't (yet) gain all the benefits that the XOR metric offers.)
David Mazières wrote to tell me that Kademlia was actually Petar Maymounkov's idea. (David gets credit for it only because he's Petar's professor.)
Yikes. Visits to the Mnet page continue to increase.
Mnet hacking -- I hope to have a nice stable release of Mnet ready before I go to the O'Reilly Emerging Technologies Conference.
The air outside my window is thickly filled with large snowflakes. Irby is watching Sesame Street. I think he has scarlet fever, but he seems healthy and happy so I'm not worrying. He is going to visit his grandmaxo later today. Amber got an "A+" in her cryptography course.
Niels Ferguson is a friend of mine who used to work for Bruce Schneier. He wrote a response to Bruce Schneier: educator and/or salesman. Niels provided reasonable answers to the five questions when applied to Counterpane's monitoring service.
Looking back, I see that I didn't express myself very clearly in that rant. It isn't that I think Counterpane's monitoring service would fail the Five Question Test. It is that the "advertisement" for Counterpane's monitoring service didn't talk about any of the Five Questions, but instead offered some impressive-sounding numbers that were meaningless without context. It is the tone of the advertisement that struck me as so different from the tone of the educational section. (Be sure to read the original article -- there are lots of interesting things in the newsletter even aside from analyzing Bruce Schneier's marketing techniques.)
I do also harbor philosophical doubts about outsourcing network security. The thing that I love is how technology can actually reduce your total vulnerability. Outsourcing network security reduces your vulnerability to script kiddies but it increases your vulnerability to people who can subvert Counterpane's systems (including Counterpane's employees). It is not a Pareto win with regard to vulnerability. From a larger perspective, outsourcing security leads to centralization of vulnerability -- inasmuch as Counterpane is successful and attracts new clients, it constitutes a denser and denser centerpoint of vulnerability whose failure would affect many other organizations.
But after reading Niels's defense of Counterpane's services, I think that it is a reasonable choice for businesses who have significant reason to fear script kiddie vandals but little reason to fear an enemy who could attack them through Counterpane itself.
In any case, I'm sorry that I mixed my doubts about the service itself with my criticism of Bruce Schneier's style of advertising for it.
The number of hits to the Mnet sourceforge project is climbing -- it is now up to 160 hits per day from 40 hits per day a week ago. I haven't received any e-mail about it, so I don't know who these people are or what they are thinking. (Grumble: sometimes when you load that link it gives you a nice graph of the last 30 days. Sometimes it gives you an ugly table of numbers. I wanted you to see the graph! Oh well.)
one year ago today I wrote an essay for the e-lang list about ordering guarantees in the E comms abstraction. MarkM replied praising it, which made me very proud. It's a little distressing to see that a whole year has passed -- I never sent the related essay that I mention at the end of this one, and I haven't even begun implementing my ideas into an alternate comms abstraction for E.
Irby has discovered how to frown. He does it when he is displeased with something, for example if he's been told not to do something and he really wants to do it. He stands still with his hands down at his sides, faces the target directly, tilts his head forward and wrinkles his little eyebrows. It is adorable and hilarious, which is probably not the effect he intends.
David McCusker has posted another letter that I wrote to him, along with his replies. This one is about science fiction.
Amber is done with her final finals! Whoo-hoo!
Here is a fresh-from-the-camera, high-quality Irby movie: dscf0005-2002-04-23_16-31_Irby likes getting squirted with water some more.avi.
one year ago today was the genesis of my now distname.html paper. Err.. actually one year and one day ago. I was absolutely wrong that self-authenticating objects were the same thing as an immutable namespace. Somebody should have called me on that one! In truth the two are orthogonal axes: mutability vs. immutability, and self-authenticating vs. non-self-authenticating. The only big implication that I can see right now is that if you don't have self-authentication, then you can't enforce immutability. But my in-laws are here to party so I'm gonna post this and move away from the keyboard!
Amber has been studying for her finals, which are tomorrow. So aside from a bit of E hackery (see projects.html for a link to some details) I haven't gotten much "work" done recently. I've played Magic cards with myself and taken Irby outside to enjoy the outdoors. The weather toys with us -- hovering just above the level where it is okay to take Irby out -- threatening to drop into winter at any time. Indeed I am told that tomorrow there will be a combination of snow and rain.
Thanks to aggle for helping me with my mail server configuration issues. Everything is working now (although another power fluctuation at Irby's Aerie would disable incoming e-mail), and I hope anyone who got a bounce will resend.
Hm. aggle's web log says "Mailed Zooko about his backup MXes being broken.", but I didn't receive any such e-mail. Damn. This explains why I haven't gotten very much any e-mail in the last few days.
Today's log entry grew so big it had to be broken into subsections.
In Bruce Schneier's latest Crypto-Gram newsletter, he begins by educating the reader on how to evaluate security proposals in five easy steps: ask "What problem does it solve?", "How well does it solve the problem?", "What new problems does it add?", "What are the economic and social costs?", "Given the above, is it worth the costs?". He then rightfully censures the airline industry and its regulatory overseers for rushing to implement every security measure they could think of, when all but two of them would fail this simple test.
Towards the end of newsletter, he reports on some publicity that his company, Counterpane has generated, including big articles in Business Week and Internet Week: "30 billion network events monitored and analyzed. 57,000 potential intrusions researched. 10,000 intrusions detected and prevented. Attack success rate while Counterpane was watching: 0.006%. Attacks that succeeded long enough to cause damage while Counterpane was watching: 0.00%.".
It's such a change in tone from his lead article! What new problems does Counterpane's "managed network security" add? (A rotten employee of Counterpane, Inc., can probably take advantage of your network.) What are the economic and social costs? (Centralization of vulnerability from multiple formerly independent companies into Counterpane headquarters.) I hope people who are both customers of Counterpane and readers of Crypto-Gram ask Mr. Schneier the five questions. The numbers quoted above suggest thousands of hackers crouched over their keyboards. More likely the 10,000 intrusions were the result of a hundred automated bots trying a hundred cracks each. And if 10,000 were detected and prevented, but the attack success rate was 0.006%, then doesn't that imply that there was 0.6 of a successful attack?
I'm not going to read the magazine articles to try to deduce answers to these questions -- I just wanted to comment on the contrast between Bruce Schneier the advocate of rational public policy and Bruce Schneier the salesman.
I've realized that turning off my mail server last night to protect it from lightning-caused power disruptions caused my mail to bounce. (I thought it would fall over to the back-up server instead.) If any mail to <zooko@zooko.com> has bounced, please re-send.
I sent a long e-mail to David McCusker about programming language syntax. He put the whole letter on his web log and replied to it! And said lots of nice things about me! That feels good. It's interesting how web logs are creeping into the realm of e-mail now. Raph Levien was just observing the same thing, on his web log, with which he is conversing with David McCusker.
The bit that I like best is this:
I'd probably use your language if it's small enough for you to grasp.
I prefer languages completely understood by one person without effort..
movie: 2002-04-19: Irby wears Mama's shoes on his arms then touches the camera.
Greg Egan is so good that I get my hopes up and become hypercritical. He can almost pull it off even when I'm in hypercritical mode, which shows his singular greatness. Compare to Iain M. Banks who is good enough to get me interested and then not good enough (or perhaps not serious enough) to keep me from tearing it all up with my hypercritical mental teeth and spitting it out. Compare to Vernor Vinge whose disarmingly amiable amateurishness keeps me from becoming hypercritical in the first place. Therefore Vinge continues to reign supreme in my little internal science fiction pantheon, although Egan's tendency to invent entire universes (starting from dimensional topology, proceeding up through sub-particle and particle physics, chemistry and biology, astrophysics and planetary evolution) can hardly be outdone as a jaw-dropping display of intellectual prowess.
DON'T READ THE REST OF THIS LOG ENTRY TIL AFTER YOU'VE READ Diaspora by Greg Egan.
I stayed up an hour after bed-time to finish Diaspora. I felt sad and also slightly cheated at the end. It seemed like the protagonist had drifted out of my frame of reference and I could no longer emphathize with ver. That, or ve had decided to stop sending ver thoughts to me. Perhaps that was the point, but I'm still not sure I liked it.
By the way, when Irby says "CAH!" it has a vowel sound more like "rat" than "baa". It's hard to spell arbitrary language sounds. We occasionally subject Irby to foreign languages hoping to prevent him from forgetting all of the phonemes that we don't use.
I'm no longer sick, and as usual I'm happily stressed about balancing my various time committments. I have the following things on my agenda (not showing daily Irbysitting and household chores) (semi-ordered...):
Amber won a Canadian national scholarship (NSERC) and now Canadian universities are sending her letters offering her money if she'll join their grad school. How gratifying! It isn't that they get the scholarship money, either -- we get the money. Also, many of them are offering her more than that amount to join them. She's already in love with the Toronto school, and she's already committed to them, and they're already offering us as much money as their competitors, so these offers are nothing but ego strokes.
Ugh. We went to visit grandmaxo's house and came down with a horrible flu-like infection. Thankfully Irby suffered only mildly compared to Amber and me. (I think that Amber provides antibodies in her breast milk which help him resist infection.)
It's interesting to look at my log entries from one year ago. I was sick a year ago. Irby had made his first "language sound". It wasn't until about one year later, i.e. now, that he started making sounds consistently with meanings. Nowadays he has two consistent words: "KAH" ("cat") and "NANA" ("mama"). I was excited about the Chord paper. I was trying to locate the persistent memory leaks in Mojo Nation. (One year later I still haven't found them and have largely given up. I figure that as people rewrite the Mnet code and pull pieces of it out and stick them into their own applications, the memory leak will eventually come to light.)
A year ago I wrote in my log that the lookup problem is really the core problem that distinguishes emergent networks from other kinds of distributed apps. Right now I don't understand what I meant, but maybe that's because I'm still sick.
Irby likes little metal cars. He pushes them around on surfaces, and if his daddy says "Vrum VRUM!" he gets a big smile. He also like crayons. He absolutely adores grandmaxo's cats, but they walk away whenever they realize that he is within touching distance, so he ends up following them all over the house pointing and saying "CAH! CAH!".
It's finally warm enough here that Irby can go outside. Amber took him out onto the football field next to our building and he ran around happily. Yesterday his Grandmaxo took him to her yard and let him slide down a plastic slide. She reported that he laughed with anticipation before sliding down.
I got a lot done yesterday, but not the E<->Java project because I went out to have relax and have fun instead. Today I'm hacking on that project again, and making progress. It's exciting to actually write code in a new programming language.
Wow! It's only 10:00 and already I've committed updates to three different projects. Next I'll start on a new, but hopefully short-term, project: integrating some E objects with some Java classes.
We've decided to move to Toronto. Boy is Amber excited about the research program, and boy are they excited about her.
When you sign up for a license to use the Google Web APIs you are entitled to one thousand automated queries per day.
You know, Google is really getting to be the biggest enemy of decentralization. It's a shame because they're so clueful, well-intentioned, respectful of the customer, and so forth. It was easier to oppose centralization of the web back when the most successful centralizers were ugly, incompetent power-grabbers like Yahoo, Netscape, Sun, and Microsoft. But that is why Google is the biggest threat -- because they are so good.
So today's project involves writing Java. It's funny how quickly I forget basic syntax like "what does a constructor look like". I don't forget things about the type system, the object model, the standard library, the reflection API, and so forth, just syntax. Maybe I'm syntax-impaired.
Seen on Wes Felter's world-famous, highly influential weblog:
<AccordionGuy> So Toronto gets another P2P programmer. <wmf> MojoBooty? Peek-a-Nation? <dnm> Peek-a-MojoBooty-Nation <dnm> Toronto has a number. <dnm> They're just not really employed. <jillzilla> employment is so passe for p2p programmers. <AccordionGuy> I'm "post-employed"
Note: we haven't yet decided to move to Toronto. I think tomorrow we'll decide to move to Toronto.
Capsule reviews of Sesame Street characters
Thanks a lot to Taral for reformatting my IPTPS02 paper into Springer-Verlag format.
I've made public a new version of my IPTPS02 paper. Here's a letter explaining what changed from the first version.
I saw that hundreds of new people had viewed my web log while we were in Toronto, so I had a look at my referrers list (it tells me what page people were viewing when they clicked on a link that took them to my page). I didn't see any grand pattern, but one particular link caught my eye:
http://www.google.de/search?hl=de&q=what+happened+to+mojonation++&meta=
Mojo Nation was big in Germany. It's nice to know that someone there misses it.
My Names: Decentralized, Secure, Human-Memorizable: Choose Two essay seems to be reaching a wider and wider audience. I've received several complimentary and thoughtful e-mails from people who said that they learned something valuable from it. This makes me proud, but at the same time it is humbling! MarkM has inserted a graphic representation into his Pet Names paper along with space for a new section about the "choose two" trade-off.
We just returned from a visit to Toronto. (I guess I forgot to warn about the scheduled log.html outage.) It was great! The University of Toronto is a top-tier institution, and they are doing the kind of research that Amber loves, and they very much want her to join them (including the economist's proof of utility: they are offering her money to join them). The city was not too bad. I have misgivings about raising Irby (for the next five or so years) in a city instead of an idyllic mountain cabin, but Toronto seems nice.
I also imagine hanging around the University of Toronto and listening in on their multitudinous seminars, speeches, and colloquia. I asked one of the computer science grad students if they had any cryptographers, and he named Rackoff, co-inventor of zero-knowledge proofs. A short while later a man came into the lounge where we sat and had his lunch. When he left, the grad student informed me that it was Rackoff. He asked if I'd like to be introduced and I expostulated "No way!". Not until I've studied for a couple of more years and understand zero-knowledge proofs.
Irby did very well. He charmed everyone we met, as usual. We stayed one night with Amber's relatives where Irby got to play with two patient dogs. He loved it.
Cool! I just found Kademlia's home page. They say that they are going to release a file-sharing app based on Kademlia's routing algorithm and eDonkey2000's download algorithm. The former is an elegant and promising DHT/emergent routing design and the latter is a mature distributed downloading system (although I wasn't previously aware that it was publically documented or reusable...).
I'm very proud of a crypto trick I proposed on the mixminion-dev list. It has yet to be reviewed by any Real Cryptographers, though. "MixMinion" is an anonymous remailer project.
Okay, e-mail to <zooko@zooko.com> is working again.
My mom has been clamoring for more Irby stories, so here is a movie. It's entitled "dscf0366-2002-03-31_09-00_Irby chews on a motorcycle, crawls under a chair, obeys instructions, and begins the sitting-down I'm-a-good-boy dance.avi".
I'm trying to set up a mail server so that I can receive incoming mail again.
Mail to <zooko@zooko.com> has been out for a few days. Does anybody know the IP address for the computer formerly known as mail.mad-scientist.com? Maybe Amber and I will allocate some of our family time to setting up our new server!
I'm a bit dejected because it is the last possible minute to submit a position paper to the first international workshop on the economics of security and I still haven't gotten very much written down. One problem is that I couldn't decide which of several positions I might take would be most relevant to the organizers of the conference. (The intersection of economics and information security is really a very broad topic.) But a bigger problem is that I haven't forced myself to stop dithering and start writing.)
Irby's comprehension is really amazing. I just announced to Amber that I was going to take a shower and Irby got up from his toys and came running to the bathroom (he loves showers and baths). I didn't really want to take a shower with him, so I spent some time distracting him. Next time, I'm going to have to spell "S-H-O-W-E-R" or use sign language behind his back or something. (What's the sign for "shower"?)
I woke up at 6:30 this morning and enjoyed the sunrise over the harbor through my large east-looking 15th-story windows.
Ahem. Sorry about the dead air here. I've been doing little -- mostly Irbysitting and playing Magic: the Gathering with myself.
MarkM called me excitedly a few nights ago to announce that he had been thinking about "Names: Secure, Distributed, Human-Memorizable: Choose Two", and he had realized that his Pet Names design uses all three workable combinations! The public keys are secure and distributed but not human-memorizable, the pet names are secure and human-memorizable but not distributed, and the "suggested nicknames" are distributed and human-memorizable but not secure!
I discovered a security bug in syncmail which can give people login access to computers when they are supposed to have only "restricted shell" access. In fact, I tested it by running commands on cvs.sourceforge.net which I wasn't supposed to be able to do. I reported it and submitted a patch, but have received no response yet. It's a problem, to try to inform users of syncmail so they can patch their systems without simultaneously informing script kiddies who could use the information to exploit them. I suppose I should write some kind of report for a security-bug-tracking list.
Hello, world.
Sunah wrote in to point out that Irby experienced grass at her house just a few weeks ago. That's true! I'd forgotten how it is already summer in the southern latitudes. Yesterday the sky was obscured by blowing snow. The high temperature was 2 degrees celsius.
Some fresh discussion is happening on the p2p-hackers mailing list. The part that I'm most interested in at the moment is about taxonomies of emergent file-thingie networks.
A Concerned Citizen wrote in to object that Irby did too experience grass, last Fall. I wasn't counting that because I think he was too young to remember it, so grass was still novel to him last week.
I'm buying tickets and trying to arrange lodgings for the O'Reilly Emerging Technologies Conference and hopefully also the Workshop on Economics and Information Security. If you want to split a hotel room at the O'Reilly conference, e-mail me.
Whoo-hoo! I've discovered another conference I could submit a paper to: the 5th international workshop on information hiding. See conferences.html.
It's too bad that it is snowy, blustery and cold outside. I enjoy the beauty of snow seen through large, thick-paned windows, but I wish to take Irby outside to play. There was a slightly-less-cold day last week when I took him outside and let him play on some grass and he was so excited! It's amazing to think that he'd never experienced grass before that.
In another year or two he'll be big enough to play outside in the snow during the winter.
Hm. I just discovered via amazon.com that there are at least six books for sale about "peer to peer".
Today's pick from the IPTPS02 workshop is "Observations on the Dynamic Evolution of Peer-to-peer Networks" by David Liben-Nowell, Hari Balakrishnan and David Karger. David Liben-Nowell said something that I had been known to mutter to my p2p-hacker pals: that a design that says "Here is the good state of the network, and if it should ever fall out of the good state, here is an algorithm to get it back into the good state pretty quickly." is fundamentally disconnected from reality. Instead, David proposes, we should develop a model that says "Here is a realistic, almost-good state. We analyze how it behaves while it is in this almost-good state, and we analyze under what conditions it can guarantee to stay in the almost-good state instead of falling into a truly bad state.".
This is yet another thing that I learned at the Workshop that is very exciting to me: that the "here's the perfect state" kind of analysis that comes with the original Chord paper is only a first step for the Chord scientists, and that a more realistic analysis is forthcoming. In fact, glancing at the the Chord technical report, it looks like they have already written down some of their new work in that direction!
I've edited distnames.html in response to a good suggestion from Aaron Swartz. I'm starting to become satisfied with this essay. Soon I will mark it as "version 1.0" and then hopefully I can stop tweaking it.
Inspired by the "open questions" paper that I referenced yesterday, I've been trying to figure out if these five different systems could be implemented as a single network that satisfies all of their various needs:
Today's choice of paper from the IPTPS02 workshop is "Routing Algorithms for DHTs: Some Open Questions", which was presented by Scott Shenker. I recommend that you read this if you are either a designer or user of any "Distributed Hash Table". During the talk, he alluded to a new design named "Viceroy" which apparently requires only O(1) links per node (although it "needs help" to maintain this state) while still providing only O(log(N)) hops per message. This seems incredible to me, unless the "needs help" part refers to the intervention of a globally knowledgable and authoritative centerpoint.
Irby is happily pushing all the buttons on a "singing, beeping, educating" birthday present toy.
Another thing from the IPTPS02 workshop: two different groups [1, 2] presented ideas about using peer-to-peer techniques to dynamically redistribute load during "flash crowds". I gave each of them the keyword "BitTorrent" and I intend to give Bram the BitTorrent Hacker a reference to them.
Yesterday Irby had a birthday party. It was pretty chaotic. He was a bit unsure about being the center of attention so much, but I think he was pleased. When I wasn't standing guard over Irby I tended to try to escape from the noise, and after the party was over I regretted that I didn't take the opportunity to visit with my relatives. Those opportunities will become much rarer soon. I did play Magic Cards with Witbee's grown-up cousin Julia, and Julia told me about her international science fair project about apple farming techniques. Julia is a bit reserved normally (perhaps she, too, was trying to avoid the chaos), but she smiled with genuine pleasure when I asked about her science fair project. It's nice to see a young person whose passion is science.
I'm reading a collection of science fiction short stories (Year's Best Sf 6 edited by David G. Hartwell) that I bought up at the airport on the way back from the conference. I've read the first five stories, and I rate them: "sort of weak", "excellent" ("Reality Check" by David Brin, originally published in Nature), "very weak", "ok", and "very weak". I was surprised to find so much crummy stuff in a "year's best" anthology. I guess über-editor Gardner Dozois has spoiled me. (Addendum: the rest of the anthology went the same way -- there were only a couple of stories worth reading.)
The electronic proceedings from the International Workshop on Peer-to-Peer Systems is up. You can find more than thirty "position papers", many of which are deeply interesting to me. I'll post comments about some of them in this log.
"Dynamically Fault-Tolerant Content Addressable Networks" by Jared Saia, Amos Fiat, Steve Gribble, Anna Karlin and Stefan Saroiu is the paper formerly known as "Censorship Resistant Peer-to-Peer Content Addressable Networks". (I have linked to it before in this log.) I am eager for the theoreticians to go down the path that this paper explores: analyzing the network's robustness in the context of an extremely powerful and knowledgeable adversary instead of in the context of random, unrelated failures. Unfortunately the network proposed in this paper has some limitations which make it less interesting for actually deployment: it requires some expensive (message broadcast) set-up to configure the initial network, and then once configured it can never grow to store more data than it was initially configured to store.
I expect that the authors of this paper and other scientists will make good progress in the next few months, and we will have a design that it is provably attack resistant while also flexible and practical. Then we can start on the really hard problem: implementing and deploying the thing so that it works, has lots of users, and still retains its theoretically-proven security. ;-)
Unfortunately some of the fundamental ground that the theoreticians are building on is little-understood. My choice for the most important theoretical paper at the Workshop is The Sybil Attack by John R. Douceur, even though I completely disagree with its premise and conclusions. I'm really glad that someone has actually addressed this fundamental and widely misunderstood issue.
Sunah suggested a clever gimmick for my log. Here it is:
I'm working through e-mail. There is a TON of e-mail in my "pending" box which is interesting, timely, and needs a response.
Updated today: projects, Mnet status, conferences.
One interesting thing that happened at the P2P Workshop was that I said during my presentation that I knew some hackers who were implementing Chord and once they got it working I intended to plug it into Mnet. This received lots of response. The authors of Chord wanted to know why my friends didn't just use their implementation, which is available under a free software license (written in C++) (I didn't know). The authors of Pastry suggested that I use Pastry instead of or in addition to Chord. They will have free software source code available RSN (written in Java). Some other random scientists wanted to know why I thought Chord would be any more robust than Mnet already is in the face of the massive churn that I had described. (I didn't know.)
Here's a recent picture of Irby (from last night). I retouched it with the GNU Image Manipulation Program to make his pupils black. (Click on it for the extremely large and unretouched version.)
Many people are asking me to tell me what happened at the P2P Workshop, and I have a difficult time deciding where to start. The broadest things I can say are that there is much active research on the topic, with a slight majority (at least at this workshop) of "systems people" rather than crypto people. Based on the ideas I heard about, both those formally presented and those bandied about in off-line conversation, I look forward to many exciting papers within the next six months.
In the coming days I'll try to write some comments in this log about specific things I learned from the workshop.
I got to meet a personal hero of mine, Ross Anderson. Here is a recent interview he gave to an info security magazine. In the interview, he says a bundle of typically incisive and commonsensical things, like:
Unfortunately, what may happen more and more is that unnecessary security technologies will be deployed as a smoke screen. The government will say, "We have given you a medical smart card, and the smart card is tamperproof; therefore, your health records are secure." And meanwhile, of course, the government copies of your health records will be sold to drug companies, and you'll have no say in the matter whatsoever. And if you protest, you'll simply be told that you have a smart card, and therefore your medical records are secure. This is the kind of thing that goes on, and, to a very large extent, "security" is used in such applications as a means of bamboozling the customer.
I have ninety-six messages sitting unread in my inbox. (Most of these are copies from mailing lists rather than messages written to me specifically.) The oldest one is from 2002-02-12. I have seven hundred and fifty-four messages sitting in my "pending" mailbox -- messages that I've already looked at but that I want to read more closely and/or answer. The oldest one is from 2000-07-27 and is about the first limited public release of Mojo Nation (revealed to selected people only) and about hushmail and digital cash.
Thanks, mom, for calling the hotel room and leaving a nice message!
The conference was immensely stimulating. I hope to write more about it. Right now it is bed time at the Aerie!
I'm in Cambridge, MA, USA today until the 10th (Sunday). I'm plenty busy on the 7th and 8th, but I may well find myself wandering around town with nothing to do on the 9th. The only way to contact me is to call the hotel at +1 (617) 494-6600 and ask for a guest named "Mr. Wilcox-O'Hearn".
Today is Packing Day!
I was on the phone with my brother Josh and he asked if Irby could say "Mama" or "Daddy" yet. I was holding Irby in my arms, so I held the phone out to him and said "Irby: can you say `DA-DA'?". He said "MWAH!" or something like that. I said "That was a good try. Can you say `DA-DA'?". He said "DA-DA!".
Josh wanted to know how long he had been doing that. I answered that this was the first time!
After the phone call I asked Irby to say "DA-DA" again. He looked away, then gave a big smile and said "DA-DA!". So that's twice, but I haven't gotten him to say it a third time yet. I'm a bit sad about going away for four days tomorrow.
Homework:
Sorry about the radio silence here. I've been spending time with family and playing with Magic cards and and hacking on a program for playing with (virtual) Magic cards.
Things for me to think about before the imminent P2P Workshop (in no particular order):
ObIrbyAnecdote: Amber says that during their walk in the park today, Irby looked at a dog and said a word that sounded like "DOG".
Rabbits! Rabbits!
Today I mostly relaxed and played with Irby. I wrote a Python program to manipulate "Magic: the Gathering" card data. This evening Amber and I have a date!
Irby's eyes, like his mama's, reflect red in camera flash. Here is a big (1.6 MB) jpeg image of Irby laughing for the camera.
I've updated distnames.html. I feel better about it now, but I am eager to get feedback, especially from people who have never read any previous version.
I've updated projects.html (it now includes a link to Mnet status page, as I did a little work for Mnet today (at Brandon's instigation)).
I want to make a list of all the different projects that I might want to work on, and ...
Okay, here it is: projects.html.
Here is a paper that I find myself recommending frequently: Wide-area cooperative storage with CFS (2001) by Frank Dabek, M. Frans Kaashoek, David Karger, Robert Morris, Ion Stoica. (That is, by the way, the Robert Morris, Jr., famous for taking down the Internet in 1988...)
Thanks to JDarcy for telling me about the broken link in the previous line.
This summary of the Python 10 Conference is good. Andrew König (a leader of the C++ development and standardization effort) says that C++ standardization should have been delayed two years. Tim Berners-Lee (a guy who invented the World Wide Web) learned Python on an airplane flight "all on one battery". This is another confirmation of my conviction that Python is the easiest language to learn. I think it takes approximately four hours for an experienced programmer to become conversational in Python. This is one of the reasons that I believe Python is destined to become one of the Big Languages of the decade.
(Thanks to Taral for reminding me that Andrew König's name is spelled with an ö.)
If you are one of the people who is waiting to talk to me on the phone, be advised that I'm taking care of Irby all afternoon today but tomorrow I'll be available in the afternoon. (I am in the Atlantic Standard Timezone, which is four hours ahead of PST and one hour ahead of EST!)
Today's I'm spending lots of time with Irby. I'll try to elicit an I'm-a-good-boy dance from him in order to satisfy the clamoring masses. During non-Irby time, I'm planning to force myself to leave Mnet alone (despite the desire I feel to work on it in response to other people using it, working on it, and talking about it) and study for the imminent Peer-to-Peer Workshop.
Here are the papers that I'm reading for my workshop homework:
I also need to talk to several friends on the phone, but that always seems to be problematic. They are asleep when I'm having morning work time, and then in the afternoon I'm often Irbysitting, which is pretty incompatible with phone calls.
Hm. I wonder if there will be a lot of people flying from San Diego to Ottawa on June 26. (And doing last minute proposals on March 1st...) conferences.html. Oh whoops! June != July.
Update: Irby is in the mood to play by himself with his toys, so I can do some reading while watching over him. (Ross Anderson's "Security Engineering")
For any programming language fans out there, this personal story by Erann Gat seems important: How I Lost My Faith.
Today I'm working on Mnet (please see the mnet-devel mailing list archives if you are interested).
James Levine has posted some pics from CodeCon, including a couple of pictures of Irby!
The beginning of the chapter on "Access Control" in Ross Anderson's book contains two quotations:
"Going all the way back to early time-sharing systems, we systems people regarded the users, and any code they wrote, as the mortal enemies of us and of each other. We were like the police force in a violent slum." --Roger Needham
"Microsoft could have incorporated effective security measures as standard, but good sense prevailed. Security systems have a nasty habit of backfiring, and there is no doubt they would cause enormous problems." --Rick Maybury
On 2002-02-11 we posted a video of us trying to get Irby to walk and him falling down. Now ten days later, here is a video of Irby walking.
I'm trying to "catch up with the Net" after being absent from it for a week. Here are a few comments or stories written by other people about CodeCon, some of which mention me:
The trip to the Bay Area was intense and exhausting. I've made a new rule that I can't visit the Bay Area unless I can stay for at least ten consecutive days. There was just too much to do, and some very important things (like visiting certain old friends) didn't get done at all.
There are a lot of people in the Bay Area who showed that they really care about us and helped us in our adventure. It felt good to have people going out of their way to take care of us and visit with us.
Irby can now toddle about as much as he pleases, and he's good at toppling gracefully.
I am pleasantly surprised at the amount of activity around Mnet. Several independent hackers have started submitting patches and ideas.
Amber has been accepted to the PhD program at the University of Colorado at Boulder. She is really thrilled. We're still waiting to hear back from the other seven schools to which she has applied.
I'm excited about my work. The next thing to do is prepare for the First International Workshop on Peer-to-Peer Systems. I started writing an essay with MarkM at his beautiful house in the La Honda mountains.
The demo didn't go very well, as I didn't prepare a machine to run it on, and spent the entire session trying to compile with one hand and give a talk at the same time. :-P The Q&A session showed that people understood the concepts and were interested in the technical details, so that was good. Now I'm visiting with all the people that I know here.
Demo imminent! See CodeCon.org for details! Watch the live DNA lounge web cam! Contribute a node to make the demo work by installing Mnet yourself!
Whoops! Missed a day! To make up for it, here is a video of Irby's parents trying to trick him into walking. (The consequence is that he falls down a lot...)
It's a cold rainy day. Mist hovers over the harbor.
Notes to self:
Whoo-hoo! I'm famous! Wes Felter mentioned me on his widely read, highly influential web log!
My résumé says that one of my skills is "choosing the right tool for the job". I just realized that during the past few weeks I've used the following programming languages: C for implementing a fast version of a new algorithm. Python for Mnet. C++ for eimp (a preliminary barely-begun interpreter for E). E for experimenting with E. Jython and the Java standard libraries for Ogres vs. Pixies.
I haven't found the right job (or the time) to use some other interesting programming languages: ocaml, Haskell, Squeak Smalltalk.
Actually, I did use Haskell for Dr. Elizabeth Bradley's Numerical Computation course in university, and it proved to be a brilliant tool for that job, allowing me to spew out complete solutions in one line while my C-using buddies were laboring over more complicated implementations.
To recap:
More snow and fog this morning. The car that got snowed under during the blizzard is still out there under a snowbank in the parking lot across the street.
I tidied up the licence_quick_ref.html file a bit and bumped the version number to v1.0. I know that there are still major issues with the Quick Ref such as omitted licenses (rows) and omitted features (columns). I would appreciate e-mail about any such major issues, and I would also appreciate e-mail about any remaining minor issues!
Thanks to various friends and family who e-mailed or called to find out if I was okay. It feels good to know that you will be missed if you drop off the Net for a week without warning.
Ahem. Hello world. There was a blizzard. Amber and Irby were sick. I was sick. My DNS queries were timing out. I was depressed. The new 2.4.17 kernel for naomi wouldn't drive the old RTL 8029 NIC. I just now checked mail for the first time since 01-28 or so, and I have 187 unread messages.
On the other hand, baby Irby was delightful as always. I finished a Python implementation of `libbase32', and then a fast C implementation (featuring a Duff's device!). I implemented part of `libzstr' so that `libbase32' could use it, and part of `libzutil' so that `libzstr' could use it. I studied the x86 asm output that GCC 3 would generate from various C idioms. I started read True Names to Amber. I played "Stratego Legends" with Luke. I improved (?) Ogres vs. Pixies, then made a new game called Cellular Automata vs. Cellular Automata, then made a new game called, um, Blue Army vs. Red Army and then a new one called Sorceror vs. Sorceror that takes place in a dungeon labyrinth.
Okay, so now I need to pick out the really urgent ones from my 187 uread messages, and figure out where the heck Amber and Irby and I are going to stay before and after CodeCon.
Here's a picture of the Wizard from Sorceror vs. Sorceror, thanks to the other Brothers Wilcox for art and concept.
network troubles....
Last night during Family Time I looked up to see baby Irby arriving at the couch, having walked from the coffee table. He must have taken two longish steps to reach it. He had both hands outstretched to reach the couch. As soon as he arrived at the couch he promptly went about playing, without looking up at us. He acts like it isn't miraculous at all.
Sorry for the missed day. Nothing much happened yesterday.
Today I used my brand new batteries which are supposed to be good for digital cameras. They are "nickel metal hydride" batteries, rechargeable, and I left them charging overnight (it takes 16 hours). I put them in my camera and made a 50 second movie. The camera then reported that the battery power was critically low. What? Is this the way this is supposed to work? I'm starting to wonder sheepishly if I bought a kind of battery that is really good for still-picture digital cameras but bad for movie digital cameras.
Movies that I want to see before they leave the big screen:
The tops of all the trees and buildings and cars are perfectly smooth blankets of white snow.
Irby took his first steps today. I had been holding him up and I pulled my hand away. He ran a couple of steps toward me and grabbed my hand. He was concentrating pretty hard on my hand, and I don't think he noticed what happened. Later I tried to make him do it again, but he was tired so instead of running and grabbing my hand, he sat down and cried.
We went and got more batteries for the digital camera, and a battery charger.
A new version of Ogres vs. Pixies is now available. Here's the contents of the ChangeLog. It still requires two players at the same keyboard, and there are still no docs.
v0.0.2: * Pixies can pick things up and drop them * Ogres can destroy rocks (rocks have 1 hit point, 2 defense dice) * when an Ogre eats a tree, all adjacent trees immediately die of fright
When we went out today, I saw that it wasn't just the top, but also the southern side of every object was coated with snow. It was interesting to see a smooth coat of snow laying "sideways" on each vertical telephone pole.
Today I started writing a base 32 encoder in Python. (If anyone already has one in Python or as a Python module, please let me know!) Also my little brother e-mailed me the game that we wrote last Xmas so now I have another project I can spend my time on!
Here is v0.0.1 of the computer game version of "Ogres vs. Pixies". It is playable, and requires: a Java 2 runtime, Jython v2.1 and two players sitting at the same keyboard. There are no instructions. Start it by invoking "OvP.py" from the command-line.
Whoo-hoo! Someone sent me a gift: True Names: And the Opening of the Cyberspace Frontier.
So far today I've been working on isolating EGTP from the rest of the Mojo Nation code base, in the hopes of making EGTP useable in other projects. Along the way, I'm also isolating and cleaning up all of the generally useful utility functions and classes that we developed for Mojo Nation and that EGTP relies on. I intend to publish them as a separately useful "pyutil" library.
Bleah. I can't figure out how to write the Swing equivalent of "Hello World" in E -- that is to write some E code that gets invoked whenever the Swing window needs to be repainted. Therefore my project of rewriting Ogres vs. Pixies in E is going very slowly...
incomplete list of Things To Do, in no order:
As a precondition for the first two I'm tweaking E to build and install nicely on my system. I'm submitting patches to MarkM so that future releases of E might fit more naturally into systems based on Linux and/or gnu `stow'.
It is extremely important to me to have friends and family to share child-raising duties with. There are currently four adults and two children who regularly (or at least frequently) spend their time playing with Irby. This means that the adults get extra hours, because one adult can take care of two children as well as one. It is also very good for Irby, I think. He is very excited and he interacts with the other children a lot. It's distressing to imagine moving away from Halifax and losing those shared times.
Irby has grown up so much in the last few weeks! Now he points at things, and demonstrates a great deal of cognizance about the world. He recognizes many words, including words that are spoken in English and are not signed. Just now, for example, I had prepared a bottle and put it on the counter next to the stove. He came into the kitchen, pushing his little stroller and wearing his Upstanding Monkey Helmet. I removed his helmet and asked him in English plus ASL if he wanted a yummy yummy yummy bottle now. He gave a big grin and started pointing toward the other counter (where bottles are normally made and where the kettle was still bubbling) while searching for the bottle with his eyes and his index finger. I said "Here it is!" and pointed to the counter that held the bottle, whereupon he looked there, saw it, and aimed his bottle-locating finger at it.
This was probably more wonderful in real life than in the re-telling, but I'm sure my mom will be glad to hear another Irby vignette.
Irby has started reaching out for a new handhold which is just barely in reach, and then carefully letting go of his previous handhold while grabbing the new one. I expect this to evolve into his first steps soon. Better charge up those camera batteries!
I'm really enjoying working on whichever project interests me at the moment. My current projects ("current" meaning I've worked on them already so far today) include:
We are having quite a party! Three adult relatives and two child relatives are visiting us this evening, for a total of five adults, two children, and a baby (he has not yet toddled without support, so he is still a baby.). Such a pleasure, and such a time-saver, to share childsitting duties!
I like this site called songfight.com, where each week they announce the name of the next song and a bunch of musicians send in songs to compete for the title. MC Frontalot's "Floating Bridge" was pretty good -- a rap chronicling the evolution of bridge building techniques.
Heh heh heh. Dilbert is unemployed. It's good that he's keeping in touch with his audience.
I was looking at Lars Wirzenius's homepage and I liked the way the hyperlinks shadow when your mouse moves over them, so I copied how he did it. What do you think? Also what about these "external links"? Did you prefer the little red arrows, or the green color, or a different color, or would you prefer external and internal links to be identical? Please send your suggestions to <zooko@zooko.com>.
Here's version 0.0.1 of my E interpreter.
I'm having fun writing an interpreter for the E programming language in C++. I will probably never finish it, and the current design isn't the best for either speed or for simplicity, but it is the best for my goal of refreshing my expertise in C++. I haven't done any serious work in C++ since 1997, and it is a pleasure to dive back into it. Programming in C++ feels very different in my head than programming in Python, Java, C, and bash (the languages that I've used most during the last several years). It's hard to describe. It's sort of like the way that I feel different in my head when listening to different kinds of music.
I'm evaluating whether the Evil Geniuses Transport Library would be a good fit for the messaging needs of a client. I'm excited about that tool becoming more widely used, since it can help connect people directly to one another, bypassing the accidental and deliberate roadblocks that are staring to pile up on the Internet.
Last night I held a sleeping baby in my arms for an hour while rocking in the rocking chair and reading a book by candlelight. What a pleasure! Daddy Irby (baby Irby's namesake) used to say "A baby on your shoulder will cure what ails ya'.".
I'm a bit of an optimist, and I enjoy thinking of the current economic winter as an opportunity for unemployed programmers to do other good things. For example, Joey de Villa was recently sacked, so now he has time to party all night and write delightful journal entries about it.
At dinner, Amber showed me a new Irby trick. She said: "Irby, where's mama's watch?", and he looked at the watch on her wrist, and then reached out to touch it. Further experimentation showed that he can touch "mama's nose", "daddy's nose", and "your spoon" when prompted, but he doesn't know where "Irby's nose" or "your nose" is. P.S. Further research has proven that Irby knows what "Irby's keyboard" or "your keyboard" is! This is so exciting! (Yes, he has his own QWERTY computer keyboard, a broken cast-off from one of his parents.)
The east wall of my office is a nice large window, 4 feet tall and 8 feet wide (that's 1.2192 by 2.4384 meters for you Metric System people). Currently it shows a dim and wavery view of the Halifax waterfront through a curtain of rainwater.
Irby stories (my mom loves these):
I have completed some optimizations for the Python interpreter. They are what I call "micro-optimizations", meaning that they are only a constant-time speedup, and in fact a pretty small constant at that. ;-) Of course, it depends on the use case. If every attribute in your Python program were named "__delattA__", "__delattB__", "__delattC__", and so forth, then my patch would make attribute lookups 3.8 times as fast! ;-)
67 different IP addresses fetched pages from http://zooko.com/ in the last 24 hours.
There was white snow blowing across the windows when I woke up, and it continued blowing all morning. Now it is afternoon and patches of blue sky are visible. I can see the shadows of clouds sliding across the city.
Amber was out to class (she's excited about the course in cryptography that she's taking!) and I was Irbysitting. I sat Irby down in the living room with his back to the bookshelf, and then squatted in front of him and asked him (in English and ASL) if he wanted me to read a book to him. He gave a big smile, turned around, crawled over to the bookshelf and looked up at me. I said encouraging happy things. He grabbed Dr. Seuss's ABC's off the shelf. Once again we read the whole thing while sitting in the rocking chair, and once again he turned the page when I said "Turn the page!".
Another grey day.
I updated the list of conferences that I am interested in attending. There is a very exciting new conference: the first international workshop on the economics of security, chaired by Hal Varian and Ross Anderson.
Here's an Irby story. It is too wonderful to be believed, but I'll tell it anyway.
Amber was out for a lecture, and I asked Irby, using English and ASL, if he wanted me to read a book to him. He smiled and did the gesture that he uses to indicate that he wants something. This gesture is made by holding his hand palm outward and opening and closing his fingers. It evolved out of reaching for things, but now he uses it abstractly to mean that he wants something. At least, I think that's what he means. Anyway, he held his hand up and opened and closed his fingers. So I picked him up and carried him over to his bookshelf and arranged several books with their covers facing out and asked him which one he wanted. He crawled forward and grabbed Dr. Seuss's ABC's, which I have read to him two times in the last couple of days. I took Irby and the book to the rocking chair and read the entire book to him from front to back. He paid attention all the way through, reaching to touch the moon on the "M" page as he always does, and to touch other pictures, and turning each page when I hold it out to him and say "Turn the page!". This is the third time in the last couple of days that he has sat and listened to the entire Dr. Seuss's ABC's. I'm so excited!
Carl Bergh has submitted some corrections to a previous story that was posted here. He'd like the world to know that (a) he doesn't remember Mr. Ludwig mentioning the career of an actuary, (b) he had Mr. Ludwig in 11th grade, not 10th, (c) he knew he wanted to be an actuary since 8th grade, and (d) he actually likes using his brain, and isn't in it simply for the money.
I was up by 7:30 this morning, but the sun never came up! The grey roof of clouds did get a bit brighter.
Here is a list of conferences that I am interested in attending.
I woke at 6:30 this morning and now, at 7:30, the red glow on the horizon is finally intensifying toward a single point. I realized yesterday that while my windows face east, the sun comes up in the south! It also raises its head very slowly -- a weakened god in winter -- and then immediately arcs west. There! Even as I wrote this log entry, a fiery rim of molten gold spilled over the ocean and into my eyes.
In the last twenty-one hours 33 people have read this log, 14 of them doing so for the first time.
I cleaned up licence_quick_ref.html, removing unnecessary words and rewriting some of the rationale.
There's this guy named Carl Bergh, who decided that he wanted to be an actuary when his 10th grade math teacher, Mr. Al Ludwig, told the class that "actuary" was the highest-paid profession in America. Now, a dozen years later, you know what Carl does for a living? He's actually an actuary! I wonder what I wanted to be when I was in 10th grade. I don't remember. Popular, I think, except also I didn't want to be popular, or I wanted to be unpopular, or something. I think my intended professions at that time included "poet", and "prophet". Anyway, nowadays I'm a "security and distributed systems engineer", which is a subtype of "software engineer". During 2001 I think software engineers had a higher average salary than actuaries. Perhaps in 2002 actuaries will be back on top. Poets and prophets are still pretty low on the totem pole of average salaries.
Again I woke at 7:30, and while Amber and Irby slept I moved our computers into the newly organized office. I also got USB mass storage working under Linux 2.4.17, so I can now instantly transfer digital camera pictures and movies to my computer. Stay tuned...
I ordered a new computer! ASUS A7V266-E! 1.33 GHz Athlon XP! 512 MB PC2100 DDR RAM from the `Crucial' brand! Whoot!
Some people who were my friends in high school have tracked me down and invited me to visit with them in Colorado Springs this coming summer. This feels very weird to me, because I can't sort out which feelings are from my 27 year old self and which are from my 17 year old self. If you are, or know how to contact, the following people please e-mail me: Rebecca Addington, Heather Fitzgerald, Penny Patterson, Mike Zamiatala.
I woke up with the sun today, 7:30. Looking east from my fifteenth-story windows, I see that beneath the rusty brilliance of the sunrise the waterfront is dark and silent. I hear the morning calls of birds.
MarkM and MarcS have published a draft paper entitled The Digital Path: Smart Contracts and the Third World. It is exciting, but today their programme feels remote to me, almost dreamlike. I believe in incremental development -- building on proven success. I want to see large improvements in Third World social systems growing out of smaller ones. I'm always thrilled to receive e-mail from Mojo Nation users in distant lands. Last week I received such an e-mail from Ukraina.
Perhaps I could imagine the Digital Path more vividly if I had experience in these societies today. Suddenly, I feel wanderlust! Let's go to Mexico together! To Russia! To China!
My e-mail is back. <zooko@zooko.com>
Happy New Year, all you Gregorian calendar people.
When Julius Caesar established the calendar that (with Pope Gregory XIII's modifications) we now follow, he wanted the year to begin on the Spring Equinox or the Winter Solstice, but as a political compromise he allowed it to remain January 1 -- the date that the Roman Senate traditionally convened. Here's a nice summary of our calendar's history.
