spike

this page contains a compilation of my anime reviews. it is compiled from my myanimelist. reviews are sorted by rating, then alphabetically within ratings.

many of the factors impacting my reviews are largely external to the shows themselves. all are excellent in their own right, but the best are often colored by the when and where they found me.

10/10

excellent shows that are at once deeply thought provoking, personal, and culturally foundational.

Akira
Cowboy Bebop
Horimiya
Jin-Rou
Memories
Pale Cocoon
Perfect Blue
Ping Pong the Animation
Redline
Serial Experiments Lain
Shinseiki Evangelion
Shinseiki Evangelion Movie: Air/Magokoro wo, Kimi ni
Sonny Boy
Sousou no Frieren
Steins;Gate

9/10

amazing, even flawless shows, which nonetheless lack that special spark that would otherwise make them masterpieces.

Bakemonogatari
Bocchi the Rock!
Byousoku 5 Centimeter
Death Note
Hotaru no Haka
Houseki no Kuni
Howl no Ugoku Shiro
Katanagatari
Kimi no Na wa.
Kiseijuu: Sei no Kakuritsu
Kizumonogatari III: Reiketsu-hen
Majo no Takkyuubin
Mushishi
Mushoku Tensei: Isekai Ittara Honki Dasu
Mushoku Tensei: Isekai Ittara Honki Dasu Part 2
Nichijou
Ookami to Koushinryou
Ousama Ranking
Owarimonogatari 2nd Season
Sen to Chihiro no Kamikakushi
Shingeki no Kyojin Season 3 Part 2
Shingeki no Kyojin: The Final Season - Kanketsu-hen
Sono Bisque Doll wa Koi wo Suru
Vinland Saga
Vinland Saga Season 2

Horimiya

My all time favorite comfort media.

Years ago, back when I was a bit more prone to bouts of weekend depression, I remember telling friends: “If you hear anything about me re-watching/reading Horimiya, it’s a call for help.”

And though I no longer watch it as often anymore (both because I’m in a much better place mentally now, thanks for asking; and because it got to the point where I started memorizing whole episodes line by line), Horimiya still holds strong in my mind as one of the best oiled pieces of story telling machinery ever put to animation.

Pale Cocoon

There’s a definition of the word ‘creative’ that I heard a long time ago that I really liked. It went something like

When faced with a new idea, an uncreative person (who is otherwise perfectly intelligent), will come out with just as many ideas as they are told.

A normal person might bump this up to 2 — equivalent exchange. They can extend an idea, but the sum number of novel ideas in the world remains roughly the same.

For a creative, this number is 10 fold. Leave a creative a tiny little interesting breadcrumb, and they’ll retort back with new, interesting ideas so fast that you can hardly believe they hadn’t heard the thing before.

Certain works have the power to make me feel like a creative. Pale Cocoon is one such case. For 20 minutes, every second, and every scene, and every piece of technology and skyline and faded photograph and facial expression and fragment of history, tastes, like how I imagine, one of those creative breadcrumbs tastes.

Sonny Boy

I didn’t really enjoy watching Sonny Boy.

I get that that might sound weird seeing the 10 right next to this review, but hear me out.

Sonny Boy is a show that’s good only after you watch it.

There’s a decent number of shows that I remember fondly for nostaligic reasons — Gurren Lagann, Steins;Gate, FLCL, etc. These shows are obviously good, but my liking of them are biased, both because of the personal impact they had at the time, and how my life has colored their memory in the years since I’ve watched them.

And I rewatch these shows, I’m almost always disappointed. They never live up to how good they are in my memories. The hue just isn’t right. In some sense, what made these shows good wasn’t their actual text, but my memories of them. Of course, the shows didn’t become an instant 0 or whatever upon rewatch, but that brings me to my point:

Sonny Boy is a show exclusively made of memories.

When I graduated high school, I felt this a void well up inside me where I thought an emotion should have been.

“Shouldn’t I be a bit more proud? Or accomplished? Or something?”

Finishing Sonny Boy felt like much the same. Everything about the show felt like it was one of the greats. Yet when credits rolled, I didn’t really feel much of anything.

Some time after I finished Sonny Boy, out of nowhere, I suddenly remembered the date of my graduation.

My parents didn’t show up that day, busy with work, what not. I didn’t mind. I picked my graduation certificate, listened to a droning speech by the principal, and went about the rest of my day without much of a thought.

But I knew something significant had happened, I just hadn’t felt it yet.

In the years to come, as old friends and memories drifted, reuinted, then drifted again, I slowly pieced together what that feeling was supposed to be.

Sonny Boy was the first piece of media to make me feel exactly like that brewing feeling I have about graduation. It gave me a new piece of emotional vocabulary I could call back on, and touch, and talk about — like I’m doing right now.

Bakemonogatari

I’ve watched Bakemonogatari three times.

Once, as a young teenager, on the recommendation of my older brother. I got the impression that it was a very weird show by very weird Japanese people whose editing style made me press the pause button a whole ton to catch all of the text dumps.

Twice, as a college freshman, revisiting the whole Monogatari series as a literary work after reading a ton of essays. I found the themes of dealing with mental illness quite intruiging. The weird editing and characters had also grown on me in the mean time. It was then that I realized that the Monogatari series was one that definitely got better on rewatches. After this second rewatch, I promised myself that I would learn Japanese for the sole purpose of one day properly watching Bakemonogatari.

Then thrice, finally, with subtitles off, a month after college graduation. This time around, I found Bakemonogatari to be an incredibly endearing love story. This became even more poignant as episode 12 came along, and supercell’s kimi no shiranai monogatari started blasting; I realized that the lyrics were actually telling the story as a love letter, from Senjougahara’s point of view. I was love struck before, and I fell in love with the series all over again at that moment.

Byousoku 5 Centimeter

Perfectly bittersweet.

The art is just utterly gorgeous. The airial shots of Japanese cities at night look like Shinkai just ripped them straight out of my memory.

The first part by itself is an easy 10/10, though the writing and presentation falls off a little bit for the second and third parts. As a whole, I’d say the piece sits comfortably at a nine-ish.

Part of it is that I’m not really sure how a space ship, or surfing, or much of the rest of it fit together with the main thread of Takeki, Akari and Kanae’s awkward little injective romance. They definitely made for some good imagery, so who cares right?

This is Makoto Shinkai’s best work in my eyes, even though I’d say I enjoyed watching Your Name more overall.

Katanagatari

Katanagatari is, in many ways, Nishio Isshin-lite.

The story is much more straight forward, the characters a lot more transparent, and the themes a lot more pronounced. The story is about collecting 12 swords. The title, 刀語(katana gatari), literally just means ‘sword-story.’

It’s so straightfoward that maybe it’s honestly a more interesting work meta-contextually than it is textually. Much of the fun comes from it playing with your expectations, then slyly making fun of you for having been played, and then you playing along anyway because WHO spoils a major plot beat out of nowhere like 5 times, then has the absolute gall and confidence to still make the spoiled episode just as good anyway.

Fun on first watch, and I suspect it’ll be just as fun on rewatch.

Staring Into The Singularity

Fri, 14 Aug 2026 08:16:57 GMT

STARING INTO THE SINGULARITY

From The Low Beyond. 1996, 1999 by Eliezer S. Yudkowsky

The short version:

If computing power doubles every two years, what happens when computers are doing the research?

Computing power doubles every two years. Computing power doubles every two years of work. Computing power doubles every two subjective years of work.

Two years after computers reach human equivalence, their power doubles again. One year later, their speed doubles again.

Six months - three months - 1.5 months ... Singularity.

It’s expected in 2035. (Oops, make that 2025.)

The End of History
The Beyondness of the Singularity
The Definition of Smartness
Perceptual Transcends
Great Big Numbers
Smarter Than We Are
Sooner Than You Think
Uploading
The Interim Meaning of Life
Getting to the Singularity

The End of History

It began four billion years ago in a pool of muck, when a molecule made a copy of itself and so became the ultimate ancestor of all earthly life.

It began two and a half million years ago, when the first human awoke to consciousness.

Fifty thousand years ago with the rise of the Cro-Magnons. Ten thousand years ago with the invention of civilization. Five hundred years ago with the invention of the printing press. Fifty years ago with the invention of the computer.

In less than forty years, it will end.

Vernor Vinge saw it first. At some point in the near future, someone will come up with a method of increasing the maximum intelligence on the planet - either coding a true Artificial Intelligence or enhancing human intelligence. An enhanced human would be better at thinking up ways of enhancing humans; he would have an “increased capacity for invention”. What is this increased ability going to be directed upon? Why, the next generation of enhanced humans, of course.

And what will that doubly enhanced intelligence do? Research methods on triply enhanced humans, or build AI - Artificially Intelligent - assistants or even independent AI researchers who operate at computer speeds. And an AI researcher would be able to reprogram itself, directly, to operate even faster - and better still, smarter. And then our crystal ball explodes, everything we know is out the window, Life As We Know It is over, the “old models break down and new ones must be applied”. Hence the phrase: Singularity.

There are multiple paths to the Singularity. Nanotechnology - the ability to build computers atom by atom and brains neuron by neuron. Artificial Intelligence - if we can create programs that can match our intelligence, they shall shortly thereafter exceed us in speed if not in ability, because computers are currently increasing in speed by 55% per year. Even a mildly “intelligent” program, with the ability to notice some meaning in text, could greatly increase the rate of scientific progress by making knowledge more accessible. We could bootstrap our way to the Singularity via the relatively mild enhanced humans produced by Algernon’s Law. Something completely unanticipated could occur, such as this decade’s invention of the Scanning Tunnelling Probe, which advanced the nanotechnology timetable by about ten years.

If the current trends continue - if we don’t run up against some unexpected theoretical cap on intelligence, or turn the Earth into a radioactive wasteland, or trip on one of the hazards of truly advanced technology - the Singularity is inevitable. A planet scoured of life and a superintelligence are the only two stable states, and the Universe is filled with stable things; Life As We Know It is unstable, and it will shortly be over one way or the other. The generally accepted estimate has been and remains 2035 - less than forty years! - although many, including I, think that the Singularity may occur substantially sooner.

Some terminology, due to Vinge’s Hugo-winning A Fire Upon The Deep:

Power - an entity from beyond the Singularity. Transcend, Transcended, Transcendence - The act of reprogramming oneself to be smarter, reprogramming (with one’s new intelligence) to be smarter still, and so on ad Singularitum. Also the metaphorical area where the Powers live, or belonging to that area. Beyond - The grey area between being human and being a Power; the domain inhabited by entities smarter than human, but not possessing the technology to reprogram themselves directly and Transcend.

(In association with Amazon.com.)

The Beyondness of the Singularity

“I imagine bugs and girls have a dim perception that Nature played a cruel trick on them, but they lack the intelligence to really comprehend its magnitude.”- Calvin

But why should the Powers be so much more than we are now? Why not assume that we’ll get a little smarter and that’s it?

Consider the sequence 1, 2, 4, 8, 16, 32... In other words, the iteration of F(x) = (x + x). Every couple of years, computer performance doubles. (That’s actually gone up recently, to 55% per year.) That is the proven rate of improvement as overseen by constant, unenhanced minds, progress according to mortals.

Right now the amount of computing power on the planet is equal to the power of a human brain (10^11 to 10^26 operations per second, with consensus at 10^17) multiplied by the number of humans. The amount of artificial computing power is so small as to be irrelevant, not because of the number of humans, but because of the sheer power of a human brain. At the old rate of progress, computers reach human-equivalence levels - 10^17 floating-point operations per second or one hundred petaflops - at around 2035. That’s actually a bit long - since there are one-teraflops machines around now [as of 1996; it’s now up to 3.2 teraflops], which wasn’t expected until 2000 or so, and since computers started improving at 55% per year instead of 40% or so. Once we have human-equivalent computers, the amount of computing power on the planet is equal to the number of humans plusthe number of computers. The amount of intelligence available takes a huge jump. Ten years later, humans become a vanishing quantity in the equation.

That doubling sequence is actually a very pessimistic projection, because it assumes that computer power continues to double at the same rate. But why? Computer speeds don’t double due to some inexorable physical law, but because researchers and engineers find ways to make them faster. If some of the researchers and engineers are computers...

A group of human-equivalent computers spends 2 years to double computer speeds. Then they spend another 2 subjective years, or 1 year in human terms, to double it again. Then they spend another 2 subjective years, or six months, to double it again. Six months later, the computing power goes to infinity.

That is the “Transcended” version of the doubling sequence. A Transcended version of a sequence {a0, a1, a2...} is a function where the interval between an and an+1is inversely proportional to an . (If there’s a pre-existing mathematical term for this, let me know.) So a Transcended doubling function starts with 1, in which case it takes 1 time-unit to go to 2. Then it takes 1/2 time-units to go to 4. Then it takes 1/4 time-units to go to 8. This function, if it was continuous, would be the hyperbolic function y = 2/(2 - x). When x = 2, (2 - x) = 0 and y = infinity. The behavior at that point is known mathematically as a singularity.

And the Transcended doubling sequence is a fairly pessimistic projection, not a Singularity at all, because it assumes that only speed is enhanced. What if the quality of thought was enhanced? Right now, two years of work - well, these days, eighteen months of work. Eighteen subjective months of work suffices to double the speed of computers. Shouldn’t this improve a bit with thought-sharing and eidetic memories? Shouldn’t this improve if, say, the total sum of human scientific knowledge is stored in predigested, cognitive, ready-to-think format? Shouldn’t this improve with short-term memories capable of holding the whole of human knowledge? A human-equivalent AI isn’t merely “equivalent” - if Kasparov had had even the smallest, meanest automatic chess-playing program integrated solidly with his intuitions, he would have beat Deep Blue into a pulp. That’s The AI Advantage: Simple tasks carried out at blinding speeds and without error, conscious tasks carried out with perfect memory and total self-awareness.

I haven’t even started on the subject of AIs redesigning their cognitive architectures, although they’ll have a far easier time of it - especially if they can make backups. Transcended doubling mightrun up against the laws of physics before reaching infinity... but even the laws of physics as now understoodwould allow one milligram (more or less) to store and run the entire human race at a million subjective years per second. (Without even using quantum computing.)

Let’s take a deep breath and think about that for a moment. One milligram. The entire human race. One million years per second. That means, using only this planet for computing power, it would be possible to support more people than the entire Universe could support if we colonized every single planet. It means that, in a single day, this civilization would have lived over 80 billion years, several times older than the age of the Universe to date.

The peculiar thing is that most people who talk about “the laws of physics” and hard limits on Powers would never even dream of setting the same limits on a (merely) galaxy-spanning civilization of (normal) humans a (brief) billion years old. Part of that is simply a cultural convention of science fiction; interstellar civilizations can break any physical law they please, because the readers are used to it. But part of that is because scientists and science-fiction authors have been taught, so many times, that Ultimate Unbreakable Limits usually fall to human ingenuity and a few generations of time. Powered flight, faster-than-sound, space travel - all proved impossible.

We know that change crept at a snail’s pace a mere millennium ago, and that even a hundred years ago it would have been impossible to place correct limits on the ultimate power of technology. We know that the past could never have placed limits on the present, and so we don’t try to place limits on the future. But with transhumans, the analogy is not to Lord Kelvin, nor Aristotle, nor to a hunter-gatherer - all of whom had human intelligence - but to a Neanderthal. With Powers, to a fish. And yet, because the power of higher intelligence is not as publicly recognized as the power of a few million years; because we have no history of naysayers being embarassed by transhumans instead of mere time; some of us still sit, grunting around the fire, setting ultimate limits on the sharpness of spears; some of us still swim about, unblinking, unable to engage in abstract thought, but knowing that the entire Universe is wet.

To convey the rate of progress driven by smarter researchers, I needed to invent a function more complex than the doubling function used above: T(n). You can think of T(n) as representing the largest number conceivable to someone with a class n brain. More formally, T(n) is defined as the longest block of 1s produceable by any halting Turing Machine with n states acting on an initially blank tape. If you are familiar with computers but not Turing Machines, consider T(n) to be the largest number produceable by a computer program with ninstructions. Or, if you’re an information theorist, you can think of T(n) as the inverse function of complexity; it produces the largest number with complexity n or less.

The sequence produced by iterating T(n), S{n } = T(S{n- 1}), is constant for very low values of n. S{0} is defined to be 0; a program of length zero produces no output. This corresponds to a Universe empty of intelligence. T(1) = 1. This corresponds to an intelligence not capable of enhancing itself; this corresponds to where we are now. T(2) = 3. Here begins the leap into the Abyss. Once this function increases at all, it immediately tapdances off the brink of the knowable. T(3) = 6? T(6) = 64?

T(64) = vastly more than 1080, the number of atoms in the Universe. T(10^80) is something that only a Transcendent entity is ever going to be able to calculate, and that only if Transcendent entities can create new Universes to supply the necessary computing power. I would venture to guess that even T(64) will never be known to any strictly human being.

Now take the Transcended version of S{n}, starting at 2. Half a time-unit later, we have 3. A third of a time-unit after that, 6. A sixth later - one whole unit after this function started - we have 64. A sixty-fourth later, 10^80 or whatever. An unimaginably tiny fraction of a second later... Singularity.

Is S{n} really a good model of the Singularity? Of course not. “Good model of the Singularity” is an oxymoron; that’s the whole point;the Singularity will outrun any model a human could have formulated a hundred years ago, and the Singularity will outrun any model we formulate.

Also, T(10^17), or T(human), should presently equal 10^12, or the power of a computer, and S{n} should equal S{n-1} + T(S{n-1}). The main objection, though, would be that S{n} is an ungrounded metaphor. The Transcended doubling sequence models faster researchers. It’s easy to say that S{n} models smarter researchers, but what does smarter actually mean in this context?

The Definition of Smartness

Smartness is the measure of what you see as obvious, what you can see as obvious in retrospect, what you can invent, and what you can comprehend. To be a bit more precise about it, smartness is the measure of your semantic primitives (what is simple in retrospect), the way in which you manipulate the semantic primitives (what is obvious), the way your semantic primitives can fit together (what you can comprehend), and the way you can manipulate those structures (what you can invent). If you speak complexity theory, the difference between obvious and obvious in retrospect, or inventable and comprehensible, is somewhat like the difference between NP and P.

All humans who have not suffered neural injuries have the same semantic primitives. What is obvious in retrospect to one is obvious in retrospect to all. Four notes: First, by “neural injuries” I do not mean anything derogatory - it’s just that a person missing the visual cortex will not have visual semantic primitives. People who lose their visual cortex forget what it is like to see. Second, theorems in math may be obvious in retrospect only to mathematicians - but anyone else who acquired the skillwould have the ability to see it too. Third, to some extent what we speak of as obvious involves not just the symbolic primitives but very short links between them. I am counting the primitive link types as being included under “semantic primitives”. When we look at a thought-sequence and see it as being obvious in retrospect, it is not necessarily a single semantic primitive, but is composed of a very short chain of semantic primitives and link types. Fourth, I apologize for my tendency to dissect my own metaphors; I really can’t help it.

Similarly, the human cognitive architecture is universal. We all have the same sorts of symbolic structures. The nature of these structures is not known, no more than we know what symbols are made of, but our ability to communicate with each other indicates that, whatever we are communicating, it is the same on both sides. If any two humans share a set of symbols, any structure composed of those symbols that is understood by one will be understood by the other.

Different humans may have different degrees of the ability to manipulateand structure symbols; different humans may see and invent different things. The great breakthroughs of physics and engineering did not occur because a group of people plodded and plodded and plodded for generations until they found an explanation so complex, a string of ideas so long, that only time could invent it. Relativity and quantum physics and buckyballs and object-oriented programming all happened because someone put together a short, simple, elegant semantic structure in a way that nobody had ever thought of before. That is being a little bit smarter; that’s where revolutions come from. Not time. Not hard work; although hard work was usually necessary, others had worked far harder without result. Raw smartness.

Now think about the Singularity. Think about a chimpanzee trying to understand integral calculus. Think about the people with damaged visual cortices who cannot remember what it was like to see, who cannot imagine the color red or visualize two-dimensional structures. Think about a visual cortex with trillions of times as many neuron-equivalents. Think about twenty thousand distinct colors in the rainbow, none a shade of any other. Think about rotating fifty-dimensional objects. Think about attaching semantic primitives to the pixels, so that one could see a rainbow of ideas in the same way that we see a rainbow of colors.

Why does anything exist at all? Nobody knows. And yet the answer is obvious. The First Cause must be obvious. It has to be obvious to Nothing,present in the absence of anything else, formed from -blank-. What is it that evokes conscious experience, the stuff that souls are made of? We are made of conscious experiences. There is nothing we experience more directly. How does it work? We don’t have a clue. Two and a half millennia of trying to solve it and nothing to show for it but “I think therefore I am.” The solutions operate outside the semantic primitives and the semantic structures we can use.

Our descendants, successors, future selves will figure out the semantic primitives necessary and alter themselves to perceive them. The Powers will dissect the Universe and the Reality until they understand why anything exists at all, neurons until they understand qualia. And that will only be the beginning. It won’t end there. Why should there be only three hard problems? After all, if not for humans, the Universe would apparently contain only one or two hard problems - how could a non-conscious thinker formulate the hard problem of consciousness? Might there be states of existence beyond mere consciousness - transsentience? That’s what the Singularity is all about.

So before you talk about life as a Power or the Utopia to come - a favorite pastime of transhumanists and Extropians is to discuss the problems of uploading, life after being uploaded, and so on - just remember that you probably have a much better chance of solving all three hard problems than you do of making a valid statement about the future. This goes for me too. I’ll stand by everything I said about humans, including our inability to understand certain things, but everything I said about the Powers is almost certainly wrong. “They’ll figure out the semantic primitives necessary and alter themselves to perceive them.” Wrong. “Figure out.” “Semantic primitives.” “Alter.” “Perceive.” I would bet on all of these terms becoming obsolete after the Singularity. There are better ways and I’m sure They - or It, or [sound of exploding brain] will “find them”.

Perceptual Transcends

I would like to introduce a unit of post-Singularity progress, the Perceptual Transcend or PT.

[Brief pause while audience collapses in helpless laughter.]

I’m not trying to get it right, just make a point.

A Perceptual Transcend occurs when all things that were comprehensiblebecome obvious in retrospect, and all things that were inventablebecome obvious. A Perceptual Transcend occurs when the semantic structures of one generation become the semantic primitives of the next. To put it another way, one PT from now, the whole of human knowledge becomes perceiveable in a single flash of experience, in the same way that we now perceive an entire picture at once.

Computers are a PT above humans when it comes to arithmetic - sort of. While we need to manipulate an entire precarious pyramid of digits, rows and columns in order to multiply 62305 by 10358, a computer can spit out the answer - 645355190 - in a single obvious step. These computers aren’t actually a PT above us at all, for two reasons. First of all, they just handle numbers up to two billion instead of 9; after that they need to manipulate pyramids too. Far more importantly, they don’t notice anything about the numbers they manipulate, as humans do. If a human multiplies 23704 by 14223, using the wedding-cake method of multiplication, he won’t multiply 23704 by 2 twice in a row, just steal the results from last time. If one of the interim results is 12345 or 99999 or 314159 he’ll notice that too. The way computers manipulate numbers is actually lesspowerful than the way we manipulate numbers.

Would the Powers settle for less? A PT above us, multiplication is carried out automatically but with full attention to interim results, numbers that happen to be prime, and the like. If I were designing one of the first Powers [and I am - ‘99], I would create an entire subsystem for manipulating numbers, that would pick up on primality, complexity, and all the numeric properties known to humanity. A Power would understand why 62305 times 10358 equals 645355190, with the same understanding achieved by a top human mathematician who spent hours studying all the numbers involved. And at the same time, the Power will multiply the two numbers automatically.

For such a Power, to whom numbers were true semantic primitives, Fermat’s Last Theorem and the Goldbach Conjecture and the Riemann Hypothesis might be obvious. Somewhere in the back of its mind, the Power would test each statement with a million trials, subconsciously manipulating all the numbers involved to find why they were not the sum of two cubes or why they were the sum of two primes or why their real part was equal to one-half. From there, the Power could intuit the most basic, simple solution simply by generalizing. Perhaps human mathematicians, if they could perform the arithmetic for a thousand trials of the Riemann Hypothesis, examining every intermediate step, looking for common properties and interesting shortcuts, could intuit a formal solution too. But they can’t, and they certainly can’t do it subconsciously, which is why the Riemann Hypothesis remains unobvious and unproven - it is a conceptual structureinstead of a conceptual primitive.

Perhaps an even more thought-provoking example is provided by our visual cortex. On the surface, the visual cortex seems to be an image processor. In a modern computer graphics engine, an image is represented by a two-dimensional array of pixels (picture elements, spots of color). To rotate this image, each pixel’s rectangular coordinates {x, y} are converted to polar coordinates {theta, r}. All thetas, representing the angle, have a constant added. The polar coordinates are then converted back to rectangular. There are ways to optimize this process, and ways to account for intersecting and empty pixels on the new array, but the essence is clear: To perform an operation on an entire picture, perform the operation on each pixel in that picture.

At this point, one could say that a Perceptual Transcend depends on what level you’re looking at the operation. If you view yourself as carrying out the operation pixel by pixel, it is an unimaginably tedious cognitive structure, but if you view the whole thing in a single lump, it is a cognitive primitive - a point made in Hofstadter’s Ant Fugue when discussing ants and colonies. Not very exciting unless it’s Hofstadter explaining it, but there’s more to the visual cortex than that.

For one thing, we consciously experience redness. (If you’re not sure what “conscious experience” a.k.a. “qualia” means, the short version is that you are not the one who speaksyour thoughts, you are the one who hears your thoughts.) Qualia are the stuff making up the indescribable difference between redand green.

The term “semantic primitive” describes more than just the level at which symbols are discrete, compact objects. It describes the level of conscious perception. Unlike the computer manipulating numbers formed of bits, and like the imagined Power manipulating theorems formed of numbers, we don’t lose any resolution in passing from the pixel level to the picture level. We don’t suddenly perceive the idea “there is a bear in front of me”, we see a picture of a bear, containing millions of pixels, every one of which is consciously experienced simultaneously. A Perceptual Transcend isn’t “just” the imposition of a new cognitive level; it turns the cognitive structures into consciously experienced primitives.

“To put it another way, one PT from now, the whole of human knowledge becomes perceiveable in a single flash of experience, in the same way that we now perceive an entire picture at once.”

Of course, the PT won’t be used as a post-Singularity unit of progress. Even if it were initially, it won’t be too long before “PT” itself is Transcended and the Powers jump out of the system yet again. I exerted all my ability to write an even briefly plausible description of progress beyond the Singularity, and yet the Singularity is as far beyond me as it is beyond any other human, and my PTs will be as worthless a description as the doubling sequence discarded so long ago. Even if we accept the PT as the basic unit of measure, it simply introduces a secondary Singularity. Maybe the Perceptual Transcends will occur every two consciously experienced years at first, but then will occur every conscious year, and then every conscious six months - get the picture?

It’s like the “Birthday Cantatatata...” in Hofstadter’s book Godel, Escher, Bach. You can start with the sequence {1, 2, 3, 4 ...} and jump out of it to w (omega), the symbol for infinity. But then one has {w, w + 1, w + 2 ... }, and we jump out again to 2w. Then 3w, and 4w, and w2 and w3and ww and w^(ww) and the ordinal e0 , which includes all exponential towers of ws.

The PTs may introduce a second Singularity, and a third Singularity, and a fourth, until Singularities are coming faster and faster and the first w-Singularity is imminent -

Or the Powers may simply jump beyond that system. The Birthday Cantatatata... was written by a human - admittedly Douglas Hofstadter, but still a human - and the concepts involved in it may be Transcended in the very first PT.

The Powers are beyond our ability to comprehend.

Get the picture?

Great Big Numbers

It’s hard to appreciate the Singularity properly without first appreciating really large numbers. I’m not talking about little tiny numbers, barely distinguishable from zero, like the number of atoms in the Universe or the number of years it would take a monkey to duplicate the works of Shakespeare. I invite you to consider what was, circa 1977, the largest number ever to be used in a serious mathematical proof. The proof, by Ronald L. Graham, is an upper bound to a certain question of Ramsey theory. In order to explain the proof, one must introduce a new notation, due to Donald E. Knuth in the article Coping With Finiteness. The notation is usually a small arrow, pointing upwards, here abbreviated as ^. Written as a function:

int arrow (int num, int power, int arrownum) {

int answer = num;

if (arrownum == 0) return num * power;

for (int i = 1; i < power; i++)answer = arrow(num, answer, arrownum - 1);

return answer;

} // end arrow2^4 = 24 = 16.

3^^4 = 3^(3^(3^3)).

7^^^^3 = 7^^^(7^^^7).

3^3 = 3 * 3 * 3 = 27. This number is small enough to visualize.

3^^3 = 3^(3^3) = 3^27 = 7,625,597,484,987. Larger than 27, but so small I can actually type it. Nobody can visualize seven trillion of anything, but we can easily understand it as being on roughly the same order as, say, the gross national product.

3^^^3 = 3^^(3^^3) = 3^(3^(3^(3^...^(3^3)...))). The ”...” is 7,625,597,484,987 threes long. In other words, 3^^^3 or arrow(3, 3, 3) is an exponential tower of threes 7,625,597,484,987 levels high. The number is now beyond the human ability to understand, but the procedure for producing it can be visualized. You take x=1. You let xequal 3^x. Repeat seven trillion times. While the very first stages of the number are far too large to be contained in the entire Universe, the exponential tower, written as “3^3^3^3...^3”, is still so small that it could be stored if a small percentage of all the computers on the Web cooperated.

3^^^^3 = 3^^^(3^^^3) = 3^^(3^^(3^^...^^(3^^3)...)). Both the number and the procedure for producing it are now beyond human visualization, although the procedure can be understood. Take a number x=1. Let x equal an exponential tower of threes of height x. Repeat 3^^^3 times, where 3^^^3 equals an exponential tower seven trillion threes high.

And yet, in the words of Martin Gardner: “3^^^^3 is unimaginably larger than 3^^^3, but it is still small as finite numbers go, since most finite numbers are very much larger.”

And now, Graham’s number. Let x equal 3^^^^3, or the unimaginable number just described above. Let x equal 3^^^^^^^(x^s)^^^^^^^3. Repeat 63 times, or 64 including the starting 3^^^^3.

Graham’s number is far beyond my ability to grasp. I can describe it, but I cannot properly appreciate it. (Perhaps Graham can appreciate it, having written a mathematical proof that uses it.) This number is far larger than most people’s conception of infinity. I know that it was larger than mine. My sense of awe when I first encountered this number was beyond words. It was the sense of looking upon something so much larger than the world inside my head that my conception of the Universe was shattered and rebuilt to fit. It felt as I imagine a mountain looks to people who don’t appreciate nuclear weapons: something forever beyond us to subdue. All theologians should face a number like that, so they can properly appreciate God. My happiness was completed when I learned that the actual answer to the Ramsey problem that gave birth to that number - rather than the upper bound - was probably six.

Why was all of this necessary, mathematical aesthetics aside? Because until you understand the hollowness of the words “infinity”, “large” and “transhuman”, you cannot appreciate the Singularity. You must knowthat even appreciating the Singularity is as far beyond us as visualizing that number is to a chimpanzee. Farther beyond us than that. No human analogies will ever be able to describe the Singularity, because we are only human.

The number above was forged of the human mind. It is nothing but a finite positive integer, though a large one. It is composite and odd, rather than prime or even; it is perfectly divisible by three. Encoded in the decimal digits of that number, by almost any encoding scheme one cares to name, are all the works ever written by the human hand, and all the works that could have been written, at 200 words per minute, over the age of the Universe raised to its own power a thousand times. And yet, if we add up all the base-ten digits the result will be divisible by nine. The number is still a finite positive integer. It may contain Universes unimaginably larger than this one, but it is still only a number. It is a number so small that the algorithm to produce it can be held in a single human mind.

The Singularity is beyond that. We cannot pigeonhole it by stating that it will be a finite positive integer. We cannot say anything at all about, except that it will be beyond our understanding.

If you thought that Knuth’s arrow notation produced some fairly large numbers, what about T(n)? How many states does a Turing machine need to implement the calculation above? What is the complexity of Graham’s number, C(Graham)? Probably on the order of 100. And moreover, T(C(Graham)) is likely to be much, much larger than Graham’s number. Why go through x= 3^(x ^s)^3 only 64 times? Why not 3^^^^3 times? (That’d probably be easier, since we already need to generate 3^^^^3, but not 64.) And with the extra space, we might even be able to introduce an even more computationally complex algorithm. In fact, Knuth’s arrow notation may not be the most powerful algorithm that fits into C(Knuth) states. Again, T(n) is the metaphor for the growth rate of a self-enhancing entity because it conveys the concept of having additional intelligence with which to enhance oneself. I don’t know when T(n) passes beyond the threshold of what human mathematicians can, in theory, calculate. Probably more than n=10 and less than n=100. The point is that after a few iterations, we wind up with T(4294967296). Now, I don’t know what T(4294967296) will be equal to, but the winning Turing machine will probably generate a Power whose sole purpose is to generate a really large number. That’swhat the term “large” means.

Smarter Than We Are

It’s all very well to talk about cognitive primitives and obviousness, but again - what does smarter mean? The meaning of smart can’t be grounded in the Singularity - I haven’t been there yet. So what’s my practical definition?

“The toughest challenge for a writer is a character brighter than the author. It’s not impossible. Puzzles the writer needs months to solve, or to design, the character may solve in moments. But God help the writer if his abnormally bright character is wrong!”

Larry Niven

“Of course, I never wrote the ‘important’ story, the sequel about the first amplified human. Once I tried something similar. John Campbell’s letter of rejection began: ‘Sorry - you can’t write this story. Neither can anyone else.‘”

Vernor Vinge

Smartness is that quality which makes it impossible to write a story about a character smarter than you are. You can write about super-fast thinkers, eidetic memories, lightning calculators; characters who learned a dozen languages in a week, who can read a textbook in an hour, or who can invent all kinds of wonderful stuff - as long as you don’t have to produce the invention, that is. But you can’t write a character with a higher level of emotional maturity, a character who can spot the obvious solution you missed, a character who knows (and can tell the reader) the Meaning Of Life, a character with superhuman self-awareness. Not unless you can do these things yourself.

Let’s take a concrete example, the story Flowers for Algernon(later the movie Charly), by Daniel Keyes. (I’m afraid I’ll have to tell you how the story comes out, but it’s a Character story, not an Idea story, so that shouldn’t spoil it.) Flowers for Algernon concerns a neurosurgical procedure for intelligence enhancement. This procedure was first tested on a mouse, Algernon, and later on a retarded human, Charlie Gordon. The enhanced Charlie had the standard science-fictional set of superhuman characteristics; he thought fast, learned a lifetime of knowledge in a few weeks, and discussed arcane mathematics (not shown). Then the mouse, Algernon, gets sick and dies. Charlie analyzes the enhancement procedure (not shown) and concludes that the process is basically flawed. Later, Charlie dies.

That’s a science-fictional enhanced human. A real enhanced human, of course, would not have been taken by surprise. A real enhanced human would realize that any simple intelligence enhancement will be a net evolutionary disadvantage - if enhancing intelligence were a matter of a simple surgical procedure, it would have long ago occurred as a natural mutation. (This goes double for a procedure that works on rats!) As far as I know, this never occurred to Keyes. (I selected Flowers, out of all the famous stories of intelligence enhancement, because, for reasons of dramatic unity, this story shows what happens to be the correct outcome.)

Note that I didn’t dazzle you with an abstruse technobabble explanation for Charlie’s death; my explanation is two sentences long and can be understood by someone who isn’t an expert in the field. That’s the difference between a fictional genius and an actual enhanced human such as myself. I wouldn’t have been taken by surprise, and for that matter I wouldn’t have been that dramatically upset if I had. Normal author no enhancement, real enhancement no story.

Do I hear the audience demanding an explanation? Well, a full explanation is elsewhere; the brief version is that I am a Specialist, with a neurological perturbation (best guess: pathway severed from the right mammillary body or amygdala) that indirectly resulted in cognitive resources being over-allocated to a few favored abilities, including causal analysis and combinatorial design. This is of course a net evolutionary disadvantage (in accordance with Algernon’s Law), since an evenly balanced set of abilities is the design optimum.

From my perspective, of course, I’m perfectly normal. Other humans have this odd blind spot or slowness when it comes to seeing certain “obvious” answers, although they have no trouble understanding them. Algernon’s Law, for example. I can’t imagine how anyone who read Flowers for Algernon, much less the author, managed to miss it.

I also have blind spots, wherever cognitive resources have been diverted. Although I have no trouble understanding certain concepts, I cannot formulate them myself. Often just hearing the name given to a concept - and chunking it into a symbol rather than a group of ideas - causes everything else to fall into place. I can’t name my ideas except over the course of years; usually someone else winds up naming them. Why? Obviously I have a much fuzzier picture of my blind spots than my abilities, but I think they have to do with symbol formation, spotting non-causal similarities, and formulating linear strings of goals. (I’m sure you’re not interested in the specifics; I just want you to know that they’re there.) The point is...

It’s hard to convey what the term smarter means to someone who’s never seen their own or someone else’s blind spot. That is why I am the most fanatic Singularitarian on the planet; because I have some slight, infinitesimal, actual experience with how intelligence enhancement works. Because I have a visceral appreciation of how utterly, finally, absolutely impossible it is to think like someone even a little tiny bit smarter than you are. I know that we are all missing the obvious, in every day, and every way. There are no hard problems, only problems that are hard to a certain level of intelligence. Move the smallest bit upwards, and some problems will suddenly move from “impossible” to “obvious”. Move a substantial degree upwards, and all of them will become obvious. Move a huge distance upwards, and Lord knows what.

And I know that my picture of the Singularity will still fall short of the truth. I may not be modest, but I have humility - if I can spot anthropomorphisms and gaping logical flaws in every alleged transhuman in every piece of science fiction novel, it follows that a slightly higher-order Specialist (much less a real transhuman!) could read this page and laugh at my lack of imagination. Call it experience, call it humility, call it self-awareness, call it the Principle of Mediocrity. I know, in a dim way, just how dumb I am.

I’ve tried to show the Beyondness of the Singularity by brute force, but it doesn’t take infinite speeds and PTs and ws to place something utterly beyond us. All it takes is a little tiny bit of edge, a bit smarter, and the Beyond stares us in the face once more. I’venever been through the Singularity. I’ve never been to the Transcend. I, like any Specialist, just staked out an area of the Low Beyond. This page is devoted to communicating a sense of awe that comes from personal experience.

From my cortex to thine; every concept here was born of a mere human - and any impression it has made on you was likewise born of a mere human. Someone who has devoted a bit more thought, or someone a bit more fanatic; it makes no difference. Whatever impression you got from this page has not been an accurate picture of the far future; it has, unavoidably, been an impression of me. And I am not the far future. Had this page been written by a Power, you would have gained an accurate impression. But it isn’t, and wasn’t, and so you didn’t. Take whatever emotion this page evoked, and associate it not with the Singularity; associate it with me, the mild, quiet-spoken fellow infinitesimally different from the rest of humanity. Don’t bother trying to extrapolate beyond that. You can’t. Nobody can - not you, not me.

2035, Probably earlier.

Sooner Than You Think

We don’t lack the technology; the Singularity could happen tomorrow.The Internet has enough power, if properly reprogrammed, to run a human brain, and perhaps a smaller computer has enough power to run a seed AI. We possess machines capable of producing arbitrary DNA sequences. We have bacteria capable of turning DNA into protein. 100K of information from the future could specify a protein that built a device that would give us nanotechnology overnight. 100K could contain the code for a seed AI. We don’t have to receive that information from the future, either. One breakthrough - just one major insight - in the science of protein engineering or atomic manipulation or Artificial Intelligence, one really good day at Cycorp or Zyvex, and the books of history close. That could happen at any time. There is a major breakthrough in some scientific field once per day, and that statistic is from years ago.

Drexler has written a detailed, technical, how-to book for nanotechnology. After stalling for thirty years, AI is making a comeback. Computers are growing in power even fasterthan their usual, pedestrian rate of doubling in power every two years. Quate has constructed a 16-head parallel Scanning Tunnelling Probe. Last but not least, I’m starting to work out methods of enhancing human intelligence and coding a transhuman AI.

The exact time of Singularity is customarily predicted by taking a trend and extrapolating it, much as The Population Bomb predicted that we’d run out of food in 1977. For example, population growth is hyperbolic. (Maybe you learned it was exponential in math class, but it’s hyperbolic to a much better fit than exponential.) If that trend continues, world population reaches infinity on Aug 17, 2027, plus or minus 1.8 years. Although it is impossible for the human population to reach those levels, some say that if we can create AIs, then the graph might measure sentient population instead of human population. These people are torturing the metaphor. Explain to me who designed the population curve to take into account developments in AI. It’s just a curve,a bunch of numbers. It can’t distort the future course of technology just to remain on track.

If you project on a graph the minimum size of the materials we can manipulate, it reaches the atomic level - nanotechnology - in I forget how many years (page vanished), but I think around 2035. This, of course, was before the time of Scanning Tunneling Microscopes, which has recently been used to make an atomic-scale abacus on which actual calculations were performed, or the completely unanticipated artificial atom (“You can make any kind of artificial atom - long, thin atoms and big, round atoms.”), which has in a sense obsoleted mere molecular nanotechnology - the surest sign that nanotech is just around the corner. I believe Drexler is now giving the ballpark figure of 2013.

Similarly, computing power doubles every two years eighteen months. If we extrapolate forty thirty years ahead we find computers with as much raw power (10^17 ops/sec) as somepeople think humans have, arriving in 2035 2025. Does this mean we have the software to spin souls? No. Does this mean we can program smarter people? No. Does this take into account any breakthroughs between now and then? No. Does this take into account the laws of physics? No. Is this a detailed model of all the researchers around the planet? No.

It’s just a graph. The “amazing constancy” perhaps entitles it to consideration as a thought-provoking metaphor of the future, but nothing more. The Transcended doubling sequence doesn’t account for how the faster computer-based researchers can get the physical manufacturing technology for the next generation set up in picoseconds, or how they can beat the laws of physics.

Mathematics can’t predict when the Singularity is coming. Well, it can, but it won’t get it right. Even the remarkably steady numbers, such as the one describing the doubling rate of computing power, (1) describe unaided human minds and (2) are speeding up, perhaps due to computer-aided design programs. Statistics may be used to predict the future, but they don’t modelit. What I’m trying to say here is that “2035” is just a wild guess, and it might as well be next Tuesday.

In truth, I don’t think in those terms. I do not “project” when the Singularity will occur. I have a “target date”. I would like the Singularity to occur in 2005, which I think I would have a reasonable chance of doing via AI if someone handed me a billion dollars a year. I would really, really like the Singularity to arrive before nanotechnology, given the virtual certainty of deliberate misuse, misuse of a purely material (and thus amoral) ultratechnology powerful enough to destroy the planet. You cannot just sit back and wait. To quote Michael Butler, “Waiting for the bus is a bad idea if you turn out to be the bus driver.”

The most we can say about 2035 is that it seems like a reasonable upper bound, given the current rate of progress. The lower bound? Thirty seconds.

Uploading

Maybe you don’t want to see humanity “replaced” by a bunch of “machines” or “mutants”, even superintelligent ones? You love humanity and you don’t want to see it obsoleted? Well, tough luck. But just because humans become obsolete doesn’t mean you will become obsolete. You are not a human. You are an intelligence which, at present, happens to have a mind unfortunately limited to human hardware. That could change. With any luck, all persons on this planet who live to 2035 or 2005 or next year or whenever - and maybe some who don’t - are going to wind up as Powers.

Transferring a human mind into a computer system is known as “uploading”; turning a mortal into a Power is known as “upgrading”. The prototypical upload is the Moravec Transfer, proposed by Dr. Hans Moravec in the book Mind Children. The Moravec Transfer gradually moves (rather than copies) a human mind into a computer. You need never lose consciousness. The key assumption of the Moravec Transfer is that we can perfectly simulate a single neuron, which Penrose & Hameroff would argue is untrue. Let’s assume that either the laws of physics are computational or we can build a trans-Turing computer that does the same thing a neuron does, to which P&H would have no objection. Note that the details which follow have been redesigned and fleshed out a bit (by yours truly) from the original in Mind Children.

A neuron-sized robot swims up to a neuron and scans it into memory. The computer starts simulating the neuron. The robot waits until the neuron perfectly matches its simulation inside the computer, and then replaces the neuron with itself as smoothly as possible, sending inputs to the computer and transmitting outputs from the simulation of a neuron inside the computer. This entire procedure has had no effect on the flow of information in the brain, except that one neuron’s worth of processing is now being done inside a computer instead of a neuron. Repeat, neuron by neuron, until the entire brain is composed of robot neurons whose guts are inside the computer.

Despite this, the synapses (links) between robotic neurons are still physical; robots report the reception of neurotransmitters at artificial dendrites and release neurotransmitters at the end of artificial axons. Phase two replaces the physical synapses with software links. For every axon-dendrite (transmitter-receiver) pair, the electrical inputs are no longer reported by the robot; instead the computed axon output of the transmitting neuron is added as a simulated dendrite to the receiving neuron. At the end of Phase Two, the robots are all firing their axons, but none of them are receiving anything, none of them are affecting each other, and none of them are affecting the computer simulation. Finally, we disconnect the robots. You have now been placed entirely inside a computer, bit by bit, without losing consciousness. In Moravec’s words, your metamorphosis is complete.

If either of the phases still seems too abrupt, the transfer of an individual neuron, or synapse, can be spread out over as long a time as necessary. To slowly transfer a synapse into a computer, we can use weighted factors of the physical synapse and the computational synapse to produce the output. The weighting would start as entirely physical and end as entirely computational. Since we are presuming the neuron is being perfectly simulated, the weighting affects only the flow of causality and not the actual process of events. Slowly transferring a neuron is a bit more difficult. The robot would have to surround the neuron and suddenly replace the axons and dendrites with robotic tentacles without disturbing the neural cell body. (That’s going to take some pretty fancy footwork!) If this turns out not to be feasible, the robot can enclose the neuron entirely. At this point, the robot accepts weighted outputs from both the neuron and the computer. Once the weighting has shifted entirely to the computer, the neuron is discarded.

Assuming we can simulate an individual neuron, and that we can replace neurons with robotic analogues, I think that pretty thoroughly demonstrates the possibility of uploading given that consciousness is a function of neurons. And if we have immortal souls, then uploading is a realsnap. Take soul out of brain. Put soul in new substrate. Upload complete.

At this point it is customary to speculate about how one goes about eating, drinking, walking around; people state that they are unwilling to give up physical reality, worry about whether or not they will have sufficient computational power to simulate a hedonistic world of their wildest desires, and so on and so on ad nauseam. Even Vinge himself, discoveror of the Singularity, has gone on record as wondering whether one’s true self would be diluted by Transcendence.

I hope that by this point in the page you have been sufficiently impressed by the power and scope and incomprehensibility and general Transcendence of the Singularity that you see these speculations for what they are. If you wish to remain undiluted, you will be able to arrange it. You will be able to make backups. You will be able to preserve your personality regardless of substrate. The only folks who have to worry about being unwillingly diluted are the first humans to Transcend - but we’ll take the risk.

Of course, it may be that any being of sufficient intelligence wantsto be diluted. Exercising anxiety over that possibility seems spectacularly pointless, analogous to children worrying that, as adults, they will no longer want to be thoughtlessly cruel to other children. If you wantit, it’s not something wrong that you should worry about.

The human brain has a finite number of neurons and therefore a finite number of states. Eventually, you will die, go into an eternal loop, or Transcend. In the long run... the really long run... mortality isn’t an option.

Maybe, after Transcending, you will perhaps be a bit changed. If that is so, it is both inevitable and morally right. Given that absolutely nothing you can do will change that, why worry? Save your anxiety for what you can affect.

(The same goes for worries about hostile superintelligences wiping out humanity. I cannot guarantee this won’t happen, since I’m not superintelligent. However, I am reasonably (95%) certain that the Powers will be ethical;I just don’t know what really is “ethical”, the true moral right. If wiping out your creators is morally wrong, it won’t happen. If we do get wiped out, it will be because, were we upgraded to superintelligences instead, we would see the moral necessity and commit suicide. If that’s the case, there is absolutely nothing we can do about it, both in the long run (see above) and the short run (see below); and, even were we granted the opportunity, it would be morally wrong to take it. Observer-dependent morality is a chimerical artifact of evolutionary competition; it dissolves under the application of sufficient intelligence. If you think it would be wrong to kill you, and you are rationally correct about it, presumably the Powers will see the same argument and you’ll be perfectly safe. Again, why worry?)

The Interim Meaning of Life

“You know, I don’t understand why humans evolved as such thoughtless, shortsighted creatures.”

“Well, it can’t stay that way forever.”

“You think we’ll get smarter?”

“That’s one of the two possibilities.”- Calvin and Hobbes

I know. You don’t want things to move so fast. You want a slowSingularity, a soft Singularity, one that isn’t quite so frightening. You would like all of humanity to enter the first stages together, in a smooth, synchronized Transcendence. You wish a serene song of rising intelligence, rather than a sudden shock. A gentle seduction.

Believe me, if that was possible, I’d do it. Not for myself, I’ve already said my goodbyes to life as we know it. I’d do it to reassure all the people on whose cooperation the Singularity depends. My allegiance is to the Singularity, but I don’t think it makes much of an intrinsicmoral difference when Earth’s Singularity happens, as long as it’s within the next ten thousand years or so. I am in such a tearing hurry for exactly one reason: Every delay increases the possibility that humanity will exterminate itself before reaching Singularity.

Nanotechnology, in my humble opinion, poses the most urgent threat. Complete control over the molecular structure of matter, via tiny self-reproducing robots, would make it too easy to deliberately wipe out all life on the planet. “Active shields” might suffice against accidental outbreaks of “grey goo”, but not against hardened military-grade nano, perfectly capable of attacking active shields with fusion weapons. And yet, despite this threat, we can’t even try to suppress nanotechnology; that simply increases the probability that the villains will get it first. (However, most nanotechnologists (e.g. Zyvex ) intend to sell generalized assemblers or otherwise publish their success. If you have the opportunity to politely explain to these people that they are being dumbfoundingly, suicidally naive about the benevolence of their fellow humans, by all means do so.)

Although nuclear war would almost certainly leave enough survivors to get civilization started again, I think it would probably alter the balance between AI and nanotech in the wrong direction. A major economic collapse would also slow AI more than nanotech; nanotech requires a single laboratory, while a seed AI requires an Internet and probably thousands of programmers. Those, then, are the three major “deadlines”, as I call them: Nanotech, nuclear war, and economic collapse.

Yudkowsky’s Threats:

The future is not a playground; it is a minefield. This isn’t to say that there isn’t any fun or happiness in the pre-Singularity future. But it isn’t a Utopia and you have to be careful. Blind fear will get you killed even faster than blind enthusiasm. This includes all forms of technophobia. It includes any panic, no matter how terrible the threat. Attempting to suppress a technology only inflicts more damage. It develops unevenly, or the compensating benefits are denied, or someone else gets it first. You can’t even try to regulate it or slow it down. This holds true of any technology, no matter how dangerous. Technologies with military applications are always misused. This is human nature. Human nature, to a cognitive engineer, is a thing that can be changed; but until it changes, there will be someonewilling to take global risks for personal power. It is easier to destroy than defend. Ever since the invention of nuclear weapons, offensive technology has been overwhelmingly more powerful than defensive technology. Unless a new technology can defend against nuclear weapons produced and augmented by that technology, this will remain true. Nor is the possibility of destruction the only reason for racing to Singularity. There is also the ongoing sum of human misery, which is not only a practical problem, not only an ethical problem, but a purely moral problem in its own right. Have you ever read P.J. O’Rourke’s description of a crack neighborhood? If I had the choice of erasing crack neighborhoods or erasing the Holocaust, I don’t know which I’d pick. I do know which project has a better chance of success. I also know that the victims, in retrospect if nothing else, will probably prefer life as a Power to life as a junkie.

Have you ever pondered the Great Questions of Life, the Universe, and Everything? Have you ever wondered whether it really matters, cosmically speaking, if you stay in bed this morning? Have you ever stared into the hard problem of ethics, or consciousness, or reality, and felt yourself slowly going insane as you realized that there is no justification for subjective experience, getting out of bed, or anything existing at all? How can we do anything, set any goals, without knowing the Meaning of Life? How can we justify our continued participation in the rat race if we don’t know why we’re running? What’s it all for?

We don’t know. We have to guess, and act on our best guesses. Regardless of the absolute probabilities, superintelligence has a better chance of discovering the true moral right, having the power to implement it, and wanting to implement it. The state where superintelligence exists is, with a very high degree of probability regardless of the True Meaning of Life, preferable to the current state. That’s the Interim Meaning of Life, and it works well enough... but it’s a long, long way from certainty, or really knowing what’s going on!

I have had it. I have had it with crack houses, dictatorships, torture chambers, disease, old age, spinal paralysis, and world hunger. I have had it with a death rate of 150,000 sentient beings per day. I have had it with this planet. I have had it with mortality. None of this is necessary. The time has come to stop turning away from the mugging on the corner, the beggar on the street. It is no longer necessary to close our eyes, blinking away the tears, and repeat the mantra: “I can’t solve all the problems of the world.” We can. We can end this.

And so I have lost, not my faith, but my suspension of disbelief.Strange as the Singularity may seem, there are times when it seems much more reasonable, far less arbitrary, than life as a human.

I suppose the ancient philosophers might have had it worse - they, after all, had no knowledge of physics. Their world must have seemed arbitrary indeed. Why is this pink lump a hand? Why does it respond to my will? Why is it soft and not hard? But knowing physics changes nothing. The world remains arbitrary. I know my hand is a collection of atoms - but why should I care about this pink lump? What makes anything important? Why? Why? Why?

And while we make and stake our best guesses - “human life”, “free will”, “new ideas” - the sum of human suffering goes from bad to worse. And I knowthat there is a better way! Why rationalize this life? Why make it seem bright and happy? There is an alternative!

Our fellow humans are screaming in pain, our planet will probably be scorched to a cinder or converted into goo, we don’t know what the hell is going on, and the Singularity will solve these problems. I declare reaching the Singularity as fast as possible to be the Interim Meaning of Life, the temporary definition of Good, and the foundation until further notice of my ethical system.

Getting to the Singularity

This page isn’t a call to arms in the ordinary sense. I am not raising a political cause or forming a social group. Perceiving those elements of the mind, I prefer to stay away from them. I do not command. I upgrade my readers’ minds until their goals match my own. I suppose that in our insane culture that sounds arrogant, but it’s not one-tenth as arrogant as the condescenscion quietly implicit in exerting coercion.

This page is a call to awareness. “Incoming! Heads up! Watch out! In ten years it’s all over!” If this makes you want to join the Cause or rebuild your ethical system or have a midlife crisis or send someone money, wonderful. But the main idea is that you should have trouble keeping a straight face whenever someone talks about “a hundred years down the road”, issuing 50-year bonds or retiring in 2030.

That said, what can we do to accelerate the Singularity?

Some of my readers will be neurologists and researchers and programmers; for them, the correct course is to be ready when the Singularity requires you. Make brain-tampering and Artificial Intelligence your hobby, keep up with the forefront of the most modern methods and advances, and maybe even try to contribute your own ideas every now and then.

If you want to help and all you’ve got is money, probably a lot of researchers on paths to the Singularity are spending valuable time writing grant proposals or doing things that could be done by lab assistants. It would be a fine thing if there were a Singularity Support Foundation to ensure that these people weren’t distracted. There is probably one researcher alive today - Hofstadter, Quate, Penrose, Chalmers, Drexler, Lenat, Moravec, Minsky, someone just graduating college, or even me - who is the person who gets to the Singularity first. Although some conceptual breakthroughs may be dependent on the laboratory, others may be dependent on how much time is spent in thought. Every hour that person is delayed is another hour to the Singularity. Every hour, six thousand people die, and most of the survivors are unhappy for an hour. Perhaps we should be doing something about this person’s spending a fourth of his time and energy writing grant proposals.

One major problem faced by a new movement is turning into a cult, a mutual admiration society, or a bunch of crackpots. Drexler barely prevented this from happening to nanotechnology. I have two major hints as to how to go about this.

First, think of yourself as a business venture. Plan to make a profit. Bootstrapyour way to the Singularity instead of begging for spare change. Ask for venture capital and have something specific in mind. There’s a lot of money to be made in AI. Even a little bit of semantics in a computer can result in huge increases in usability and power.

Second, don’t go Utopian. Don’t describe Life after Singularity in glowing terms. Don’t describe it at all. I think the all-time low point in predicting the future came in the few brief paragraphs ofUnbounding the Futurethat I read, when they described a pedestrian being run over and his hand miraculously healing. That’s ridiculous. Pedestrian? Run over? Hand?Biological bodies, cars, in a nanotech world? Why not just hire a bunch of apes to describe the ease of getting bananas with a human mind?

In the words of Drexler:

“I would emphasize that I have been invited to give talks at places like the physical sciences colloquium series at IBM’s main research center, at Xerox PARC, and so forth, so these ideas are being taken seriously by serious technical people, but it is a mixed reaction. You want that reaction to be as positive as possible, so I plead with everyone to please keep the level of cultishness and bullshit down, and even to be rather restrained in talking about wild consequences, which are in fact true and technically defensible, because they don’t sound that way. People need to have their thinking grow into longer-term consequences gradually; you don’t begin there.” [Emphasis mine.]

The problem with people expounding their Utopian visions of a nanotech world is that their consequences aren’t wild enough. Looking at stories of instantly healing wounds, or any material object being instantly available, doesn’t give you the sense of looking into the future.It gives you the sense that you’re looking into an unimaginative person’s childhood fantasy of omnipotence, and that predisposes you to treat nanotechnology in the same way.

Worse, it attracts other people with unimaginative fantasies of omnipotence. There’s no better way to turn into a bunch of parlor pinks, sipping coffee and planning the Revolution without actually doing anything. I suppose I shouldn’t be too harsh on the nano-Utopia types. Some of them may be actual researchers or science-fiction writers or other people doing useful things, some of them may be rank-and-file sincerely trying to make it happen who just got caught in the general lack of imagination, and of course none of them have been to the Low Beyond. Once you’ve read this page, though, there’s no excuse.

This page is about staring into the Singularity. It is about awe, the Beyond, the end of history, and things beyond human comprehension. It is intended to invoke a sense of future, and I hope that my readers will be inclined to view nanotechnology, artificial intelligence, neurology, and all the other paths to the Singularity in the same way - as part of the future. I hope that attracts the right sort of people.

In a moment of insanity, I subscribed to the Extropian mailing list. These people know what “Singularity” means. In theory, they know what’s coming. And yet, even as I write [in ‘97 - they’ve improved a bit in ‘99, maybe even due to my prodding], folk who really ought to know better are arguing over whether transhumans will have enough computing power to simulate private Universes, whether the amount of computing power available to transhumans is limited by the laws of physics, whether someone uploaded into a trans-computer is really the same person or just an amazing soybean imitation, and - least believably of all - whether our unimaginably intelligent future selves will still be, er... let’s say, “interested in the old-fashioned method of reproduction”.

Why is this our concern? Why do we need to know this? Can it not be that maybe, just maybe, these problems can wait until afterwe’re five times as smart and some of our blind spots have been filled? Right now, every human being on this planet who has heard of the Singularity has exactly one legitimate concern: How do we get there as fast as possible?What happens afterward is not our problem and I deplore those gosh-wow, unimaginative, so-cloying-they-make-you-throw-up, and just plain boringand unimaginative pictures of a future with unlimited resources and completely unaltered mortals. Leave the problems of transhumanity to the transhumans.Our chances of getting anything right are the same as a fish designing a working airplane out of algae and pebbles.

Our sole responsibility is to produce something smarter than we are; any problems beyond that are not ours to solve.

How do we keep the world economy from disintegrating for at least another ten years? How do we keep first-generation nanites from eating the planet? If an enhanced human is evil, intelligent enough to outrun us, and not smart enough to be good, how do we defeat him? If the Internet woke up tomorrow, how would we know, how could we talk to it, and how could we make backups? If a nascent Power with ill intentions originates on the Internet (probably because some moron violated the Prime Directive of AI and imposed Asimov Laws), what containment procedures will be necessary and how much damage could it do? Even these aren’t very practical questions, but at least they are our concern, things we may need to deal with using our naked wits.

How do we get a multi-billion-dollar Singularity Support Foundation up and running? Who’s willing to fund an AI project? Who do we need to recruit for an AI project? Will OpenSourcing the AI help, and is it safe? How can we disable the standard technophobic backlash? How can we find the pre-existing Specialists needed to create the basic design patterns for the AI?

These are the practical questions that will be faced in the immediate future. The correct questions, and the answers, are the proper concern of mailing lists. I don’t object to letting the imagination run free; it may produce useful ideas. But don’t get so emotionally involved in it, don’t even think about trying to claim that your position has a chance of being correct, and spend your time coding a transhuman AI - or simply making money - instead.

why-python_listing1

Fri, 14 Aug 2026 08:16:57 GMT

Listing 1. fetchmail Configuration File
set postmaster "esr"
set daemon 300
poll imap.ccil.org with proto IMAP and options no dns
    aka snark.thyrsus.com locke.ccil.org ccil.org
       user esr there is esr here options fetchall dropstatus warnings 3600
poll imap.netaxs.com with proto IMAP
       user "esr" there is esr here options dropstatus warnings 3600
skip imap.21cn.com with proto IMAP
       user esr here is tranxww there options fetchall
skip pop.tems.com with proto POP3:
       user esr here is ed there options fetchall
skip mail.frequentis.com with proto IMAP:
       user esr here is imaptest there with options fetchall

why-python_listing2

Fri, 14 Aug 2026 08:16:57 GMT

Listing 2. fetchmailrc
fetchmailrc = {
    'poll_interval':300,
    "logfile":None,
    "postmaster":"esr",
    'bouncemail':TRUE,
    "properties":None,
    'invisible':FALSE,
    'syslog':FALSE,
    # List of server entries begins here
    'servers': [
    # Entry for site `imap.ccil.org' begins:
    {
        "pollname":"imap.ccil.org",
        'active':TRUE,
        "via":None,
        "protocol":"IMAP",
        'port':0,
        'timeout':300,
        'dns':FALSE,
        "aka":["snark.thyrsus.com", "locke.ccil.org", "ccil.org"],
        'users': [
        {
            "remote":"esr",
            "password":"Malvern",
            'localnames':["esr"],
            'fetchall':TRUE,
            'keep':FALSE,
            'flush':FALSE,
            "mda":None,
            'limit':0,
            'warnings':3600,
        }
        ,        ]
    }
    ,
    # Entry for site `imap.netaxs.com' begins:
    {
        "pollname":"imap.netaxs.com",
        'active':TRUE,
        "via":None,
        "protocol":"IMAP",
        'port':0,
        'timeout':300,
        'dns':TRUE,
        "aka":None,
        'users': [
        {
            "remote":"esr",
            "password":"d0wnthere",
            'localnames':["esr"],
            'fetchall':FALSE,
            'keep':FALSE,
            'flush':FALSE,
            "mda":None,
            'limit':0,
            'warnings':3600,
        }
        ,        ]
    }
    ,
    # Entry for site `imap.21cn.com' begins:
    {
        "pollname":"imap.21cn.com",
        'active':FALSE,
        "via":None,
        "protocol":"IMAP",
        'port':0,
        'timeout':300,
        'dns':TRUE,
        "aka":None,
        'users': [
        {
            "remote":"tranxww",
            "password":None,
            'localnames':["esr"],
            'fetchall':TRUE,
            'keep':FALSE,
            'flush':FALSE,
            "mda":None,
            'limit':0,
            'warnings':3600,
        }
        ,        ]
    }
    ,
    # Entry for site `pop.tems.com' begins:
    {
        "pollname":"pop.tems.com",
        'active':FALSE,
        "via":None,
        "protocol":"POP3",
        'port':0,
        'timeout':300,
        'dns':TRUE,
        'uidl':FALSE,
        "aka":None,
        'users': [
        {
            "remote":"ed",
            "password":None,
            'localnames':["esr"],
            'fetchall':TRUE,
            'keep':FALSE,
            'flush':FALSE,
            "mda":None,
            'limit':0,
            'warnings':3600,
        }
        ,        ]
    }
    ,
    # Entry for site `mail.frequentis.com' begins:
    {
        "pollname":"mail.frequentis.com",
        'active':FALSE,
        "via":None,
        "protocol":"IMAP",
        'port':0,
        'timeout':300,
        'dns':TRUE,
        "aka":None,
        'users': [
        {
            "remote":"imaptest",
            "password":None,
            'localnames':["esr"],
            'fetchall':TRUE,
            'keep':FALSE,
            'flush':FALSE,
            "mda":None,
            'limit':0,
            'warnings':3600,
        }
        ,        ]
    }
    ]
}

why-python_listing3

Fri, 14 Aug 2026 08:16:57 GMT

Listing 3. Metaclass Function
def copy_instance(toclass, fromdict):
# Initialize a class object of given type from a conformant dictionary.
    class_sig = toclass.__dict__.keys(); class_sig.sort()
    dict_keys = fromdict.keys(); dict_keys.sort()
    common = intersect(class_sig, dict_keys)
    if 'typemap' in class_sig:
    class_sig.remove('typemap')
    if tuple(class_sig) != tuple(dict_keys):
        print "Conformability error"
#       print "Class signature: " + `class_sig`
#       print "Dictionary keys: " + `dict_keys`
        print "Not matched in class signature: " + `setdiff(class_sig, common)`
        print "Not matched in dictionary keys: " + `setdiff(dict_keys, common)`
        sys.exit(1)
    else:
    for x in dict_keys:
        setattr(toclass, x, fromdict[x])

why-python_listing4

Fri, 14 Aug 2026 08:16:57 GMT

Listing 4. Code that Calls Metaclass Function
# The tricky part--initializing objects from the
#   configuration global
# `Configuration' is the top level of the object
#   tree we're going to mung
Configuration = Controls()
copy_instance(Configuration, configuration)
Configuration.servers = [];
for server in configuration[`servers']:
    Newsite = Server()
    copy_instance(Newsite, server)
    Configuration.servers.append(Newsite)
    Newsite.users = [];
    for user in server['users']:
        Newuser = User()
        copy_instance(Newuser, user)
        Newsite.users.append(Newuser)

Why Python?

Fri, 14 Aug 2026 08:16:57 GMT

Why Python?

Software

by Eric Raymond on April 30, 2000

My first look at Python was an accident, and I didn’t much like what I saw at the time. It was early 1997, and Mark Lutz’s book Programming Python from O’Reilly & Associates had recently come out. O’Reilly books occasionally land on my doorstep, selected from among the new releases by some mysterious benefactor inside the organization using a random process I’ve given up trying to understand.

One of them was Programming Python. I found this somewhat interesting, as I collect computer languages. I know over two dozen general-purpose languages, write compilers and interpreters for fun, and have designed any number of special-purpose languages and markup formalisms myself. My most recently completed project, as I write this, is a special-purpose language called SNG for manipulating PNG (Portable Network Graphics) images. Interested readers can surf to the SNG home page at http://www.catb.org/~esr/sng/. I have also written implementations of several odd general-purpose languages on my Retrocomputing Museum page, http://www.catb.org/retro/.

I had already heard just enough about Python to know that it is what is nowadays called a “scripting language”, an interpretive language with its own built-in memory management and good facilities for calling and cooperating with other programs. So I dived into Programming Python with one question uppermost in my mind: what has this got that Perl does not?

Perl, of course, is the 800-pound gorilla of modern scripting languages. It has largely replaced shell as the scripting language of choice for system administrators, thanks partly to its comprehensive set of UNIX library and system calls, and partly to the huge collection of Perl modules built by a very active Perl community. The language is commonly estimated to be the CGI language behind about 85% of the “live” content on the Net. Larry Wall, its creator, is rightly considered one of the most important leaders in the Open Source community, and often ranks third behind Linus Torvalds and Richard Stallman in the current pantheon of hacker demigods.

At that time, I had used Perl for a number of small projects. I’d found it quite powerful, even if the syntax and some other aspects of the language seemed rather ad hoc and prone to bite one if not used with care. It seemed to me that Python would have quite a hill to climb as yet another scripting language, so as I read, I looked first for what seemed to set it apart from Perl.

I immediately tripped over the first odd feature of Python that everyone notices: the fact that whitespace (indentation) is actually significant in the language syntax. The language has no analog of the C and Perl brace syntax; instead, changes in indentation delimit statement groups. And, like most hackers on first realizing this fact, I recoiled in reflexive disgust.

I am just barely old enough to have programmed in batch FORTRAN for a few months back in the 1970s. Most hackers aren’t these days, but somehow our culture seems to have retained a pretty accurate folk memory of how nasty those old-style fixed-field languages were. Indeed, the term “free format”, used back then to describe the newer style of token-oriented syntax in Pascal and C, has almost been forgotten; all languages have been designed that way for decades now. Or almost all, anyway. It’s hard to blame anyone, on seeing this Python feature, for initially reacting as though they had unexpectedly stepped in a steaming pile of dinosaur dung.

That’s certainly how I felt. I skimmed through the rest of the language description without much interest. I didn’t see much else to recommend Python, except maybe that the syntax seemed rather cleaner than Perl’s and the facilities for doing basic GUI elements like buttons and menus looked fairly good.

I put the book back on the shelf, making a mental note that I should code some kind of small GUI-centered project in Python sometime, just to make sure I really understood the language. But I didn’t believe what I’d seen would ever compete effectively with Perl.

A lot of other things conspired to keep that note way down on my priority list for many months. The rest of 1997 was eventful for me; it was, among other things, the year I wrote and published the original version of “The Cathedral and the Bazaar”. But I did find time to write several Perl programs, including two of significant size and complexity. One of them, keeper, is the assistant still used to file incoming submissions at the Metalab software archive. It generates the web pages you see at metalab.unc.edu/pub/Linux/!INDEX.html. The other, anthologize, was used to automatically generate the PostScript for the sixth edition of Linux from the Linux Documentation Project’s archive of HOWTOs. Both programs are available at Metalab.

Writing these programs left me progressively less satisfied with Perl. Larger project size seemed to magnify some of Perl’s annoyances into serious, continuing problems. The syntax that had seemed merely eccentric at a hundred lines began to seem like a nigh-impenetrable hedge of thorns at a thousand. “More than one way to do it” lent flavor and expressiveness at a small scale, but made it significantly harder to maintain consistent style across a wider code base. And many of the features that were later patched into Perl to address the complexity-control needs of bigger programs (objects, lexical scoping, “use strict”, etc.) had a fragile, jerry-rigged feel about them.

These problems combined to make large volumes of Perl code seem unreasonably difficult to read and grasp as a whole after only a few days’ absence. Also, I found I was spending more and more time wrestling with artifacts of the language rather than my application problems. And, most damning of all, the resulting code was ugly—this matters. Ugly programs are like ugly suspension bridges: they’re much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code.

With a baseline of two dozen languages under my belt, I could detect all the telltale signs of a language design that had been pushed to the edge of its functional envelope. By mid-1997, I was thinking “there has to be a better way” and began casting about for a more elegant scripting language.

One course I did not consider was going back to C as a default language. The days when it made sense to do your own memory management in a new program are long over, outside of a few specialty areas like kernel hacking, scientific computing and 3-D graphics—places where you absolutely must get maximum speed and tight control of memory usage, because you need to push the hardware as hard as possible.

For most other situations, accepting the debugging overhead of buffer overruns, pointer-aliasing problems, malloc/free memory leaks and all the other associated ills is just crazy on today’s machines. Far better to trade a few cycles and a few kilobytes of memory for the overhead of a scripting language’s memory manager and economize on far more valuable human time. Indeed, the advantages of this strategy are precisely what has driven the explosive growth of Perl since the mid-1990s.

I flirted with Tcl, only to discover quickly that it scales up even more poorly than Perl. Old LISPer that I am, I also looked at various current dialects of Lisp and Scheme—but, as is historically usual for Lisp, lots of clever design was rendered almost useless by scanty or nonexistent documentation, incomplete access to POSIX/UNIX facilities, and a small but nevertheless deeply fragmented user community. Perl’s popularity is not an accident; most of its competitors are either worse than Perl for large projects or somehow nowhere near as useful as their theoretically superior designs ought to make them.

My second look at Python was almost as accidental as my first. In October 1997, a series of questions on the fetchmail-friends mailing list made it clear that end users were having increasing trouble generating configuration files for my fetchmail utility. The file uses a simple, classically UNIX free-format syntax, but can become forbiddingly complicated when a user has POP3 and IMAP accounts at multiple sites. As an example, see Listing 1 for a somewhat simplified version of mine.

Listing 1

I decided to attack the problem by writing an end-user-friendly configuration editor, fetchmailconf. The design objective of fetchmailconf was clear: to completely hide the control file syntax behind a fashionable, ergonomically correct GUI interface replete with selection buttons, slider bars and fill-out forms.

The thought of implementing this in Perl did not thrill me. I had seen GUI code in Perl, and it was a spiky mixture of Perl and Tcl that looked even uglier than my own pure-Perl code. It was at this point I remembered the bit I had set more than six months earlier. This could be an opportunity to get some hands-on experience with Python.

Of course, this brought me face to face once again with Python’s pons asinorum, the significance of whitespace. This time, however, I charged ahead and roughed out some code for a handful of sample GUI elements. Oddly enough, Python’s use of whitespace stopped feeling unnatural after about twenty minutes. I just indented code, pretty much as I would have done in a C program anyway, and it worked.

That was my first surprise. My second came a couple of hours into the project, when I noticed (allowing for pauses needed to look up new features in Programming Python) I was generating working code nearly as fast as I could type. When I realized this, I was quite startled. An important measure of effort in coding is the frequency with which you write something that doesn’t actually match your mental representation of the problem, and have to backtrack on realizing that what you just typed won’t actually tell the language to do what you’re thinking. An important measure of good language design is how rapidly the percentage of missteps of this kind falls as you gain experience with the language.

When you’re writing working code nearly as fast as you can type and your misstep rate is near zero, it generally means you’ve achieved mastery of the language. But that didn’t make sense, because it was still day one and I was regularly pausing to look up new language and library features!

This was my first clue that, in Python, I was actually dealing with an exceptionally good design. Most languages have so much friction and awkwardness built into their design that you learn most of their feature set long before your misstep rate drops anywhere near zero. Python was the first general-purpose language I’d ever used that reversed this process.

Not that it took me very long to learn the feature set. I wrote a working, usable fetchmailconf, with GUI, in six working days, of which perhaps the equivalent of two days were spent learning Python itself. This reflects another useful property of the language: it is compact—you can hold its entire feature set (and at least a concept index of its libraries) in your head. C is a famously compact language. Perl is notoriously not; one of the things the notion “There’s more than one way to do it!” costs Perl is the possibility of compactness.

But my most dramatic moment of discovery lay ahead. My design had a problem: I could easily generate configuration files from the user’s GUI actions, but editing them was a much harder problem. Or, rather, reading them into an editable form was a problem.

The parser for fetchmail’s configuration file syntax is rather elaborate. It’s actually written in YACC and Lex, two classic UNIX tools for generating language-parsing code in C. In order for fetchmailconf to be able to edit existing configuration files, I thought it would have to replicate that elaborate parser in Python. I was very reluctant to do this, partly because of the amount of work involved and partly because I wasn’t sure how to ascertain that two parsers in two different languages accept the same. The last thing I needed was the extra labor of keeping the two parsers in synchronization as the configuration language evolved!

This problem stumped me for a while. Then I had an inspiration: I’d let fetchmailconf use fetchmail’s own parser! I added a —configdump option to fetchmail that would parse .fetchmailrc and dump the result to standard output in the format of a Python initializer. For the file above, the result would look roughly like Listing 2 (to save space, some data not relevant to the example is omitted).

Listing 2

Python could then evaluate the fetchmail —configdump output and have the configuration available as the value of the variable “fetchmail”.

This wasn’t quite the last step in the dance. What I really wanted wasn’t just for fetchmailconf to have the existing configuration, but to turn it into a linked tree of live objects. There would be three kinds of objects in this tree: Configuration (the top-level object representing the entire configuration), Site (representing one of the sites to be polled) and User (representing user data attached to a site). The example file describes five site objects, each with one user object attached to it.

I had already designed and written the three object classes (that’s what took four days, most of it spent getting the layout of the widgets just right). Each had a method that caused it to pop up a GUI edit panel to modify its instance data. My last remaining problem was somehow to transform the dead data in this Python initializer into live objects.

I considered writing code that would explicitly know about the structure of all three classes and use that knowledge to grovel through the initializer creating matching objects, but rejected that idea because new class members were likely to be added over time as the configuration language grew new features. If I wrote the object-creation code in the obvious way, it would be fragile and tend to fall out of sync when either the class definitions or the initializer structure changed.

What I really wanted was code that would analyze the shape and members of the initializer, query the class definitions themselves about their members, and then adjust itself to impedance-match the two sets.

This kind of thing is called metaclass hacking and is generally considered fearsomely esoteric—deep black magic. Most object-oriented languages don’t support it at all; in those that do (Perl being one), it tends to be a complicated and fragile undertaking. I had been impressed by Python’s low coefficient of friction so far, but here was a real test. How hard would I have to wrestle with the language to get it to do this? I knew from previous experience that the bout was likely to be painful, even assuming I won, but I dived into the book and read up on Python’s metaclass facilities. The resulting function is shown in Listing 3, and the code that calls it is in Listing 4.

Listing 3

Listing 4

That doesn’t look too bad for deep black magic, does it? Thirty-two lines, counting comments. Just from knowing what I’ve said about the class structure, the calling code is even readable. But the size of this code isn’t the real shocker. Brace yourself: this code only took me about ninety minutes to write—and it worked correctly the first time I ran it.

To say I was astonished would have been positively wallowing in understatement. It’s remarkable enough when implementations of simple techniques work exactly as expected the first time; but my first metaclass hack in a new language, six days from a cold standing start? Even if we stipulate that I am a fairly talented hacker, this is an amazing testament to Python’s clarity and elegance of design.

There was simply no way I could have pulled off a coup like this in Perl, even with my vastly greater experience level in that language. It was at this point I realized I was probably leaving Perl behind.

This was my most dramatic Python moment. But, when all is said and done, it was just a clever hack. The long-term usefulness of a language comes not in its ability to support clever hacks, but from how well and how unobtrusively it supports the day-to-day work of programming. The day-to-day work of programming consists not of writing new programs, but mostly reading and modifying existing ones.

So the real punchline of the story is this: weeks and months after writing fetchmailconf, I could still read the fetchmailconf code and grok what it was doing without serious mental effort. And the true reason I no longer write Perl for anything but tiny projects is that was never true when I was writing large masses of Perl code. I fear the prospect of ever having to modify keeper or anthologize again—but fetchmailconf gives me no qualms at all.

Perl still has its uses. For tiny projects (100 lines or fewer) that involve a lot of text pattern matching, I am still more likely to tinker up a Perl-regexp-based solution than to reach for Python. For good recent examples of such things, see the timeseries and growthplot scripts in the fetchmail distribution. Actually, these are much like the things Perl did in its original role as a sort of combination awk/sed/grep/sh, before it had functions and direct access to the operating system API. For anything larger or more complex, I have come to prefer the subtle virtues of Python—and I think you will, too.

Resources

All listings referred to in this article are available by anonymous download in the file ftp.linuxjournal.com/pub/lj/listings/issue73/3882.tgz.

Why Python? Eric Raymond is a Linux advocate and the author of The Cathedral & The Bazaar . He can be reached via e-mail at (esr@thyrsus.com).

spike

about

supaiku dot com §

about §

aesthetics

supaiku dot com §

aesthetics §

websites §

anime reviews

supaiku dot com §

anime reviews §

10/10 §

9/10 §

Horimiya §

Pale Cocoon §

Sonny Boy §

Bakemonogatari §

Byousoku 5 Centimeter §

Katanagatari §

archive

supaiku dot com §

archive §

Staring Into The Singularity

STARING INTO THE SINGULARITY §

The End of History §

The Beyondness of the Singularity §

The Definition of Smartness §

Perceptual Transcends §

Great Big Numbers §

Smarter Than We Are §

Sooner Than You Think §

Uploading §

The Interim Meaning of Life §

Yudkowsky’s Threats: §

Getting to the Singularity §

why-python_listing1

why-python_listing2

why-python_listing3

why-python_listing4

Why Python?

Why Python? §

Software §

Resources §

supaiku dot com

about

supaiku dot com

aesthetics

websites

supaiku dot com

anime reviews

10/10

9/10

Horimiya

Pale Cocoon

Sonny Boy

Bakemonogatari

Byousoku 5 Centimeter

Katanagatari

supaiku dot com

archive

STARING INTO THE SINGULARITY

The End of History

The Beyondness of the Singularity

The Definition of Smartness

Perceptual Transcends

Great Big Numbers

Smarter Than We Are

Sooner Than You Think

Uploading

The Interim Meaning of Life

Yudkowsky’s Threats:

Getting to the Singularity

Why Python?

Software

Resources