Spending Your Triangles

by Rudy Rucker

September 7, 2001

Recently I’d been worrying that e-business — whether booming or busting — might eat up all available mind share for thinking about computers. Even though the e-bust has come, the media continues to slobber over the same trivial, dead-end, greed-headed stuff. Must e-tailing and networking forever dispossess such wonderful aboriginal CS topics as Virtual Reality, Chaos, Fractals and Other Gnarl?

I’ve been less worried about this since last spring, when I went to the Game Developers’ Conference in my native San Jose, California. Everything great about computers is still alive and well in the world of videogames. Here, for your consideration, is my report.

Outside in the park are some homeless San Jose people. Our fair city. A chunky blonde street-girl is chewing an enormous shiny nail, like a ten-penny nail, its head sticking out of her mouth. Two phrases cross my mind: “Tough as nails,” and, “Biting her nails.”

I’m Artificial Life aficionado from way back. Called “Alife” for short, this field studies how to create computer simulations of things that behave like living creatures. In a thorough-going Alife simulation, the creatures will even breed and evolve. Alife was big in the 1980s, but it’s kind of died out. Like Artificial Intelligence, Alife failed to deliver on its initial wild-eyed promises. Simulations don’t in fact evolve into cool things very fast. If you regard Earth as a large, specialized computer, you’ll observe that it’s been running for billions of years, parallel processing itself at every point of space, pumping along at an update speed limited only by things like Planck’s constant and the speed of light. Kind of hard to match that on your desk-top machine.

So I’m excited to see that the conference has a tutorial on Artificial Life in Games. (Info on the talks for this tutorial is at www.jfunge.com/gdc2001.) The game community still hasn’t really picked up on Alife. The tendency is to have games that behave in predictable, replicable ways --- unlike living things. It would be great if Alife could rise out of academia and break into the lively, moneyed world of videogames. Finally an application!

A University of Toronto professor named Demetri Terzopoulos gives a talk on an Artificial Fishtank he made; it’s a virtual world populated by simfish, or simulated fish. The program isn’t really a game, or if it is a game, it’s a zero-player game, meaning that it’s just something you look at. Nor is Terzopoulos’s program a product you can buy, it’s only been presented in museums and big conference demos. He shows us a slide of a Japanese girl in big shutter glasses inside a portable Virtual Theater peering at his simfish. Ah, the eternally cyberpunk quality of the Japanese. But I digress.

Although the simfish aren’t all that important in and of themselves, lets talk about them for a bit, by way of getting somewhat up to speed on the general principles of how one generates virtual realities for use in games.

Like many virtual critters, the simfish are based on skeletons something like a few wire squares and triangles hooked together. These skeletal shapes each have lump masses at their corners, and their edges are “viscoelastic elements,” which are like springs coupled with dampers. As well as acting like bones, the edges act like muscles.

To make the critters look good, the flat faces of their skeletal squares and triangles are replaced by smooth mathematical surfaces, like car fenders, say. One of the most commonly used computer-graphical surface is in fact named a Bezier patch, after a Monsieur Bezier who designed fenders for Renault in the Fifties.

How do you draw a smooth surface? Well, you tessellate it, which means you break it into lots of small triangles. Tessellation is a theme that comes up over and over in the conference. Basically it’s all about triangles, all of the time. Why not squares? The graphics cards like triangles better. When a card draws a 3D object, it moves all the triangles into position in a virtual 3D space, calculates how they would project onto your computer screen, and then colors the triangles in, taking into account any lights or fog that you may have placed into your virtual scene.

So when you look at a simulation of a 3D object in a videogame, you’re actually looking at a mesh of triangles that are artfully filled in with colors. The colors within any single triangle can vary from corner to corner and across the triangle’s face, so that it can become virtually impossible to tell where the individual triangle borders are. The process of turning a model into a screen image is known as rendering, an odd word, really, given that “to render” also refers to the process of melting the fat out of animal carcasses skeletons by heating them.

A computer can only handle so many triangles per second, and if your simulation runs at slower than something like thirty updates a second, it sucks. A too-slow simulation looks jerky and clunky. So you need to keep the number of triangles down to the bare minimum needed to make something look nice. Thanks the ability to color the triangles in artful ways, you can get by with fewer triangles than you might imagine. An appropriately shaded icosahedron of twenty triangles, for instance, can look almost like a sphere. If you use something called Phong shading instead of Gouraud shading, you can even make a cube look like a sphere. But this is more than you want to know.

As well as the graphical appearance of the simfish, we also have to worry about their behavior, which comes down to sensing, thinking, and acting. This is where AI comes in. Whatever compute time a game doesn’t spend on its triangles, it spends on its critters’ Artificial Intelligence. You share your energies between creating your world and thinking about it.

In order to think, of course, a critter needs to know what’s going on around it. To speed things up, you can let them cheat and look up the other critters’ positions in an “oracle” that is simply the program’s data. Or they can do it the hard way, like humans do, and ray-trace lines into the world and see what the lines run into.

Each of Terzopoulos’s simfish has an AI mind based on mental states called Hunger, Libido, and Fear. A simfish fears collisions. predators, and above all the walls of the tank. The fear of a wall is absolute, deeper than any emotion, the simfish can’t overcome it. What if “fear” was all that really did keep you from walking through a wall, what if one’s impression of the world’s solidity was just a weird kind of innate behavior? Looking at these simulated worlds sets the mind off down odd pathways indeed.

Figuring out the AI for your game creatures is a big deal. Given that the creatures have to update thirty times a second, the AI has to be fast, though you can in fact let a critter think a little slower than it moves. Like maybe he only looks around and thinks after every three graphics updates, and if he sometimes sticks part of a finger inside of a wall, who’s looking that closely. Whatever it takes to stay over thirty frames per second.

Terzopoulos shows us fish mating, chasing each other, running away and so on. Rather than actually writing the code for their AI, he let the behaviors evolve over time by the genetic operators of reproduction, mutation, and selection, which is what Alife researchers like to do. “To hell with programming, we’ll let the answer evolve!” Over human-scaled periods of time, this doesn’t actually work very well on real-world problems. But its good enough for little toy worlds like the simulated aquarium. The fish move pretty good. The scene that sticks in my mind the most is a demo of virtual fishing, where a hook hauls a simulated fish out of sight. Imagine the horror of this for the simfish!

During a break in the talks, I chat with a guy named John Nagel who happens to be sitting next to me. He’s one of the founders of Autodesk, Inc., where I worked for a few years in the Nineties. Nagel is a genius and an eccentric, loaded with interesting, skewed ideas. He remarks that the main thing that makes money is the marketing, not the tech, that’s why cool things aren’t emerging as fast as they could. He comes up with a great bon mot regarding why we are working so hard these days: “Better technology helps workers about as much as better weapons help soldiers in a war.” The new tech just makes it worse for the workers, it spews out more crud for them to deal with. The generals love the new weapons, but all they do for the soldiers is kill more of them. You know that nostalgic, wistful feeling you get when you look at a Forties movie and nobody is using a lousy, stinking computer? Must have been nice.

The next day I ambitiously start in on an all-day tutorial on how to take advantage of accelerated graphics cards for your 3D rendering. Some guys from the Nvidia graphics card company are explaining how to use the special 3D graphics protocols known as OpenGL, formerly the property of Silicon Graphics but now a lingua franca across all kinds of platforms including the great King Kong of Windows. The Nvidia guys are, it turns out, not talking about OpenGL in general so much as they are talking about some special new OpenGL functions that are only going to work on their new $600 graphics card called GeForce 3. They show us a cripplingly complex demo, an animated chameleon who changes from chrome to glass to colors while crawling along a branch. “After what we tell you today there’s no reason you can’t write a demo just like this,” says the introducer. Rrright. I look over at the twenty-something graphics hacker next to me. We exchange grins like students in a class that’s harder than we expected.

The first speaker talks about how to go about deleting more and more vertices of an object’s mesh as it gets farther away --- so as to not be wasting compute time on unnecessary detail. He talks about “not spending too many triangles.” He uses the acronym LOD, for Level of Detail. The problem that exercises this guy is how to dynamically change the LOD tessellation without what he calls “popping,” which would be an unrealistic-looking abrupt change when a ball, say, goes from using twenty triangles to using two hundred triangles. He’s spewing out primo buzzwords here. “More highly tessellated.” “Water-tight tessellation.” It’s very close to gibberish, with eighty percent of the words technical. “Polynomial patches, vertex shading, alpha blending, shadow buffers, bump mapping.” “T&L” for “transform and lighting.”

The speaker shows a demo that zooms in on a blue glass banana slug shape, the lighting is continuous, and the image isn’t “popping” because extra triangles are invisibly seeping out of seams in the slug when you make it bigger. “Isn’t that great?” he says.

I’m boggled by the intricacy of the gyrations we are forever going through to make our simulations run fast. This will never end, I suddenly realize. The real world is inexhaustible. More computational ability just makes us do harder simulations. It’s like the thing Nagel said. More guns, and the soldiers are more desperate than ever. Enough OpenGL for today.

I check out a tutorial on “Interactive Storytelling.” How do you tell a story in a game? A screenplay is totally different from a story, it’s all about showing instead of telling. But a game, it’s not even about showing. It’s about letting people find stuff. And somehow you have to herd the gamer along a dramatic trajectory. How to do this is a mind-boggling question. But the speaker doesn’t know the answer. The audience is the most interesting thing in this session, they’re not at all the same crowd as in the Advanced OpenGL session. I begin to grasp that the game developer community is a veritable university, with designers, programmers, writers, artists, businessmen and marketers.

I find some of the artists at a “Conceptual Design” session I wander into. The speaker here is an artist, who illustrates his talk with detailed marker drawings that he does on a sheet of paper that sits beneath some TV cameras, a high tech overhead projection set up. Long periods of silence while he draws. How wonderfully realistic his hands look. He is explaining how to draw shiny things so they look cool. It’s great. The audience is even less like programmers than the writers were. Inarticulate artists, one asks a question like “Why is something that of sticks up kind of shiny? Why is it dark at one edge?” I learn a lot. There’s an interesting reversal in this talk. Rather than focusing on how to draw a 2D picture that looks like a 3D object, the guy is really talking about how to deform the meshes of a 3D object so that the 2D image in the game rendering will look cool. He talks, for instance, about putting a pooched-out “bone” on a surface to make a nice reflection line. He uses the word “pooch” a lot.

Will Wright, the head of Maxis, gives a talk called “Design Plunder.” It’s in the civic center, a huge crowd is there. Wright is the designer of SimCity and recently The Sims, which is kind of like a live doll-house with humanoid Sims you move around and do things to. The Sims do things on their own as well, you can sit back and watch a situation play itself out.

Will --- somehow you can’t call this guy by his last name --- gives a great talk. First thought on seeing him come out: what a geek. Hawaiian shirt, a Charlie Chaplin mustache, skin so bad you can tell from thirty rows back, lank dirty hair. There’s a big screen behind him that shows his head and screens on either side showing slides of his Power Point slides. I should mention that everyone but everyone uses Power Point nowadays, slides that are inside their portable computer and which come out on the video projector.

Will talks about Christopher Alexander’s book, A Pattern Language, a chunky old $60 tome from Oxford University Press. Everyone keeps hyping this book to me, I gotta check it out. It’s one of the inspirations for the biggest new buzz in the software engineering community: Software Patterns. Will talks about the patterns of Hierarchy, Network, Landscape. He has a good line about makes a good user interface. “A user interface isn’t done until there’s nothing left to remove.”

As a complete non-sequitur he throws in a slide of some woolly animal and says, “The Vicuna is a relative of the Llama.” Inside joke: Wright’s business card calls him “High Llama,” as in Dali Lama.

He starts showing some of the “stories” people have made up with their Sims. Like kids having stories about their Barbies or G. I. Joes, except now they’re in computer form. There are sites where you can get “skins” to make your Sims look however you like, e.g. you can dress them in bondage outfits or in spring break ski vacation outfits. Will shows a story where some kids are in a ski lodge and one of them dies and they hide the body inside a snowman. He dreams of having computer software to recognize a developing story and help along by, perhaps, putting in obstacles to the goal so that the story gets more complex.

It’s all good, but after awhile I can’t listen to any more talks. I go to see a demo reel of some of the best visuals from this year’s games. A Chinese girl in a grotty tenement. It’s high time for the Chinese to be cyberpunk like the Japanese. Forget about all the historical stuff, get with the Western program, yes! It’ll come. Now the reel shows gothic devils by a lake of lava. A man in a top hat, ah, the wonderful sinister quality of a top hat. Creatures with three legs, I notice a number of these in different people’s games. “I dare to dream of three legs!” Laser beams with hoops of emphasis around the beam. A cartoon world with a woman who gets out of a coffin, like Sleeping Beauty or Snow White, and she has the most extreme, voluptuous figure I’ve ever seen, sexier even than Jessica Rabbit’s. Then the reel shows a hooded man in the rain, it’s a Japanese game with long credit sequence like a Noir movie. The game is called Metal Gear Solid, terrific Japanese-style name, the way they always get the words in slightly the wrong order. The reel shows a world called Exmachina with cool funky dirigibles and a screaming fat woman with blue pig-tails. “I dare to dream of blue pig-tails!”

There’s also a reel of demos from a European movement called the “Demo Scene,” with more info at www.scene.org/dog. These are small executables that produce images and sounds. They try for 64K exe size. Seem to be written in BASIC, my dear. Shocking. They’re like loops you’d see in a European disco. The programmers have names like KKowboy, The Popsy Team, and Byter. The demos are weak, but it’s always great to see high-tech stuff get out on the street.

Time to hit the Expo Hall. First thing I notice is that most booths are giving away toys. The developers call it “schwag.” There’s Slinkies, clackers, Hackey-sacks and, ah, Silly Putty. I get five green Silly Putties from Nvidia. I’ve always wanted enough Silly Putties to completely fill up the plastic egg it comes in. My egg is so full it bounces when I drop it. Tactile feed-back.

There’s some pretty odd game add-on equipment in the lesser-frequented booths. Plastic sheets to lay over your keyboard in case you can’t remember the fifty or a hundred possible key commands. A quarter-dome of white cloth with colored lights inside it; the lights supposedly flash in synch with the events of your game, like red for an explosion or yellow when you shoot your gun. I’m already overloading on the shooting all around me. Your gun, your gun, your gun.

There’s virtual tactile-feed-back to be had. A company has a joystick with a motor in its base so it can push back. The screen shows a ball on a trampoline. As you wiggle the stick you feel the trampoline sag, give, then pitch the ball up. And the stick slams back when the ball comes down.

The worst product of all is from digiSCENTS(TM) with the iSmell(TM) technology, a little humidifier-like thing that sits by your computer and pulses out a waft of scent in synch with your game. They have the most impressive “booth-bunnies” in the whole hall, sexy women in skunk suits. But it’s not enough to make any rational developer or gamer want to touch this product. So far as I can tell, the product hasn’t actually been synched to any real game, I think they’re still looking for more funding. A guy gives me a demo where you’re playing Doom. The shot-gun blasts smell like, he claims, daffodil, the extra bullets like wintergreen, the enemies like butterscotch, but really all you feel is the wind of the puffs of air. Surely this guy must know that his company is doomed.

I have a momentary wave of revulsion. Virtual Reality is alive and well here, but it’s being used for such crappy purposes. It’s like having a million dollar synthesizer and playing Whitney Houston songs. One guy is demoing a design program in which he’s produced a best called Bubba. Bubba has eighteen thousand triangles and has surfaces made up of these very cool mathematical functions called NURBs. But Bubba’s a completely cruddy and moronic looking monster, like Disney at his generic worst. NURBs and eighteen thousand triangles to be just as stupid as ever.

I calm down by watching a Microsoft demo of the software compiler they’re calling VisualStudio.NET, also known as Visual Studio 7.0. To the palpable relief of the programmers around me, version 7.0 looks very much like the version 6.0 that we’ve all been using for the last two years. One never knows when Microsoft is going to choose to work you over with their latest Brave New World of compatibility issues.

Above the Microsoft demo area is a giant poster, a banner really, of a guy with a nose ring and a Maltese Cross piercing in his tongue, his mouth open screaming, this is for their DirectX software library. How odd to think that this is how one of the world’s largest companies sells tools to serious programmers! How far we’ve come from the suit-and-tie company-men of the 1950s.

I cruise the Expo Hall a lot more over the coming days and I begin to have more and more fun. I watch some developers playing the demo games set up. One is a Japanese game called Jet Grind Radio about skater painting graffiti. Amazingly antisocial. I talk to the guy playing with it. “I like how they make it look like cels,” he says. “Each figure has a thick dark line around it like in a cartoon.” Another game being played is Samba Amigo, with an interface that is, yes, a pair of maracas. “That’s the most brain-dead game I’ve ever seen,” I say to a developer. “Yeah, but it’s awesome,” he said. “I’ve been playing it a lot.” The game is to shake the maracas in patterns indicated by circles that have dots appearing in them, you follow the dots. In the background is an endless procession of colorful shapes, like a three-day ecstasy trip or something, hot-dogs in serapes, grinning amigos, cute computer-graphics girls with game-world bodies.

I meet some Irish guys from a company called Havok who have a physics package for games, it basically solves spring equations and the like in real-time so that you can have bouncing hair, flapping cloth, and spinning rocks with accurate collisions. This used to be supercomputer Virtual Reality, and now it’s a plug-in package for game developers. They’re asking a pretty penny, though, $75K for the full game developer’s kit. Oddly enough, Havok’s biggest competitor is a company called Karma out of Oxford University. Back in the Old World, they really teach students something.

Sony is there with a pen full of Aibos, their robot dog. I reach in and snap my fingers, an Aibo comes over and sniffs me, I pet its head, it sits back on its haunches and whines, I’m in love. A Japanese programmer shows me something that looks like a videocassette with little levers in its sides. In his broken English he is giving me to understand that this cassette-sized box is the inner hardware of the Aibo, and that I could develop my own shell to put onto the box, Sony is looking to license to developers. I have a flash of a world in which all the creatures and people I interact with are in fact armatures of triangle meshes tacked onto these Sony boxes. Someday the meshes disappear, and my office-mate at school is revealed to be a black box with levers sticking out of it. The triangles are scattered across our office floor. “Are you Jon Pearce?” I say to the box, and the lever in front goes up and down nodding yes.

I keep walking around the Expo hall, more and more into it. I’m better able to see things now, with familiarity it’s less of an overwhelming jangle. One thing I totally notice is that they have some women dressed in black up on stages dancing, two different stages. Each women has reflective beads attached to her cat-suit, maybe fifty of them. Around the stage are computer monitors showing realtime moving wireframe models of the women. The developers are interested in this, both in the dancing women and in the moving wireframe models. They hardly know which to stare at the most.

I listen to the presentation at the Vicom Motion Capture stage. Around the stage are eighteen megapixel digital video cameras shooting 25 frames per second. The dancer is Megan. She has dark lips, a perky smile, a messy pinned-up ponytail that’s in the wireframe models as well. She yawns, dances, poses while the pitchman talks. She’s as ceaselessly active as the tendrils of a sea anemone. She leans, the epitome of grace, on the partition separating the stage from the pit where two programmers sit running programs to clothe her wireframe bod in rendered triangles. She has one arm akimbo. What a gulf between this live California girl and the male programmers thinking about how best to “spend their triangles” on her rendering. Are there any women game developers?

At the tail end of the conference, I catch a talk by Michael Abrash, who’s working on the Microsoft Xbox, a ballyhooed new gaming platform on the horizon. It has Nvidia graphics hardware. Abrash has been testing it for a year. He’s a super-programmer, the co-author of the classic first-person shooter game Quake. The hall is filled shoulder-to-shoulder with hardcore techie game developers, maybe a thousand of them, there’s not a single woman in view, not so much one single triangle of femininity as far as I can see. Abrash lets loose like a fire hose. A complete geek info-spew. The Xbox is to deliver 125 million triangles per second! All this to draw Megan’s arm akimbo. After his talk Abrash is besieged by questioners, they’re like dogs fighting over a piece of meat, which is Abrash’s brain. Being under a Microsoft Non-Disclosure Agreement --- and you can imagine what that must be like --- he can’t give them as much as he’d like to.

As it turns out, I’m having dinner with Abrash, along with two of his Microsoft cohorts. They want to pick my brain about wild computer-science ideas for video games.

On the way to meet them at the restaurant I stop in at St. Joseph’s cathedral. A humble party of working-class San Joseans is gathered there, one of the church officials is prepping them for a wedding they’re going to have at noon tomorrow. The richness of this space, the murals, the dimensionality. The grains of the wood and the marble. The humanity of the people in the wedding party. Will the geeks of a hundred years from now be volumetrically modeling wood and character animating better sims of people? Why, why, why?

At dinner, Abrash is brilliant and intense, a man looking for another big score. I make some suggestions about videogame things I’d like to see. Having just finished writing a novel about the fourth dimension, I’m particularly eager to see a four-dimensional videogame. The glass screen of your computer could as easily look onto a simulation of hyperspace as onto a simulation of regular space. Abrash is resisting this, though, he’s more attracted by the siren song of Cellular Automata, which are a wondrously gnarly precursor of Artificial Life. I happen to have some opinions about this too; it’s great to be talking to someone who might actually do something with them. An undulating surfscape made of continuous-valued Cellular Automata --- now that would be worth spending your triangles on!

All in all, the Game Developers Conference was a vastly energizing experience, like a brief immesion in a floating university. These guys totally get the old-time hot-rodding aspect of what computers are for. They’re not for delivering groceries, for God’s sake. They’re for speeding like hell to places nobody’s ever seen.