Pipes 161 programs to be built out of simpler programs. Pipes allow programs to be used in unplanned and unanticipated ways. Pipes allow simple implementations.” Unfortunately, chanting mantras doesn’t do Unix any more good than it does the Hari Krishnas. Pipes do have some virtue. The construction of complex systems requires modularity and abstraction. This truth is a catechism of computer science. The better tools one has for composing larger systems from smaller sys- tems, the more likely a successful and maintainable outcome. Pipes are a structuring tool, and, as such, have value. Here is a sample pipeline:3 egrep '^To:|^Cc:' /var/spool/mail/$USER | \ cut -c5- | \ awk '{ for (i = 1 i = NF i++) print $i }' | \ sed 's/,//g' | grep -v $USER | sort | uniq Clear, huh? This pipeline looks through the user’s mailbox and determines which mailing lists they are on, (well, almost). Like most pipelines, this one will fail in mysterious ways under certain circumstances. Indeed, while pipes are useful at times, their system of communication between programs—text traveling through standard input and standard out- put—limits their usefulness.4 First, the information flow is only one way. Processes can’t use shell pipelines to communicate bidirectionally. Second, pipes don’t allow any form of abstraction. The receiving and sending pro- cesses must use a stream of bytes. Any object more complex than a byte cannot be sent until the object is first transmuted into a string of bytes that the receiving end knows how to reassemble. This means that you can’t send an object and the code for the class definition necessary to implement the object. You can’t send pointers into another process’s address space. You can’t send file handles or tcp connections or permissions to access particular files or resources. At the risk of sounding like a hopeless dream keeper of the intergalactic space, we submit that the correct model is procedure call (either local or remote) in a language that allows first-class structures (which C gained during its adolescence) and functional composition. 3Thanks to Michael Grant at Sun Microsystems for this example. 4We should note that this discussion of “pipes” is restricted to traditional Unix pipes, the kind that you can create with shell using the vertical bar (|). We’re not talking about named pipes, which are a different beast entirely.

162 csh, pipes, and find Pipes are good for simple hacks, like passing around simple text streams, but not for building robust software. For example, an early paper on pipes showed how a spelling checker could be implemented by piping together several simple programs. It was a tour de force of simplicity, but a horrible way to check the spelling (let alone correct it) of a document. Pipes in shell scripts are optimized for micro-hacking. They give program- mers the ability to kludge up simple solutions that are very fragile. That’s because pipes create dependencies between the two programs: you can’t change the output format of one without changing the input routines of the other. Most programs evolve: first the program’s specifications are envisioned, then the insides of the program are cobbled together, and finally somebody writes the program’s output routines. Pipes arrest this process: as soon as somebody starts throwing a half-baked Unix utility into a pipeline, its out- put specification is frozen, no matter how ambigious, nonstandard, or inef- ficient it might be. Pipes are not the be-all and end-all of program communication. Our favor- ite Unix-loving book had this to say about the Macintosh, which doesn’t have pipes: The Macintosh model, on the other hand, is the exact opposite. The system doesn’t deal with character streams. Data files are extremely high level, usually assuming that they are specific to an application. When was the last time you piped the output of one program to another on a Mac? (Good luck even finding the pipe symbol.) Pro- grams are monolithic, the better to completely understand what you are doing. You don’t take MacFoo and MacBar and hook them together. —From Life with Unix, by Libes and Ressler Yeah, those poor Mac users. They’ve got it so rough. Because they can’t pipe streams of bytes around how are they ever going to paste artwork from their drawing program into their latest memo and have text flow around it? How are they going to transfer a spreadsheet into their memo? And how could such users expect changes to be tracked automatically? They cer- tainly shouldn’t expect to be able to electronically mail this patched- together memo across the country and have it seamlessly read and edited at the other end, and then returned to them unscathed. We can’t imagine how they’ve been transparently using all these programs together for the last 10 years and having them all work, all without pipes.