The Long View
The Long View
Blue
According to an oft-repeated story, there was a meeting at Apple in 1988, at which specifications for new versions of the operating system were divided into two sets, and written on cards. Pink cards contained a description of a future objected-oriented operating system, which later evolved into the Taligent project, which never led to a Macintosh OS. Blue cards contained features to be included in the next version of the Macintosh operating system, that was finally released in May 1991, as System 7.
It would be interesting to see what was written on the blue cards. I think it is safe to guess that there were a lot of things added to the project over the three years or so that it was in development. But I also think it likely that the cards were basically a list of unrelated features, not bound by a single underlying theme or philosophy. System 7 was not intended to be a fundamental revision of the Macintosh OS. It was just a set of features that various people wanted added to the existing OS. The really revolutionary stuff was on the other cards-- the pink ones.
The Macintosh operating system prior to System 7 had also undergone a progressive addition of features. The biggest changes had occurred with the introduction of the 128K ROMs in the Macintosh Plus and 512Ke, but there had been a series of additional features introduced since then, each of which appeared as a set of functions that programmers could call on to do additional things. These new features were explained in new chapters of Inside Macintosh. The features of the 128K ROMS were described in Volume IV, those introduced with the Macintosh II line of machines were in Volume V. System 7 would be documented in a huge new tome, Volume VI. There would never be a Volume VII.
Where does it all go?
Where would all this new code for System 7 be put? One obvious solution would be to make a new version of the Toolbox ROM, as was done at the time of the 512ke/MacPlus release. But this would have rendered all previous Macintoshes unable to run the new operating system. The Macintosh II line was released in March 1987, about the same time as the legendary meeting with the cards. A new ROM for System 7 would have set off a riot among Macintosh II owners. Of course, many Macintoshes were designed for ROM upgrades. Apple must have at least considered the possibility of adding new features to new ROMS and selling (or giving away) ROM upgrades. I don’t know why this was ruled out, but it was probably because the ROM sockets couldn’t handle the address space required for the feature explosion that happened in System 7 and thereafter. It was also at least partly due to the ease of distributing the new code and patches to old code on disk, as a part of the operating distribution. The original Mac operating system consisted of the Toolbox ROM and 2 disk files, System and Finder. System extensions (called INITs because they were loaded during the boot process) and control panel modules (called CDEVs) were added during the System 1-6 progression. But these were intended for particular purposes, and their proliferation had already been identified as a problem (more about this later). The Finder was really just an ordinary program, so it was ruled out. So most of the new code had to go into the System File. The System file is best thought of as a directory rather than a single file because it is a collection of resources. Like menus, strings, and a lot of other kinds of data, Macintosh code was packaged in resources. The Resource Manager could pack a bunch of resources into a file, and they could be read into memory individually when they were needed. There were already a variety of ways to add new code to the System File, and to use it. The Package Manager was the original way to do this. For some reason, several pieces of the original Macintosh API were not on ROM, but were distributed in the System File as Packages. For example the floating point emulation software (called SANE) was a set of subroutines stored in the System File, calls to which would be dispatched by the Package Manager. The Package Manager was part of the ROM Toolbox, and provided a mechanism for loading and executing code loaded in packages. Packages were stored in the System File as resources of type PACK. Another pre-existing mechanism for adding new code was in the form of ROM patches. Calls to routines in the ROM were dispatched using a set of unimplemented 68k instructions. These were intercepted by the system and handed to a trap dispatch table stored in the System Heap. That table had the address (usually in ROM) for the corresponding routine, and the dispatch code would pass control to the requested Toolbox routine. To make fix bugs in the ROM, or to improve performance, or make some other needed modifications, a mechanism was provided to make changes in the trap dispatch table to branch to code located in RAM. Adding new code in RAM, and adding corresponding entries in the trap dispatch table, was a simple method for adding new code to the Macintosh operating system. Most of the new features in System 7 were added as new packages (e.g. the Color Picker, Data Access Manager, Help Manager, Apple Event Manager, Pict Utilities) or as patches (e.g. 32 bit Quickdraw). Others were programs that were loaded into the System Heap (like the DAHandler).
Networking
Networking was new in the 1980‘s, being mostly found only in experimental environments like that at Xerox PARC. The Macintosh operating system was a pioneer in this regard, and had networking for a long time. LocalTalk networking using the RS-422 serial ports and software support through AppleTalk were introduced early in the life of the Macintosh, and was promoted as the basis for the Macintosh Office in 1985. The first Apple laser printer, also introduced in 1985, used AppleTalk networking as its primary form of communication, and was available to all Macs on its network without the need for a print server. Because AppleTalk supported dynamic addressing and resource discovery (way back then), setup was easy and AppleTalk networks required practically no configuration or maintenance. Apple offered file server software, AppleShare, starting in 1987, and System 6 had built in client support for AppleShare in the Chooser. AppleShare could run on nearly any Macintosh, but the server could do no other task while AppleShare was running. Mac users wanted a peer-to-peer system for sharing, and third party file sharing software was starting to appear. An INIT called Public Folder allowed Macintoshes running System 6 to place a single folder on the network for other to see. In the meantime, progress in using local networking had been made in the Unix community, and a number of new uses for networking (like email) were being exploited by them.
System 7 expanded the networking capabilities of the Macintosh. The underlying AppleTalk software was expanded and improved, and several programming interfaces and user-level features were added. The most notable was built-in file sharing on every Macintosh. A file sharing control panel allowed users to activate file sharing and control access by remote users in a way that is not much different from that seen on today’s Mac OS X. The Finder allowed sharing individual directories. A Mac OS X user sitting down to use file sharing on System 7 would know exactly what to do, because Mac OS X uses the basics of the System 7 file sharing user interface. But in addition, the file sharing control panel included another new feature, called program linking. This turned in a mechanism called Program to Program Communication, that allowed programs to establish communication channels and request services from, and offer services to each other. It did not matter whether the programs were on the same Macintosh or were on different ones connected via the local area (AppleTalk) network. Unix programmers reading System 7 documentation would recognize this as an equivalent of services that were being developed and exploited in that operating system at the same time. Of course unix offered no user interface for these things. They were strictly accessed and controlled by text configuration files. System 7 came with a great user interface feature, called the PPC Browser, that allowed programs to prompt users to control the connections to other computers.
Aliases
Aliases seem like simple things. They are files that are placeholders for other files. You want a file to appear at more than one place in the directory tree, but you don’t want two copies of the file. The original Macintosh file system didn’t have a directory tree. It was flat, and had no possible use for anything like that. But starting with the Hierarchical File System on the 512Ke/MacPlus, there was a use for file placeholders, and there was no mechanism. There was a mechanism on most other operating systems at the time, including Windows 3.1 where they were called Shortcuts, and Unix, where they were called links. Shortcuts were an essential part of a nauseating kludge built into Windows 3.1 called the File Manager (no relation to the Macintosh File Manager). The shortcut file contained the commands required to open the file, and so could only be used as a user interface item. Links in Unix are much more versatile, and there are two varieties, called soft and hard. Hard links are a form of cross-listing of the file at the very bottom of the file system. That is, a hard link is the same file as the file linked to. Both refer to the same data in the same way and are equal in the eyes of the file system. If you delete either one of them, it has no effect on the file’s data. You have to delete them both to delete the contents of the file. Soft links (also called symbolic links) in Unix are more like Windoze shortcuts. They just contain information about how to locate the real original file. If you delete the symbolic link it has no effect on the original data. But deleting the file linked to deletes the data and renders the symbolic link file useless.
Macintosh aliases, introduced in System 7 and still in use today, are like symbolic links in unix. They contain information about how to find the file to which they point. If the original file is deleted, the alias will still be there, but will be useless. Apple improved on the Unix design, however. Instead of just storing the location of the linked-to file (or directory) in the file system, Macintosh aliases contain a resource called an alias record. The record contains complete information about the location of the file, including volume information. So, for example, if the volume is not mounted at the time you (or your program) try to use an alias, the alias manager will mount the volume for you if possible (e.g. it is a network shared resource) or prompt you to mount it. Also unlike the Unix symbolic link, if the target file is moved, the alias is automatically updated. This last feature makes Macintosh aliases much more useful than any of their brethren on other operating systems. This accounts for their continued use in OS X, despite that availability of regular unix hard and symbolic links. In OS X, the unix hard link mechanism has been leveraged to help keep track of files when the move, making Macintosh aliases even more resilient to the fact that user move files around in their directory tree.
Users like and use aliases to cross-index information in the file system. But aliases records, the information stored in the alias file, are also heavily used by other parts of the system software, especially the Edition Manager (publish and subscribe) and by interprocess communications via AppleEvents, and the AppleScript system, all of which were also introduced in System 7.
AppleEvents and Applescript
Unix programmers moving to the Macintosh would have recognized the Program to Program Communication system, but they would not have recognized what became the main use for it. AppleEvents offered a method for any program to send a message to any other one, without the necessity of establishing a low-level communication channel. The first user for this was the Finder. Before System 7, the Finder (or any other program) could launch another program, and could leave that program a message about what it wanted done in a small data area associated with its application globals. But once it had started, the launched program was on its own. There was no good general form of communication between parent and child process. Other operating systems used similar launch mechanisms. Starting in System 7, the Finder could launch a program and then direct its activities using a series of AppleEvents. For example, if the user told the Finder to print some document, the Finder could launch the program, send it AppleEvents telling it to print the documents, and then tell the program to Quit. Programs could define their own AppleEvents for specialized services that they offered, and other programs could be designed to take advantage of those services. This was particularly useful for programmers who were creating suites of software expected to work together. Programmers of such programs could design an arbitrary language of AppleEvents by which they could work together. AppleEvents were, in my opinion, the most creative and important System 7 innovation.
In what turned out to be an amazing breakthrough, later versions of System 7 introduced a script language, AppleScript. It was used to control the Finder and other programs, orchestrate data flow between programs and generate entire new user interfaces building on services offered by existing programs. Nothing like this existed in Windows, Unix, or even NeXTSTEP. The last is particularly important, because when Apple bought NeXT, and based it’s new operating system on NeXTSTEP, it was necessary to either abandon AppleScript or somehow port it into the new OS. I’m sure there were those who wanted to deep six AppleScript. Luckily, somehow AppleScript was saved, and both AppleEvents and AppleScript have continued to be supported in almost their original System 7 form in Mac OS X.
Virtual Memory
Virtual Memory, Memory Protection, and Preemptive Multitasking were computer industry mantras of the 1990s. I suspect that most of the people talking about these things, especially in the computer industry press didn’t then and still don’t know what these words mean. You know who you are, so listen up. These things are related but they are not the same, and it is possible (but usually suboptimal) to have one with out the others. The absence of Memory Protection in the Macintosh operating system became a real serious problem in System 7. In a single job operating system, as in the original Macintosh, it makes perfect sense for a program crash to crash the machine. After all, there’s nothing else running that you could hope to survive. Starting with the Multifinder, but especially in System 7, it just didn’t make sense to users that when one program crashed, they would have to reboot their entire computer and lose all unsaved work in all the other programs. If nothing else was running, there was at least the Finder that ought to survive. The only solution to runaway programs is to run each program in its own memory space and to protect all the others (including the Finder and the System Heap) from being clobbered by disallowing writes to their memory.
A second and almost unrelated issue is the restriction imposed by limited physical memory. The ability to swap the contents of some parts of memory to disk when not in use (Virtual Memory) was a luxury that was appreciated by users who couldn’t afford enough memory to use their Macintoshes the way they wanted to. But of course most users were also short of disk space, and they didn’t want the computer to slow down appreciably because of Virtual Memory. Users had unrealistic expectations about what Virtual Memory could do for them.
The Macintosh could load more than one program into memory and switch between them, but it could only do a context change when the running program called WaitNextEvent to collect it’s next instructions from the User or from other programs. Macintosh programs had to call WaitNextEvent pretty often to work right, so this was not really a problem, unless a program hung, or got too busy doing some really intense calculation. One twist was that Toolbox routines didn’t call WaitNextEvent, so a user could basically prevent a context change indefinitely by holding the mouse button down on a menu or control. Aside from that, Preemptive Multitasking by itself wouldn’t have changed the user’s experience on the Macintosh in any really key way. Users and pundits hoped that it might somehow help with a well-known flaw in Finder’s implementation. If the Finder was doing a file copy or other operation, it would basically become unresponsive till the job was done. This problem was solved in System 8 by the threaded Finder, without Preemptive Multitasking.
Everything looks clearer when you get a little distance on it. I don’t know if I would have known what do do in 1987. But looking back at it, I think it is clear that Apple should have worked on Protected Memory. But apparently neither Protected Memory or Virtual Memory were really important issues at Apple during this time. In his brief account of events at Apple during his time as technical lead on the System 7 project, Darin Adler says:
One of the engineers from the Pink group, named Phil Goldman, and his partner, Rick Daley, decided that virtual memory for the Macintosh would be easy. Since no one in system software was willing to work on it, they decided to do it in their spare time. They had a lot of spare time, since their part of the Pink project was the only one that was ahead of schedule -- they were waiting for other people in Pink to catch up. They got virtual memory to the point where it worked -- not ready to ship, but clearly demonstrating that it could be done. I took over virtual memory along with the help of someone named Joe Buczek. I worked on it nonstop for weeks and got it much closer to shippable, and then handed it off to Joe.
Nobody in system software wanted to work on virtual memory? Seriously? That says it all. I guess virtual memory was not on any of the blue cards. Phil Goldman, you remember, was one of the authors of the original Multifinder. He and Rick Daley put the virtual memory system for System 7 together in their spare time while working on Pink. Too bad Darin didn’t ask them to use a little of their spare time to design a protected memory system for System 7.
Anyway, the virtual memory system included in System 7 was a good demonstration for users that virtual memory was not the answer for the things that they didn’t like about the Macintosh. The computer press was flummoxed by it though. They had flayed Apple because it didn’t have Virtual Memory. Now they had to admit that the Macintosh had virtual memory, and Macintosh users could load more programs than they had memory to accommodate. The way it worked was to swap out parts of program’s application heaps so that a program could work in a larger memory partition than there was physical memory to accommodate. Hard disks were slow and space on them was short. Virtual memory in System 7 worked fine and did what it was supposed to do, but it caused a lot of thrashing. My advice if you are running System 7 today is to get a bunch of real memory and keep the virtual stuff turned off. If you are using an emulator, it is already off, because none of them support it.
New Sound Manager
The new Sound Manager was not really new with System 7, but was available starting with system 6.0.7. But most of the software that used it was written for System 7, and it was documented in Inside Macintosh Volume VI. The Macintosh had great sound at the time of its introduction, including speech, but it was provided by a driver with a relatively primitive programming interface. The Sound Manager made creation and playing of sounds on the Macintosh easy for programmers. It included a set of routines for recording sound with an attached or built-in microphone that presented the user with a standard user interface, built-in support for AIFF files and SND resources, and a single function that would play the contents of either one. On Macintoshes with stereo support, stereo recording and playback was automatically available. Sounds and software for recording and playing sound exploded after the release of System 7. Sounds could be copied into the scrapbook, and sound files could be played by double-clicking them in the Finder. Users replaced Apple’s alert sounds with ones of their own choice, and exchanged sounds on internet sites. It was like the current ringtone craze. The ultimate outcome was SoundJam, then iTunes.
Data Access Manager
You can’t win them all. The Data Access Manager is gone and long forgotten. It was an API for accessing databases. It wasn’t that this was a bad idea. It was just that this kind of thing wasn’t destined to be platform-specific. The idea is alive and well, as Open Database Connectivity, which is used by Mac OS X and everybody else.
Palette Manager
Almost all color drawing on computers now is done in ‘true color’, which means specifying 24 bits of color per pixel. If a programmer wants to specify a color, it is done in the same way artists do it, by mixing so much red, so much blue, and so much green (or cyan, magenta, yellow and black). But artists don’t have every color of paint mixed up at the same time. They mix up a few colors, and save them on a palette. At the time of System 7, video boards could represent only 256 colors (8 bits per pixel). Programmers would select 256 colors that they wanted to use, and assign them to locations in a table with 256 entries. Then, the colors would be designated by their index in that table, and each pixel on the screen could be specified by one byte. Actually, 256 colors are not bad, if you select them carefully, and users didn’t really want for colors. But if your program mapped all 256 colors, then all the other program on screen got recolored using your look-up table. If you were writing a gray-scale photography program using 256 shades of gray, the Finder and everything else turned gray when your program came to the front. The Palette Manager was an API that allowed you to code your program’s use of color in a way that had minimum impact on the multi-program environment of the Macintosh’s screen. If you could get by with say 64 colors, you could put them in a palette, and leave the rest for the system to use for everything else. You specified colors by their entry in the palette. The idea of a color palette was familiar to programmers at the time, who were used to working in a limited color environment (many systems used 16 colors) and Apple’s implementation was excellent. Its utility disappeared when 24 bit color systems became common.
Worldwide Software
Apple had long been interested in localization. Even early on, there were versions of the Macintosh operating system localized for a variety of languages. Even before System 7, there were specialized versions of the Macintosh operating system that used diverse character sets, including Arabic, Japanese, Chinese, Hebrew and Cyrilic, but these were put together in an ad hoc fashion. The Script management system in System 7 unified all the aspects of software localization. It produced a system of creating and installing new scripts (meaning character sets), ways of writing numbers and dates, keyboard layouts, and calendars. If you are wondering why Apple didn’t just use Unicode, remember that System 7 was developed between 1987 and 1991, and the first draft of the Unicode standard was released in 1990. Apple led the way in software localization, and the Worldwide software system introduced as a part of System 7 continues to live as an important part of Mac OS X, as a framework that includes Unicode support.
Publish and Subscribe
I’ve already gone on enough about this, but I can’t seem to get beyond the second stage of grief about losing the services of the Edition Manager. Briefly, this System 7 innovation made it possible for a document to publish a selection in one of its documents,called an Edition, or to embed an Edition made by another program. These were references to graphic, text, or other (e.g. sound) elements of documents, rather than the data themselves. Users could change the Edition using the program that created it (the Publisher) and the changes would propagate to all the documents that link to it (Subscribers). This feature was designed to benefit makers of compound documents. Of course, programs that are intended to act together, such as Adobe’s Illustrator, Photoshop and inDesign, provide a way of doing this. For example, I can place a link to a image from Photoshop into a larger graphic in Illustrator. Then I can place a link to the resulting compound document in an inDesign document. If I change the image in Photoshop, inDesign will detect this change, invite me to update the graphic, and display the changed graphic. This won’t happen with graphics embedded (e.g. by cut and paste) into a Pages document, or one from practically any other Mac program. Here’s what I liked about Publish and Subscribe. You could link document fragments between practically all programs; they didn’t have to have been designed to work together. If you had one image created in Canvas, and another from Photoshop, you just created an Edition of each, and subscribed to it in any other program that could display graphics. If you wanted to edit either of these images, you just made your changes in the program that created it. This was a system-level service that was easy to implement in a program, and it made all your favorite programs work together as if they were designed to do so.
Publish and Subscribe was an important client of the Alias Manager. Publishers, Subscribers and Editions kept up with each other using Alias Records, and so did not lose track of each other when files were moved around by the user. It was available starting in System 7, and was a mainstay through the System 9. It was not a part of Carbon, and so removing this useful feature became a big part of the work of porting any Macintosh program to OS X when it came out.
Organizing the System Folder
Control Panels
NeXTSTEP was developed in the 1980’s and followed the System 6 design. When Mac OS X was created, it initially used the NeXTSTEP preference panel window, that had a scrolling list of icons representing the various panels (except running horizontally rather than vertically as in System 6). But that had the same problem as the System 6 control panel, that is, too many control panels. The current version of Mac OS X uses an entire (special kind of) window as a menu of icons representing control panels (err-- I mean preference panels). When one is selected, it replaces the icon menu, but you can always go back to the menu view in one click.
Drag and Drop
The right way to open a document on the Macintosh has always been to double click its icon in the Finder. The Finder’s desktop database kept a record of what application created that document, and used its creator to open the file. But with the proliferation of useful standard document formats, it became common to open a file using an application different from the one that created it. This problem persists in OS X, although it has been changed a little. Now (meaning OS X 6) it isn’t the creating program that is used to open a double-clicked document, but a standard one for that document type, as indicated by the three-letter file extension [I know it sounds crazy]. The solution we use today is one introduced in the System 7 Finder. In System 7, if you drag a document’s icon onto an application’s icon, that application will try to open the document. This is an elegant solution, and it beats the alternative by a mile. The alternative, of course, is to start the application you want to use, and select Open from the File menu. Then Navigate to the file you want to open using the Standard File Dialog (aka NSOpenPanel Dialog), and open it there. If you are paid by the hour, that might just work out fine for you.
In later versions of System 7, Drag and Drop got generalized into an API for programmers. This let any programmer add this kind of user interface, not just for icons, but for text, graphics, and other things.
Window Dressing
Color did not start with System 7. Color Quickdraw was available as an INIT in System 6, and color programs were commonplace on Macintosh II’s for 3 years before System 7 was released. But the standard window decorations, menus and controls were all in black and white. The Finder had color labels for folders, but it only colored the outline of the folder. System 7 came with new menu, window, and control definition functions that looked much better. These were no doubt influenced by the NeXTSTEP operating system, which was released in 1988, and had very beautiful user interface items in 16 shades of gray (color starting in 1990). The user interface features in System 7 were very beautiful, and subdued, without any of the embarrassing primary colors and garish features of its main competition at the time, Windows 3.1. Macintosh users were taken with the appearance of System 7, and that was enough. After all, if they didn’t care about the way things looked, they probably wouldn’t have been Macintosh users (did I mention that Windows 3.1 was butt ugly?).
System 6 did not have color icons. What I mean is, the resource that defined icons only supported 1 bit descriptions. But they could use the old quickdraw 8-color system, and System 6 icons could have some color in them. System 7 included a format for color icons, bringing color to nearly every item on the desktop.
Bad Rap
System 7 is often blamed for the problems that Apple experienced in the 1990s. It is true that some of the problems with the operating system that should have been addressed were not. It is also true that effort was wasted adding unnecessary features to the operating system. System 7 was considered a resource hog at the time, but in retrospect it was as frugal as it could have been. It was, and still is, very reliable despite its complexity. And most ironically, most of its critics used an operating system that was a flagrant copy of System 7. Even popular versions of unix got most of their user interface features directly by copying those of System 7. The basics of the System 7 user interface stayed intact, in many cases unmodified, throughout the System 8 and 9 era. After the NeXT reconquista, attempts to impose the NeXTSTEP user interface on Macintosh users failed. The interface users wanted? System 7 and its offspring. Today’s Mac OS X user would find System 7 more familiar than NeXTSTEP.
-- BG (basalgangster@macGUI.com)
Wednesday, June 30, 2010