The Macintosh Toolbox routines were stored in read-only memory.  In keeping with its name, the code in the ROM was immutable.  When your program used a toolbox call, it passed parameters into the toolbox routines in registers or on the stack, and temporary data used in the course of execution of the routines were stored the same way.  If the function returned a result, it would  return that on the stack (or in a register) too. Each time a function like that is called, it gets its own place on the stack to store parameters, temporary storage, and its return value.  So in principle a toolbox subroutine ought to be able to call itself, or to be interrupted after partial completion and resumed after another was called, with no error. But most toolbox calls also did something else as well, like drawing on the screen, making sounds, or writing data to disk.  Many of these jobs required some information about the state of the computer.  The variables that contained that state information needed to be kept in some place where they could be accessed by all the other code.  The Macintosh designers chose put this kind of state information in the first ~2800 bytes of memory, the famous Low Memory Globals. If the toolbox routines had only read the values stored there, everything would have been okay.  But they also sometimes changed the content of those global variables.

Global Variables Considered Hazardous

    Of course all Macintosh programs also kept their own global variables, containing the program's state.  Even if you were some kind of functional programming purist and you didn't allocate any global variables for your program, you still had some anyway, because each program was given a set of globals when it started up, whether it wanted to or not.  Those global variables, the Quickdraw globals, contained the state of that program's screen-drawing context.  On the Motorola 68k computers used in the early Macs, a program's globals, including its Quickdraw globals, were accessed relative to register A5. They were thus referred to as the program’s A5 World.  Each program had an A5 value that showed it where to look for its globals.  But no register was required to find the low memory globals.  Being in the very bottom of memory (within a 16 bit offset of memory location 0), they could be accessed directly by their addresses, which Apple shared with developers.

    Global variables are a hazard in a multitasking environment.  Consider what would happen if a Toolbox routine wrote a value to a low memory global, planning to store it there so it could be used later.  While it is executing, it gets interrupted by another task, and it calls the same Toolbox routine, which writes something else in that global.  When control returned to the original process, it  finds a value written in the global that is different from the one it wrote there.  Except in very special circumstances, this would be a bug.  The problem is that there is just a single copy of that memory location, not one for every instance of the subroutine's execution. Of course, sometimes it is intended that way. 

    It turns out that nearly every low memory global was a unique hack, meant to be the solution to some special problem.  Some of them are less hazardous than others.  For example, ScrapCount (0968) is used to keep track of changes in the scrap.  It was designed to solve a multitasking problem that arose from the very beginning because of Desk Accessories.  The value of ScrapCount changes when a program changes the contents of the scrap.  Although the original Mac OS ran only one program at a time, Desk Accessories could alter the scrap behind the back of the regular program, and so ScrapCount was a way for programs to know if that had happened.  While ScrapCount is intended for communication, other globals really ought to be specific to each program.  For example, PaintWhite (09DC) determines whether windows are erased before the update event that tells the program to redraw the contents of its windows. If two programs were running, you might want them to be able to have different values for PaintWhite.  But ScrapCount is a single location in memory, owned not by the program but the system.  Programs could interfere with each other a little by changing PaintWhite, but they couldn’t do each other any real harm that way.   TempRect (09FA) is a different thing altogether.  This is a scratch rectangle used by the Window Manager when doing stuff like interactively growing a window.  If it was possible to drag two windows simultaneously by time slicing between two processes, these would rectangles would clobber each other.  A more likely source of conflict is the Window Manager's Port WMgrPort (09DE).  The Window manager creates this GrafPort when you initialize windows (by calling InitWindows) at the start of your program.  Window Manager calls that draw on the screen modify this data structure.  At the beginning of Window Manager toolbox calls, the WMgrPort is configured for the drawing calls that come later.  If operations on one window controlled by one program could interrupt the same functions for another program's windows, there would be some serious problems.  Really, every program that’s running ought to have its own Window Manager Port.  But because the Macintosh was supposed to run only one program at a time, it was okay to put the Window Manager Port in a Low Memory Global.  At least it was at first.

Low Memory Globals Are Not All

    The Low Memory Globals did not occupy the only impure data area in the Macintosh’s low memory.  Another danger lurked in the very mechanism that called the Toolbox and Operating Systems routines in ROM.  When a program called those routines, it did it by attempting to execute instructions that didn’t exist.  Motorola m68k instructions were 16 bit words, so there were in 65,535 possible instructions in the set.  Of course, there were not that many instructions, and a large set of the possible instruction words were not implemented.  Macintosh Toolbox and OS functions were assigned to a set of these unimplemented instructions.  When they were encountered by the processor, it recognized that they were illegal, and branched to a function that looked up the illegal instruction number in a table (called a Trap Dispatch Table) also stored in low memory, just above the Low Memory Globals. From that table it got the memory location (usually in ROM) for the particular Toolbox routine, and branched to that location.  This made it possible for Apple to fix errors in the ROM by entering the address of RAM-resident code in the table, and so diverting execution to an alternative version of the function.  No problem there, but user programs were allowed to do the same thing, and many did. Application programmers patched Toolbox calls for a variety of reasons, but usually trying (often in vain) to get improved performance.  There was even a Toolbox function that could change the Trap Dispatch Table.  That makes the Toolbox self-altering code.  Very convenient for applications programmers, but come on, guys, were you just trying to make it hard for yourselves? 

   All of these things are now and were then recognized as practices that were incompatible with efficient design of multitasking operating systems.  But the Macintosh team were not designing a multitasking operating system.  They were trying to make a small and efficient single job operating system.

Desk Accessories and Switcher

    Switcher only loaded multiple programs in memory and switched between them.  It didn’t have to worry about programs interrupting each other.  Programs got switched out by the explicit command of the user.  The program that made the switch was Switcher itself, so it had control at the time the switch occurred, and could make sure it did not interrupt anything at a problematic moment (like during a Window Manager operation involving TempRect or WMgrPort.  But programs did have to be kept from altering the Low Memory Globals and Trap Dispatch Table used by other programs.  When one program was switched out and another switched in, Switcher saved the values of all the registers for the program switching out.  That included register A5, so the program could later find its variables.  It also had to save the values of the Low Memory Globals, and any changes that had been made in the Trap Dispatch Table.  This meant allocating some memory somewhere, copying all that stuff there, and keeping track of where it was put.  Then the Low Memory Globals and patched trap entries for the program switching in had to be copied into the correct locations in low memory.  Nothing I can find written about it says what globals were changed out and which were not, but I’m sure they couldn’t all be changed.  ScrapCount, for example, should be shared among all programs for communication about the state of the scrap, which they all shared.  So Switcher couldn’t just do a big block move of all that stuff from wherever it was to low memory.  The time required to do a switch between applications must have been mostly used up just swapping out those data.

    Desk Accessories came and go even more rapidly than switches, but unlike programs in Switcher, Desk Accessories periodically got some time to execute even if not in the front.  But they lived in the same memory space as the host program, shared the same Low Memory Globals and Trap Dispatch Table, and had to make sure they didn’t do anything to mess up the main program.  There were opportunities for crazy interactions between programs and Desk Accessories because of the shared stuff in Low Memory.  I’m surprised there weren’t more problems than there were.  But because time for Desk Accessories to run was controlled by the Toolbox call SystemEvent, they did not interrupt the main program at unpredictable times. 

Vertical Retrace

  You could argue (and it has been) that Macintosh Desk Accessories were not real multitasking because they did not interrupt the main program at arbitrary times.  Instead, they got time to run when the main program called SystemTask, which it could do or not do at its pleasure.  Most programs called SystemTask once at the beginning of each transit through the event loop, at a time when it wasn’t deeply engaged in any particular task.  This is a relatively safe time to hand over control to somebody else.  But there was one multitasking mechanism whose credentials have never been questioned. Sixty times per second, the running program was interrupted to blank the electron beam on the screen during its trip back to the beginning of the raster scan.  It was possible to attach a small piece of code to that interrupt, and have it run on every retrace.  Because that code did interrupt all manner of Macintosh activities, it was forbidden from doing many things that were required for full-on Macintosh programs.  For example, vertical retrace tasks could not allocate memory using the Memory Manager.  It is possible that a vertical retrace task could be executing part way through the task of compacting memory.  If that happened, the entire Macintosh mechanism for keeping track of allocated memory could get messed up.  The long list of  limitations that are placed on vertical retrace tasks are a hint at how difficult it would have been to have designed a post-hoc preemptive multitasking system on the Macintosh.

Multifinder

    Still, pretty good multitasking was introduced relatively early in the Macintosh’s career.  Multifinder was introduced in October 1987.  It was written by Erich Ringewald and Phil Goldman, who had been responsible for Switcher maintenance after Andy Hertzfeld.  Hertzfeld also independently released a freeware beta of a Multifinder-like program, named Servant.  Servant showed that the Switcher idea could be combined with Desk Accessory-style multitasking to extend the Macintosh operating system into a place it had never been intended to go.  Multifinder worked with most existing programs, because it captured calls to SystemTask to give background programs a chance to execute.  As in Switcher, there was still one foreground process, and it was the one whose windows were frontmost on the screen.  Switching between foreground tasks was done by the user, by selecting the program from a task menu (this was a lot like Switcher) or by bringing a program to the front by clicking on one of its windows.  When programs came to the front, it was done as in Switcher, by copying their Low Memory Globals into the low memory area, and their patches into the Trap Dispatch Table. But properly configured background programs could run in the background, and could receive update events and redraw their windows when activity in front of them had dirtied them.  When the front program had no important events to handle for a couple of cycles though the event loop, Multifinder would pass the opportunity to execute to background tasks.  This included the ability of those tasks to draw things in their windows, so it was necessary to restore their Low Memory Globals and Trap Patches, even if they were not being brought to the front.  Thus a special set of circumstances, including a lull in activity in the front process and its following of correct Desk Accessory etiquette was required to keep up the rotation of control among background tasks.  This was called Cooperative Multitasking, because it relied on programs being designed to cooperate in this way.  As in so many Macintosh technologies, Cooperative Multitasking was borrowed from Smalltalk-80.  But yielding the processor to another task in Smalltalk was a relatively lightweight operation, compared to the Macintosh.

    For programs that were built after the introduction of the Multifinder, there was a set of procedures that could be followed for making programs cooperate better.  SystemTask was omitted, and passing control to background processes was built into the event loop using a single call, WaitNextEvent.  Because Multifinder was optional, programs needed to check to see if WaitNextEvent was available (meaning that Multifinder was active) or if the user was running the traditional Finder, and so the program should do the event loop in the old way.

    Multifinder was an amazing retrofit of the Macintosh.  Fundamentally, the operating system could not have been designed worse for multitasking.  Making it work was an astonishing feat, and Multifinder worked great, considering.  Of course, lots of cycles were wasted juggling between tasks (as it was called), but that only happened when the user wanted to bring a new program to the front, or when the front program wasn’t very busy.  Programs did not cooperate in performing single tasks, so nobody cared about benchmarks on how many context switches Multifinder could make per second.  On the face if it, the worst thing was that programs had to be written to work differently depending on whether it was present or not.  And this was fixed in System 7, when the Multifinder was made mandatory at all times.  But as amazing as it was, it did not mitigate the underlying problem with multitasking.

System 7

    When they started thinking about a major new version of the operating system, the problem with context switching must have been at the front of everyone’s mind.  It must have gone like this.  Ok, here’s what has to be done.  Get rid of the low memory globals.  Developers were told from the beginning that they should not use them directly. For some of them there are functions that can be used instead.  Some of them (like PaintWhite) should be in the application’s Quickdraw globals.  Some of them shouldn’t exist at all, because they are part of the reason the Toolbox calls not reentrant.  Secondly, disable Trap Patching by applications.  This was another thing developers have long been told to avoid because it would someday become illegal.  Last, and admittedly hardest, rewrite the Macintosh ROM to make all the routines reentrant and safe for multitasking.  Maybe a system stack for the Toolbox routines to use for scratch memory would help.  However you do it, fixing these things will make a generation of Macintosh computers that are compatible with a modern version of the operating system, that can be improved in increments later.  Finally, since this new operating system won’t work on the old equipment anyway (because it requires the new ROM), include a paged memory management chip and remap memory so that every application sees the same addressing space as all the others, instead of being stacked into a single memory space like sardines in a can.  This way, a runaway program won’t be able to overwrite memory that belongs to another program, or to the system.

   Maybe somebody said that, maybe not.  System 7 was an opportunity for Apple to fix the design flaws that prevented it from being a modern multitasking operating system.  The previous Macintosh operating system was an elegant successor to the Apple II.  But it was a small system, not a full-featured one like the Lisa. Now the Lisa project had been cancelled, there was no successor to it, and users expected the Macintosh to rise to the occasion and be Apple’s flagship computer.  The Macintosh had been designed to be the little brother of the Lisa, not its successor.  It had already succeeded way beyond its design purpose.  Brilliant programmers like Andy, Phil and Erich had come up with astonishing hacks in Switcher and Multifinder that made it seem to be something it wasn’t.  But even though it seemed like that,  it wasn’t. In 1991, the Macintosh operating system ought to have been at the end of its project cycle, but it was clear that it wasn’t going to give way to the next big thing.  It had to be the next big thing.  Still, fixing its internals may not have been the highest priority at Apple.

   There would have been a lot of resistance from applications programmers to making a fundamental transformation that would break every System 6 program.  System 7 had to be something really new and revolutionary, but it couldn’t be so new that all old programs would quit working.  I think it would have been possible to do that.  Many programs did access low memory globals, but programmers knew this was risky and would have understood having to do a minor rewrite to accommodate this.  As it was, System 7 required a minor rewrite of most programs, and many great programs were broken by it, never to return.  Trap patching was common among the big programming outfits like Microsoft, but they had a lot at stake and teams of programmers that could make the necessary changes.  The biggest job would have fallen on Apple’s shoulders.  To make a thread-safe version of the Memory Manager and Resource Manager and other parts of the ROM, without changing their interfaces, would have been a big job and would have taken a while.  All that work would be under the hood.  It would not have been appreciated by users or programmers directly.  The only thing Apple would have to show for it would be a machine that still worked with existing code and could handle the challenges for the next two decades.  There would be time later for perfecting and fine tuning, it like making a really great scheduler that would handle the special problems of interactive programs or providing multiple user accounts on individual machines.  In short, it would probably have been a disappointment to users, who were hoping for some whiz-bang new features.  It would have been like Snow Leopard, except with high expectations.

   I don’t know what determined the decisions that were made about what should be included in System 7.  The problematic fundamentals were not changed.  The old part of the ROM was left mostly unchanged, and the essence of the Multifinder preserved.  So much so, that System 7 could run on computers as old as the Mac Plus.  A paged memory management chip was an option in the early Macintosh II machines, and was used later to implement a half-hearted virtual memory system that gave no memory protection.  Ultimately, the need not to make fundamental changes paralyzed the development of the Mac OS, and led to its final end and replacement by NeXT.  On the other hand, System 7 offered a host of cool new features, some of which were truly revolutionary, but too many of them  never caught on.  Applescript and Apple Events were transformative.  There was nothing else like them, and they continues to pay off for the Mac in OS X.  The Edition Manager was a mixed blessing, but I loved it and still miss it.  Above all, System 7 was beautiful to look at, and a joy to use.  And that aspect of it just got better and better as it glided along to its disastrous end.

  -- BG (basalgangster@macGUI.com)