The Long View
Building Photoshop
On February 13, 2013, the Computer History Museum reported that they had received the source code for version 1.1 of Photoshop from Adobe, and had permission to release it for non-commercial use. This is the second time recently that something like this has happened, the first being the release of the Quickdraw and MacPaint source code by Apple last year. In both cases, there was a nice commentary by Grady Booch, software design legend and trustee of the Computer History Museum. Both times, Booch reported having a look at the code and found it to be elegant and clear, and outstanding examples of how to write software. In his commentary on the Photoshop source code, Booch says "Software source code is the literature of computer scientists, and it deserves to be studied and appreciated".
Writers need to study the writing of others, and usually they do. Computer programmers? not so much. There is a lot written about programming, and most programmers read that stuff (including Grady Booch's books), but there are very few opportunities to actually read an acknowledged Great Work of programming from the past. So now we have the sources to Photoshop, an indisputable software masterpiece. How can we make use of it? Shall we read through the 100,000 or so lines of code in 179 files to see how Thomas Knoll made it do its magic? Maybe Grady Booch can just look at the code and immediately understand the thread of genius that runs through it, but I can't. Source code may be the literature of computer scientists, and each program’s source code may tell the story of its function, but that story is written in a scrambled, crazy order and it makes no sense. To understand their stories, computer programs must be read in the order of their execution, not the order of the text in the files. For Photoshop, as for MacPaint, that means the code has to be compiled and running when we read it. We need to see it in its natural order, and while it is executing, so we can compare what it says to what it does. Only then can we learn its story.
MacApp version 1.1
Photoshop was created for the Macintosh computer in the late 1980's, and it looks a little strange to today's programmers. Anyone can see (and many are surprised) that it is written in Pascal. Maybe it is not well known that 25 years ago the Pascal language saw a lot of use, even for teaching programming in college. Pascal is not taught much these days, and that must be why almost nobody notices that Photoshop is not written in ordinary Pascal, but rather in Object Pascal. Even fewer would recognize that it is written using Apple's MacApp object library. The version of Photoshop that was released to the public is not a complete set of sources. If you are going to build Photoshop from the distributed files, you need a Macintosh Object Pascal compiler, its associated libraries, and the appropriate version of the MacApp class library. MacApp was not included in the source code release, and could not have been, because Adobe does not have a license to distribute MacApp™.
It would have been helpful if Apple had decided to release the source code for the appropriate version of MacApp at the same time as Photoshop. Why not? The source for MacApp is not a trade secret. The class library was distributed to programmers as source, and they were expected to compile it themselves. The version of MacApp in question is version 1, released in 1987 and replaced by a totally rewritten version in 1990. The original was distributed as a set of 2 floppy disks. MacApp version 2 was the first version to be distributed on CDROM. In 1990, Apple wanted programmers to port their programs to version 2, and they didn't include version 1 on the version 2 CD or on any of the later developer distributions. I think it is probably pretty rare. Of course, some people no doubt still have an old copy of MacApp on floppy disks. I didn't. I started using MacApp at version 2, in 1990, and never had a copy of version 1. Miraculously, a pair of MacApp v1.1.1 diskettes was for sale on ebay at the time the Photoshop code was released. 
I bought them, realizing that at their age they and might or might not be readable. They mostly were. One of the disks contains all of the Pascal and assembly source code for the MacApp library. The other has everything else needed to build and test the library, including the resource files, the one object file for which Apple did not share their source (called wwDriver.c.o), and the MPW script called MABuild that is used to build MacApp programs from their make files. There is also a compiled MPW tool, called PostRez, that is used to post-process the code. It connects menu GUI elements compiled by the resource compiler Rez to the object code that they are supposed to evoke. The second diskette also contains the classic MacApp example programs, like DrawShapes and Nothing. I am lucky; the diskettes I bought were mostly readable. The second diskette was completely error-free. The source code diskette had one unreadable file, that contained seven bad sectors. The file that could not be read was UTTEView.inc1.p, which contained the implementation of an object called TTEView that is a baby text editor. It was a relief to find that Photoshop did not use this component of MacApp, but I wasn't surprised. TTEView was not used much outside of Apple's demos, because it was limited to editing text blocks of size 32K or smaller.
The MacApp that I bought was complete enough to compile Photoshop. But if you happen to have a copy of UTTEView.inc1.p, I'd be interested in acquiring a copy of that file, just to complete the library. Don’t worry about Apple’s lawyers. I have a license for MacApp.
I tried to compile MacApp and a couple of examples (the ones that didn't use TTEView) to make sure my MPW setup was working with MacApp. It wasn't. A quick look around the inter-webs yielded this post on comp.sys.mac.programmer from Apple's Keith Rollin in 1989, after the release of MacApp 2.0 (thanks to macgui.com for archiving these old usenet posts). Basically, it says that if you want to keep on using MacApp 1.x (and you shouldn't), you have to use MPW version 2.x and its Pascal compiler, not version 3.x. Of course I was trying to use MPW 3.0. It is the earliest version I have. I haven’t used MPW 2 on any computer since about 1991. And like MacApp v1, MPW 2 was only distributed on floppy diskettes. I still have the MPW 2.0 diskettes, but in a moment of bad judgement in the mid 1990's I reused them to store some files that seemed more important at the time (but don’t seem so anymore). There were no MPW 2 diskettes that I could find for sale anywhere. According to the usenet post, I could use MPW 3.0 under system 6 with some changes to the MacApp makefiles and to the application resources. It says that might suffice for builds that do not evoke MacApp's built-in debugger. But according to Keith Rollin, the MacApp debugger will not work if MacApp is compiled under MPW 3. I tried the Nothing and DrawShapes examples both ways; the nodebug versions worked and the debug versions crashed. I could build the nodebug version of Photoshop. But what good is that? I don't just want to just use some old version of Photoshop, I want to run it in the MacApp debugger. MPW version 2 didn't have a source level debugger. Apple’s first source debugger for MPW (SADE) was released with version 3. I think Thomas Knoll must have created Photoshop without the use of any source level debugger other than the one that was part of MacApp. If I want to see it the way he saw it (and I do), I would need to compile MacApp with debugging turned on. Luckily, the problem with the debugger was fixed with only a small change in the MacApp source code.
Getting MacApp to Build
If you build a MacApp v1.1 program with debug on, it enables code for the Writeln window, names in code, range checking, the MacApp debugger interface, and subroutine tracing. Tracing subroutine calls, when on, means that the MacApp debugger keeps track of entry into and exit from every subroutine. When trace is then activated in the debugger, every subroutine call is recorded to the Writeln window, similarly to the trace function in Lisp. If the code was compiled for the debugger, whether tracing is turned on or not, MacApp calls a couple of weirdly-named functions called %_BP and %_EP respectively (implemented in unit UTrace.inc1.p) on entry and exit to every subroutine. Some subroutines that should never be seen in a trace (like the ones that write trace information to the Writeln window) are exempted from %_BP and %_EP using the directive {$D+}, which leaves subroutine names in code but does not trace. The directive {$D++} turns tracing back on for the next subroutine. In MacApp v1.1, both %_BP and %_EP call a subroutine, named MeasureTally. For some reason, MeasureTally was not exempt from the trace. I don't know what would happen if it was compiled with the MPW 2.0 Pascal compiler, but MPW 3.0 compiled this into an infinite recursion, as it should. At startup any program compiled with debugging on would immediately hang, stay hung for a while, and then the stack would collide with something vital in the heap (probably some code segment) and there would be a Bad Crash. It was an easy bug to track down. As soon as it hung but before it crashed, I hit the programmer's switch and fell into TMON. I was in %_EP. Both %_EP and MeasureTally are short, and I could step through them and see them call each other in turn. The problem was fixed by exempting MeasureTally from the trace.
Fixing the Files
The files in the Photoshop distribution are not ready to use. Firstly, all the text files are contaminated by Windows-style newline characters. Every text line in Windows (and in DOS before it, and in CPM and DEC RT-11 before that) was terminated by a pair of characters, a carriage return (ascii 13) and a linefeed (ascii 10). If you were printing the text using a teletype machine these two characters would first move the print head to the left limit, and then advance the paper one line. This made sense as a line terminator back in the days of the PDP-11, but by the time the Macintosh was released it was already vestigial. Macintoshes were never expected to be connected to teletype machines. They needed a character to represent newline, but one was enough. So Apple omitted the linefeed and kept the carriage return. Somebody at Adobe or at the Computer Museum must have thought we would be using this code with Windows, or some other operating system intended to work with teletype machines. To build Photoshop, the first thing you have to do is to strip out all those extraneous linefeeds. Next, it is necessary to attend to the copyright blurb at the top of each of the resource files. Every Pascal and every Rez file has a little snippet of legalese at the top, encased in curly brackets, like this:
{Photoshop version 1.0.1, file: About.r Computer History Museum, www.computerhistory.org This material is (C)Copyright 1990 Adobe Systems Inc. It may not be distributed to third parties. It is licensed for non-commercial use according to www.computerhistory.org/softwarelicense/photoshop/ }
Curly brackets delimit comments in Pascal, so these are fine for the Pascal files. For the assembly language files, the curly brackets have already been replaced with semicolons at the start of each line, which is exactly right. Semicolons start a comment line in MPW assembly (as in many others). But the resource files are in the Rez language, whose conventions are apparently unknown to the archivers, because they retained the curly bracket Pascal comments. Rez resembles C, so curly brackets mean something altogether different. Comment lines in Rez begin with //, and so all of these need to be added. Finally, the MPW make files and other shell scripts used a unique lexicon of special characters. Comment lines began with #, as they do in most unix shell scripts, and this part has been done right in the sources as distributed. But all the MPW scripts are mangled by the use of a backslash (\) instead of option-d (∂) as the escape character (for line continuation), and all instances of option-f (ƒ) to indicate target-dependency in makefiles have been replaced with colon (:), I suppose because that is what is used in unix makefiles. Fixing all that was easily done with a couple of little MPW scripts.
There are some additional small version issues to address before compiling Photoshop in MPW 3.0. The Photoshop code uses a Pascal interface file called Quickdraw32bit.p. The contents of this file were folded into Quickdraw.p at the time of release of MPW 3, so you have to to remove the reference to Quickdraw32bit.p in several places. The MPW 3.0 version of Quickdraw.p left out the definition of a couple of constants that were in Quickdraw32bit.p, however, so I had to borrow a later version of Quickdraw.p from MPW 3.2. Also, I think Knoll must have been using a particularly early version of the Quickdraw32bit.p file. He refers to a field of the ColorTable record as transindex. That field is called ctFlags in Inside Macintosh V, and in the MPW 3.x interface files. I changed references to transindex with ctFlags everywhere it appeared in the Photoshop code.
Knoll’s Modifications of MacApp
One of the great things about MacApp was that it was distributed as source code. The first thing you had to do was to build the library yourself. What was so great about that? Well, it meant you could improve it if you wanted to. In this particular case that was not so great. I do not have a copy of Thomas Knoll's version of MacApp, so I was hoping that he had used an unaltered version, but I knew it wasn't likely. A lot of things you can readily see happening in Photoshop were not implemented in MacApp v1. For example, that floating palette window with the tools in it. Floating windows like that were (in)famously not part of MacApp v1. Another well-known difficulty with MacApp at that time was its lack of support for "mouse-up tracking”. This problem was even mentioned by Kurt Schmucker in his 1986 book on MacApp, "Object-Oriented Programming for the Macintosh".
To understand the problem with mouse-up tracking, you have to understand MacApp's support for mouse tracking. Mouse tracking means that your code is drawing and redrawing something on the screen in response to movements of the mouse. Normally this happens only when the mouse button is down. When your user was drawing, or defining a selection or dragging something, or for whatever reason moving the mouse around in a window with the button pressed, mouse monitoring code (in the TFrame object) would call your command object (subclass of TCommand) and tell it where the mouse was and where it had been most recently. A command object that handled mouse tracking was called a tracker. You were expected to subclass TCommand to implement each specific behavior of your program during mouse-down tracking, but you didn't expect to have to change TFrame at all. TFrame would call your tracker periodically while the user held the mouse button down, so you could update what was needed and draw stuff on the screen accordingly. When the user let go of the mouse button, your tracker was called one last time, with the advisement this was the end of the series. At that time you were supposed to clean up and draw the final state of your window as it should appear after the user-interaction.
It was a cool system, because you just made a command object and didn't worry about how it got called. Stuff that happened in between the time that the TApplication object intercepted the user's action and the time your tracker was called stayed completely invisible to you as a programmer. The down side was you couldn't easily keep tracking the mouse after a mouse-up event. The TFrame would think that the action was over and wouldn't call your tracker again till the mouse button was pressed to start a new interaction. But let’s say you wanted to let the user draw with the mouse up. For example, imagine you wanted the user to be able to define a polygon by clicking on a series of vertices, and have the program drag out a dynamic line between your last clicked point and the current mouse position as the user moved around with the mouse button up. Mouse-up tracking was used in the polygon-drawing tools in MacDraw, SuperPaint, and Canvas, and lots of other drawing programs. If you wanted to do mouse-up drawing in a MacApp v1 application, you could not do it purely in your command object. You would have to bore down deep and alter the normally untouched MacApp code in class TFrame that called command objects. These days this kind of problem is solved using multiple inheritance, or (even better) by delegation, so you never have to alter the guts of the framework, and this is why Apple doesn't have to share the class framework source code with you.
It turned out that the floating palette window was not a problem. Knoll coded that window's behavior by subclassing the existing MacApp TView class. That's one of the things I'd like to study at when I get everything running. And Photoshop almost did not use mouse-up tracking. But it did, just for one small thing. I am a long-time Photoshop user, but I didn't know or had forgotten about this feature. I learned about it because I got an error at compile-time. A subclass of TCommand (a tracker) in Photoshop, called TLassoSelector, overrides a TCommand method called TrackMouseUp. The compiler knew that in my version of MacApp, TCommand does not have a message called TrackMouseUp. In Thomas Knoll’s version, there must have been.
The Lasso tool uses mouse-up tracking, but only sometimes. Try this in any version of Photoshop. Start to make a selection using the Lasso tool. Part way through the selection, press the option key, and then release the mouse button. Photoshop drags out a straight line between the place you released the mouse button and the current location of the mouse. If you release the option key, it closes the selection with a straight line. Photoshop v1.1 also did that, Knoll had to change MacApp to make this happen, and I didn't have a copy of his changes. Of course it is easy to put a dummy method called TrackMouseUp in TCommand; that would satisfy the compiler. But it would never get called, and the mouse-up tracking would not happen. My version of Photoshop would compile, but it would not be authentic. Not good enough. I had to add a call to TrackMouseUp somewhere in just the right spot. I would know it was the right spot, because if I did it right, TrackMouseUp in Knoll’s lasso command object would do the right thing. Of course, I also needed a default do-nothing version of TrackMouseUp in TCommand so it could be overriden. I’m pretty sure TFrame is the object that has to call TrackMouseUp. In MacApp v1 (but not in later versions), objects of class TFrame provides scrolling to views in windows. They also monitor clicks and drags in their screen territories, and send messages to tracker command objects that are supposed to be drawing in those spaces. As pointed out by Grady Booch in his commentary, there are almost no comments at all in the Photoshop code; there was no hint left anywhere by Knoll. By the looks of it, TrackMouseUp in the LassoSelector was supposed to take over for TFrame after TFrame gave up tracking the mouse. I stuck a call to TrackMouseUp in the part of TFrame.TrackInContent that runs at the end of tracking, when the mouse has just been released. It works, but I can't be sure whether this is the same change that was made to MacApp for the original Photoshop. A similar change to another part TFrame was also required because of another mystery override. Photoshop overrides TFrame.CalcSBarMin, another MacApp method that does not exist. There is a TFrame.CalcSBarMax in TFrame.AdjustSBars, and so I put the call to CalcSBarMin there too. I don't actually know what the function of this is, so I can't even be sure it's working. I’ll find that later when I study it in action. Photoshop also expects to find a procedure called FlashButton. I thought it was apparent what this should do and wrote something accordingly.
EVEITF.o
Photoshop expects to link an object file called EVEITF.o. The file is not there. It apparently provides the code for 5 functions, called EVEStatus, EVEReset, EVEEnable, EVEReadGPR, and EVEChallenge. These functions are called by a global function, VerifyEve, which is present, and is called at startup time. Linking Photoshop fails because of these unresolved references. Examination of VerifyEve shows that none of the EVE functions are called unless Photoshop has a resource of type 'Eve ' (that’s E-v-e-space). There is no such resource in the distribution's resource description file. So these functions would never be called anyway. The source code distribution is designed to link EVEITF.o into the final build, but not to call it. It is dead code. But adding an appropriate ‘Eve ‘ resource would bring it back to life without a rebuild. I think VerifyEve must have had something to do with serial number registration, because it is called immediately after RegisterCopy, which checks whether or not a valid registration number has been entered. After commenting out the EVE functions, the program runs and RegisterCopy works fine. It successfully accepts valid Photoshop serial numbers and rejects invalid ones, even without VerifyEve. So VerifyEve remains a mystery. The personalization information, including the serial number if correct, is stored in the program in a resource of type 'Reg '. The debug version of Photoshop includes av alid 'Reg ' resource with registration information, and allows the program to start up fine, as if it were already registered. It shows personalization for Thomas Knoll at Knoll Software. Cool. Photoshop thinks I am Thomas Knoll. Who else would be running it with the debugger enabled?
Things Were Tougher Then
I built Photoshop on a stock Mac IIci with my ‘040 accelerator removed and the cache card put back in place, to see what the experience would have been to do the build on a high end Macintosh in 1989. The full build, compiling MacApp and the program together, took 14 minutes. If MacApp was already built, it was reduced to 10 minutes.
The debug version of Photoshop starts up normally, and the only immediate differences from the release version are the Debug menu and the presence of the Writeln window (labeled Debug Window) at the bottom of the screen. 
The Debug window is a regular window you can drag around and resize. A few things are initially written there, and any Writeln command in the code will write into it, but it is not immediately obvious how you use it as an interactive debugger. In MacApp v2 there was a selection in the Debug menu that would stop the program and drop you into an interaction with the Debug window. There is no such item in version 1. I don't have a version 1 manual, so I had to look up in the MacApp source code to find out how you were supposed to enter the debugger. It turns out the answer is to press the command, option, and shift keys at the same time. At that point you get a prompt (≥). Typing a ? gets a list of one-character commands and what they do.
Some of the commands are pretty obviously useful. The Stack Crawl command prints the name of functions on the stack, and addresses of objects whose methods are on the stack. It is also possible to display a block of memory, a list of recently run subroutines, information about memory allocation and some other useful things. You can step through the program. But this isn't a source level debugger in the sense of being able to examine the source file from inside this debugger. One really useful-sounding command is Inspect, which will show the contents of fields of an object. In MacApp version 2 this feature has its own window that presents a list of every object in memory and can inspect any of them. In this version, if you know the address of an object, it will print the values stored in all the fields, in hex. Not as useful as you might hope. Turning on trace (T) and starting the program again (G) will create a storm of activity in the Debug Window as subroutines fire off and run their course. By default, the Debug Window only stores 55 lines of text, so it is important to redirect the output to a file. This is probably the most useful general debugging feature.
Seriously, did Thomas Knoll write and debug Photoshop with just this? I don’t know for sure, but maybe so. Programmers were tougher back then. Most had never used a source debugger; they relied on debugging in object code, using something like TMON or Macsbug. Probably a lot of debugging of Photoshop happened this way. I suspect Knoll was not a big believer in the MacApp built-in debugger. I think this because he didn't code for it. If you were going to use the Inspect command in MacApp, you needed to override a special method, called Inspect, in all your classes. In Inspect you would format the values of all the fields you had added to your applications classes so they could be inspected in the MacApp debugger. If you didn't do this, Inspect would only show the fields of the most immediate superclass that did implement it. Nowhere in the Photoshop code is there any single override of Inspect. There are quite a few direct calls to Writeln, so that the debugging version of the code reports some information about what was going on. I think that means he was using the MacApp debugger. If so, I'm sure that there were lots more of these calls to Writeln when something was not working during the debugging of Photoshop, and that these were removed after the problems were solved.
One of my first lessons learned while reading Photoshop is an appreciation for the power and convenience of the tools we have available for coding today, and the skill of the programmers who made big programs like this without them not too long ago. I'm sure there are a lot more lessons waiting for me, now that the book of Photoshop is open and ready to read.
BG
Saturday, March 30, 2013
