Amazing 3-D Games Adventure Set

I wanted to write a bit about a book I've had sitting either on my bookshelf (or, at times, packed away in a box) for the past 22 years.

"Amazing 3-D Games Adventure Set" by Lary L. Myers, published in 1995.

To summarize the contents of the book briefly, it basically walks through a game engine that the author had written called "ACK-3D" (short for "Animation Construction Kit 3D") for building Wolfenstein 3D-like games. Actually, I'd guess it would be more accurate to say it's for building Blake Stone 3D-like games, since his engine also includes features like textured floors and ceilings, and light shading effects, and so on (unlike Wolfenstein 3D). But still at it's core, it's the same 2D grid, raycasting approach used by Wolfenstein 3D. The author takes you through the internals of his game engine, explaining the math and how it works along the way. Honestly, having re-read chunks of it recently for the first time in many years, he does a pretty good job of explaining things overall and his writing style is good at not putting you to sleep (unlike many math books I find). It's not anything along the lines of a super in-depth math textbook at any rate, as the math required for a raycaster like this is not that complicated at the end of the day. At the time that this book was published, this style of game engine was already "old news" since Doom had been out for over a year by then.

My grandmother bought this book for me while I was in a bookstore with her sometime in 1997 (possibly 1998?). I noticed it while looking at the computer books section and was totally blown away by the idea of being able to create 3D games. I was 13 or 14 at the time and had only been programming for two years or so (almost entirely QBasic, tiny bit of C by that point). Unbeknowst to me at the time, was that the level of complexity that that book was going to bring to the table was still a little bit beyond my grasp. But all the pretty screenshots on the back and in the colour gallery pages at the front of the book had me hooked.

Sure enough after getting home with it and proceeding to read through it, I struggled to understand a lot of the math (math was never my strongest subject and trigonometry was not something that was taught until one or two years later in school where I was). Perhaps most importantly was that I could not follow along with the code being discussed because I did not own any C compilers that were required to build the code included on the CD the book came with. At this point in time, I'd taught myself a little bit of C with the book "C DiskTutor" my dad had bought for me. This book included a stripped down version of Watcom C 8.5, and unfortunately was not suitable for this game engine code.

The demos on the included CD were fun to play with though, and it included a map editor as well with a fairly large library of game artwork to use in your levels.

Over the years I occasionally would pick up this book and flip through it, always feeling disappointed that I couldn't do anything with the code (even as my math knowledge improved and more of the book's explanations started making sense to me).

More recently, since I've fallen into this admittedly somewhat bizarre path of collecting obsolete software development tools, I realized when I picked this book out of a box in my closet that "hey, I could probably do something with this code now after all these years!"

I no longer have the original CD that my copy of the book came with (guessing it was an accidental victim of one of my previous "throw out all the junk" sessions which inevitably end up with my realizing I should not have thrown something out) but thankfully it is available still via Google search.

Anyway the author of the book, Lary, included multiple different versions of his ACK-3D game engine on the CD.

• DOS version built with Borland C++ 4.0 (compatible with 4.5), found on the CD under /ACK/DOS/BORLAND.
• DOS version built with Watcom C/C++ 9.5, found on the CD under /ACK/DOS/WATCOM. Additionally, the "FDEMO" and "MALL" demo projects under /ACK/DOS/FDEMO and /ACK/DOS/MALL both have source code intended to be built with Watcom C/C++ under those respective directories.
• Windows version built with Borland C++ 4.0 (compatible with 4.5), found on the CD under /ACK/WIN. All the Windows projects/code under that directory are for Borland compilers.

As a teenage-hobbyist self-taught programmer in the 90's, you were probably most likely to be using Borland Turbo C++ as your C compiler, and not Borland C++ (sans "Turbo"). This is an important distinction as you would not be able to build the Borland DOS code with Borland Turbo C++. Why? Because Lary (quite understandably) utilized the Borland PowerPack for DOS addon which allowed you to write 32-bit DOS code that ran under Borland's DPMI extender. From what I can tell, this PowerPack addon was not included in Borland C++ (certainly it is nowhere to be found on my Borland C++ 4.02 CD) and instead had to be bought separately. It was also probably not likely that you were using Watcom C/C++ as a hobbytist programmer since it was more expensive (at least before it became Open Watcom around 2000/2001) and was not nearly as user-friendly as the Borland tools were.

So since I like Watcom C/C++ myself, I naturally decided to start with that version of the ACK-3D code. The included "FDEMO" and "MALL" executables worked perfectly fine and clearly were built with Watcom since they displayed the familiar DOS 4GW banner on startup.

For reasons unknown to me, Lary decided to use his own MK.EXE tool (included on the book's CD) instead of normal Makefiles using WMAKE or just MAKE. It's not the end of the world though, as his custom .MAK format is still perfectly readable (if a bit verbose). I was able to build the source code myself with Watcom C/C++ 10.0 completely without a problem, but running the built executable was another story.

Running it through a debugger indicated it was crashing in one of the assembly drawing routines. Great, yet another author that was probably incredibly rushed in the lead up to his book being published and made some mistake along the way in preparing the files for the CD, probably copying the wrong version of the source code over or something. *sigh*

Lary doesn't specifically mention what version of Watcom C/C++ he used anywhere in the book (many parts of the book make it feel like he was primarily developing with Borland and the Watcom version was only a side-venture), but included on the CD in a couple places are compiler .MAP files that included a Watcom version banner indicating he was using version 9.5. So I decided to search around the internet for a copy of that specific version and to try building the code with that. Alas, no such luck, my own executables built with the same version of the compiler Lary used fail in the same way.

So, since as I mentioned above, the book left me with a strong impression that Lary was using Borland C++ primarily in developing ACK-3D, so maybe I should give that version of the code a try. I installed Borland C++ 4.02 and the PowerPack for DOS on a Windows 95 Pentium MMX machine I built a while back (probably will write a post about this machine later, maybe) and the ACKLIB and FDEMO projects both compiled. Well, not so fast actually. I had to first fix up some hardcoded path's that were set in the project files (no big deal), but more importantly, I also had to fix some bizarre issue with TASM which seemingly could not be invoked successfully by the Borland C++ IDE. Specifically, this seems to be an issue with TASM32.EXE, as after some experimentation, I found that adding an alternate .asm to .obj "translator" (Borland's terminology for it) using TASMX.EXE worked fine. Trying to manually run TASM32.EXE from the command line seemed to highlight the source of the problem ... it seems that under Windows 95, TASM32.EXE 4.0 (which I am using) is unable to read any files (always fails with an error that it is "unable to locate [file]"). Oh well, at least I have a workaround. The only difference between the two TASM executables is that one can include 32-bit debugging information and the other 16-bit debugging information (at least according to my TASM manual anyway), so it should not matter to me for now.

With build issues out of the way, I was able to successfully build an executable for the FDEMO project. But it failed to run under Windows 95 with an error message "This DPMI32 module cannot be run on Win32." Ok, well, Borland C++ 4.0 (and the PowerPack) was released before Windows 95, so I guess it's perhaps understandable that it's DPMI extender might not be totally compatible with it. I rebooted into MS-DOS mode and tried to run again and while it worked, it didn't look exactly right.

As a bonus, it crashes when you use the FDEMO project's keys to toggle ceiling/floor drawing on/off. I didn't experiment with this version of the code much since I knew I was unlikely to pursue using the Borland DOS code if I wanted to build anything off the ACK-3D engine. Perhaps though, there is a simple solution to the problems I saw.

Needless to say that so far this was all not really that encouraging. I'm kind of glad way back when as a teenager that I did not bug my parents to buy me specific versions of compilers to so I could build the code for this book... I surely would've been quite disappointed!

The only thing left to try was the Windows version of the code. Back in 1995 this probably would've been pretty exciting as that was before DirectX was a thing (DirectX v1.0 would not have been released until later that year) and writing fast graphics code for Windows was uncommon at best. I imagine most people picking up this book in 1995 would've dug into the Windows code first just out of pure curiosity if nothing else. If you were one of the few game developers who really wanted to write fast graphics code for Windows instead of just writing for DOS as pretty much everyone else was at that time, you would've had pretty much no choice but to turn to WinG, which was a precursor to DirectX. WinG basically gave you fast pixel-level access to Windows DIBs which allowed you to do all the same low-level pixel stuff you would've done under DOS. Lary's Windows version of ACK-3D used WinG.

Once again I fired up Borland C++ 4.0 and this time built the Windows ACKLIB and WIN_EXAM projects (the former being more-or-less the equivalent of the DOS FDEMO project). Once some path errors were fixed and the same weird TASM building issues under Borland C++ were dealt with, it actually worked!

No crashes that I could find. I like to imagine that my younger teenage self would be so excited at this moment. It is such a shame to me that this is the only version of the ACK-3D code that works properly out of the box from the book's included CD, because in 2019 I have little-to-no interest in building Windows 3.1/95 code... at least right now anyway.

Anyway, the ACK_EDIT project which is the Windows version of Lary's map editor also builds and runs successfully. Also the example game that Lary frequently talks about in his book, "Station Escape" builds and runs flawlessly too.

Now... yes, I am greatly disappointed in the apparent lack of quality control / verification / testing that went into the source code that was included on this book's CD. It seems pretty clear that the perfectly working DOS demo project's pre-built executables were built with some different version of the DOS source code than what was included on the CD. That would seem to be the only explanation. I've spent a lot of time tweaking build settings and whatnot and cannot get my build artifacts to match the ones Lary included on the CD (he included .OBJ files as well as .EXE). They're always off by several KB or more. I thought at first that it was the difference of debug information being included, but that turns out not to be the case. Something else is up and again, my suspicion is that, like many other book authors, Lary was probably rushed during the final preparation of the CD files and perhaps mistakenly coping the wrong files. If you imagine back to the 90's, source code version control tools were not common place as they are today. And the source control tools that did exist at that time were usually not very good. Typically for version control you'd just copy files manually (either individually or entire directories) and end up with things like "GAME", "GAME1", "GAMEBAK", "GAMEBAK2", etc. If this was also the kind of thing that Lary was doing, then I could easily imagine the wrong version of the DOS source code being included accidentally. However, that is all just a wild guess... I really have no idea after all!

Since in 2019, I am primarily interested in building for DOS, I still wanted to try to get the DOS version of the ACK-3D code running reliably. So I decided to dig into the Watcom version in a bit more detail.

I've mentioned this before in a past post, but the Watcom C++ compiler, WPP386, catches many more warnings and errors than WCC386 does. I'm not a C/C++ standards expert so maybe there is a logical reason for this, but at any rate, I've often found it useful to temporarily compile a project with WPP386 just as kind of an extra "code linter" even if my project is 100% plain C. Doing this with the ACK-3D sources sure picked up a bunch of stuff, including pointer casting mistakes. Unfortunately fixing all of these did not resolve the crashing. That sure would've been nice and easy though, heh.

Stepping through the code with a debugger, the first set of problems I encountered was in AckDrawPage function found in ACKRTN.ASM. I actually happened upon this first before I decided to step through all the code bit by bit, as I was fiddling around with viewport size settings for some reason and noticed that if I set the viewport size to be the maximum screen size,

ae->WinStartX = 0;
ae->WinEndX = 319;
ae->WinStartY = 0;
ae->WinEndY = 199;

then crashes happened far less frequently and never upon immediately running the program. That led me to the aforementioned AckDrawPage function.

This function does a check to determine if it's rendering to a smaller (not maximum screen-sized) viewport, and if so, it runs a separate render loop.

dp_smallscreen:
    mov	    eax,[_gWinStartOffset]
    add	    edi,eax
    add	    esi,eax
    movzx   eax,[_gWinStartX]
    add	    edi,eax
    add	    esi,eax
    mov	    dx,[_gWinHeight]   ; <---- PROBLEM #1
    inc	    dx
    movzx   ebx,[_gWinWidth]
    mov	    ebp,320            ; <---- PROBLEM #2
    sub	    ebp,ebx

dp010:
    mov	    ecx,ebx
    shr	    ecx,1
    rep	    movsw
    rcl	    ecx,1
    rep	    movsb
    add	    edi,ebp
    add	    esi,ebp
    dec	    dx
    jnz	    dp010

dp090:
    pop	    edx
    pop	    ecx
    pop	    ebx
    pop	    edi
    pop	    esi
    ret
    endp

The first problem would only occur at random and would depend on what was in the EDX register in prior code. But what I would see is that sometimes it would have some large 32-bit value in it (some value larger then a 16-bit number) and so the dp010 loop below, which uses DX as a loop counter could end up looping a really large number of times and end up trashing a bunch of memory in the process. Switching the mov to a movzx so the 16-bit _gWinHeight variable overwrites the entire EDX register value solves that nicely. I suspect this might have been a missed bug that was a holdover from ACK-3D's 16-bit roots.

The second problem was much more serious and always occurred for me. Using EBP as a general purpose register was a common tactic when you were trying to write a tight loop and run out of registers to use. It's tricky though as it usually means you cannot reference variables on the stack or other locals in your assembly code (your assembler usually replaces those with references to memory using EBP) and also you need to be careful to restore it's original value when you are done. In this case, the latter was not being done which always caused a crash when this function returned. Oops. I also see that this big is present in the Windows version of the ACK-3D code, but all the Windows projects worked fine for me because they all use a maximum-sized viewport (at a guess anyway... I've not returned to look at the Windows code since discovering this bug).

Fixing both of these problems solved the common crash cases. However, it still would occasionally crash at random. Argh!

Turning back to the debugger, the remaining crash always occurs (when it occurs, seemingly at random) while the 2D bitmap object rendering code is walking through the engine's linked-list of Slices in FindObject within ACKVIEW.C.

sa = &Slice[Column];

if (sa->Active)
    {
    while (sa != NULL)
	{
	if (j <= sa->Distance)
	    {
	    sa2 = sa;
	    while (sa2->Next != NULL)
		sa2 = sa2->Next;            // CRASHES HERE

	    saNext = sa2->Prev;
	    while (sa2 != sa)
		{
		memmove(sa2,saNext,sizeof(SLICE)-9);
		sa2->Active = saNext->Active;
		sa2 = sa2->Prev;
		saNext = saNext->Prev;
		}

	    sa->Distance = distance;
	    sa->bNumber	 = ObjNum;
	    sa->bColumn	 = BmpColumn;
	    sa->bMap	 = omaps;
	    sa->Active	 = 1;
	    sa->Type	 = ST_OBJECT;
	    sa->Fnc	 = WallMaskRtn;
	    break;
	    }

	if (!sa->Active)
	    break;

	sa = sa->Next;
	}
    }
else
    {
    if (j <= sa->Distance)
	{
	sa->Active = 1;
	saNext = sa->Next;
	memmove(saNext,sa,sizeof(SLICE)-9);
	sa->Distance = distance;
	sa->bColumn  = BmpColumn;
	sa->bNumber  = ObjNum;
	sa->bMap     = omaps;
	sa->Type     = ST_OBJECT;
	sa->Fnc	     = WallMaskRtn;
	saNext->Active = 0;
	}
    }

The first thing here that caught my eye was the memmove calls with the curious size value being used. Looking at the SLICE definition answered that question and one more that I had.

typedef struct _slicer {
    UCHAR     **bMap;
    UCHAR     *mPtr;
    short     bNumber;
    unsigned  short bColumn;
    short     Distance;
    short     mPos;
    unsigned  char Type;
    void      (*Fnc)(void);
    unsigned  char Active;        /* Keep these last 3 fields in this order */
    struct    _slicer *Prev;      /* a memmove is done on sizeof(SLICE)-9 */
    struct    _slicer *Next;      /* to move everything above these fields. */
} SLICE;

I had noticed before that the existing build files for all of the ACK-3D projects (DOS and Windows) specified byte-alignment for structures and I was curious about that, figuring it was just an oversight, but I always left it alone just incase. And this code here explains at least one reason why it's like that. The sizeof(SLICE)-9 wouldn't work otherwise.

Unfortunately this is where I am currently stuck! This particular crash happens seemingly at random (I'm sure there is a pattern to it, but I cannot tell what it is yet). The links in the global Slice linked-list appear to be getting broken somewhere along the way. Other then the initial memory allocation, this loop here seems to be the only place that directly manipulates the linked-list via the Next and Prev pointers, so unless I've missed some other place (possibly in the assembly code somewhere if anywhere), then it would seem to be likely that some accidental pointer misuse or something is trashing part of the linked-list's memory occasionally. I ensured that all memory allocations that the engine does at startup get zeroed out and in this case the Next pointer it crashes on is definitely not NULL.

I intend on continuing to try getting this to work ... I feel like I owe it to my younger self, heh. My last resort is to try back-porting the Windows code to DOS since that version of the ACK-3D engine is newer (based on the timestamps of the files on the CD).

UPDATE: I wrote a follow-up post here.