This is the development log of Snowdrop OS. It was developed primarily during evenings, some longer and some shorter. Outside of this, I often spent time thinking about what to do next and how to do it.
Re-familiarized myself with boot loaders and BIOS booting process.
Read real-mode interrupt primer.
Start thinking about organization of segments (stack, etc.).
Start thinking about where applications will be loaded, and how they'll return to the OS.
Installed Bochs, a virtual machine provider which I hope to use as my development environment.
Put together toolchain - it assembles and creates boot loader floppy image from one script.
The floppy imaging tool tries to be extra smart and injects its own bootloader. dd'ing the image to insert my own bootloader simply corrupts things.
For now, I can either create a floppy image with files on it, or a bootable, empty floppy image.
About five different tools later, I can boot, as well as have valid files on the floppy image.
The problem could be that I'm missing the BPB boilerplate. I added it, and it boots in Bochs, but I can't read the FS from a DOS 6.22 VM. However, in VMWare, I can read the FS, but of course, doesn't boot.
It turns out I was missing one byte from the beginning of my BPB (BIOS Parameter Block). Once I added it, my floppy disk image booted in both Bochs and VMWare. It also became readable in both DOS and Windows XP. That one byte took 3 hours to find.
Of course, as soon as I try to copy even one file to the floppy image, it stops working (can't be read by DOS or Windows XP).
I reached a compromise where DOS no longer can read the floppy, but Windows XP can. Of course, it boots in both Bochs and VMWare.
I refused to give up and took a look at an online example of a bootloader which specifies the PBP in order to make a "proper" floppy image. While the FAT12 specification says that the first 3 bytes of the boot sector are ignored, this is not true. With a 0 there, DOS 6.22 doesn't recognize the floppy as valid. Instead, I changed the 0 to a nop (0x90) instruction, and it started working. I don't know if this is a DOS-specific thing, or not, but the generated floppy image now boots in Bochs and VMWare, and is recognized as a valid floppy disk in both Windows XP and DOS.
The build script now injects a few test files into the floppy image.
Started reading about reading from a floppy disk, since the plan is to read FAT12 and locate a file with a specific name which will contain the kernel.
Wrote some calls to help me debug FAT12 operations.
I don't know if I will be able to find a suitable debugger for Windows which will work with Bochs, so I'm going to code up some functions to dump registry values.
Found a good LBA-to-CHS translation routine and incorporated it in the code. I've tested it using a converter utility website. Looks like it's working well, but I'd like to clean it up again. The point of this routine is to be able to map a logical disk sector number (0 to n-1) to a physical Cylinder-Head-Sector tuple which is required by the BIOS's interrupt 13h (disk services). The ultimate purpose is for the loader to load the kernel.
The simple "print hex byte to screen" routine is already paying off.
TODO: Clean up LBS-to-CHS code (save results of division)
TODO: Make sure head/cylinder calculation makes total sense before moving on
I have a feeling I'll run out of space with the boot loader, so I'm refactoring the LBA-to-CHS function.
Saved 4 bytes by switching to using the stack rather than temporary registers.
LBA-to-CHS head/cylinder calculation makes sense fully now, and I've tested it using an online converter.
Saved 2 bytes from the LBA-to-CHS routine. This was possible because there's no point in preserving the ax register, since it will be set in preparation for the int 13h call anyway, right after returning from the LBA-to-CHS routine.
The code is now debug printing first three entries in the root directory.
Wrote the fragment which finds the root directory entry which belongs to the kernel file. The boot loader file size is getting uncomfortably close to 512 bytes. It will lose a bit of "weight" once I remove some of the text output functions. Found a damned bug, where if a file has the kernel file name, but a wrong last character, the code considers it the kernel.
The bug was fixed shortly after. Tested that kernel file is found regardless of whether there are other files around it on the floppy.
TODO: Convert the debug_print function to work without a character count (by expecting 0-terminated strings). Still allow character count to be specified, to dump chunks of memory more easily.
My floppy image hasn't been booting in VMWare for a while. Introduced a second build method, which builds the floppy as an flp image, using ImDisk.
Fixed a small but in the root directory search code which incorrectly left 2 bytes on the stack
I was printing out the 32-byte root directory entry to help me debug, and I was changing the DS register, but not changing it back. Since it was late and night and was tired, this mistake cost me one hour.
Learned about how FAT works in more detail. I can now load the first cluster of the kernel file.
Learned that the data area on FAT12 actually starts with cluster 2. This caused problems with offsetting to locate the first cluster storing the kernel file, before I found this fact online.
Got the loader (load then move to next cluster) code working, adapting an online example. However, it can be tidied up a bit.
Saved 3 bytes by removing a useless mov.
Saved 2 bytes by converting a mov to a dec.
Saved 3 bytes by removing another useless mov.
Saved 2 bytes by removing a push/pop pair and using a different register.
Saved 1 byte by using dec to compare to 0 a register whose values can only be 1 or 0.
I might want better messages and better floppy error handling, which is why I'm tightening up the code.
I've started writing out an example of the cluster chain calculation in comments. Basically, if I'm able to explain it, it means that I've understood it.
I'm trying to understand FAT12 bit packing, and while it's working, I'm uncomfortable that I don't fully understand it. I'm tired now, and will try again tomorrow.
This morning I found the missing link, which is IBM PC's little endianness. When you read the word 0x34 0x12 from memory, you get 0x1234. I've finished writing the example, and everything makes sense now.
I've been cleaning up the boot loader code, with cleaner comments. I also have to remove some debug/dump functions I no longer need. I'll probably add them somewhere else, perhaps a stand-alone file, to serve as examples.
In the mean time, I wrote a quick "hello world" kernel and verified that control is passed to the kernel correctly.
One thing I have not accounted for is floppy disk retries. Real floppy drives (as opposed to those emulated by VMWare and Bochs) often require read operations to be retried. Unless I find a computer with a floppy drive, I will leave it as it is.
Saved 5 bytes by converting segment register assignments to be done via the stack (push->pop instead of mov->mov).
I found that an old USB floppy disk drive I had worked as a boot device, so it's worth trying to get a real bootable floppy.
I downloaded dd for Windows (to copy the floppy image to an actual floppy), and because it doesn't show progress by default, I thought it kept getting stuck, and I kept cancelling the process and re-trying. However, after cancelling, I once tried to see if Windows can read the floppy. Although it had been cancelled half way through, the first sectors, which contain my test files did get copied, and the floppy worked.
After this, I realized dd has a --progress flag, which will show the copy progress in bytes. I created a floppy fully, and verified that the boot loader is loaded.
However, my code currently doesn't retry floppy disk reads, and it needs to, if it is ever to work off of a real floppy. I will get to work on this now.
Fixed a problem where BX was not zeroed, and by luck alone had a value of 0 from before the boot loader was entered.
I got the retry code around every int 13h call, and only have a few bytes to spare. This is because I wanted each int 13h call to display a string so that I could debug any failures which occurred on real hardware.
The loader now has retry code around every floppy operation, and every step is logged via a message to the screen.
Even better, I've tested the boot loader on 2 different floppy drives, on three different computers, and it worked everywhere. This was very good news, as I've been investing a lot of time in making sure the floppy image is as "proper" as possible.
I really wanted the boot loader to initialize all segments for the kernel, but I had to remove that code since I need all the bytes I can get, to have better messages.
Began working on the kernel. The loader only needs some clean up and comments, but is otherwise done.
The kernel registers its first interrupt, 80h, which prints a string. There will be more, dealing with the disk, most likely. They will be made available to apps.
I'm bringing some floppy drive functionality into the kernel, ultimately exposing it via interrupts. I had a choice between this and working on the user console (command line interface). I chose the floppy routines because they're still fresh in my head from the boot loader.
The kernel now exposes interrupt 81h, which loads a file from the floppy disk. I still have to decide where to transform a file name to the uppercase, blank space-padded version, as is stored in the root directory of the floppy. Currently it just loads a hardcoded file name.
More work around making the floppy drive code more reusable, by passing in a pointer to the destination where the file data will be loaded. Int 81h (load file) now returns the success status of the operation.
Decided to reduce the size of the kernel by having the console be a separate app, loaded on startup by the kernel.
Created a caller-app contract, which specifies values of registers, etc. that the consumer of an app must set before running an app. It also mentions rules the app must follow.
The kernel now loads and runs the "startup app" (whose name is still hardcoded).
Wrote some code around restoring whichever video mode was current when the kernel began executing. The idea is to reuse this code for the console app, so that it restores the video mode if any app exits with a different video mode current.
Cleaned up my build scripts and source directory.
I'm writing the routines needed to convert from a lower case string representation of an app's name to a FAT12 file name.
For a change of scenery, I opened up the aSMtris source code to look for calls to int 21h. Int 21h is the entry point into the system services provided by DOS. Since many hobbyist x86 assembly programs assume DOS is present (either natively, or via Windows's NTVDM), int 21h almost ubiquitously makes it into source code, forum posts, tutorials, etc. aSMtris is in the same bucket, but I want to make it run on Snowdrop, because every operating system needs games. As such, I must essentially implement specific int 21h functionality aSMtris uses.
Here are the TODOs for these:
TODO: implement a "clear keyboard buffer" interrupt
TODO: implement "get total timer ticks" interrupt by hooking int 8h
TODO: implement "get next random number" by relying on the get ticks interrupt above and seeding by current RTC seconds value
TODO: implement "print string at coordinates" interrupt
Implemented the "clear keyboard buffer" interrupt
Changed aSMtris "print string" routine to no longer rely on int 21h (since there is no DOS), and instead, rely on Snowdrop's int 80h, which also prints a string.
Hardcoded aSMtris's file name in the floppy load file routine. The kernel loaded it, and then ran it, but aSMtris filled the screen with characters. Quickly after, I realized int 21h expects strings to end in $, whereas Snowdrop expects them to end in a 0 (NULL) character. After updating all strings, aSMtris starts and shows Game Over immediately. This is because Snowdrop doesn't yet implement the routines needed for timing and random numbers.
It felt very satisfying to boot Snowdrop and run aSMtris on real hardware.
Finally removed the hardcoded file name inside the "load file" floppy disk routine
There's a bug in the floppy load file routine... When I try to load the config file, it loops infinitely, and this is not good.
The bug took one painful hour to fix. Not only was I overwriting the disk buffer, butchering the stored FAT, but I also wasn't popping all values on the stack when returning successfully from the config file parser.
Fixed an off-by-one bug which prevented 8-character startup app names from being considered valid.
Startup app names can now be specified using either lowercase or uppercase characters in the kernel config file.
Cleaned up all error messages.
Tested the config file reader code thoroughly.
Added to the app contract a clause on app exit. Upon startup, the caller specifies whether the app is allowed to exit or not. The reason for this is that in some instances, it is undesirable for an app to exit. One such example is the startup app (whatever that may be), which should never exit.
Updated aSMtris to adhere to the "allowed to exit" app contract clause.
Added an interrupt handler for interrupt 1C, which is invoked automatically by interrupt 8 (system timer interrupt).
Noticed that in Bochs, the system timer interrupt occurs significantly more often than 18.2 times a second. I fixed this by configuring Bochs's system timer to be bound to the host machine's system timer.
Rewrote the aSMtris delay routine to use Snowdrop's int 84h. aSMtris now runs as intended on Snowdrop OS, both on virtual machines and real hardware.
Started working on the console app, beginning with correctly handling "special" keys, such as backspace, enter, etc.
The console app now enforces app file name length limit, and is able to load files to memory.
If told to load the console app, it will do nothing.
Backspace and enter keys are now properly handled.
The console app can now load and run other apps.
TODO: better error messages for when apps can't be loaded.
Greatly improved console error messages.
Arrange source files in directories, so that the src tree is cleaner now.
Rewrote the spin wait delay routine in aSMtris into a much simpler version which has no irregularities (unlike the previous version), which ended up as a "delay" interrupt that the kernel exposes.
Introduced a "get random number" interrupt. Ultimately, Snowdrop will rely on both system timer and real time clock to generate random numbers, so that apps don't have to maintain and calculate their own random numbers.
TODO: use values read from the real time clock at boot time to seed the random number generator
Updated aSMtris to no longer calculate its own random numbers, and, instead, rely on the OS-provided interrupt.
TODO: adjust the random number formula to not alternate between even and odd numbers
TODO: manually test aSMtris for random number fairness
Wrote a test program to help me analyze even/odd distribution across a few tens of thousands of random numbers. There is some variation, but nothing too bad.
Started writing the random number initialization routine, which uses the CMOS RTC's current time.
Manually tested aSMtris for random number fairness.
The kernel now initializes the random number generator at boot time based on the RTC (real time clock). When used as a startup app, aSMtris now starts with different blocks every time, as it should.
Went through all source files and added a header which describes the purpose of the file, along with any contracts that the file must respect.
Went through all source files, trimming lines down to 80 characters and removing commented-out code.
Tested most functionality to ensure things still work after the cleanup.
Removed unnecessary files, including the FLP stack, which I haven't used in a while. It seems like the IMG stack produces the exact same results, but has the advantage that it requires no installed Windows programs. This means that I can distribute all tools needed to build the Snowdrop OS floppy image in a single zip which requires no installation.
I just realized that nasm (the assembler I use) optimizes by default. I prefer that the assembler generate exactly what I tell it to, so I turned on the switch -O0 which tells nasm to perform no optimizations. Immediately after doing so, the boot loader exploded in size to 523 bytes. That's 11 bytes I now have to trim.
Since the boot loader had generous amounts of space dedicated to user messages, I was able to shorten some of the messages to gain the 11 bytes back. I was glad I could avoid any further code changes to the boot loader, to keep things stable.
My source code is now assembled without any kind of modifications, such as optimizations.
Cleaned up the build scripts, deleting a bunch of REM'd out trials from back when I was still trying to get the floppy image bootable, as well as accessible via DOS and Windows.
Deleted about 60% of the tools I had downloaded, also from back when I was trying to properly build the floppy image. These tools were no longer used, and I only kept what I needed.
Implemented a kernel interrupt which loads the root directory into the specified memory location. This will be used by the APPS app to print the list of available apps.
Fixed a problem with the build script which erroneously copied the boot loader file to the floppy image.
Hooked up the "dump X bytes" routine to be accessible via an interrupt.
The apps app is now almost complete, and only needs some aesthetic changes.
Finished the apps app.
Wrote a mandatory hello app, which displays "Hello, world!". The hello app is meant to show a very basic example of a Snowdrop OS app which respects the app contract.
Made a few comments changes across the apps.
Started researching tools to convert the floppy image to an ISO CD-ROM image, so that I can also make a bootable Snowdrop CD-ROM as well.
mkisofs seems to be the most popular way of creating CD-ROM images. Unfortunately, mkisofs is a Linux tool, so I will try to find a Windows port.
Found a Cygwin build of the cdrtools suite, which includes mkisofs. Fortunately for me, this build included cygwin1.dll, which removes the requirement that Cygwin be properly installed. I was happy about this because it means that my build scripts continue to require no installed applications in Windows; everything needed to assemble and to create floppy and CD-ROM images of Snowdrop OS are encapsulated in the single Snowdrop OS download.
I've tested that I can boot a VMWare VM using the CD-ROM image I can now generate.
Burned the current Snowdrop CD-ROM image to physical media, and successfully booted it on real hardware.
Found a bug where when the kernel config file is not there, the kernel treats it as if it were.
It turned out to not be a bug; I had forgotten entirely to handle the "kernel config file not found" case. It was easy to fix.
Wrote an app which shuts down host computers which support APM (Advanced Power Management). It didn't take very long to get it working and I've always found it cool how software can shut down hardware.
Tested the shutdown app on real hardware.
TODO: in the console app, change tab characters to print out blanks.
The console app now prints blanks for tabs.
Began cleaning up the directory structure. The point here is to reduce the number of available build scripts, and to hide away the run scripts (those which I used for testing) in their own directory. People who download this will likely be more interested in building Snowdrop than seeing how I ran it, on my particular development environment. However, I am still keeping them for my own sake.
Added a scary "make sure you know what you're doing" message to the "write image to real floppy" script, since it can destroy media if the target drive is not the intended target floppy drive for Snowdrop.
Cleaned up the directory structure even more.
Added a kernel interrupt which offers internal speaker functionality.
Wrote a test app which plays Pachelbel's Canon in D.
Started creating a diagram showing both components and flow of the Snowdrop OS boot and initialization process.
Improved how cluster loading progress is shown by the boot loader.
Finished the sequence/flow diagram for Snowdrop OS.
The Canon in D app now prints to the screen the note being played.
Wrote the bear app which displays a bear.
Made further refinements to the build scripts, in preparation for release.
Moved most of the build script smarts inside the build directory, leaving only two scripts at the project root level: "build all" and cleanup scripts.
Started writing the documentation for Snowdrop OS, contained in the "docs" directory.
Documented all kernel-provided services (via interrupt handlers).
Documented the build process, along with potential issues.
Documented the basic steps to create apps for Snowdrop OS, including setting them as startup apps to run automatically after booting
Documented the recommended steps to extend the Snowdrop kernel.
Wrote an introductory readme.
Performed an online virus scan of the Snowdrop distributable package.
Generated the distributable floppy and CD-ROM disk images and tested them on two virtual machines, and on several real hardware computers.
Tested that the Snowdrop package works out of the box on a fresh installation of Windows XP 32bit.
Snowdrop OS is now ready for distribution!
(the first version of Snowdrop OS was released here)
Started writing the PS/2 mouse driver. I'm going off of various sources of PS/2 mouse documentation and guides.
I got the "reset" workflow to complete, but I'm still missing many pieces.
The difficulty here arises from the wildly different behaviours between PS/2 implementations of virtual machines, real hardware, etc. I'm getting different results all over the place.
Struggled for a whole day with the "get PS/2 status byte and then turn on IRQ12" step. For some reason, printing the PS/2 status byte - right after I asked for it - causes it to no longer be available to read. Since I had plenty of debug printing, this was a pickle to find.
Completed the initialization workflow, and am now looking at creating an IRQ handler.
I was happy to see that IRQ handlers for IRQ12 are registered the same way as all interrupt handlers Snowdrop OS already registers. The vector number for IRQ12 is 74h.
New problem: my IRQ12 handler seems to freeze after the first invocation, even with the proper EOI (End Of Interrupt) being acknowledged to the PIC. It turned out to be that I was not actually reading the PS/2 data port, causing no further IRQs to happen. Tested that the CPU wasn't actually halted by printing something from my timer interrupt handler.
Another difference: real hardware sends a lot more IRQ12s than Bochs. I'll have to see if differentiating by bit 5 of the status byte is enough.
On two virtual machines and two real machines, I get four different results. I am no longer confident that the initialization code will function properly on a variety of computers.
Mitigation strategy: allow user a few seconds during boot to disable the mouse driver. This is to ensure that Snowdrop OS remains functional despite driver-hardware incompatibilities.
Created the mouse interrupt contract. User programs who wish to use the mouse will register a handler which is called every time the driver accumulates 3 "state changed" bytes from the PS/2 mouse.
Wrote a routine which waits for user input (with a timeout), and doesn't load the mouse driver if the user wishes so.
Started on the idea of a "raw" stack, and a "managed" stack.
The raw stack is more complicated to work with, but gives a user program access to raw PS/2 mouse data.
The managed stack is simpler to use, implements a bounding box algorithm, and exposes less data to the consumer.
I'm trying to make it work in more than just 2 test computers. I'm adding timeouts, as that seems to be an accepted practice online. Initial results are not promising.
Now it works well in the 2 VMs, but not on real hardware (on one keyboard doesn't work, and on the other, mouse events are out of sync).
Still working well in 2 VMs, and only intermittently on a 1996 computer that has real, hardware-based PS/2.
It now works reliably on the 1996 computer! A longer timeout during mouse RESET command gave the mouse hardware more than enough time to reset itself.
To my very pleasant surprise, the above fix made it also work on a 2008 computer that has emulated PS/2 (via SMM).
These are great successes; my driver now works on 2 VMs as well as 2 real computers!
It continues to not work on a 2014 computer. I'm not very concerned, since with newer hardware I am at the mercy of their legacy PS/2 implementation. From what I've read, some such implementations are buggy.
Wrote a test program which exemplifies how to poll raw mouse data from a user program.
Wrote a test program which shows how to register a custom interrupt handler, in order to be notified when mouse events take place, and be given the raw mouse event data.
I'm learning about FPU (floating point) x87 opcodes, and how they work. This is the first time I've seen them, and they may be useful for the "managed" mouse mode, to perform divisions when converting to the user-specified coordinate system.
One important fact is that regardless of the coordinate system, a delta X or Y of 1 must always move the mouse position by 1, to allow for the most granular movements in any coordinate system. Higher delta values can scale according to the coordinate system.
Split out the mouse driver source code into two files: communication and higher-level callbacks and polling.
Wrote the mouse manager initialization interrupt.
Created the "poll mouse manager" contract, and wired up interrupts.
Implemented buttons state tracking in the mouse manager.
Added a test program which polls the mouse manager (button state and current location), and displays them on the screen.
Started on the code which translates mouse deltas to locations in user coordinates.
FPU opcodes and operating mode are different than those of the "regular" x86. The x87 FPU operates on a register stack, which I find more confusing than the more traditional "independent" registers paradigm.
I'm making headway into operations such as division and multiplication, needed for the translation to user coordinates.
Translation to user coordinates code is completed. I now have linear mouse movement tracking within a specified bounding box.
Fixed a bug where button clicks registered as +1 movements to the right and downwards.
TODO: Write a graphical example app, to test whether the mouse "feels" good, and adding acceleration if it doesn't.
I'm working on a "mouse GUI" app, which shows how to display a mouse pointer.
Improved the messages displayed by the mouse test apps.
Still working on the mouse GUI app.
More work on the mouse UI app. It's taking a bit longer because I am also writing new routines which draw bitmaps on the screen (for the mouse cursor).
I now have all the bitmap buffer manipulation routines I need to draw the mouse without "erasing" the background behind it.
The mouse pointer now moves on the screen. There is a wrapping issue I have to fix, whereby the mouse appears on the other side of the screen, when it's at the right-most edge.
The mouse is too fast. I have to slow it down.
Simplified mouse-to-box factor calculation by making cursor equally fast vertically and horizontally. This also made it "feel" better.
The mouse cursor no longer "wraps around" the screen when near the right-hand or bottom edges. The mouse GUI test program is now complete.
Introduced acceleration of the mouse cursor when the delta X or Y from the mouse hardware are above a certain threshold.
I will have to adjust the threshold and acceleration amount until they feel right.
Found a sweet spot where I'm happy with how the mouse feels.
Drew a diagram showing the components and interactions of the PS/2 mouse driver.
Documented and cleaned up mouse driver code.
Documented the new interrupts.
(the second version of Snowdrop OS was released here)
Spent some time thinking about how the scheduler would work, and what compromises are needed.
Tightened up kernel code to free up two 64kb memory segments for the memory manager.
Started writing the memory manager. So far it can allocate entire 64kb memory segments.
Wrote a memory allocation test app.
Started writing the scheduler.
The scheduler can now accept tasks, and start them. It still needs task interrupt/resume functionality.
Hooked up the code that starts the startup app to rely on the scheduler and memory manager rather than hardcode things.
The scheduler now supports task yielding. I don't yet have a good way to test it; I will once I spin off console2, which will start apps using the new memory manager and scheduler functionality, rather than just "calling far" into them, and expecting a "return far".
The only way I can test it right now is using one task (the console), which keeps yielding to itself.
TODO: Give each task its own stack.
Spun off console 2 app from the original console. It's still missing at least some error checking.
Started writing a small app to exemplify multi-tasking.
Minor cleanup around the code I recently added.
The two console apps behave differently with respect to programs returning. One calls into the program it loaded, expecting it to return upon completion (old app contract), the other registers the program as a task with the scheduler, then yielding to it. The scheduler later returns to the console either when a loaded app yields to it, or when all loaded apps have exited.
I'll probably upgrade all apps to respect the new contract, and then discontinue the first version of the console.
I'm also considering removing the "don't terminate" part of the app contract, as it isn't technically interesting. In the rare case that somebody may want an app other than the console to run and not terminate, they can provide a non-terminating version to run on startup.
Implemented "task exit" functionality, which allows a program (task) to notify the scheduler that it has finished.
Fixed a bug in console2 which caused a crash when memory could not be allocated for a new task.
Changed the scheduler to allocate each task its own stack.
Fixed a bug in the console which did not free memory when the app failed to load.
In other news, my mouse driver demonstration app no longer works for some reason...
Had a hard drive failure and lost some work.
Investigating why the mouse driver stopped working took me about six gruelling hours. It uncovered an issue with how the flags register was initialized when adding a new task to the scheduler.
Started working on a virtual display manager, so that each task receives its own virtual display.
Basically, once multi-tasking is implemented, most resources have to be virtualized. I will begin with the display, as an example.
Continued work on the virtual display manager.
Continued work on the virtual display manager.
Virtual displays are now initialized and can be recalled into video ram.
TO DO: inspect all places in kernel code where string operations (rep stosb, etc.) are used, and explicitly cld (with pushf/popf) right before.
Updated all kernel procedures which relied on string operations (rep, repe, etc.) to cld right before doing so, and to save/restore flags. This is so changes to the direction flag from user programs do not affect the operation of the kernel.
Started work on the "print string to virtual display" routine.
The "print string" routine is not trivial; it has to update the cursor, as well as write to video ram if the target virtual display is active.
This requires the introduction of the idea of an "active" virtual display, along with a way to "activate" a task's virtual display.
The "print string" routine has a bug in the scrolling code. Once scrolling occurs, no further characters are printed, for some reason...
Fixed the scrolling bug above.
TO DO: add support for the special characters 13 (carriage return) and 10 (line feed).
Added support for CR and LF in the "print string" routine.
Wrote a "move hardware cursor" routine which writes to the 6845 CRT controller to restore the position of the cursor when the active virtual display changes.
More work on wiring up the scheduler and virtual display manager into user-addressable interrupts.
Fixed some bugs which made everything crash upon trying to switch the active virtual display.
Wrote a "print character to virtual display" routine which relies on the existing "print string to virtual display" routine.
TO DO: write a "move cursor in virtual display" routine
More work on the virtual display manager.
Writing to an active virtual display will now write exclusively to the video ram. This is to cover the case of child tasks writing directly to video ram via BIOS, such as in the case of console-started apps that don't require their own virtual display.
With a few hard-coded interrupt calls, the console now starts an app and switches to its virtual display. The app writes a few strings and then exits, re-activating the console's virtual display. This is promising and I'm getting closer to my goal of having multi-tasking and virtual display support.
Wrote a "get hardware cursor position", needed for direct-to-vram string output.
TO DO: decide what happens when the task whose display is active exits (maybe store a task's parent task?)
Fixed an issue whereby the hardware cursor position was not saved when changing the active virtual display.
Migrated the console2 app to use strictly virtual display interrupts to output text, and away from BIOS "write text" interrupts.
Fixed a bug where a non-preserved register would cause a mysterious "0" to appear when the hardware screen scrolled.
While still using some hard-coded calls, the console2 loads and switches to the display of a test app, and then back to itself.
Added (rudimentary, but good enough for now) support for printing out the backspace character. As opposed to the BIOS version which just moves the cursor, my version also erases the previous character.
TO DO: consider moving an app's stack into the first 100 bytes or so (needs all apps to be ORGed differently)
Each task now stores a reference to its parent, that is, the task that started it. This lets the kernel switch to the display of the parent task of a task that's exiting, if the display of the exiting task is active.
New bug, as I'm transitioning out of debug hard-coded calls into changing out of the exiting task's display during a task exit call: cursor is not set properly, and I get some artifacts on the screen.
Spent on hour fixing the above bug. As with most bugs in assembly, it had to do with a register holding a different value than planned.
The only hard-coded debug calls now are the ones that set the active display on app load. The rest is all wired up.
Added logic to store IDs of tasks started by the console app.
Added logic to allow the user to switch between running tasks' virtual displays via ALT+F1 through ALT+F10.
Tested that virtual displays are switched properly upon task exit. Also verified that the user cannot switch to a task's virtual display once the task has exited.
Added a message informing the user of how to switch to the virtual display of a newly started task.
Factored out display driver functionality into 3 source files.
Deprecated the first version of the console app.
Deprecated the memory allocation test app.
Deprecated the first version of the multi-tasking test app.
Updated all apps to use the new "task exit" interrupt when their execution finishes.
Updated all apps to no longer rely on the now deprecated "should return" contract.
Updated all apps which change the video mode to also restore it to its initial value upon exiting.
Found a bug in how console2 tracks completed tasks. Reproduction: run the hello app a few times in a row.
Found a bug in the display driver. Reproduction: run the hello app a few times in a row.
Fixed the bug which prevented the re-use of task list slots in the console2 app.
A bunch of my apps don't output text properly because some of my older interrupts relied on BIOS. I'm converting these interrupts to use my display driver instead, for feature parity with my virtual display routines.
Also, my display driver works only in text mode, so I must change apps which use mode 13h to print text through BIOS.
TO DO: Change aSMtris and mousegui apps to use BIOS for text output in a graphics video mode.
TO DO: Change the apps app to no longer use next segment as data buffer. (either use an in-segment buffer, or allocate memory via the kernel).
Updated the aSMtris and mousegui apps to use BIOS for text output in a graphics video mode.
Updated the apps app to no longer allocate next segment as data buffer. FAT12 root directories have a fixed size, and will fit in the same segment as the app's code segment.
Added error checking in the console2 app when the user tries to switch to the display of a task that exited.
Cleaned up the multitasking test app.
After having observed bunch of people automatically reach for the arrow keys when aSMtris starts in front of them, I updated aSMtris to use left and right arrow keys to move the falling pieces.
Changed any remaining 0x syntax numbers to 0h syntax, for consistency.
Moved routines from kernel.asm into the kernel component source files which needed them. This eliminated two-way dependencies between source files.
Updated the Snowdrop app contract text in all source files.
Encountered a very frustrating bug whereby the graphical mouse test app stopped working. All other mouse apps worked, but not that one. This started happening after routine refactoring in unrelated areas.
Turns out I was only allocating one byte (instead of two) for the scheduler's initial FLAGS register value (upon starting a task). Two bytes are, of course, needed because FLAGS is a 16-bit register. While that single byte was just before a filename buffer, it overran into the first byte of the buffer, running into a happy value. As soon as I moved it into the scheduler, it overran into a bad value, causing undefined behaviour.
Documented the new interrupts, and updated the existing ones.
Cleaned up some code comments.
Drew a diagram showing the components and interactions of the multi-tasking eco-system.
Wrote some documentation.
Modified mousegui and aSMtris apps to switch to their own virtual display upon startup. This prevents the clobbering of the parent task's virtual display.
Renamed console2 to snowshell.
Began implementing a "drop piece" feature in aSMtris.
Finished the "drop piece" feature in aSMtris.
Also switched all controls to the arrow keys. aSMtris can now be played with only one hand. Space bar can still be used to rotate, allowing for two-handed play, as well.
Improved aSMtris's "last chance movement" code, to make the game feel smoother when a piece lands.
The mouse GUI test app doesn't work on real hardware, to my surprise and disappointment. At a first glance, it has something to do with the interrupts becoming disabled at some point. Sure enough, this app works in two virtual machines.
The problem was due to my test machine starting out with the interrupt flag disabled. That value was causing the scheduler to start apps with the interrupt flag off.
Packaged the new version of Snowdrop OS, and re-tested on real hardware.
Re-visited the APM power off code, in hopes of getting it to work on machines other than Bochs.
The power off code now works in Bochs, VMWare, and an old laptop (late 90s). Unfortunately, newer computers don't seem to have APM any longer, causing the code to fail.
Improved aSMtris to allow "last chance" movement after a drop.
Factored APM power off functionality into the kernel, offering it to consumers via a kernel interrupt. Updated the shutdown app.
Found a tool that can create bootable USB keys from my floppy image. I'm still experimenting with it, and may include it in the download pack.
Reverted aSMtris "last chance" after a drop, because I found it to cause me to make more mistakes.
Added to the tool set a tool which creates bootable USB keys from the Snowdrop OS floppy image.
Found a bug in aSMtris which I'm fixing before releasing: pieces which spawn right on top of a tall stack are not rendered, and the "last chance movement" counter takes longer than it should.
Fixed the aSMtris bug. The counter which allowed pieces to be slid was not initialized properly when the piece spawned right on top of a tall stack.
(the third version of Snowdrop OS was released here)
I want add delete/write capabilities to the FAT12 code, and shape it into a nicer file system driver.
Started on FAT12 "delete file" functionality.
Small cleanups to a run script.
Wrote the "mark root directory entry as deleted" part. I'm happy to see that booting a DOS image with the Snowdrop as the second floppy does not show the deleted file via "dir". Similarly, the APPS app no longer shows the deleted app.
Similarly, mounting the floppy image in Windows XP also no longer shows the deleted file, confirming the method.
Spent some time refactoring floppy code. The complexity and duplication was getting a bit out of hand, so it was time it got cleaned up and modularized into proper, reusable routines.
Completed the "delete file" workflow. It still needs some testing, but it looks like it's working for empty files, single-cluster files, and multi-cluster files.
More refactoring in the floppy code. It's time consuming and requires concentration, but it's satisfying when the resulting code is easier to read.
I finally broke down the large and scary "load file" routine into reusable parts, reducing its size to less than half of what is was.
Improved both "delete file" and "load file" to account for how zero-sized files store their first cluster number in the root directory entry.
I'm now writing routines meant to support the "write file" workflow.
Started with a simple "count free clusters", and was happy to see that the in-use and free cluster counts added up to the right total amount.
I think the best way to test either delete or write workflows will be when both are available.
I'm continuing on with "get free root directory entry" and "count free root directories".
Finished the "get free root directory entry" routine.
Finished the "count free root directories" routine.
Added better error messages for fatal floppy failures.
I now have all the parts I need to write the "write file" workflow.
Since the "write file" workflow is complex, I've started by writing out pseudo-code.
"load file" workflow now also returns the file size in bytes.
I've begun implementing the "write file" workflow.
After about two thirds in, I have a bug whereby every other free cluster is skipped, for some reason.
I fixed the bug. I was using a dummy value that exceeded 12 bits when building the cluster chain. This corrupted nearby cluster values in FAT, causing the cluster-skipping behaviour.
The "write file" workflow is now completed, but what I get is a crash, and no file yet.
The issue was an uninitialized pointer. Test file is now created. I can read its contents via DOS, but not via Windows XP, for some reason.
Another symptom of this problem is that Snowdrop's delete code dies trying to delete this file.
If I write a zero-sized file, the problem does not occur.
Snowdrop "delete file" code reads an unexpected value from FAT from the first cluster of a written file. The bug must be with how I update the FAT.
I've isolated the root cause of the FAT write bug to only odd FAT cluster entries. This is promising.
Fixed the bug. I had intended to shift left by a nibble (0xABCD to 0xBCD0), and shifted by one bit, instead of 4.
Windows XP now shows the written file properly, without any complaints.
A lot more testing is needed, including edge cases of full floppies, large files, etc.
Factored out the large FAT12 driver file into four.
Hooked up disk service interrupts and began writing a test app to exercise the disk routines.
Fixed a bug caused by a missed segment assignment.
Ran some basic tests of file creation on real hardware, via both USB and real floppy disk.
Wrote two test apps which fill the floppy data area (with large files), and which fill the root directory with many small files, respectively.
Wrapped vram and virtual "set cursor" functionality and exposed it via a new interrupt.
Implemented a "write attributes to screen" interrupt.
Updated the "get task status" interrupt to also return whether the task's display is currently active.
This interrupt can now be used by tasks that wish to take control, and not yield again, but only when their display is made active.
Implemented a "clear screen" interrupt.
Started work on a file manager, which can be used to browse and manage files.
Contributed further to the file manager, mainly around the UI.
More work on the file manager, focusing on paging.
More work on the file manager paging. Rudimentary page back/forward works now.
Added code to handle the "disk empty" case.
Re-structured code to allow re-initialization, effectively allowing user to work with multiple disks by swapping and re-initializing.
Fixed a multiple month-old bug which manifested itself if keys were pressed during kernel initialization.
The bug had no effect on some machines, froze the keyboard on others, and froze the mouse on others.
Tested the fix on virtual machines and real hardware.
The file manager can now delete the highlighted file, after waiting for user confirmation.
Initialization code is probably lacking something, as deletion from pages 2 and on bugs out the state of the program after re-initialization.
I was incorrectly restoring dirty register values after a re-initialization. The problem has been fixed.
Factored out box and box title drawing routines.
Continued work on the file manager UI.
More work on the file manager UI.
Implemented an unsigned 32bit "integer to string" routine.
The file manager now reports file sizes.
Extended the "integer to string" routine with two formatting options.
Fixed a buffer overrun bug in the "add commas" string format routine.
Fixed a number formatting bug whereby some commas were missing.
File manager now reports the amount of free disk space, relying on a newly-added kernel interrupt.
File manager now reports the amount of available file slots on disk.
Added indicator labels for previous and next pages.
Factored out "read line of text from user" functionality from the shell into an interrupt.
Unfortunately, despite my intentions, the "switch virtual display" workflow of the shell is preventing a bit of code re-use I was aiming for. The shell doesn't just read user input into lines of text; it also has to respond to "change virtual display" key commands.
Added two more formatting options.
Started working on a "page X of Y" label at the top of the file box, in the file manager.
Finished the "page X of Y" label, for the file manager.
The file manager is now complete for the time being.
Started work on the text editor app.
Implemented a string utility which converts 8.3 dotted file names to FAT12 space-padded uppercase format.
Started factoring out useful routines into a set of "common" source files that can be easily included in and re-used by Snowdrop OS apps.
Work continues on the text editor app. User file prompt is now finished.
Added a "dump virtual display to buffer" interrupt.
Added a "dump characters to virtual display" interrupt.
Spent half an hour debugging an issue caused by one wrong letter (used DI register instead of SI).
Wrote the transformation code which runs when the user saves the file.
Fixed a bug in "dump characters" routines which caused erratic behaviour for character counts of zero.
Fixed a bug in "read user input" routines which caused the escape key to be recorded as a key stroke.
TODO: disallow zero-length file names in the text editor.
TODO: write a 8.3 filename validator
Added an interrupt which validates 8.3 filename strings. I was pleased that I wrote it in one go, and the unit tests showed no bugs.
The text editor app now re-prompts user if file name input was invalid.
Factored out some re-usable screen functionality.
Bad news: sometimes, the File Manager app shows an extra, ghost file on the last page. This bug started manifesting itself as the size of the app changed due to the "common" routines changing.
A weird bug has surfaced. I have been testing the file manager with two hundred 10 byte files. Out of all these, one of the files, on page 5 of 14, shows a 3.5Mb size, instead of 10 bytes.
The bug from yesterday is proving to be nasty. Adding a byte before the "root directory" buffer shifts the bytes which represent the erroneous file size forward.
It's hard to debug, because I can't insert debug statements without running the risk of hiding the error.
Skipping some calls before the area where the file size is printed required padding with NOPs, to maintain the same binary size, to keep the error occuring.
This bug has already cost me a few hours, but I have a good lead.
I converted the 3Mb figure into bytes, and it they were 32, 49, 52, 0 (note that x86 is little endian).
I tried ASCII, and noticed that 32, 49, 52 represent the string " 14". During my 10 byte file test, I have 14 pages of files, so this was a hint to the routine which prints "page X out of Y".
Inspected the "page X out of Y" code, and if I change "itoa's" formatting option from 3 to 2, the problem goes away.
Looks like the culprit is the "remove leading zeroes" fragment, invoked when formatting an integer with options 3 and 4 (for interrupt 0A2h).
I have finally fixed the nasty bug from two days ago.
The fix involved changing a single letter. An incorrect data segment meant that the routine clobbered the caller's memory rather than use its own temporary buffer.
Improved the "dump X characters to screen" call to print question marks for non-printable characters such as backspace, line feed, etc.
Replaced aSMtris's ugly hand-drawn arrows with nicer, ASCII ones.
Brought the debug utilities to allow inclusion in apps as well.
Wired together existing "get cursor position" routines, and exposed this functionality via an interrupt.
Started working on cursor movement routines for the text editor.
Work continues on the text editor, on handling special keys, such as home, escape, etc.
More work on the text editor, around character printing.
Fixed a bug in the kernel's "output string routine", whereby a backspace character printed on column 0 would erroneously print a character rather than behave like backspace should.
The text editor can now save.
The text editor is good enough for now, as it does everything I had planned.
The two test apps which fill the disk with files now prompt the user for confirmation at the beginning.
Documented the new kernel service interrupts, and cleaned up source code comments.
(the fourth version of Snowdrop OS was released here)
I would like "format disk" functionality in Snowdrop OS.
Additionally, I want an utility which makes Snowdrop OS bootable disks, similarly to how the DOS SYS command works.
For "format disk", my plan is to use DOS or Windows to format a floppy disk, and then grab the first 33 sectors, saving them as a file that I can then raw-write from within Snowdrop.
For "make bootable", it will be like above, plus the 512-byte loader, plus straight file copying (kernel, configuration file, etc.).
Implemented a "format disk" service interrupt, and a small app to test it.
I don't yet have a proper FAT12 image, so I tested with a hand-created text file of the same length at the FAT12 image.
Used existing tools in my toolchain to make a script which generates a blank floppy disk image, and then extract the first 16.5kb.
The first 16.5kb of an empty 1.44Mb floppy disk image is sufficient to format a new floppy disk by simply raw-writing it starting at logical sector 0 on the disk.
I've always had a strong preference for very simple toolchains. Being able to generate the FAT12 image with no additional tools was very pleasant.
The format app now prompts the user for confirmation.
Wrote a "write boot loader" service interrupt.
Started on the "make boot" app, which makes a floppy disk bootable, and copies core Snowdrop OS files to it.
Refactored FAT12 driver initialization code.
The FAT12 driver now stores the disk's boot sector upon initialization to make OS core file transfer to a new disk easier.
The "make boot" app now copies the Snowdrop OS core files to a new disk, prompting the user to insert the correct disk at every step.
Although I still have to test on real hardware, the "make boot" app behaves well in virtual machines.
For now, I am happy with it, and am moving on to adding support for copying files to the file manager.
Implemented support for reading a file in the file manager.
Fixed an integer-to-string bug whereby after adding commas, the wrong few bytes of memory would be clobbered.
The cause was incorrectly nested push-pop pairs.
Implemented the "write loaded file" workflow.
Adjusted the "empty disk" workflow to take into account that a file may have been loaded from before.
Save must work even when the disk is empty, to support copying a file to another, empty disk.
File manager now reports an error message when it cannot allocate memory.
Created a test app which exercises 32bit integer-to-string formatting.
I decided to make this test app because I've found and fixed more bugs in those routines than I was comfortable with.
I decided I wanted basic support for serial port communications.
Interacting with the serial port is proving to be easy. I can already initialize, as well as transmit (via Bochs, to a file on the host computer) and receive (via HyperTerminal connecting to Bochs).
Arranged all serial port debugging scripts into their own directory inside runscripts\
Wrote a test app which registers a user interrupt handler to receive serial port bytes.
Added a "get serial driver status" system call, just like for the mouse driver.
This is because the user can choose to not load this driver, just like the mouse one.
Added a kernel interrupt that can be used by consumers to register custom interrupt handlers.
Converted kernel initialization and apps to rely on the above call to register their handlers.
The "receive bytes from serial port" app now also sends key presses over the serial port.
Wrote a small test app which sends a message over the serial port.
It looks like seemingly unrelated work has affected the file manager app, which crashes when it loads a file, in preparation for a copy.
The cause was a common library routine which destroyed a register, causing the file manager to clobber the kernel space, likely the FAT12 driver.
Inspected all common library routines for similar problems; found no additional occurrences.
My end-to-end format and "make boot" test includes starting from an unformatted disk, and finishing with a bootable Snowdrop OS disk, which boots into aSMtris.
The initial test has failed. All I get is a disk showing all sorts of 300Gb file entries.
It looks like the formatting utility reads the FAT12 image from the disk to which it is supposed to write it, rather than from the disk current when the operation started.
I'll have to fix that tomorrow.
Split formatting system call into a read-write pair.
After combinations of computers and wires, serial port communications finally worked on real hardware. The problem was either a faulty DB9 gender changer, or an overly sensitive USB-to-serial converter.
As soon as I used a no-frills null-modem cable, I had two 90s laptops talking to each other via the serial port.
Tested, organized, and documented the new scripts.
Documented the new kernel service interrupts.
(the fifth version of Snowdrop OS was released here)
Since Snowdrop OS can now communicate via the serial port, it's time to make a multiplayer game.
However I choose to handle the networking part, I need a way to buffer received bytes.
Implemented a byte-based circular queue to be used to store bytes received via the serial port.
Tested and cleaned up queue routines.
Started work on the multiplayer game. It will be a snake game where each player tries to not collide with the enemy, walls, or their own snake.
Work continues on the snake game. To begin, I want to be able to control the snake using the keyboard.
The "local" snake can now be moved with the arrow keys.
The "remote" snake can now be moved via HyperTerminal.
Began work on the networking routines.
Wrote a service interrupt which gets the ASCII character and attribute byte at a specified row-column location in the virtual display.
Implemented collision detection and transmission of "win" or "lose" packets.
I have been testing using HyperTerminal as the client, and looking at the bytes I receive, while sending directional keystrokes.
Implemented game won and lost workflows.
Implemented the game start sync for server mode.
When both snakes collide at the same time, the winner is now randomly chosen.
Started on the client mode of the game.
This is a crucial part of the development, as it is when the two bridge halves have too meet up and connect.
I've tested all I could without actually copying the executable to a floppy and trying it on my two laptops connected via serial cable.
In client mode, the game now updates snake positions based on data from the server.
In client mode, the game now handles "game over" packets.
Cleaned up code a bit.
Running the game on real hardware shows that the client redraws the snakes only sporadically.
The issue was that bytes were dequeued from the buffer and thrown away if the wrong procedure got to them first.
Fixed it by introducing a "queue peek" procedure and relying on that.
The snakes are now synchronized!
Fixed an issue where the client would render the snakes' heads in the position where they collided, overlapping borders, each other, etc.
Cleaned up colours and messages.
Added obstacles which cause you to lose the game on collision.
Users can now cancel out of "waiting for client/server", exiting the program.
Tested the "peek" queue functionality more thoroughly.
Documented new interrupts.
(the sixth version of Snowdrop OS was released here)
Started adjusting the text editor, to create a new slide show presentation app.
The Presenter app is now ready. It only took me a few hours to write, as I've used the text editor as a starting point.
Created a few sample slides.
Updated the apps app to list apps in multiple columns. This was necessary because the addition of the Presenter app, a single-column display would've scrolled off screen.
The Presenter app now lets the user specify the prefix of the slide file names, to allow loading of multiple presentations from the same disk.
The Presenter app now provides better error information to the user when the first slide cannot be found.
Added an interrupt which converts FAT12-format file names to 8.3 dot file names.
(the seventh version of Snowdrop OS was released here)
Began writing a debug system call which dumps the values of all registers to video ram.
Finished the "dump registers" call. It now also displays up to 4 bytes at the top of the stack, while not crossing segment boundaries.
Added a scheduler interrupt which allows a task to exit, but keep its resources.
It will be useful for programs which provide services to other programs via the interrupts they register.
Decided to change approach from a "exit and keep" interrupt to a "set task lifetime" interrupt, with the task invoking the same interrupt as when exiting normally.
This keeps the app contract unchanged, not increasing its complexity further.
Since this was implemented with an additional task state entry, I was thankful to my past self for leaving a number of unused bytes in each task state entry; it made this change painless.
Implemented a "set task lifetime" system service interrupt.
TODO: NOOP handlers for custom user interrupts
TODO: "get current handler" call
TODO: finish install/call example
Added NOOP custom user interrupt handlers.
Finished install/call example apps.
Improved the example chained interrupt handler to properly maintain all registers when invoking the previous (old) interrupt handler.
This required some meticulous stack operations.
Further improved the chained interrupt handler by reducing its complexity by a third.
Improved chained interrupt handler by reducing its size by another third. This was possible by no longer using the DS register to access memory.
Spent some time re-learning stack frames, since that is relevant material to my recent work involving stack manipulation via BP.
(the eighth version of Snowdrop OS was released here)
I intend to make Snowdrop OS more approachable for game development. As such, I've decided to create a software sprites library that is easy to use by consumers.
Currently, the mousegui app already shows an object (in this case the mouse pointer) that can be moved "over" a background.
I've used it as the starting point, and am now in the process of generalizing some of the app's routines.
Continued work on the sprite library. It now has enough functionality that I transitioned the mousegui app to use the library.
Started adding a simple parallel (LPT) port driver. I'm not yet sure how far I'll take it.
I can now output bytes through the parallel port and see them in a dump file in Bochs. I haven't yet tried it on real hardware.
Factored out graphics mode initialization and restoration logic into a graphics module.
Moved non-sprite specific code into the graphics module.
Renamed and commented all graphics and sprites functions written so far.
Added sprite hide/show routines.
Cleaned up and commented parallel port driver code. Also registered an interrupt handler which sends bytes to the parallel port.
Began writing an app which outputs a byte to the parallel port, while letting the user modify the byte's value.
Adding overlapping sprites revealed some issues with how background rectangles were saved and restored.
In response, I made fundamental changes to how background rectangles are saved.
The sprites of the "sprite demo" app now move in a cycle.
The "parallel port test" app now also allows rotation of the bits of the number being output.
Removed the "wait for port ready" step, since it made real hardware freeze.
The parallel port driver now reports when the computer does not have a parallel port.
I built a small circuit which connects 8 red LEDs to the data pins of the parallel port.
Through the "parallel port test" app, the LEDs can now be turned off or on!
I'd like to be able to load graphics from BMP files, so sprites can be created from them.
I've started reading about the BMP format.
I'm initially going to not support multiple palettes - just the default VGA palette.
I found that nobody has made a 256-colour test BMP which has a correct (matching that on a computer) VGA palette.
I've corrected a test BMP whose palette was almost VGA-proper. It is now suitable as my test BMP file. It is also a good starting point to create further BMPs, as it has the correct VGA palette.
Started working on a test app which displays my test BMP image.
Cleaned up "parallel port test" app.
Fixed reading mismatched height and width from BMP headers.
Experimented with BMPs created with both GIMP and MSPaint.
Wrote a function which reverses the order of all horizontal pixel lines of a loaded BMP file. This is so that they are in the same order as when copied to video memory.
Snowdrop OS can now render a 256 colour BMP file! However, palette loading is not yet supported.
Modified the "sprite test" app to load sprite data from a BMP file.
Introduce a vsync procedure in the graphics module. This is meant synchronize with the monitor's vertical retrace, preventing sprites from flickering.
Began work on loading palettes from BMP files.
I can now load 256-colour BMP palettes, but there's a mismatch between 8-bit BMP palette entries and 6-bit VGA palette entries. This causes images to look very strange at this time.
Converting 8-bit colour to 6-bit colour was as simple as shifting each palette colour right by 2 bits.
I'm having an issue with BMP files because they may or may not contain a so-called "colour space information".
Resolved my previous issue by relying on the "remaining header bytes" header entry, which helps offset correctly to the beginning of the palette data.
Factored out newly-added graphics and BMP routines into their respective modules.
Moved 8-bit to 6-bit conversion to the BMP-specific routine, fixing a bug in the save-then-load palette workflow.
Added a routine which fades the screen to black via palette modification.
Simplified palette loading and saving routines.
Cleaned up kernel initialization order. This was long overdue.
Split out the fade effect routine into a new "graphics effects" module, so apps who don't need effects don't waste memory on the effects module's buffers.
Made the fade effect much smoother by relying on subtraction rather than bit shifts.
Tested loading of BMP images created by various editors (GIMP and MS Paint).
Created BMP test image.
Converted all sprite code to work with 16x16 pixels sprites rather than 8x8. The problem now is that they flicker. It takes too long to render 50 sprites of 256 pixels each. I have to find a solution to this.
I've tested my flickering sprite code on multiple systems, and the results are strange: Old laptop doesn't flicker, new laptop does. Bochs VM doesn't flicker, VMWare does...
I'm starting to think this is related to whether vsync is properly raised.
I'm trying to update all sprite code so that it works with any size square sprites (up to a limit).
I've hit a problem where it seems like I can't use all the memory I'm setting aside for sprites. Something is overflowing somewhere, and it's very difficult to troubleshoot.
Reverted to a "last known good". It will probably be easier to start again than to get out of the rabbit hole I am in.
While making progress again, I identified the issue to be a loop index which wasn't preserved.
Fixed a small issue whereby sprites near the edges of the screen became garbled.
Tested "destroy sprite" functionality.
The sprite library now works with square sprites of any size up to 16x16 pixels. This means that they can moving objects as large as characters and as small as bullets.
TODO: Don't crash when the parallel port is not available
Simplified background generation routine for the "sprite test" app.
Factored out "allocate memory or exit" routine, since it is common to several apps.
While working on making the parallel port driver perform a NOOP when the port is not present, I found that simply entering the "send byte" interrupt freezes computers without the serial port.
The issue was a simple oversight on my part: the handler wasn't installed when no parallel port was present.
Introduced a "get parallel port driver status" interrupt, to keep consistent with other driver status interrupts I've introduced so far. Updated test app to take driver status into account.
TODO: Factor out the high-level workflow of displaying a bitmap on screen, directly from a BMP file.
Clean up and document new features.
(the ninth version of Snowdrop OS was released here)
My plan for the next version is to modify the system timer, so that it is faster than the default 18.2Hz. This will allow for more control over delays used in games and other applications.
Wrote the 8253 PIT interface code, which sets the PIT to a new frequency based on a divisor.
I now have to experiment with a higher frequency, to see how different computers (as old as mid 90s) behave.
Settled on 100Hz as a frequency for the system timer.
Re-organized kernel initialization so that the timer is ACTUALLY the first module which initializes.
Updated all kernel modules references to timer routines, changing them to call timer interrupts. This improves encapsulation.
Simplified scheduler initialization routine.
Verified that no other kernel module calls timer routines directly. Also verified that timer routines do not rely on any interrupts. This ensures that the timer has no co-dependencies with other interrupt handlers.
Added sternly-worded warnings to make future development less likely to break anything timer-related.
Increased the delay value of all kernel references to int 85h.
Updated all references to the previous 55ms timer tick length (it is now 10ms) in kernel comments.
Updated all references to the previous 55ms timer tick length in documentation.
Updated all delay amounts in apps.
(the tenth version of Snowdrop OS was released here)
My next feature is animated sprites.
I have begun modifying the sprites library to allow it to cycle through multiple animation frames for each sprite.
Made rendering faster by caching a pointer to the pixel data of the current animation frame.
Wrote a procedure which sets sprite animation parameters.
Sprites can now animate!
Adapted the "sprite test" app to create an "animated sprite test" app.
Wrote a "clear screen to colour" graphics routine.
Created a nice bouncy ball animation.
Implemented support for sprite animation start/stop.
My next area of focus are sounds.
I already have a routine which can turn on the speaker, but I want apps to be able to generate sounds of various durations without worrying about delays.
This requires a driver which takes in requests for sounds and then manages the playback time of each sound.
For my ZX Spectrum game "Husband Chores", I had created sound playback routines to cope with the computer's lack of a sound chip. They would basically slice a sound over multiple video frames, taking up a fraction of the CPU each video frame, and using it to output sound.
My intention here is to port some of the sound queueing capabilities to Snowdrop OS, so that games can play sounds over multiple animation frames, by queueing them up.
This boils down to porting Z80 assembly to x86 assembly, and then modifying sound hardware-specific parts so they work on IBM PCs.
I ported the sound routines from Z80 to x86. They rely on Snowdrop OS's two sound system calls to start and stop speaker sound output.
I currently have an issue of a previously-enqueued sounds not being thrown out of the queue properly.
Wrote a test app for the new sound routines.
Wrote utilities for displaying 32bit numbers to the screen.
Sound routines test app now displays sound queue information.
Sound playback now supports frequency-shifted sounds, to create nice sweep effects.
Polished the sound test app a bit.
Experimented with moving the "continue playing sounds" routine so that it's invoked by system timer ticks.
It seems to work well, and it would allow the internal speaker driver to be moved into the kernel. The best thing about this is that sounds will now have real time durations, rather than durations relative to the game's frame duration.
It also removes any need for any initialization from user apps.
Moved all sound queueing and playing into a kernel module.
Other than the "normal" (queued) way of playing sounds, I've introduced two more: immediate and exclusive.
A sound added with the "immediate" mode will start playing immediately.
A sound added with the "exclusive" mode will remove all other sounds from queue and keep the queue locked (preventing new sounds from being added) until it finishes playing.
Fixed a bug where a pointer was treated like an offset, causing wacky behaviour for "immediate" mode sounds.
Fixed a bug where sounds which resumed playing after an "immediate" mode sound interrupted them would just be silent.
Documented all "include" directives from the main kernel file.
Cleaned up the sound test app.
TODO: draw diagram of sound driver
Documented new interrupts.
Fixed a bug whereby an empty sound queue would stop the speaker on every system timer tick, rendering the direct-to-speaker interrupts useless.
(the eleventh version of Snowdrop OS was released here)
The last missing piece for Snowdrop OS's game development-oriented capabilities is keyboard handling. The default BIOS keyboard services are not enough to deal with checks such as "is key X currently pressed?", which are vital for video games.
My plan is to create my small keyboard driver, which can be asked about individual key status.
It will likely be based on an interrupt handler sitting in front of the default BIOS handler, chaining to the BIOS handler once it finishes storing key information based on scan codes read from the keyboard controller.
Wrote an interrupt handler which reads the most recent scan code from the keyboard controller.
Read scan code documentation.
Decided to keep things simple, and not care about the extended (escaped) scan codes. This greatly simplifies handler code, with a small cost (not being able to tell left Control key from the right Control key, for example).
Added a list of constants covering the scancodes of the entire 101 key US keyboard.
Started writing a keyboard driver test app which displays the status (pressed/not pressed) of all supported scancodes.
Hooked up a new "get key status" kernel service interrupt.
Completed keyboard driver test app. It has an issue whereby the BIOS keyboard buffer fills, causing the keyboard to become unresponsive and beep.
Clearing the BIOS keyboard buffer on every display loop iteration seems to fix this, but I would like a more elegant solution.
Investigated the keyboard freezing issue further. The problem seems to be a general BIOS approach of locking the keyboard if the buffer is full.
I noticed a similar behaviour in aSMtris as well, so it isn't just related to how my new driver intercepts scan codes.
I'm still deciding how to proceed. I will likely take an approach where a user app which expects multiple, concurrent keystrokes can choose to disable the BIOS keyboard driver before beginning operation. This will keep the buffer empty throughout the app's operation.
The other option is to completely re-write the BIOS keyboard driver - a much larger piece of work, and straying from my goal.
I've chosen to take an approach whereby the keyboard driver can be configured to not call the previous (BIOS usually) handler and issue its own EOI.
I now have a bug whereby skipping BIOS, and then changing the mode to not skip BIOS freezes the keyboard for some reason.
The cause seems to be disabling the BIOS handler invocation while an ALT scan code is still registered as pressed. This is because I'm pressing ALT+F2 to switch to the keyboard test app.
The solution was to modify the "set keyboard driver mode" code to block until all keys are released, before actually changing the mode. This way, the switch between delegating and not delegating to the previous (BIOS) handler is only done when no keys are pressed.
(the twelfth version of Snowdrop OS was released here)
This new version of Snowdrop will revolve around a game which showcases most functionality of game development-oriented libraries.
Fixed a small issue where a file name in the animated sprites app had an extra character.
I have set up a small skeleton for the game. I don't yet know what the game will be.
For now, I'm doing some game design in my head, trying to shape an idea.
The game will be called Storks and will be about delivering babies.
Started drawing a flying stork animated sprite.
Found a nice forest photograph that I can use as a background, once I apply some filters to make it look less realistic.
Since Snowdrop OS is "fully homebrewed", I stopped near a small forest on my drive home, and took a few photos myself.
I will choose the best photo to become the background image for Storks.
Finished all graphics for Storks.
Split out the defines which define sprite library limits (size and count), so that consumers can choose from multiple sprite sizes and counts.
Development continues on Storks.
Interestingly, the last few colours of the VGA palette are different on my test laptop. This caused white pixels to appear wherever colour 255 was used in the background BMP.
Factored out a function which, given BMP file data, makes the BMP's palette current.
Storks now makes the palette of the background BMP active.
Found a bug whereby a sprite that moves diagonally (both up and down) within the same frame will improperly restore the rectangle behind it.
The bug was caused by the incorrect preservation of old X and old Y when a sprite moved twice within the same frame. It's now fixed.
Found another bug related to sprite movement, specifically when moving twice along the same axis within the same frame. The symptoms are similar to the previous bug: sprite background is not preserved.
Modified the sprite test app to use more diversity in how it moves sprites, to catch issues like the two above more readily. In essence, this app is now a better test.
Moving a sprite multiple times before redrawing it now no longer corrupts the background.
Factored out the "disable BIOS keyboard handler" functionality, for apps that wish to only use Snowdrop OS's keyboard driver to check for key status.
Storks now has a wing-flapping stork that can be controlled via the keyboard, on top of a full-screen background.
TODO: Make the random number generator better. Test with the sprite test app.
Started writing a random number generator test app, meant to help visualize sequences of random numbers.
Improved the random number generator to be less streaky.
Improved the random number generator test app, so that additional sequences can be generated and accumulated on-screen.
I generated a pair of "before and after" screenshots, showing how random number sequences have changed.
The "before" screenshot shows clear bias for some values.
The "after" screenshot shows a much more uniform distribution.
I was happy to see such an improvement with just one change: a left bitwise rotation by one position.
Work continues on Storks. The game now has a menu->game loop.
Re-organized and cleaned up some procedures.
Fixed a bug in the kernel delay routine which didn't treat "wait for 0 ticks" as NOOP.
The stork's movement is now properly bounded.
Fixed a few apps which incorrectly set AX to a sprite number, instead of AL.
Wrote 2D rectangle collision detection routines in a new common module called "geometry".
Introduced a new sprite library function which checks whether two sprites are colliding, by calling into the rectangle collision function described above.
Introduced sprite show/hide functionality.
Extended the sprites test app to allow the user to show and hide a sprite, to test newly-added functionality.
Generalized bmp module's "flip vertically" routine, and promoted it to the graphics module. This is in preparation of supporting vertically flipped sprites.
Created a "flip horizontally" routine.
Simplified sprite show/hide functions greatly. As a result, show/hide no longer sometimes cause all visible sprites to flicker.
Introduced a new sprite flag: "is horizontally flipped".
Implemented a "draw transparent rectangle horizontally flipped" function, which is called when the sprite being drawn is horizontally flipped.
Introduced sprites functions to horizontally flip (and restore) sprites.
The "sprites test" app now allows the user to flip a certain sprite horizontally.
Fixed a bug which caused horizontally flipped sprites that were partially off-screen to render incorrectly.
Work can now continue on Storks.
The stork now turns when moving left.
Implemented wish (that is, what the stork must bring to a hollow) generation logic.
Cleaned up wish location in relation to each hollow.
Started on the bundle pick-up logic.
I'm getting some flicker when all sprites are visible. I can either try to reduce the number of sprites used, or improve the sprite rendering code.
Fixed a few sprites functions which were not preserving ES register.
Added a "destroy all sprites" function.
Saved up to 20 clock cycles in line drawing functions by pushing/popping specific registers instead of relying on pusha/popa.
Fixed a bug in the sprites collision detection code which sometimes corrupted the stack.
Pick-up and dropoff now work properly.
Moved all initialization into routines, to make it easy to re-initialize the game from the menu.
A new, very strange, bug has surfaced; simply requesting a random number from the kernel seems to destroy wish generation altogether.
It was fatigue; I hadn't noticed that AX register was needed farther down from where the random number service call destroyed its value.
The action key will now only work when being pushed, as opposed to also when being held down.
Bundle objects now relocate randomly vertically upon pick-up.
Added a module with functions which render text in graphics mode. The consumer can select the colour of the text.
The font I used is the same font I created for my ZX Spectrum game programming library, libzx.
The font is not complete, but has enough letters and symbols to be useful for games.
Added a convenience call which centres text at a specified Y location.
Added support for CR and LF characters when printing text in graphics mode.
Factored out ASCII code definitions out of the keyboard common include file and into a new include file.
Created a few more ASCII font characters to fill in some gaps.
Converted the Storks menu screen to only use Snowdrop OS text output routines.
Combined the two "print text in graphics mode" calls into a single function that currently supports two flags: "double width" and "centre text".
The menu now has two animated storks flanking the title.
Added finishing touches to the menu.
Holding a bundle will now prevent you from picking up another bundle of the same type.
Wrote a utility function which renders a progress bar. Storks will use it to most likely display time remaining and progress.
More work on Storks' progress and time indicators, including labels, and hooking up to actual progress and time values.
Added a frame to the animation of the "wish bubbles", to make them more visible.
Added a counter which guarantees the creation of a "wish bubble" if enough times passes without one being randomly created.
Added detection and behaviour for when the player loses and for when he wins.
Started working on a way to deal with backgrounds changing while sprites are active.
Designed the way to deal with backgrounds changing by introducing a "prepare" and "finish" function pair, which should enclose any background changes made by the sprites consumer.
Implemented pattern in Storks, and it looks good.
Introduced a way to configure the sprite library.
The first supported configuration is for the library to automatically wait for vsync before refreshing the screen.
The second is for the library to automatically animate all sprites before refreshing the screen.
These two configurations allowed for some simplifications in consumer code, as well as waiting for vsync on background changes.
Fixed a bug which prevented the final rendering of the progress and time indicators upon game end.
Changed the menu colour scheme to green-based.
Spent some time playing the game, to get a sense for its balance. It is not meant to be difficult.
The game now feels balanced to play, so I'm not planning any further changes to time and scoring.
Started on adding sounds, beginning with an inventory of what events will cause sounds to be played.
All sounds (including frequency sweeps, groups, etc.) will be in the key of D. This should keep things sounding pleasant.
The pick-up sound is a D to A frequency sweep.
The drop-off sound is an A major chord.
I chose these so that the pick-up -> drop-off sequence sounds pleasant.
Fixed a bug which caused the "wrong bundle type" sound to be played when attempting to drop off a bundle on a hollow whose wish had been previously cleared.
Decided on a pleasant G major for the victory sound, and a diminished C# for the loss sound.
Tested Storks on multiple computers.
(the thirteenth version of Snowdrop OS was released here)
This version will clean up certain aspects of the kernel which have been bothering me for a while.
Refactored parallel and serial port driver initialization to also consider BIOS Data Area's port counts.
The serial port driver no longer assumes the COM1 base address; instead, it reads it from the BIOS Data Area, just like the parallel port driver.
Abstracted serial port base address away from main kernel file, and into the serial port driver file.
Snowdrop OS now boots faster because the serial port driver no longer gives the user a few seconds to skip its initialization; that is automatic now, based on the detected port count.
Tested driver changes on all available computers.
Interestingly, only my oldest computers correctly set BIOS Data Area parameters when serial and parallel ports are disabled.
Next will be a better way of handling configuration files, including reading properties set within.
Implemented and tested a string compare function.
Started working on a whole new configuration file parsing stack.
The old configuration stack was extremely crude, and I've wanted to replace it for a long time.
Wrote an iterator function which iterates over property names and values contained in a memory buffer. This memory buffer is loaded with the kernel config file.
Moved the routines which run the startup app after kernel initialization into their own file.
Startup app file name is now reported in 8.3 format instead of FAT12 format.
Factored out code which handles kernel crashes. Tested all usages.
Wrote an atoi function.
Length of the user choice prompt timeout (during kernel initialization) is now configurable via the kernel configuration file.
Generalized and decoupled the property-parsing stack from kernel configuration file.
Factored out property-parsing stack into its own file and renamed to "parameter" everywhere.
Found a bug in the parameter-parsing stack: while the code correctly finds all properties, it returns empty strings.
The bug was due to an improperly initialized segment register. It now works again.
User choice choice prompt timeout now displays time remaining as decimal numbers rather than hexadecimal.
Random number generator tester app now uses a more pleasant yellow and has better instructions.
Storks "wish" graphics now make it easier to tell between wish types.
Used the generalized parameters stack to introduce task parameters. These are passed in when creating a task and represent program arguments.
Tested and verified that task parameters are correctly truncated past the limit. Also tested happy cases.
Began work on snowshell, so that it supports parameter data after the app name.
Snowshell now passes in parameters to the tasks it spawns.
Introduced a system call which reads parameter values for the current task.
Wrote a "params test" app which prints to the screen the values of parameters it receives.
New bug: reading the parameter value that's longer than the limit (64 characters) crashes the kernel.
Reading parameter names or values that are longer than the limit now no longer crashes the kernel.
Increased the limit for parameter values.
Updated snowshell startup instructions.
Upon initialization, the keyboard driver now configures the keyboard to be most responsive.
Upon detecting an unrecoverable error, disk drives are now given a few seconds to spin down before interrupts are disabled.
Tested on real hardware that the disk spin-down interval is long enough.
The text editor now displays the name of the file being edited.
The text editor now reads the file name from the command line.
Factored out reading and validation of an argument which must contain a valid 8.3 file name.
Clean up the text editor, since it no longer needs to display a file name entry box.
Converted the present app (slide show) to take in slide names from the command line.
Finally implemented a feature requested by many people who have tried Snowdrop OS: "previous command" (via up-arrow) in the shell.
Also implemented erasure of the current line in the shell (via down-arrow).
Wrote a file deletion app called fdelete.
Wrote a file copy app called fcopy.
Tested fcopy error cases and 0 size case.
Wrote and tested a file rename app called frename.
Display driver functions which output to the virtual display of the current task now output directly to video ram if the scheduler has not yet started.
Kernel initialization now allows the user to override the startup app configured in the kernel configuration file with a manually entered one.
Found a nasty bug which sometimes causes a kernel crash when outputting characters from two concurrent tasks.
It seems like the issue occurs whether or not the virtual display of any of the two concurrent tasks is active (on-screen).
Narrowed the problem down to one of the concurrent tasks outputting a backspace (ASCII value of 8) character.
This bug has existed undetected for a year: an incorrect jmp from a "virtual display" function into a "hardware display" function.
Corrected the bug, and the issue no longer manifests itself.
Tested printing a backspace character further and found another issue: printing a backspace character while the cursor is on the left-most column corrupts the stack and crashes the kernel.
Narrowed the problem down to inactive virtual displays only.
Fixed the issue, fixing another issue which short-circuited the "virtual display" print string routine when an inner character was backspace.
This year-old bug took me one hour to find and about ten minutes to fix.
The internal speaker can now be disabled via a kernel configuration property.
(the fourteenth version of Snowdrop OS was released here)
This release will make minor changes to parallel port driver functionality.
I am planning an app which will output text to an LCD1602 16-column, 2-row LCD.
"Get parallel port driver status" call now also returns base address.
Development proceeds on two fronts: hardware and software.
On the hardware front, I have the following questions to answer:
- can the LCD1602 work at all with 4 volts?
- using the breadboard, what is the total current drawn? (to see if it can be powered from the parallel port)
- can the backlight work with 3 volts? (if so, I can power it from a compact CR2032 3V watch battery, reducing the PCB footprint)
On the software front, I have to:
- map parallel port control register pins to LCD1602 control pins (data pins match from the start)
- start with a barebones program which outputs to the parallel port control register
- measure voltages on parallel port pins to check levels and whether they're inverted or not (documentation is inconsistent)
- expand program to where it takes in a string, which it prints to the LCD1602
The backlight can be run from 3 volts.
Worked out the full initialization sequence for the LCD1602.
Much literature states that it's a better idea to sink current into the parallel port than to source from it. If the LCD1602 current draw is small enough, I'll still attempt to source current from the parallel port.
I have a barebones program which outputs to the parallel port control register
Measured voltages on parallel port pins to check levels and recorded they're inverted or not.
Built a breadboard prototype, driven by mechanical switches.
It can initialize LCD1602 and write characters.
I've experimented more with various power sources.
Button-cell batteries simply didn't have enough juice for my needs, so I settled on three AA batteries (4.5V) for my external power supply.
Made the parallel port ribbon connector, and setup breadboard to be connected to PC.
Initialization code is not complete yet, but I can output characters to the LCD.
The test program now brings all parallel port pins low after all text has been output, so current is not unnecessarily drawn from the port.
Work continues on the driver app, mostly code and comments clean-up.
TODO: bring all parallel port pins low upon kernel driver initialization
Went back to the text editor app and restored the workflow which let a user specify the text file name in a dialog rather than forcing him to enter it on the command line.
I realized that one of the use cases of the text editor is to allow the user to fix a broken SNOWDROP.CFG kernel configuration file, in the case that the startup app misbehaves.
The user - once again - can now override the configured startup app with the text editor, to repair the kernel configuration file as needed.
The app continues to accept a text file name parameter on startup.
Soldered all components on a PCB and discovered that extra, garbage characters were displayed on the LCD.
After some investigation, the problem was found to be electromagnetic interference (EMI), generated when enough pins changed from low to high or vice-versa. It was causing the ENABLE (clock) pin on the LCD1602 to be pulsed, displaying extra characters.
A filtering capacitor on the ENABLE pin solved the EMI problem well enough to be acceptable.
Cleaned up the LCD1602 app source code.
(the fifteenth version of Snowdrop OS was released here)
This release will introduce Snowdrop OS's graphical user interface (GUI) framework.
It will allow applications to present a consistent visual interface to the user while abstracting mouse/keyboard events and GUI component behaviour (buttons, etc.).
Introduced an interrupt handler that is called by the mouse manager when mouse hardware raises events.
Started writing the skeleton of Snowdrop OS's GUI library.
The first building blocks will be the event queue plus rudimentary mouse events.
First, I want to be able to enqueue UI events from interrupt handler which responds to mouse hardware interrupt requests.
Implemented most of the logic for event addition and removal from the GUI event queue.
Spent two hours trying to understand why simple mouse button clicks are sometimes seen as erratic mouse movements.
I've narrowed it down (while fixing other bugs) to my laptop's touch pad either not playing well with either Bochs, or its Windows XP host.
If after initializing the mouse manager, a left click is the first even that occurs, everything goes bad.
I can't reproduce this in VMWare.
When adding in a keyboard interrupt handler which queues up keystroke events alongside mouse events, everything goes bad.
This problem is present in VMWare as well. I have to investigate this.
The problem is more fundamental than I thought.
When Snowdrop's keyboard driver is enabled, mouse events sometimes interfere with keyboard events.
One possible solution is to allow consumers to switch between BIOS and Snowdrop keyboard interrupts via the existing configuration interrupt.
Modified keyboard driver to either pass through or to ignore previous (BIOS) handler.
Improved GUI event queue atomicity.
I confirmed that even with no Snowdrop keyboard code running anywhere, the mouse still behaves erratically sometimes.
I've also noticed that after focusing outside my development VM, the mouse goes back to working well.
This means that, almost certainly, incorrect mouse behaviour is caused by Bochs running inside VMWare.
The next thing I want to do is devise a simple test app that I can try out on several real computers.
Its purpose is to validate the way GUI framework raises mouse/keyboard events.
There's a nasty bug I found while trying out the kbtest app.
On exit, something's going wrong with either PS/2 controller buffer or the BIOS keyboard buffer.
Right after I exit kbtest, the keyboard freezes, in both Bochs and VMWare.
This bug is extremely irritating. kbtest app exhibits the problem, but storks game does not. They are more or less the same in how they interact with the keyboard.
The difference came from the fact that kbtest app had to be switched to.
From snowshell, that was accomplished via ALT+F2, and the "clear BIOS keyboard buffer" code didn't explicitly wait for special keys (shift, ALT, CTRL) to be released.
BIOS buffer clearing routine now waits for special keys to be released, as well.
Before I continue, I want to ensure that mouse and keyboard handling work well.
Thus, I am going to prepare two test apps and try them out on all of my test computers.
Other apps will be given a quick surface test, as well.
In order to more easily allocate (static) memory for the storage of elements (button, etc.), I've decided to store each type in its own array.
To speed up development, I've decided to create so-called templates.
A template is a file containing data and functions which implement more complex data structures (such as an array).
The programmer performs a few clearly-indicated find-and-replace operations, after which the resulting code can be included verbatim in a program.
An example is a program which works with multiple arrays. In a sense, this is a rough version of OO instantiation.
It should be noted that this approach trades memory for speed and ease of consumption.
Wrote a "struct array" template, "instantiated" it as the GUI buttons array, and tested it.
Started work on GUI buttons.
Work continues on buttons, this time focusing on initial rendering and how the consumer adds a button.
More work on button styling and making it easy for the consumer to choose button dimensions to match text well.
Designed the structure and wrote the code which handles those events that are applicable to buttons.
Buttons now render released and depressed (clicked).
Fixed a bug whereby the "add button" call didn't properly return the button handle.
Added functionality for button disabling, enabling, and deletion.
Started wiring up the callback invocation workflow.
Wired up click callback invocation fully.
Added function for consumers to set and clear click callbacks.
Added a hover mark to buttons. It is displayed when the mouse cursor is hovering over an enabled button.
The hover mark displays in the top-left corner, where it is least likely to be covered by the bulk of the mouse cursor.
Introduced a convenient way for a consumer to clear the GUI completely, to help with multi-screen applications.
Buttons are complete for now.
I've now started adding checkboxes.
TODO: Call checkbox state change callback on explicit sets.
Completed all rendering of checkboxes.
Fixed two boneheaded bugs in checkbox disable and checkbox enable functions.
Setting checkbox checked state now raises "checked set" events, just like when it is manually clicked.
Fixed a bug which caused checkbox events to dequeue more bytes than intended, consuming the bytes of other events.
Started work on images.
Fixed a bug whereby checkboxes incorrectly registered a click when the mouse button went down outside the checkbox.
Images now respond to both left click and right click events.
Images now render fully.
NEW BUG: mouse cursor erases images when moved within them ("ignore transparency" mode only)
NEW BUG: button deletion no longer removes the button from the screen
Isolated button deletion bug to only when a button is deleted and another is added from an on-click callback.
Root cause was that a button scheduled for deletion was seen as an empty slot and was overwritten by the subsequent addition of a button (within the same callback).
Public functions of all GUI components are now NOOP when called with an invalid (or non-existent) handle.
Found the root cause of the mouse cursor erasing images to be how the underlying sprites library restores background rectangles.
Modified sprites test app to include a strip of the transparent colour, demonstrating the bug.
Fixed the bug by using opaque rectangle drawing instead of transparent.
Introduced selected state for images.
Added "selected state" changed event stack for images.
Images now display a mark when selected.
I've added image selection because I foresee images being useful in simple video games, or as icons, etc.
Images can now be configured to not show the mark when selected.
Images now only use a single pixel of the mouse cursor for collision detection.
Images functionality is complete for now.
Started implementing radio buttons.
Completed radio button styling and basic functionality.
Fixed a bug whereby GUI components would handle an event right after their deletion.
Implemented radio group functionality, which allows at most one radio button within a group to be checked at the same time.
Radio functionality is complete for now.
Added a function which changes an image's data (the actual pixels).
Moved mouse rendering and event handling into the mouse module.
Application title is now displayed in a title bar. Title is hardcoded for now.
Created the global X button, which causes GUI shutdown on left click.
Introduced a way for the consumer to select an alternate mouse cursor. Compared to the default cursor, its colours are inverted, so it can be more easily seen on certain surfaces.
Introduced an "on shutdown" callback for the consumer to clean up allocated memory, etc.
Introduced an "on initialized" callback for the consumer to start modifying the background.
TODO: Introduce a way for consumers to draw to the screen while keeping GUI framework synchronized.
Setting application title is now supported.
Decided on a maximum title size.
If a title isn't specified, or if it's empty, GUI framework will use a default.
Introduced a a way for consumers to draw to the screen while keeping GUI framework synchronized.
Created the hellogui app, which contains a "Hello, World!" example for GUI framework applications.
Started factoring out component limit constants, so consumers can either configure the GUI framework to use more components, or reduce the number of components to use less memory.
Also factored out GUI framework colour palette, so consumers can change the colour scheme.
Button labels are now always centered.
Started work on a comprehensive GUI framework test app.
All mouse collisions are now calculated for a single pixel in the top-left of the mouse cursor.
Work continues on the GUI framework test app.
Buttons tests are complete.
Cleaned up source code before moving on to next test suite.
I've started on the checkbox tests.
Completed checkboxes tests.
Started on radio tests.
Fixed checkbox tests and radio tests to not re-create dummies when the "create" button is clicked.
Completed radio tests.
Started on image tests.
Work continues on image tests.
Fixed a bug in the BMP library which returned bogus instead of image loading success status and image byte size.
My test image from a BMP file is not rendering properly, unfortunately.
Fixed the problem, which was an undefined image canvas width.
Work continues on image tests.
Image tests are almost done. They are taking longer because the images' API is the largest of any GUI component type so far.
TODO: Sync indicator text upon target creation.
In all test suites, creating the target now refreshes any indicator text or components.
Completed image tests.
Switched most applications to a "take control of display immediately" model.
Fixed a minor issue in sshell which required the user to release the ALT key between toggling one virtual display to the next.
Removed GUI framework debug code and cleaned up all source files.
Properly wrapped text output in the guitests application in gui draw begin/end markers.
GUI framework test apps now add components only after the framework has initialized itself.
guitests app now also shows how to use the shutdown callback.
TODO: Add a keyboard shortcut to allow exiting GUI framework applications without the mouse.
Added a keyboard shortcut which causes the GUI framework to shut down.
GUI applications now show a "loading..." notice between common_gui_prepare and common_gui_start. This notice is visible while the application is performing long running initialization, such as reading from disk, etc.
Created diagram for the GUI framework.
Tested all applications for regressions.
Tested GUI framework application on several real hardware configurations.
Documented system calls changes.
(the sixteenth version of Snowdrop OS was released here)
The next version will add a minesweeper-type game.
Set up the application shell for the new game.
Work continues on sweeper.
More work on sweeper grid initialization, which now renders.
TODO: GUI framework should no longer rely on common\queue.asm
GUI framework now uses its own custom event queue, to allow consumer application to rely on common\queue.asm without collisions.
Mine generation algorithm is complete. The game guarantees that the player will not explode a mine on his first click.
NEW BUG: Upon clicking "new game", enabling already clicked cells leaves a "selected" contour around them.
GUI framework components now also become not hovered when they become disabled, fixing previous day's bug.
The game now supports left clicks fully. It uses solid colour place holders (green for a safe cell, yellow for unexploded mine, red for exploded mine).
Added support for flagging (via right-click).
Flagged cells are no longer left-clickable.
Wrote adjacent mine count logic. I'm not happy with how large it is.
Re-wrote adjacent mine count logic. It's now three times simpler.
Started creating the graphics for the tiles (0, 1, 2, etc.)
Work continues on graphics.
Graphics have been completed.
Started writing the logic which auto-reveals areas of zero-adjacent mine cells.
More work on the auto-reveal logic.
Fixed a bug whereby revealed cells weren't flagged as such.
Auto-reveal now looks like it's working, but it needs more testing.
NEW BUG: Correctly flagged mines are marked as incorrect upon game over.
TODO: Call reveal_cell only in one place during auto-reveal.
reveal_cell is now only called once in the "recursive" auto-reveal.
Fixed the bug which caused incorrect flagging at game over.
Wrote the victory detection and handling routine.
Fixed a bug which caused not all cells to auto-reveal due to the queue being too small.
Renamed sweeper to snowmine.
Introduced a way to configure the mouse driver so that mouse movements are quicker or slower.
(the seventeenth version of Snowdrop OS was released here)
I want a barebones BASIC interpreter, as a tip of the hat to my roots as a programmer: BASIC on the ZX Spectrum 8-bit computer.
I've started with the tokenizer.
Tokenizer can now iterate over tokens in a single line.
Tokenizer now recognizes line breaks and treats double-quoted string literals as single tokens.
Started writing some arithmetic utilities such as detecting if a string contains a signed number, signed integer itoa, etc.
The good news is that I can reuse some of Snowdrop's existing unsigned integer routines.
Wrote most of the storage and management code of numeric variables.
Re-organized BASIC functionality into multiple, smaller files.
Factored out routines for dealing with signed integers into a more general include file.
Added routines to check validity of variable names.
More work on validation routines (BASIC labels, etc.)
I've made the design decision to not support line numbers in Snowdrop OS BASIC. Instead, labels can be used to reference specific lines, just like in assembly languages.
The reason for this is most line numbers are useless overhead, as my intuition tells me that under 20% of lines are referenced by branch instructions.
Not to mention that beginners often don't leave enough "room" between line numbers, causing tedious re-numbering passes during program editing/re-factoring.
The only reason to have numbered lines is if the programmer's editor is restricted to operating on a single line at a time. In such a case, the programmer must have "handles" to EACH line to delete, edit, etc.
Started on string variable storage and management.
TO DO: implement numeric and string variable deletion; modify test code to also delete
Added variable deletion functionality for both numeric and string variables.
One nice pattern emerged here: zeroing out fields on create (rather than destroy) means that setter routines don't have to check if the array entry is in use. If it's not, as soon as it's allocated, any garbage written there will be zeroed out.
Modified string variable value getter to fill a provided buffer rather than return a pointer to its internal storage.
Fixed a tokenizer bug whereby UNIX line endings would cause the line end token to be skipped.
Started building out the BASIC entry point routine, along with line counting and stop condition.
Modified token parsing to no longer require expensive zeroing out of token buffers.
Fixed a tokenizer bug whereby characters immediately before or after double quotes would be counted as part of the string literal token.
Added support for instruction delimiters in the tokenizer. This will allow for multiple instructions on a single line.
Further work on utility routines such as for dealing with quoted string literals.
Began work on state transitions and instruction accumulation. This part is difficult.
TO DO: introduce an artificial newline at the end of the program text, to force execution of last instruction.
After all tokens have been read from the program text, the interpreter adds a newline token, to ensure the execution of the last program instruction.
Started working on a debug stack which will help me debug the implementation of the supported BASIC keywords.
Wrote the ok/error message stack, which allows errors and messages to bubble up to the highest level, which will display the specific message, along with a line number.
Cleaned up instruction token storage.
Invalid state transitions are now reported.
Simplified how end messages are set and stored.
BUG: instruction counter is incorrectly reset by the artificial newline inserted at the end.
TO DO: consider switching to an artificial semicolon, only when the last token in the program text is not a semicolon, newline, or label.
TO DO: label validation should check label uniqueness
Whether an artificial instruction delimiter token is added can be decided more easily based on the interpreter state.
It should be added only if the interpreter state is "within instruction", meaning that the current (last of the program) instruction hasn't yet been executed.
Fixed a bug with misreported line numbers when program text ended in newlines.
End of interpretation messages now also include token number.
The interpreter now asserts label uniqueness.
TO DO: report last read token to the user, in case of an error
Extended numeric variables with properties needed by for-loops.
Designed an easy way for the interpreter to change the continue point in the program text.
Wired up execution stack to the point where it calls the procedure meant to handle the PRINT keyword.
Started work on expression evaluation.
PRINT instruction can now output single tokens (literals and numeric/string variables).
Expression evaluator now supports concatenation of strings.
TO DO: test string concatenation routine more thoroughly
Implemented concatenation of string and number.
Implemented two-operand one-operator arithmetic evaluation. It supports addition, subtraction, multiplication, integer division (quotient), and modulo.
Rewrote keyword validation routine to no longer be naive.
Added PRINTLN instruction.
Started on the LET instruction.
The LET instruction is complete.
This is a good milestone, because the ability to define variables makes testing easier.
TO DO: require REM instruction (coming soon...) comments to be quoted string literals. This is a small cost to pay to keep token lookup simple.
Added the GOTO instruction.
Completely modified how current line number and current instruction number are managed.
Instead of being tracked, they are now calculated.
This fixed issues the old method had with branching.
Fixed an off-by-one bug in the current line calculation algorithm which took place when the program text ended in a newline.
TO DO: load program in a different segment, to test that there are no segment assumptions and that register preservation is correct.
I just had a very interesting idea: I can create a BASIC linker that, given a .BAS source file, can produce a standalone, executable app by simply including the linker's own binary, with a modified initial jmp, so that it jumps into code which starts up the interpreter, pointing it at the appended .BAS file.
Program behaves strangely when the program text is loaded in a segment other than CS. I have to find the issue.
Thankfully, there was only one instance of where ES was not set to CS. BASIC programs are now interpreted successfully from a different segment.
I'm taking a break from writing instructions and am now working on the linker.
BASIC linker now handles gracefully source files that are too large, by printing an error message to the screen.
TO DO: Fix bug which causes the last GOTO in the program to not be executed when it is not followed by a ; or a newline.
Fixed the bug which incorrectly ended the program if the last instruction was a branch instruction.
BASIC linker now reports maximum supported source file size.
The linker is complete for now.
Fixed a bug whereby certain conditions caused an infinite loop while parsing.
Branching via GOTO now relies on a cache to potentially avoid an expensive label resolution.
CTRL-Q now breaks program interpretation, allowing the user to exit.
Added the REM instruction.
The next instruction I want to add is FOR.
I've spent some time researching how Sinclair BASIC (my first programming language and main source of inspiration) handles FOR loop redefinition, no-iteration cases, and NEXT position rules.
Began work on FOR-NEXT loops.
Fixed a bug in the token lookup code which erroneously reported it found a certain token when it didn't.
FOR-NEXT will support STEP to specify increment.
FOR and NEXT now work. They were more difficult to design and implement than the other keywords so far.
Fixed a bug whereby an overflow during NEXT step application caused an infinite loop.
Implicit STEP value is now always +1 (even when initial value is larger than the final value).
FOR counter variables can no longer be references from TO or STEP.
Started on the IF instruction.
IF now correctly identifies the token index range of the branch (THEN or ELSE) to take.
Went over the instructions I've already implemented and ensured they all properly preserve registers.
IF-THEN-ELSE now chooses and executes the right branch.
TO DO: consider whether to limit the amount of sub-IFs/THENs/ELSEs in an IF instruction.
Nested IFs are now no longer supported and will report an appropriate error.
Implemented arithmetic comparison operators.
Wrote a reusable stack, a dependency of CALL and RETURN.
Implemented CALL and RETURN.
Fixed a bug in the stack code which dereferenced stack pointer's address only once (instead of twice) when popping off stack.
Removed some unnecessary buffers and reduced token length. The result was a significant reduction in the interpreter's binary size.
Started on the evaluation logic for [function] [argument] expressions.
Numeric literals are now checked for overflow.
Work continues on wiring up [function] [argument] evaluation.
[function] [argument] evaluation is now wired up.
Implemented the LEN (string length) function.
Implemented the RND (random number) function.
Implemented the INPUTS (read string from user) keyword.
Implemented the INPUTN (read number from user) keyword.
Added support for string comparison.
Implemented the AT keyword, which sets the cursor position on the screen.
Factored out expression evaluation of instructions with two numeric expressions separated by comma.
This greatly simplifies AT and future two-numeric-argument instructions.
Implemented the COLOURS keyword, which sets the current font and background colour for text output.
PRINT now takes current colours into account.
TO DO: bound hardware version of "write attributes to display" to the 4000 bytes of mode 03h.
Created a helper which gets the numeric value of the expression of a single-expression instruction.
Implemented the PAUSE keyword, which blocks execution for the specified number of hundredths of second.
Implemented the WAITKEY keyword, which blocks execution waiting for a keypress, which it populates into a variable.
Implemented the KEY function, which checks (non-blocking) whether or not a specified key is pressed. Its purpose is video games input checking.
Implemented the NOOP keyword, which does nothing.
Fixed an improperly preserved register, which caused expression arguments to be evaluated improperly.
Implemented logical NOT, AND, OR, and XOR.
Fixed a bug which incorrectly overflowed FOR loop counters (halting execution) instead of just ending the FOR loop.
Implemented the BEEP keyword, which plays a sound (non-blocking) over the internal speaker.
For frequency numbers near zero, Bochs crashed. I'm not sure if this behaviour is shared by real hardware, so I limited (slightly) the smallest allowed frequency number. This limit has little impact, since sounds are barely audible at those frequencies.
Implemented the STOP keyword, which immediately halts program interpretation.
Added a kernel interrupt which clears the sound queue.
Updated the SNDTEST app to also allow user to clear the sound queue.
At the end of interpretation, the BASIC interpreter now clears the sound queue, silencing any sounds played by the program.
Cleaned up includes.
Implemented the BEEPW keyword, which behaves just like BEEP except it blocks while the sound plays.
The COLOURS instruction now supports the eight "bright" font colours as well.
Generalized tokenizer's "stop character" concept. This removed the requirement of blank characters around argument-separating commas.
WAITKEY now properly reports user break requests via CTRL-Q, halting program.
Implemented the CHR function, which returns a string with a single character whose ASCII value was specified as an argument to CHR.
Implemented the ASCII function, which returns the ASCII numeric value of a single-character string.
Implemented the VAL function, which takes a string argument containing the representation of an integer, and returns that integer.
Snowdrop OS kernel now sets text video mode 3 at startup. This has been the de-facto text video mode throughout Snowdrop OS, and this change makes this explicit.
Wrote the BASICRUN application, which takes a BASIC source code file and runs it.
Started work on the "run BASIC program" workflow of the text editor.
The text editor now supports running BASIC programs.
Removed all debug utilities and left-overs from the BASIC interpreter.
Started work on a sample BASIC program which moves an object on the screen.
Snowdrop OS BASIC will include a few simple examples out of the box, which I'm now writing.
Fixed a bug in the keyword INPUTS, which didn't preserve a pointer in DS:SI while calling a routine which outputs in SI.
Fixed a bug in the line:instruction reporting routine which assumed DS = CS.
BASIC examples are now complete.
Documented BASIC keywords, syntax, etc.
(the eighteenth version of Snowdrop OS was released here)
In this version, I plan on allowing installation of Snowdrop OS on hard disks.
The essence of this change is to exclusively read single sectors via int 13h.
This must be applied to both boot loader and kernel. The boot loader will surely need some further size optimization to still fit in 512 bytes.
Started work on transitioning the boot loader from bulk sector reads to single sector reads.
The reason for this is to be able to force custom CHS geometry on hard disks.
From the looks of it, I'll need to further reduce the size of the binary; it has to be kept within 512 bytes after these additions.
Replaced some zero assignments with xor, to save one byte per occurrence.
Removed unnecessary comparison to zero after a zero flag-altering opcode.
Restructured loops to save on some jumps.
Removed an unnecessary clc after a zero assignment via carry-clearing xor.
Saved some bytes relying on an additional segment register, no longer needing some push/pops.
Saved one byte from an unnecessary pop right before a "kernel not found" fatal error is reported.
Tested boot loader changes by moving the location of the kernel file within the generated floppy disk image.
On the kernel side, I must modify the FAT12 file system driver to read and write single sectors.
Writing seems to all go through a floppy_write_sectors routine, which I can rename to _single, and then create a new floppy_write_sectors that iterates.
Reading is harder; it looks like I'll have to factor out a "read sectors" routine, or perhaps use the boot loader's.
Factored out "read sector" into a single routine, referenced by all FAT12 routines (read FAT, read root directory, etc.).
Added another layer to "write sectors" such that single sector writes are exclusively used.
Fixed an off-by-one "print string" bug created by overzealous optimization of the boot loader.
Wrote code which finds the geometry (CHS) of the first hard disk and reports it.
I tried this on multiple physical machines. Those with very large hard disks reported maximum numbers for cylinders (1023), heads (255) and sectors (63).
Generated a 10Mb hard disk image for Bochs to use during development.
Updated Bochs run scripts to use this image.
Began writing the kernel initialization step which allows the user to install to hard disk.
Work continues on installation to hard disk logic, including user confirmation, visual progress indicator, etc.
Wrote final checks and error handling.
Wrote main "copy sectors" loop.
After a quick and dirty test on real hardware, Snowdrop OS booted from a hard disk for the first time!
Cleaned up boot loader style (variable names, etc.).
Active (boot) drive number is now displayed at boot time.
Cleaned up BASIC samples.
(the nineteenth version of Snowdrop OS was released here)
This version of Snowdrop will include improvements to the text editor, to make it more suitable for BASIC development.
Modified the text editor to allocate and load the text file in an allocated segment. This change allows for larger files to be loaded.
Switched the text editor to use a separate (allocated) segment for the rendered file text.
Switched the text editor's initial rendering to be page-based.
NEW BUG: multi-page text files corrupt first page on initial rendering.
Fixed the multi-page bug. It was a simple display problem, not a text processing one.
Multi-page BASIC programs are now run successfully.
Removed the old way of displaying a whole screen, as it is no longer used.
Converted page-based viewport to line-based, for smoother scrolling.
Started updating cursor and page key functionality.
TODO: Review all functions for proper DS setting
Updated all functions which used DS:SI and ES:DI pointers to properly initialize and preserve DS and ES. They also no longer assume DS=ES=CS.
Added error checks for loading files that are too large.
Arrow up/down and page up/down now work with paging and respect maximum file size.
TODO: print keyboard command help on console upon startup
Commented all functions.
The overlay at the bottom of the screen now shows the current line number.
Updated text editor's save functionality to work with multiple pages.
Fixed an insidious bug in the text editor's "read user input" loop, which slowly ate away at the stack.
Added a "delete line" text editor command.
Added an "insert line" text editor command.
Added copy/paste functionality to the text editor.
The text editor has enough functions for now, so I'm turning my attention to the BASIC interpreter for some improvements.
The BASIC interpreter now displays last parsed token and last parsed instruction when an error occurs.
Implemented the PARALLELW (parallel port write) keyword.
Tested PARALLELW on real hardware using my LED parallel port device.
Text editor's "cursor end" command now moves cursor to right after the last printable ASCII on that line.
Text editor's "cursor home" command now moves cursor to right before the first printable ASCII on that line.
Implemented the BASIC function BIN, which converts a string containing a binary number to the respective numeric value.
Implemented the BASIC bitwise operators BITAND, BITOR, BITXOR.
TODO: Add a "BASIC help" screen in the text editor (F5 maybe?)
Implemented the BASIC bitwise operators BITSHIFTL, BITSHIFTR, BITROTATEL, BITROTATER.
Added a BASIC documentation text page which can be displayed from the text editor.
Started working on the serial port read/write in BASIC.
Implemented the SERIALW (write byte to serial port) keyword.
Started on a way to read from serial port in a blocking fashion, while still allowing user to break.
Finished the SERIALR (blocking read from serial port) keyword.
Tested SERIALW and SERIALR on real hardware.
TODO: Make CTRL-L no longer move cursor to next line in the text editor.
Added function SERIALDATAAVAIL which tests whether data is available via serial port.
Checking this before SERIALR will ensure SERIALR will be non-blocking.
Introduced atomic (hardware interrupt-safe) wrappers for all functions of the common queue.
CTRL-L now brings the cursor home on the current line after clearing it.
Started work on adding support for two-argument functions.
Added function CHARAT which returns the character printed on the screen at the specified row and column.
Started work on adding support for three-argument functions.
Started work on a substring function.
Added function SUBSTRING.
TODO: document SUBSTRING in basichlp
Tested that SERIALDATAAVAIL works well with SERIALR on real hardware.
Documented all added functionality.
Added function STRINGAT, which returns a character from a string.
Identified two similar bugs which cause an infinite loop when:
- (consistent reproduction) assigning a numeric value to an existing string variable containing the empty string
- (inconsistent reproduction) assigning an empty string value to an existing numeric variable
I'm chasing the first one and it's going to be tricky. If I insert a nop at the beginning of the numeric variable allocation function, it works.
After two hours, I've isolated further to:
- LET var = "" at any point in the program causes undefined behaviour either in preceding or following LETs, irrespective of variable to which they assign, but only for numeric values.
TODO: clean up common_string_copy.
Eliminated expression evaluation from the possible culprits list.
My hypothesis is that at variable re-assignment time, variables may be deleted.
The pointer used to look up the "to be deleted" variable MIGHT point at the LET value instead of variable name.
For non-empty string values, variable lookup fails.
However, for empty string values variable lookup finds an empty spot since initially all variable names are empty strings (zeroed out).
This causes undefined behaviour, which I fixed by properly setting the pointer to the variable name instead of the expression value.
One question remains: why does basicNumericVars_get_handle return garbage when the variable name is the empty string?
I was wrong: variables are actually cleared to 0FFh, and not to 0.
However, there was a SECOND bug, this time in basicNumericVars_get_handle, which performed one more comparison than it should have (ja instead of jae), overrunning variables storage.
This, coupled with the fact that the first byte immediately after storage was a zero (encoded from "call basicNumericVars_find_empty_slot"), caused the comparison to match the empty string input.
NOTE: this is why adding a nop made it work - the first byte was no longer zero, so comparison failed.
This invalid (overrun) offset was returned, causing subsequent logic to overwrite executable code with data.
When the program flowed into that area, it caused undefined behaviour.
I think this is the best bug I've seen and fixed so far, so I'm going to document it further.
Fixed the overrun for numeric variables.
Fixed the overrun for string variables.
This bug took around four hours to find and fix. It was due to a perfect storm of:
- SPECIFIC CASE: variable value set to "" (empty string) via LET var = "";
- BUG1: pointer pointing to value instead of variable name; used for variable lookup by name
- BUG2: variable lookup overran variable storage (ja instead of jae)
- BY CHANCE: executable code immediately after variable storage starts with a 0 byte, matched to the input empty string
- executable code is then overwritten as if it were part of variable storage
- undefined behaviour is caused
Cleaned up common_string_copy and re-tested.
(the twentieth version of Snowdrop OS was released here)