Accessing Squeak's image from the outside

   After some time playing with SqueakNOS, you'll notice you can't save the image from the inside of the VM. Also, you may want to introduce some code from the outside,  and if you are like me, you just like working in normal squeak and then migrate the changes to the VM. So there has to be a way of modifying the image.
   To do so, you have to mount the Hard disk/ISO image in a folder, read/write the .image and .changes files and then unmount.

Mounting the virtual hard drive

In case you are using the VM appliance, you'll mount the VM disk's file system inside your real file system. You can do that with this command (which gets installed with VMWare Player)

$ vmware-mount SqueakNOS.vmdk /vmdisk

You have read/write access here, so you can work with normal squeak, save the image and finally copy it overwriting SqueakNOS.* inside the mounted directory. When you are done you do

$ vmware-mount -k ../SqueakNOS.vmdk

to unmount. When you restart the VM you'll see all the changes you did from the outside.

Accessing the ISO image

Here we get into trouble: ISO images can only be mounted as read-only. This means that you'll be able to read the files inside it, but wouldn't be able to write to it a new image in case you make a change from the outside. To solve this problem, you can regenerate the ISO with a new image.

Unpack the .iso in some place, overwrite the .squeakNOS.image/changes files and then issue:

~/isocontent/ $ cd platforms/SqueakNOS
~/isocontent/platforms/SqueakNOS$ make iso

that will generate in release folder a new bootable .iso with the image you just placed.

Getting started with SqueakNOS

The idea of this article is to explain what are the different bundles that are available to deliver SqueakNOS.
You probably already know that SqueakNOS is(n't) an operating system. Then, the way to use it is by booting it's code from some media, like a CD or a hard drive. The good news is that it's not necessary to alter your hard drive in any way, nor reset your PC each time you want to test SqueakNOS, and that's possible thanks to virtualization. The idea of virtualization is to simulate the existence of an entire machine (processor, memory, hard disk and other devices), where you can install an entire operating system. The software used to do this is called a virtualizer, and there are many of them available, including VMWare Player, Sun's VirtualBox, Bochs, QEmu, etc.

SqueakNOS bundles

SqueakNOS comes bundled ready to be used in two ways:

• as a bootable CD, actually an ISO image, which you can burn and boot in your machine or load into the virtualizer to simulate a CD (as you'll see in the next section).
• as a small bootable virtual hard drive, prepared to run inside the virtualizer.

It isn't necesary to download them both, you should choose the one you feel more comfortable and use it. The virtual harddisk has the benefit of allowing you to mount it read/write in your filesystem if you need. With the ISO you can mount it read-only, and if you want to change any file you have to regenerate the image (which gets done with one simple command).

To download any of the bundles go to SqueakNOS files section in SourceForge.
As of 19-feb-2010, the hard disk image is some days more recent than the ISO image, but that should change quickly, we are preparing an update right now.

SqueakNOS in a virtualized environment

The easier way to test and work with SqueakNOS is through virtualization. That means that with the help of a virtualizer you'll create a virtual machine where you can install and boot the SqueakNOS easily. I use VMWare Player because it's a the fastest virtualizer I know, and also because it comes with vmware-mount tool which is useful to access disk images outside the VM. But there are many others supported, including Sun's VirtualBox, Bochs, QEmu, Microsoft's VirtualPC and Parallels.
Both ISO and hard drive images come with a .vmx file. This is a simple text file that VMWare uses to describe the virtual hardware. By opening it with VMWare Player you can run SqueakNOS. It takes just seconds to load the image, after this you should see the video mode changing.

SqueakNOS in a real system

The easier way of trying SqueakNOS in a real system would be to burn the bootable ISO to a CD and boot from it. Installing it to the hard drive is a bit more complex and will be addressed in another article.

Phase 3: Give it awareness [Almost done]

Make keyboard and mouse work. This let us actually try SqueakNOS. Make hard drives work, this will let us code SqueakNOS from within SqueakNOS.

Having keyboard and mouse opened the door to playing with the booted system. Now, we want to be able to save the changes, so we need some persistency. One idea (taken, again, from the old SqueakNOS), was to implement serial port support, which is already done, and some type of File System over it.

Other possibility is to implement hard disk drivers, which is also done, and after that basic FAT32 support. This will let us open .sources and .changes files.

Phase 2: Make it breath [Done]

Add graphics mode support, and native IRQ setup and handling

Graphics mode

After the nice suprise that GRUB had graphic mode switching almost ready to be used, we jumped into making it work for us, and we chose a graphic mode that is really nice AND maps 1 to 1 from Squeak's representation to video RAM: 1024x768x32 bits is our choice although any 32 bits mode would suit the needs. Hacking a ioShowDisplay() that works was easy: it's no more than a bounded memcpy(). With this we got some graphic output in SqueakNOS' screen!

With the old SqueakNOS we had some problems at this point, mainly because Squeak does not refresh the screen upon entering, so this time we solved it really fast, by doing a "self currentWorld fullRepaintNeeded" after a small Delay, and voila! Squeak's full screen was shown!

Now we were facing the most delicate part: making IRQs properly work. Subtasks: set a void handler for every IRQ and enable IRQs (see that it doesn't halt). Add an infinite loop handler for the keyboard IRQ (see that it hangs on a keypress). Set a visible handler for the timer interrupt (see that it draws something on the screen... you can still see it in the upper right corner of SqueakNOS). Finally change all void handlers for code that will trigger a Squeak Semaphore and exit, also add the primitive to register the Semaphores (see that it doesn't hang, then see that the Semaphore gets triggered from inside Squeak). At last, loop waiting on the semaphore for keyboard's IRQ, read keyboard's port, and signla IRQ end all from inside Squeak. [Done]

SqueakNOS IRQ Handling

Squeak VM lets you signal Squeak Semaphores from the native world by calling signalSemaphoreWithIndex(). We want to serve IRQs from Squeak, using interpreted code, not native code. We seriously think that with hardware close to 1000 times faster than 20 years go we should be able to do it without any problems. Of course we cannot set the native IRQs handlers to jump to Squeak code, so our idea is to have a different Semaphore for every IRQ and have a Squeak Process with highIOPriority blocking on the Semaphore.

The code inside Squeak, taken from InterruptRequestDispatcher>>installOn: looks like:

installOn: aComputer
self registerSemaphore.
process := [
 [
  semaphore wait.
  self handleOn: aComputer.
  aComputer interruptController signalEndOfInterrupt: interruptNumber.
 ] repeat
] fork priority: Processor highIOPriority.
aComputer interruptController enableIRQ: interruptNumber

And the native code to signal the Semaphore, taken from ints.h, is similar to:

 void irq_1_handler();
asmlinkage void ISR_1() {
 if (0!=IRQSemaphores[number])
   signalSemaphoreWithIndex(IRQSemaphores[number]);
}

asm(
    ".text"  
    ".align 16"
    "irq_1_handler:"
    "pusha"
    "call ISR_1"
    "popa"  
    "iret"  
)

There is an interesting detail in all this: the software interrupt ending (IRET) and the hardware IRQ ending (outb(0x20,0x20)) are detached in SqueakNOS, were they are almost always done at the same time in every other code we saw. This gives a really desirable result: The IRET lets the software continue, going back to the interpreter and letting the Process waiting on the Semaphore be rescheduled, however, the hardware part (Interrupt Controller) still thinks the IRQ has not been served yet, and will wait until the Squeak side of the handler (shown above) signals the end of the interrupt (aComputer interruptController signalEndOfInterrupt: interruptNumber.

We seriously believe that with computers close to 1000 times faster than 20 years ago this should be ok, however we are not sure yet if it may bring any problems later, and we are open, although reluctant, to the possibility of coding some glue code, or even complete "device drivers" natively (or hopefully using Exupery). Probably for sound and video I/O, we'll see.

We were successfully using the very same scheme in the old SqueakNOS, so we just ported it to the new code base.

Phase 1: Make it boot [Done]

Make it boot and start interpreting the .image with text output for tracing and debugging.

The old SqueakNOS used a custom made mutant boot loader, borned from mixing LILO, the Linux kernel boot loader, and our dirty fingers. We had quite a few problems with it, and it was really complicated to put the needed environment to compile it. So we chose to replace it for something better and more modern: GRUB.

We first thought on GRUB because it, of course, handles all initialization, switches to flat 4G mode can load huge kernles in high memory, and also has support for modules loading (module and modulenounzip GRUB commands). We wanted to detach the native part and the Squeak .image, which were glued in a single file in the old SqueakNOS, making it easier for Squeakers to touch the .image without having to set up the compiling environment.

We needed to implement the switch to graphics mode before starting jumping to the interprenter. In the old SqueakNOS we did it from real mode (that's why we had so poor video support). But this time GRUB gave us control already in protected mode, so we couldn't just call the right INT. Luckily, GRUB already has the roots for supporting graphic video mode selection, and although it is not usable in the standard GRUB distro, we could solve the problem quite easily, by copying the testvbe command into the setvbe command, and removing everything after the command switches the video mode. We later found out that the video memory address changes from box to box, so we also hacked the vbeprobe command to show the video memory address.

We also learned that GRUB supports part of the multiboot protocol and is able to load ELF binaries, this led us reduce the amount of assembly code to barely around 20 line. It's not that we are afraid of assembly, but if multiboot was supported in some other platform (Apple/PowerPC), porting SqueakNOS' kernel should be really easy. [Never tried]

At this point we had GRUB loading the kernel and loading the .image all in memory, we had some glue code to make the .image look like a file so Squeak's intepreter could start interpreting it. We coded console text output support, hacked some debugging messages in the interpreter loop, some stack traces here and there and figured out what Plugins needed to be compiled for Squeak to start up: we finally managed to make the interpreter loop run loose.

Implementation idea

We want to get rid of the operating system backends of Squeak/Pharo. For this we know we will need to implement lots of low level things, we are not afraid, but we want to take a little different approach: Implement the bare minimum as native code (a mix of assembly and C), and then do everything else in Squeak.

Currently, we have implemented in Smalltalk drivers for mouse, keyboard, serial port, ATA devices (hard disks), some network devices, and others.

A little history of the implementation

The people originally in the group only knew low level PC stuff, so we started with this, however there is already people participating that want to give Apple/PowerPC a try!

There are two incarnations of SqueakNOS, one was pretty much abandoned in 2001, and had support for 640x480x1 or 640x480x4, keyboard, mouse, serial port and a pseudo file system over serial port, although some of this features never saw the light. In May 2006 we restarted SqueakNOS, and a new version was released in May 16th, with support for 1024x768x32, and bare IRQ handling.