Mostrando entradas con la etiqueta history. Mostrar todas las entradas
Mostrando entradas con la etiqueta history. Mostrar todas las entradas

miércoles, 21 de abril de 2010

Phase 2: Make it breath [Done]

Add graphics mode support, and native IRQ setup and handling
Graphics mode

After the nice suprise that GRUB had graphic mode switching almost ready to be used, we jumped into making it work for us, and we chose a graphic mode that is really nice AND maps 1 to 1 from Squeak's representation to video RAM: 1024x768x32 bits is our choice although any 32 bits mode would suit the needs. Hacking a ioShowDisplay() that works was easy: it's no more than a bounded memcpy(). With this we got some graphic output in SqueakNOS' screen!

With the old SqueakNOS we had some problems at this point, mainly because Squeak does not refresh the screen upon entering, so this time we solved it really fast, by doing a "self currentWorld fullRepaintNeeded" after a small Delay, and voila! Squeak's full screen was shown!

Now we were facing the most delicate part: making IRQs properly work. Subtasks: set a void handler for every IRQ and enable IRQs (see that it doesn't halt). Add an infinite loop handler for the keyboard IRQ (see that it hangs on a keypress). Set a visible handler for the timer interrupt (see that it draws something on the screen... you can still see it in the upper right corner of SqueakNOS). Finally change all void handlers for code that will trigger a Squeak Semaphore and exit, also add the primitive to register the Semaphores (see that it doesn't hang, then see that the Semaphore gets triggered from inside Squeak). At last, loop waiting on the semaphore for keyboard's IRQ, read keyboard's port, and signla IRQ end all from inside Squeak. [Done]

SqueakNOS IRQ Handling

Squeak VM lets you signal Squeak Semaphores from the native world by calling signalSemaphoreWithIndex(). We want to serve IRQs from Squeak, using interpreted code, not native code. We seriously think that with hardware close to 1000 times faster than 20 years go we should be able to do it without any problems. Of course we cannot set the native IRQs handlers to jump to Squeak code, so our idea is to have a different Semaphore for every IRQ and have a Squeak Process with highIOPriority blocking on the Semaphore.

The code inside Squeak, taken from InterruptRequestDispatcher>>installOn: looks like:
installOn: aComputer
self registerSemaphore.
process := [
[
semaphore wait.
self handleOn: aComputer.
aComputer interruptController signalEndOfInterrupt: interruptNumber.
] repeat
] fork priority: Processor highIOPriority.
aComputer interruptController enableIRQ: interruptNumber

And the native code to signal the Semaphore, taken from ints.h, is similar to:
 void irq_1_handler();
asmlinkage void ISR_1() {
if (0!=IRQSemaphores[number])
signalSemaphoreWithIndex(IRQSemaphores[number]);
}

asm(
".text"
".align 16"
"irq_1_handler:"
"pusha"
"call ISR_1"
"popa"
"iret"
)

There is an interesting detail in all this: the software interrupt ending (IRET) and the hardware IRQ ending (outb(0x20,0x20)) are detached in SqueakNOS, were they are almost always done at the same time in every other code we saw. This gives a really desirable result: The IRET lets the software continue, going back to the interpreter and letting the Process waiting on the Semaphore be rescheduled, however, the hardware part (Interrupt Controller) still thinks the IRQ has not been served yet, and will wait until the Squeak side of the handler (shown above) signals the end of the interrupt (aComputer interruptController signalEndOfInterrupt: interruptNumber.

We seriously believe that with computers close to 1000 times faster than 20 years ago this should be ok, however we are not sure yet if it may bring any problems later, and we are open, although reluctant, to the possibility of coding some glue code, or even complete "device drivers" natively (or hopefully using Exupery). Probably for sound and video I/O, we'll see.

We were successfully using the very same scheme in the old SqueakNOS, so we just ported it to the new code base.

Phase 1: Make it boot [Done]

Make it boot and start interpreting the .image with text output for tracing and debugging.


The old SqueakNOS used a custom made mutant boot loader, borned from mixing LILO, the Linux kernel boot loader, and our dirty fingers. We had quite a few problems with it, and it was really complicated to put the needed environment to compile it. So we chose to replace it for something better and more modern: GRUB.

We first thought on GRUB because it, of course, handles all initialization, switches to flat 4G mode can load huge kernles in high memory, and also has support for modules loading (module and modulenounzip GRUB commands). We wanted to detach the native part and the Squeak .image, which were glued in a single file in the old SqueakNOS, making it easier for Squeakers to touch the .image without having to set up the compiling environment.

We needed to implement the switch to graphics mode before starting jumping to the interprenter. In the old SqueakNOS we did it from real mode (that's why we had so poor video support). But this time GRUB gave us control already in protected mode, so we couldn't just call the right INT. Luckily, GRUB already has the roots for supporting graphic video mode selection, and although it is not usable in the standard GRUB distro, we could solve the problem quite easily, by copying the testvbe command into the setvbe command, and removing everything after the command switches the video mode. We later found out that the video memory address changes from box to box, so we also hacked the vbeprobe command to show the video memory address.

We also learned that GRUB supports part of the multiboot protocol and is able to load ELF binaries, this led us reduce the amount of assembly code to barely around 20 line. It's not that we are afraid of assembly, but if multiboot was supported in some other platform (Apple/PowerPC), porting SqueakNOS' kernel should be really easy. [Never tried]

At this point we had GRUB loading the kernel and loading the .image all in memory, we had some glue code to make the .image look like a file so Squeak's intepreter could start interpreting it. We coded console text output support, hacked some debugging messages in the interpreter loop, some stack traces here and there and figured out what Plugins needed to be compiled for Squeak to start up: we finally managed to make the interpreter loop run loose.

A little history of the implementation

The people originally in the group only knew low level PC stuff, so we started with this, however there is already people participating that want to give Apple/PowerPC a try!

There are two incarnations of SqueakNOS, one was pretty much abandoned in 2001, and had support for 640x480x1 or 640x480x4, keyboard, mouse, serial port and a pseudo file system over serial port, although some of this features never saw the light. In May 2006 we restarted SqueakNOS, and a new version was released in May 16th, with support for 1024x768x32, and bare IRQ handling.

martes, 16 de mayo de 2006

SqueakNOS is back!

This is part of an article written almost four years ago (2006), telling some history of the evolution of SqueakNOS, with some updates mixed up in red. SqueakNOS has evolved a lot since that time but we wanted to rescue the article.

During this 5 years SqueakNOS has seen some activity from people not originally related to it, and although we never forgot about this project, this activity was what finally helped us move forward. Thanks a lot to everybody who sent emails and kind words, and specially to those who got dirty into our ugly code and successfully tried to make it come back.

Lots of things happened in the world, and we thought we would take advantage of them. First, and most important, GRUB, and of course, vmware. Also the Squeak source tree is quite nicely arranged, and the community is much more organized with SqueakPeople, SqueakMap, Monticello, etc. Some of this things we already took advantage from, and some we still have to learn how they work.

GRUB

The original bootloader was a mutant born from LILO, linux boot loader and our fingers. It was really a mess, we changed it to overcome an image size limit it impossed, and modified it to initialize the video mode before jumping to the interpreter. Now, we are using GRUB (Legacy) instead of this mutant. We also had to modify GRUB Legacy, but it's a lot cleaner than our original code, and if GRUB Legacy was not dead, our code would make into the standard GRUB release. We'll work on using GRUB2, but there are some things that we need which are still missing (we'll probably implement them).

    Why GRUB?

  • we need the bootloader to jump into protected mode and map 4G of RAM
  • we need the bootloader to load the .image from disk for us and put it in memory.
  • we need the bootloader to set the right video mode for us.
  • we need the bootloader to be able to handle a big bootable disk.
GRUB already did all this, except for a small detail.

VMWare

We were using VMWare originally in 2001, but now it's more easy to use (VMPlayer is even free to use and quite small for what it does). We are testing everything with VMWare player on Linux, but we successfuly tried it on VMWare on windows and Virtual PC on Mac OSX. We also booted it natively on some boxes, with different results.

Squeak native sourcecode

We are so incredibly happy that we started it all again from scratch (no kidding). Already changing to a new bootloader was big enough a change to push us start from scratch. But also the rearrangement of the sourcecode in subversion was so nice that we wanted to join it. We tried not to touch anything in the Common sources, nor in the generated code. And we managed to do it! We created a new platform (SqueakNOS) next to unix, Max OS, RiscOS, Common and win32 in our local copy of the main trunk. To create a new image we open Squeak and with VMMaker generate everything (only internal plugins are supported of course), and compile. This results in a SqueakNOS.obj which we move to a boot directory where we just "make SqueakNOS.iso". If you want to change the .image, you just move the new one to boot/iso.template, and that's it. As you can see, recompiling SqueakNOS and rebuilding an .iso must be really easy.

We'd love to get help here, specially in rearranging our source code so it looks more like other Squeak ports, and to check if the platform support is Ok as we implemented it (we know some things are not Ok).

Squeak community

Here we got lost. We asked around a few friends, and all recomended to start with a release and an article in SqueakPeople. Latter we'll update the swiki (this time in Squeak's main swiki, so it doesn't get lost), and upload the sources and other things somewhere, but we first need to understand what goes where.

UPDATE: here we are, now we have this site which is the official SqueakNOS site!

What's already here?

Not much yet, but already enough :-)

SqueakNOS is booting, GRUB gives us a nice 1024x768x32 color screen (this is hardcoded somewhere in the .c source code, so even if you change it with GRUB, SqueakNOS will think it's still that resolution). It already has all the primitives it had in its previous life, and some more. IRQs are handled correctly, and can be handled from Squeak (in fact, that's the only way to do it). When you boot the image you should see a quite standard Squeak (Squeak3.8-6665full UPDATE: Squeak4.1), with some add ons. You can open this very same image from a real squeak, and wonder around. There you'll find the source for the SqueakNOSPlugin implementing the primitives, and the class Computer. Although the IRQs and hardware port access are working correctly, keyboard and mouse are not yet implemented (sorry). It'll take a bit more (unless you do it!). It shouldn't bee too complicated, and in fact, it could be ported from the older version quite easily (although we prefer to reimplement it, at least in part). UPDATE: Keyboard and mouse drivers have been implemented and work very well now.

When you open SqueakNOS, you'll see lots of things, and you'll see also that if you press a key its scancode will show up in the Transcript. This is what we used to verify that IRQs and ioports are working.

This is pretty much it. We believe that everything needed to start playing inside Squeak is already done, and now everything missing should be done inside Squeak!

Near future

  • Of course, make it available in a more Squeakish form (SqueakMap? Monticello? Subversion? ???. UPDATE: Smalltalk side is maintained in SqueakSource and C side in Git.
  • Write a serie of small articles, or diary entries describing most of what we did to reimplement it. We did it all in small steps, so we think it'll be a nice serie of articles. UPDATE: We have this site now and lots of articles!
  • Implement PCKeyboard and mouse first! UPDATE: Done!
  • Implement serial port support UPDATE: Done!
  • Implement some kind of file system support over serial ports (vmware lets you set up virtual serial ports, which can be used to connect the inner world to the outer world) UPDATE: It's done, we should write an article about it. Tell us if you really really want it.
  • Our final short term goal is to be able to develop SqueakNOS from within SqueakNOS. With sources, changes and everything. UPDATE: We are almost there, have filesystem support and are almost ready to save the image from inside.

Although we think this could be contributed, we saddly think it'll be a lot easier for us to do it given that it involves going out and in of Squeak for every small test.

Mid term future

  • Add support for other hardware devices.
  • Start trying with other hardware platforms (PowerPCs on Powerbooks/eBooks)
  • Add networking capabilities (independently of the hardware support) UPDATE: Some (or all?) of this is done.

Here's where we think the community can help a lot.