download source code, PXE binary, floppy binary, CD-ROM binary

<< Newer

where we are

2008-12-18 02:14:00, dustwolf

Finnaly! The bootable CDROM image is ready.

As Kohlrak finally admits, trying to rely on other people's work has proven to be a futile excercise. Personally I have nothing against other people who contribute code, but you know.. when it doesn't work at the end of the day... well gee. I'd like to think the code we have is more reliable. For one, we really did try it with our code which is provided to you and it really really does work.

It didn't really take us all this time to make it, just life intervened. Now with this ISO headache off our heads and holidays approaching we are more likely to get more done. :)

Stay tuned.

documentation

2008-12-01 21:14:00, dustwolf

The work on the cdrom is progressing slowly as it is horribly dull work, but we intend to get it trough (in hopes that with our nifty clean GPL code, nobody will be tempted to do something like this ever again).

I was working on preparing some more documentation. Created INSTALL file with all the software requirements for assembling, testing and publishing microKernel and it's source code. Also created HACKING file with some details useful for people who may find the source code interesting to toy with. Cleaned the code up some, removing various blank or irrelevant files. Also temporarily lost hope in recruiting a maintainer I really hoped would help with the development (smoke) and removed him from the credits, except where his code was used.

Made minor modifications to this website so that you get the download links without having to look for them in the older blogs. Getting ready to publish website in some more places and hopefully get some more people take an interest in the code.

cdrom and ps/2

2008-11-25 04:05:00, dustwolf

So we are back up with no problem.

In this time we have been finnishing off the IDT, which was handled pretty much entirely by Kohlrak, while I finnished up some fancy in the pcspeaker driver as well as some details regarding the IDT.

Then we started on the mouse and keyboard drivers and CD-ROM image booting support, which was pretty much a documentation digging project. We got some nifty IBM manuals from 1990 on the PS/2 which are the very first helpful documents we have seen trough the entire development process thus far (Thank you IBM!). You know I whatched that "Triumph of the nerds" documentary again recently and there was this interview with Bill Gates telling us that IBM was the big bad generic cubicle plus marketting company. Looking at this IBM manual I can find more design genius, hands on approach and efficiency than in all of Microsoft today. Kind of prooves that all MS, Apple, and company were really good at was lying. IBM rules!

The CD-ROM image booting support I did because Kohlrak complains he has no other way of testing the microKernel on his hardware. And I would come to understand most of you probably agree with him. He wanted to go handle it all by himself originally but I figured I best do this since I made the floppy MBR too. I managed to dig up some good docs, specifically the El Torito and ISO9660 specs. Now, we have complained about sillyness in the past and little-endian and A20 really are very very ugly hacks, but this... I could give plenty of color comentary on the decision of the author of the El Torito specification to consider all numbers hex by default... I understand when you spend your life before a hex editor you kind of end up counting everything in hex, but what was the guy smoking when he decided that saying "a CD-ROM sector is 800 bytes" is a perfectly rational way of saying "a CD-ROM sector is 2048 bytes"?! This resulted in more than a few mistakes on my part and plenty of confusion, though some of it could prolly be blamed on lack of sleep.

This wasn't the worst of it though, behond: ISO reasoning! the ISO9660 is a bizzare standard, although I understand nobody really bothers to get to know what it is these days, even other kernel developers out there use premade tools for making their kernels into bootable CD images. Don't ask us why we wanted to make this ourselves too, let's just say that if we didn't want to do things ourselves we would have never started out on writing a kernel. So.. about ISO9660. Well it has these pre-defined arrays of nulls in random locations around the image. Now I can understand why somebody would leave spaces in a network protocol or the like... to have data filled in later.. but can somebody please explain to me, why would one have empty spaces in a fixed-size read-only media? I certaintly can't explain it. I don't think the ISO guys can either. It's pure nonsense. Oh and it gets better! Read this:

8.4.22 Application Identifier (BP 575 to 702)

This field shall specify an identification of the specification of how the data are recorded on the Volume Group of which the volume is a member.

If the first byte is set to (5F), the remaining bytes of this field shall specify an identifier for a file containing the identification of the application. This file shall be described in the Root Directory. The File Name shall not contain more than eight d-characters and the File Name Extension shall not contain more than three d-characters.

If all bytes of this field are set to FILLER, it shall mean that no such application is identified.

The characters in this field shall be a-characters.


Notice anything odd? Let me help you there, it's a 128 byte data structure which may only contain a 16-bit encoded DOS filename plus an additional byte. So it's a 128 byte structure that may only contain like 23 bytes tops. So what are the other 105 bytes for? Somebody obviously wasn't thinking very hard. Knowing that I guess it's not particularly suprizing that Microsoft's OOXML is an ISO standard. If you were wondering what they were thinking when they accepted it, the answer is obvious: They weren't. ISO pople don't think or plan ahead, they do company politics instead. And from this results in that %#&! double-endian format of recording offsets in ISO9660, which naturally records little-endian format FIRST. Cut down on the drugs people.

But then this frustration brings us a special kind of joy. We are developers after all. Enjoy your microKernel. :)

power down

2008-11-23 00:00:00, dustwolf

You won't be seeing this website tommorow (today, 23.11.2008) as we power down the server for maintenence on the power grid.

Worry not however, we are still working on the microKernel! Currently, as can be seen on the ChangeLog on the right, we have been working on the IDT. Kohlrak has been most helpful here, I've kinda just being sitting there giving directions, adding disclaimers and making minor corrections.

A useful tip: If you think you know interrupts, you don't understand interrupts. Some concepts surrounding them are extremely messed up. Not only that it's another silly hack of using chips outside the CPU, that there is quite a bit of bullshit about it that I have seen written in numerus textbooks of respected authors (the part about the IDT having to be on top of the memory), but also there are at least 3 different numberings of the same stuff and this is rarely pointed out.

After this is done however, the fun stuff starts. With the Clock ISR in place and multitasking on the horizon, the CPU scheduler awaits. Interesting times ahead.

the holdup

2008-11-20 02:57:00, dustwolf

Currently balancing out the need to work on other issues in our lives. With a job (or school in Kohlrak's case) taking half of your day, if we want to work on the kernel, it takes over the other half of the day too, often leaving only a small window of sleep. Not a good long-term plan.

Currently, semi-satsfied with the stability and usability of the text output, we are looking a more interesting phase of our project, implementing:
 a) output support for internal speaker
 b) input support for keyboard
 c) input support for PS/2 mouse

...this likely involves us implementing the IDT, but we assume that won't be much of an issue.

Once that is done we will likely be going into implementing usermode, along with the evil bunch: cpu scheduler, memory allocator and interprocess communication service. But then again that is still far off for now and a lot may happen until we get there.

open source

2008-11-18 01:17:00, dustwolf

Today we took some time away from our coding to comply with one of your requests, to make the project open source and to offer the binaries for download. We have never indended to keep the microKernel for ourselves since preventing leaks is not something we are good at and we really like to help the community whenever we can anyway.

There are three files:
* The source code package
* The PXE boot binary
* The floppy disk image

This is the begining. The systems are layed down to keep the downloads up to date hourly, so that we do not have to worry about preparing releases. Some documentation to help you find your way around the kernel is yet to be written, but will be added as we go along. Please also note that this is Alpha stage code, meaning it doesn't do much yet. If you want to play around with it, feel free, so long as you comply with the (GPLv3) licence.

With the source code you will find a MAINTAINERS file, which has email addresses and such if you would like to contact us. We are always open to your input.

text output

2008-11-17 02:56:00, dustwolf

The text output code is finally functional... see for yourself:

screenshot of text output

The foggyness is due to the cam on my cellphone sucking, but then you knew that. While I toyed around with a fast algorthm for making the center area black and a few other optimizations and macros for future use, as well as the text output you are looking at, Kohlrak was debugging the text output algorthm. The number output procedure is not working right as can be seen. Little-endian strikes again! Oh and he fixed "that demented K" in case anyone was wondering. I love the font, albeit it's all quite small and I'ma need a looking glass to see it on that 14inch CRT.

Letters being 8 by 8, we stored them in bit format. Of course little-endian came to hamper our efforts yet again by scrabmling bit order in the 8 bit format, so Kohlrak ended up using a Java program to make the conversion of the original common sense format into little-endian nonense.

The kernel is now almost 2kb in size, which is primarily due to the elaborate data structures required in a floppy boot record, VESA and of course the fonts.

blog up

2008-11-15 03:54:00, dustwolf

The blog system is finnaly up and since we didn't publish much until now let me take some time to quicky describe what we have been making all this time.

At the start of the project, we were disucssing much but not making much.

When I finnaly decided to go do something I started at the begining, with the boot loader, with Flat Assembler. I wen't for the floppy format because this seemed least likely to explode on me, destroying my computer in the process. While all operating systems today use a standard Master Boot Record and most users don't even bother to overwrite one when installing a different operating system, I wanted to code EVERYTHING myself, so I wrote the kernel boot loader as the MBR. I used some websites with reverse engineered MBRs as a reference. You'd never think it, but the modern MBR, just like the modern Operating System, contains a lot of legacy and a lot of nonsense. I have not copied anything, but rather interpreted the code and only wrote what was required, having great fun in the process.

Having finished the MBR I wanted to continue to making the kernel load up to 32bit or 64bit mode which is when I encountered an impass: In protected mode there was no easy way to use BIOS code to output debugging information to the screen. Working blind was not an option since those low level mechanisms are not as standard as one'd like and everything needs to be tested. I lost hope for a while.

But fortune was on my side, I managed to team up with my friend Kohlrak, whom I found on the FASM forums earlier. I have split the code so that it could be used with CVS (which seemed ideal since the two of us are from very different timezones and I had bad experience with emailing source code from the past) and we were all set. With the help of another friend, smoke (whom I have taught FASM to begin with and has since become quite an expert), we got some files from the author or FASM, that made the 64bit transition and worked. It was now our job to reverse the code and adapt it for our needs.

A few days later I decided that while very nice for programming (thank you AMD!), 64bit mode had no future with our project due to the fact that the kernel we were writing would never have all the drivers to be worthy of new computers, instead we would go to 32bit mode and aim for the low-end, from-scrap, PC configurations. Thinking about deployment, floppies being outdated and harddisks and CD boot options being a waste (the kernel is 1,5 kb at the moment), we also decided to make the kernel boot from network (PXE).

After some very stupid bugs we made in PXE (dubbed "PiXiE" by Kohlrak) it was working. I made a makefile with seperate files with includes (one with PXE and everything and one with Floppy MBR and everything) and was impressed with the portability of ASM code as despite us not paying much attention to it while coding, both versions worked flawlessly with no need for any additional hacks at all.

Working on the GDT, a table required in 32bit Protected mode was interesting since we had so many examples of the code required online and not one of them actually worked or was anything like any other. The Intel manuals on the topic were useless as they were obviously not written for people making operating systems, but instead shown memory mapping when looking from within protected mode (most useful for obscure hacks, which are the main reason why some things in Windows are so incredibly slow). There was a lot bitching and ranting over Intel's "brilliant" little-endian, which contiunes to be a source of frustration on a daily basis. What were they thinking?! Oh... don't tell... marketting. It is Intel we are talking about after all. After much bashing our heads against the walls, we came up with a macro (thank you Thomasz!) which made GDT entries easy.

Now that we were finnaly in 32bit mode, we wen't for graphics. Now the interesting point that most new operating systems don't implement anything so soon into the boot other than perhaps a splash screen IS noteworthy! We on the other hand would instead offer full graphics mode support this quick into the boot. We were really bashing our heads on this one, since no matter what we did, graphics would simply not work. Documentation was useless as, again, it was not written for OS writers, but hardware makers instead. We had some code which we eventually gave up on, until eventually ending up with a Linear Frame Buffer access implementation, which switched to the right mode trough BIOS, which was as failsafe as a brick, but even it did not work. We tried forums, but come to realize the age old truth, that whoever has enough time to post on a forum, takes no time at all for coding! We asked for assistance and all we got in return was "you're doing it wrong!". No shit! Talk about idiotic. It was really discuraging. Imagine our frustration when we, as a last ditch attempt, tried our code on a real machine as opposed to VMWare, found that our code worked just fine all along! Damn you VMWare! We switched to bochs and using the floppy version for testing.

Now, after some more work on the LFB, we realized that only half the picture was there on a real machine as opposed to bochs. How strange, don't you think? Turns out the memory mapped LFB was sitting direclty in the middle of the fameous 16 MB memory hole and the other half of it was simply innaccessible. We had to mess with A20, which was the sillyest hack we have EVER seen. I mean we thought segmentation in Protected mode was bad! Googling how to make it work resulted in more time wasted on code examples that DID NOT WORK. I don't get it? Why bother publishing an example if you don't bother to check if it works? In the end we did manage to simplify the code according to another howto to something that actually worked.


microKernel screenshot


And this is where we are now. The graphics version is up and operational. We have designed the GUI, which needed a design. This website is a direct result of that requirement, I needed to test a design and I thought, heck, might as well publish it. This is it. We are now working on a debugging level text output so that we have the debugging output I wrote about at the begining. Kohlrak is working on that while I play around with this blog engine. After this we are probably going for the drivers for keyboard and mouse and then start working on a usermode.

Stay tuned.

<< NewerOlder >>

2013-03-03 16:14:08, Jure Sah

webpage fixes

makefile (M)

2013-02-17 14:19:39, Jure Sah

debug bochs hacks and requirements

INSTALL (M)  debug.bochs (M)


2013-02-17 14:07:03, Jure Sah

ubuntu 12.04 compatibility

bochsrc (M)  makefile (M)


2012-10-27 13:00:31, Shane Tyler "Kohlrak" Yorks

Commented out the mouse (due to hang) and cleaned up the makefile, a well as fixing a bug introduced with the AVX instructions being added to FASM.

bochsrc (M)  cdrom.asm (M)  debug.bochs (M)  eltorito.asm (M)  end.asm (M)  fancy32.asm (M)  floppy.asm (M)  main.asm (M)  makefile (M)  mbr.asm (M)  pxe.asm (M)


2012-10-26 22:28:35, jure sah

mess cleanup




2009-08-11 21:59:21, convert-repo

update tags

.hgtags (A)


2008-06-23 01:03:30, dustwolf

dustwolf smoke microkernel start




2009-03-06 03:00:37, dustwolf

copyright data update

VESAflip.asm (M)  cdrom.asm (M)  floppy.asm (M)  keyboard.asm (M)  main.asm (M)  makefile (M)  mouse.asm (M)  textDebugger32.asm (M)  textResolution.asm (M)  textmodedebug.asm (M)  transition32.asm (M)


2009-03-06 02:37:59, dustwolf

fixed hex output

textDebugger32.asm (M)


2009-03-06 02:30:56, dustwolf

text mode macros now work perfectly (I think?)

textDebugger32.asm (M)  textmode.asm (M)


2009-03-06 01:59:06, dustwolf

writing up on actual text mode output

textDebugger32.asm (M)  textResolution.asm (M)  textmode.asm (M)