KFS-6 & KFS-7

After more than 500 commits since kfs-5, we're glad to present the new release of KFS !

Project rename

First of all, the project finally has a name, and a great logo ! #197

Introducing: SunriseOS

PRs: #249 and #285

Filesystem

The main focus of this release have been around creating a filesystem service for the Sunrise OS, from scratch.

Since we're a micro-kernel, the kernel is completely agnostic of concepts of files, directories, and such fancy abstractions, and also from the concept of disks, blocks, etc, as it is the drivers' job to provide a mean to read and write blocks from/to the disk. The way we do this is by having several userspace services in charge of doing just that, and let them make requests to each other via Remote Procedure Calls (RPC).

AHCI Driver

The AHCI driver is responsible for discovering disks, and exposing endpoints to read and write blocks to them.
Supports (P)ATA and SATA devices. All operations are done via DMA.

Holds a minimal PCI implementation to discover AHCI devices, and query their Base Address Register (BAR).

PR: #206
exposed API: ahci.id
documentation

FS service

The filesystem service. Think of it as the equivalent of VFS on linux: an abstract layer on top of more concrete file systems. Any user that wants to read or write a file connects to it, makes a request, and depending on the partition the file would be found on, it routes the request to the appropriate filesystem driver (e.g. ext2, fat32, nfs, ...), which in turns make requests to the ahci service to get the appropriate blocks, and interprets the raw data as structured files and directories. Right now only FAT32 is implemented.

The filesystem service supports multiple partitions on a disk if a GPT is present.

The filesystem keeps a Least Recently Used (LRU) cache of blocks in RAM, to improve performances.

PR: #390
exposed API: fs.id
documentation

Libfat

A no_std FAT12/FAT16/FAT32 compatible crate. This is the first of the supported filesystem formats.

Repo: sunriseos/libfat

disk-initializer

An utility that permit to generate a valid disk image from a file name, a size and a template directory.
The resulting disk image contains a standard GPT partition layout and a FAT filesystem.
From the disk size, it chooses a FAT filesystem that may be appropriate to fit in it.

SwIPC-Gen

In KFS-5, we introduced our IPC mechanism and had a first rough implementation of them. On the client-side, we would auto-generate clients based on SwIPC id files, while the server side relied on an object macro. It worked well, but had multiple maintenance issues:

• It was easy to have a mismatch between the client and server
• The server object macro had a lot of rough edges due to being an old-style macro
• The server object macro required putting lots of code inside macro code, which would break IDE features like auto-completion

Furthermore, there was a few documentation issues. As such, it was decided to completely revamp the IPC Server subsystem.

In the previous section you've seen some IPC definitions files, like example.id. Those are SwIPC interface definition files. When building libuser, they will be parsed, and automatically generate some rust code:

Server trait

The .id will generate a rust Trait (interface) for every exposed interface. A server only has to implement this generated trait on their own structure, and fill the endpoints with some code.

The trait will automatically implement a dispatch() method, which routes an RPC cmdid to the appropriate trait function, unpacking the arguments to pass to the function and packing the return value back to the IPC buffer. This function is then passed to the various IPC wrappers that are spawned on the event loop, to be automatically called when RPC requests are received.

This brings a huge improvement to our workflow: the lack of special macro means all standard IDE features work, and is a lot less magic and intimidating to learn. The documentation is also a lot cleaner.

Client types

Swipc_gen also generates client structs like xxxProxy for every interface in the .id. The structure will wrap an IPC session handle, and expose functions to call its known endpoints, with the appropriate parameters and return values.

Such a struct can be constructed with the new() function, which will get the sm service, and ask it to create a session to the xxx service for us, and wrap this handle in our proxy type.

See the generated structs and traits for Vi for a real life example.

Note that the documentations in the .id are used to document the generated types.

PR: #248
documentation

SwIPC-gen CLI usage

In order to help debug SwIPC, we also have a new cargo make swipc-gen rule that writes a .rs file containing the code that is automatically generated for a given .id file.

PR: #365

Futures

The main event loop of every service is pretty much the same: wait for RPC request, do some stuff, wait for IRQ to notify stuff we did is ready, do some more stuff, and eventually return from RPC. But while we wait, we don't want to be blocking, and want to be able to serve multiple RPCs simultaneously. This means that asynchronicity is at a the heart of every service.

In rust asynchronicity is done through Futures (think javascript promises), you can declare a function as async, and then .await on it. This way every RPC can be an async function, which gives back control to the event loop every time it .awaits on IRQs, and is polled again some time later when the IRQ has been received.

So, here we are, implementing a future executor !

Our service event loop is basically a light userspace scheduler (fibers/co-routines), which waits on a list of ports, sessions, and IRQ handles (like select on linux), and schedules the appropriate future when one of its handles is signaled, lets the future advance until it .awaits on another event, and loops again. Once a future is eventually finished, it is removed from the list.

All of this lives in libuser, and is available for every service. A service only has to implement the xxxAsync flavour of the trait generated from the .id for its interface instead of the synchronous xxx trait, and its RPC is now a future, congratulation.

This means that writing asynchronous services/drivers is now painfully easy, as you have the full power of rust and a bit of magic to help you behind the scenes.

Please note that our event loop lives only on a single thread, we never spawn any threads for it to work. In the future (pun intended) we might want to divide the work across a pool of worker threads.

PR: #384
documentation

Time

The time sysmodule. This sysmodule is charge of handling the RTC time and timezone.
The timezone code is based on the tz database code with some custom modifications (Leap seconds aren't handled and Tzif1 support removed)

In the future, the time service will be used to provide the filesystem service with time informations.

PR: #318
exposed API: time.id
documentation

Dynamic relocations

Userspace binaries are now relocatable. Actually they now have their own llvm target i386-unknown-none-user, and use their own linker script.

To pave the way for ASLR, they are loaded at an arbitrary address by the kernel (for now at 0x40000), and one of the first thing they do before calling main is to find their .dynamic section, parse it and apply it, effectively relocating themselves.

PR: #247
userspace linker script: userspace.ld

Threads and Thread Local Storage

In the libuser we created the threads module, which exposes an api to create and start threads, passing them arguments, akin to pthreads on linux. When we eventually port the libstd on sunrise, it will be used as a backend for std threads, instead of pthread.

But I'm afraid now is the time to talk a bit about Thread Local Storage.

On the Sunrise OS, each userspace thread has three special thread local memory regions: the kernel-allocated TLS region, which points to a userspace-allocated Thread Context, where [ELF Thread Local Storage](https://sunriseos.github.io/SunriseOS/master/sunrise_libu...

KFS-5

Screenshots

VGA:

Serial output:

Scheduler

The kernel now has a cooperative scheduler.

We are now a multiprocess and multithreading capable OS.
It is still single core for now, and the scheduler might need to be highly modified/rewritten
when we will want to support SMP, or preemption, but it works fine for now.

The schedule queue does not implement any fancy priority system, and is just a dumb FIFO.

documentation

Userspace

KFS-4 introduced a demo ring3 userspace and syscalls, with a single test program running in userspace.

Now that we are multiprocessed, we can finally become a true microkernel, and move all modules/drivers to separate userspace processes.

ELF Loading

Our userspace services are simple ELF files copied in memory by grub at boot (we use multiboot2 modules for that), and then loaded and started by the kernel after it has completed its initialization stage.

documentation

IPC

We now support Inter Process Communication between processes.

Our IPC architecture is a rendez-vous system between processes. A server accepts connexions on a 'port', and is put to sleep until a client connects to this port.
The kernel then creates a 'session', a two-ended half-duplex communication channel between two processes, and gives one end of the channel to each process.

The client can then make requests to the server through this channel, and then receive its answer on the same channel.

The IPC makes a heavy use of 'handles', that are just like unix file descriptors, but that can point on more resources than just files (thread/session/shared memory/...).

more info

Syscalls

Implemented a bunch of syscalls, based on necessity:

• svcCreateThread
• svcStartThread
• svcSleepThread
• svcExitThread
• svcExitProcess
• svcMapFramebuffer
• svcCreateSharedMemory
• svcMapSharedMemory
• svcUnmapSharedMemory
• svcQueryMemory
• svcQueryPhysicalAddress
• svcSetHeapSize
• svcMapMmioRegion
• svcCreatePort
• svcManagePort
• svcConnectToPort
• svcConnectToNamedPort
• svcAcceptSession
• svcCreateSession
• svcWaitSynchronization
• svcSendSyncRequestWithUserBuffer
• svcReplyAndReceive
• svcReplyAndReceiveWithUserBuffer
• svcCreateIrqEvent
• svcCloseHandle
• svcOutputDebugString

documentation

Services

As stated above, we now are a microkernel, and we migrated a lot of kernel features to distinct userspace services:

Vi

The Visual Interface service.

This service is called to display windows at given coordinates on the screen. Every client asks to create a window, which is just a shared memory, modifies it, and then asks again vi to draw it (refresh screen).

documentation

Shell

The read-eval loop of keyboard inputs was moved to kfs_shell.
It's a very basic interactive program that is displayed at the top of the screen, and which is solely used to display memes and run debug commands.

documentation

Clock

A small window running at the bottom of the screen that displays the current time and date.

This is composed of both a driver for the Real Time Clock, and the program that displays it to the screen.
This should be split in a service and a program in the future.

documentation

Libraries

Libuser

A light libc, that exposes wrapper functions around syscalls, IPC calls, and a whole bunch of low-level helper functions.

All services have it in their dependencies.

In the far future this might end up becoming our rust libstd, or at least be used in the glue layer in libstd.

documentation

Libkern

A library that exposes constants shared between the kernel and userspace. This mainly are syscall numbers, and kernel error values.
This should only be useful to libuser, which wraps syscalls, and matches raw errors to more rich types.

documentation

Libutils

A messy crate with various utilities shared between the user and kernel code.
Should probably be further split into several useful libraries.

documentation

Other updates

Paging

Re-wrote the FrameAllocator and Paging completely, to have better management and bookkeeping of userspace memory.

Both were made a lot more architecture agnostic, to prepare for aarch64 port.

The KernelMemory is still shared between all processes, page tables being copied at context switch time, however each process has its own 3GiB of virtual space, which is no longer managed directly in the page tables, but instead in a BTreeMap of mappings present for every process.

This enables us to save more meta-data about each mapping.
In particular, we can now do shared memory between processes, where we represent it as an Arc of the physical regions, which will be freed only when its last user unmaps it (or dies).

Note that KernelMemory (4th GiB) is still bookkeeped directly in the page tables themselves.

documentation

mkisofs

A tool to create a bootable KFS iso image.

We used to depend on grub-mkrescue, and that made compilation impossible on mac OS or Windows.
We now create the iso ourselves, and can be built on any platform.

Maintained as a separate repo.

New member

The core team (@roblabla and @Orycterope) was expanded to include a third developer, say hi to @Thog, to form the deadly triade of KFS devs, coming to get you on a deep synthwave soundtrack.

Workflow

Pull Requests and peer-review

We improved our workflow a lot. We now only submit patches by PRs, and force ourselves to go through peer-review before merging any branch into master. This has greatly helped opening conversation and constructive criticism in the last few months, and also avoid deeply broken implementations to make it to the master branch.

Tests

We started writing some unit tests for a few functions. They are compiled for i686 linux, and run on the host machine.
This meant writing a lot of mockup code and conditional compilation, but it works.

For now there's only a few tests, and a big effort should be made to write some more, but at least the infrastructure is there, and it works.

We still have no integration tests though, see #31 .

Code cleanliness

We now make heavy use of the clippy lint, to make sure our code always matches a certain standard of quality, and avoid a lot of beginners mistakes.

Updating all of our project to match the standard was a real pain in the a**, see #88

Documentation

We use a combination of rust and clippy's lints in every crate:

#![deny(missing_docs)]
#![deny(clippy::missing_docs_in_private_items)]

to make sure that every module, function, structure, and structure's field is documented, otherwise our code doesn't compile.

This ensures us that every part of the OS is fully documented, and easily understandable (looking at you, linux ರ_ರ).

Our documentation is automatically uploaded to our github pages thanks to cargo-travis

CI

We started using Travis for Code Integration. For a branch to be merged, it must pass compilation for both debug and release, the iso image must be built successfully, doc must be generated successfully, and all unit tests must pass. That way master is always 100% clean.

Issue tracking

We started using (and fell in love with) todo[bot] to automatically open issues for us, by finding "TODO" comments in our code. You can see we use it pretty much all the time, it's impressive to see how our issue count has exploded in only a few months. For an exemple, see #202.

Dependencies

We now use dependabot to automatically create a PR every time one of our dependencies is updated, so we're always up to date. See #196 for exemple.

KFS-4

This is the fourth release of KFS. Here's what we've been working on:

Interrutps

The main focus of this release has been around enabling interrupts in KFS, and write interrupts and exception handlers.

The first step to do that is to set up an Interrupt Descriptor Table, basically a table of pointers to irq/exception handler functions.

Exceptions

In the interrupts module we define functions to handle all of the x86 cpu exceptions. We put pointers to these handlers in the IDT, so that they are called when an exception occurs.

Almost all of them cause a kernel panic for now, sometimes with a more helpful context (e.g. page faulting reads cr2 to get the address that caused the page-fault).

We use llvm's x86-interrupt calling convention to backup and restore registers for us, and handle the optional errcode pushed by the cpu.

Double faulting

Stack overflowing causes a page fault exception, thanks to the page guard at the end of every KernelStack. The page fault handler is invoked, but it also runs on the same invalid stack, and this causes a page fault again, which is translated to a Double Fault. We don't want to do that a third time, otherwise we would triple fault and reset the cpu.

Because of that, the double fault handler uses a Task Gate (instead of the usual Trap Gate for every other exception), so that the cpu can switch the stack before invoking the Double Fault exception handler. This is the only place in the kernel where we use TSSs.

The double fault handler just kernel panics, because clearly something went really wrong in the kernel.

PIC

Onto interrupts now.

First of all, we wrote a Programmable Interrupt Controller driver. We configure it in cascade, with the second line cascading to another 8254. When an interrupt occurs, we acknowledge it in the irq handler.

PIT

Now that we have interrupts, set up the Programmable Interrupt Timer in periodic mode, so it delivers an interrupt every millisecond. We acknowledge the interrupt, and use it to track time when processes are sleeping for an arbitrary amount of time.

Events

We're a micro-kernel, irq handlers live in userspace. Let's consider the case of a HDD driver: the driver makes a request to read a given amount of disk sectors via DMA, and then waits for an IRQ from the disk signaling completion. The kernel unschedules the driver for now, and runs some other process while the IRQ hasn't been received. Eventually, the disk will perform the request and signal an interrupt. At this point we're still running some other process. The cpu will enter Kernel mode, and call our irq handler. What the kernel irq handler does is simply incrementing a monotonic counter for the given irq line, notice that a process was waiting on it, wake it up by adding it to the schedule queue, and return to the process it was running. When the process eventually yields, the driver will finally be scheduled, and its irq handler code will be able to run.

From the userspace's point of view, this is all represented by the IRQEvent handler type. The driver registers its interests for IRQs on a given IRQ line to the kernel by waiting on the handle, and the kernel will only wake it up after an IRQ has been signaled on the given line.

The counters are here to prevent dropping irqs: The kernel keeps for every line a monotonic counter of irqs that happened on this line, ever. An IRQEvent handle holds the number of irqs it has acknowledged on its line. If you register for an irq, sleep, and 3 interrutps are triggered before you are actually scheduled, those 2 other interrupts are not lost. When you will wait on the handle again you will be immediately waked up, incrementing the ack count on your handle. Waiting on a IRQEvent effectively consumes one interrupt event. You are expected to consume them one by one.

This is all in theory, as we don't have a scheduler yet. But we implemented it anyway, so we don't have to do it in the future.

Syscall

Finally, we implemented a Proof of Concept syscall handler. The ABI is linuxy: you pass the syscall number in eax, up to six other arguments in the other registers, and then do a

int 0x80

This will enter kernel mode and call our syscall handler function, which will unpack the arguments, dispatch to the appropriate syscall based on eax, repack the return values into the registers, and iret.

For clients, we provide the

pub unsafe fn syscall(syscall_nr: u32, arg1: u32, arg2: u32, arg3: u32, arg4: u32, arg5: u32, arg6: u32) -> u32 { ... }

function that performs the argument packing, int 0x80, and result unpacking for them.

Once again, we don't have a userspace yet, but this is all already testable in the kernel for now.

SpinLockIRQ

Because we have interrupts now, we might be interrupted at undesired moments. To prevent that, we provide the SpinLockIRQ type, that wraps and behaves like a spin::Mutex, but also disables interrupts while it is held, ensuring one's critical section won't be interrupted.

PS/2 keyboard driver

We have interrupts, we can do I/O ! We implemented a PS/2 keyboard driver, that waits for interrupts on line 1, reads the scan code(s), maps them to ascii letters or control keys (Ctrl, Shift, CapsLock, F12, Pause, www Search (yes that's a real key), Email, ...). It handles capitalization if the key shift is pressed/capslock is enabled.

It also exposes a get_next_line function to read a full line, blocking while doing so.

Font rendering

Displaying gifs is fun, but sooner or later, we'll want to display text to the screen. As always, sooner it is.

We could just take a bitmap font like GNU Unifont and blit it to the screen, but where's the fun in that ? Plus, Unifont is really ugly, and I really like the monaco typeface.

So let's do font rendering in the kernel ! We use the font-rs crate to parse the monaco.ttf file, and do the font rendering.

Rendering the 5 char gives us something like that:

Great, let's blit it to the screen !

Oh well ... Turns out font-rs doesn't handle baselines, and horizontal metrics. Unless you want to look like a serial killer blackmailing its victims, it's not acceptable. Let's fork it, export the hhea table, do some dubious geometric arithmetics, and we have a vga text logger. Using the generic logger we wrote in kfs3, we can multiplex all logs to both the serial port and the screen.

Shell

We have an Input method, the keyboard, and output method, the vga text logger. Let's do a shell !

Well ... let's do a read-eval loop that recognize three hardcoded commands.

The commands are:

• help: display the list of known commands
• stackdump: display a stack dump
• gif3: display the KFS-3 gif meme
• gif4: display the KFS-4 meme

stack dump

The stackdump command dumps the stack in a xxd fashion, going down frame by frame like (gdb) backtrace does, using the pushed esp and eip at the start of each frame to know where the next one starts.

$ stackdump
---------- Dumping stack ---------
# Stack start: 0xe01c9000, Stack end: 0xe01ccffc
> Frame #0 - eip: 0xc0004575 - esp: 0xe01cce98 - ebp: 0xe01ccf20
0xe01cce98: 0100 0000 0100 0000 f4ce 1ce0 d0dc 01c0  ................
0xe01ccea8: f0ce 1ce0 10df 01c0 e0ce 1ce0 10df 01c0  ................
0xe01cceb8: c4ce 1ce0 10df 01c0 0900 0000 20cf 1ce0  ................
0xe01ccec8: 4c4b 02c0 0500 0000 744b 02c0 0400 0000  LK......tK......
0xe01cced8: a0ce 1ce0 0400 0000 98ce 1ce0 fccf 1ce0  ................
0xe01ccee8: 98ce 1ce0 0090 1ce0 7545 00c0 0000 0000  ........uE......
0xe01ccef8: 20cf 1ce0 7545 00c0 e8ce 1ce0 2011 02c0  ....uE..........
0xe01ccf08: 0300 0000 40cf 1ce0 28cf 1ce0 40cf 1ce0  ....@...(...@...
0xe01ccf18: 28cf 1ce0 3cd7 1bc0                      (...<...
Saved ebp: 0xe01ccf60 @ 0xe01ccf20 (ebp) - Saved eip: 0xc000c165 @ 0xe01ccf24 (ebp + 4)
> Frame #1 - eip: 0xc000c165 - esp: 0xe01ccf20 - ebp: 0xe01ccf60
0xe01ccf20: 60cf 1ce0 65c1 00c0 0100 0000 1c74 02c0  `...e........t..
0xe01ccf30: 0a00 0000 2674 02c0 1200 0000 0100 0000  ....&t..........
0xe01ccf40: 3cd7 1bc0 1000 0000 0900 0000 0500 0000  <...............
0xe01ccf50: 0100 0000 0400 0000 0500 0000 0100 0000  ................
Saved ebp: 0x00000006 @ 0xe01ccf60 (ebp) - Saved eip: 0xc000c013 @ 0xe01ccf64 (ebp + 4)
> Frame #2 - eip: 0xc000c013 - esp: 0xe01ccf60 - ebp: 0x00000006
Invalid ebp
-------- End of stack dump --------

The output is really clear, it only lacks symbols resolution to display the name of the function of each frame. This will be fixed in the next release.

Bootstrap

We want the Kernel to be mapped in the high addresses, like real biggies. However, when Grub passes control to us, the paging is not enabled, and we are loaded in low addresses. From there, we can accomplish what we want in two ways:

• Have the kernel be a PIE loaded in lower addresses, which sets-up the MMU when it starts, both identity maps itself in lower, and also in higher addresses, enables the paging, relocates all its symbols in higher addresses (.dynamic ?), jumps to code in hig...

KFS-3

MMU and gifs

Who said you can't display gifs in the kernel ?

serial output:

This is the third release of KFS. Here's what we've been working on:

Virtual memory

The main focus of this release has been toward supporting virtual memory.

Physical memory allocation

First of all, for every virtual memory allocation we need to allocate some physical memory. It is the job of the Frame Allocator to keep track of which page frame is occupied and which ones aren't.

A page frame is a 4 KiB bytes physical memory block where pages will be laid out.

To do that we implement a dead simple bitmap allocator, tracking for every frame if it is free or occupied.

The 4 GiB / 4 KiB / 8 = 128 KiB array lives in the .bss.

The frame allocator returns RAII Frame types, that automatically tell the frame allocator to free them when they are no longer used (dropped).

Even though it is an astonishingly stupid design, we still managed to get it wrong, and spent several days debugging a seemingly unrelated problem, just because we were giving off occupied frames like candy:

What you're seeing here, is the gif mistakenly uncompressing itself in BIOS ROM instead of RAM, where the corrupted black strip happens. From top to bottom: RAM, BIOS ROM, and finally RAM again. Qemu is surprisingly tolerant with that, and just ignores writes to ROM. The actual hardware we tested it on, not so much...

Strongly typed pointers

We always make sure to never mangle Virtual and Physical memory addresses/pointers thanks to rust's strong type system. A physical memory pointer is wrapped in the PhysicalAddress type, virtual ones in the VirtualAddress one, and you can't cast from one into the other without deconstructing them. This is a good safety net that prevents us from doing some really stupid or dangerous stuff without noticing.

MMU

Next, we enable the MMU and use it to provide a contiguous view of the fragmented physical memory and check memory accesses.

Our paging module provides endpoints to create a mapping at a given address with the given permissions, unmap one, get the physical address of a virtual address by parsing the page tables, and find a free virtual region of any given size and alignment, used for creating mappings.

This is all provided by the PageTablesSet trait, which only requires a mean of getting a pointer to the page directory, and from the directory to a page table.

The PageTablesSet trait is implemented by 3 kinds of page table hierarchies, that differ on the way they access the page tables for writing:

• an ActivePageDirectory will want to use recursive mapping,
• an InactivePageDirectory will want to temporarily map the table in the current address space, modify it, and then unmap it,
• a PagingOffPageDirectory will point to physical memory, and directly write to it.

This way we can handle all kinds of page table hierarchies with one single api.

When KFS starts, grub hasn't enabled the paging for us, and we live in physical memory. To enable the paging we must first create a set of page tables somewhere in the physical memory, and enable the MMU by making it use this hierarchy. But as soon as we do this, the kernel itself would become unmapped, and we would page-fault. So we must not forget in this hierarchy to identity map the frames were the kernel was loaded by grub.

Thanks to our PageTablesSet trait being implemented on PagingOffPageDirectory, we can do this really easily by using the same api as we would use once the paging has been enabled.

Lands

We divide the 4 GiB virtual memory space of x86_32 as followed:

0x00000000..0x3fffffff, 1 GiB: Kernel memory
0x40000000..0xffffffff, 3 GiB: User memory

Ultimately we want the kernel to live in high memory, so we will redefine it in the future.

Heap

In kernel memory, we reserve a 512 MiB memory region for the heap. We use a linked list allocator to manage this memory. This enables us to dynamically allocate types in the kernel.

By importing the alloc crate, we can now allocate Box, Vec, and Arc native rust types. This a big step forward.

Stack

Now that we have paging, we have page guards !

We moved from having our stack in a global array that lives in the .bss to a real, allocated type KernelStack, which puts a page guard at the end of every stack so we can immediately crash upon stack overflows. Later we will be able to catch this condition in the page fault handler.

Address space switching

Paving the way for our future scheduler, we already implement switching from a page table hierarchy A to a hierarchy B.

While the 3 GiB of user memory should be different, the 1 GiB of kernel memory should stay the same, otherwise the kernel would panic as soon as it accesses any of its own allocated structures. To do this, before doing the switch, we memcpy the entries of directory A that map to kernel memory (the last ¼ of the directory) to directory B, and then tell the MMU to perform the switch. When the MMU is done, we still have the same view of the kernel memory from hierarchy B. Note that this process only copies entries, not the pages themselves nor the tables. The pages and page tables of the kernel are shared between hierarchy A and B.

This implies we need to copy from the ActivePageDirectory (A) to an InactivePageDirectory (B) that we temporarily map in A. Once again our shared paging api is extremely helpful for that.

VGA & GIF

VGA text mode is fun, but it lacks the endless possibilities for displaying memes that graphical VGA has. So let's use it !

We enable VGA, map the frame buffer into kernel memory, and draw to it.

As a proof of concept we use the image-gif crate to decode a gif frame by frame into a heap-allocated buffer, and then blit this buffer to the vga frame-buffer. This was a good test for our paging and heap allocators (see above).

PIT

Displaying GIFs introduces the need to track time between each frame. So we wrote a minimalist driver for the Programmable Interrupt Timer.

Because we don't have support for interrupts yet, we can't use it in the traditional periodic mode. Instead we program it to use the "one shot" mode, and tell it to wait for a given amount of time. This will make the PIT do the equivalent of a countdown, and our sleep function will poll the PIT until the countdown hits 0, blocking execution while doing so. When the amount of time is higher than the max countdown the PIT can do, which is almost always the case for GIFs, which deals with fractions of a second between frames, we do multiple consecutive countdowns.

Serial

Now that we configured the screen to use VGA and display memes, we now longer have our VGA text mode to log to.

So we implemented a basic rs232 logger.

Since we don't yet handle interrupts, it's really minimalist as it's polling all the time to know if it can write a character, and blocking while doing so.

The api of logging was made so the logging could be multiplexed to several backing devices (e.g. both serial and a file, or serial and a ring buffer, whatever you desire) once we have such devices in the future.

Also we now use the log crate to implement log levels (error/warn/info/debug).

Documentation

We added the cargo make doc and cargo make doc-full rules to the makefile. These will invoke rustdoc to generate an html documentation for the project from our /// doc comments above functions in the code.

How to build

This release is pretty outdated, so we provide a roadmap on how to build it.

reveal build instructions

First, make sure to to checkout the `kfs3-fixed` branch, as it contains a fix to pin the version of some dependencies:

git switch --detach origin/kfs3-fix

Cargo-xbuild

You need to downgrade your version of cargo-xbuild to v0.4.9. You must do this outside of your repo as the rust-toolchain file would try to compile it with a non-working rustc.

cd ~
cargo install cargo-xbuild --force --version 0.4.9
cd -

Cargo.lock

Because sunrise developers were so young and innocent at that time, they didn't fully realize the crucial importance of a Cargo.lock in published project. So they didn't think about including it, and it remained that way for quite a while, until they once tried to checkout an old tag and spent half a day just trying to compile it.

Unfortunately history only goes in one way, and commit logs too ( 👀 ), so we provide a Cargo.lock as DLC in the release files.

Copy it at the root of your repository, and cargo will now fetch the right versions of our dependencies.

grub-mkrescue

Because sunrise developers were lazy-ass monkeys, busy being high on smoking rolled-up intel-specs pretending it's some kind of psychotrope, they used to let all the hard work of creating an iso image be done by grub-mkrescue, while taking all the credit.

This means that depending on your distro, you might need to install:

• grub
• mtools

This went on until Thog joined them and created mkisofs-rs, and effectively rescued them, before they fall victim to 8042 withdrawal, and OD trying to snort an A20 line, while attempting to rel...

KFS-2

Segmentation and stack dump

This is the second release of KFS. The main focus around this release has been around supporting segmentation on x86_32.

bypassing supporting segmentation

Since all memory protection in KFS will be handled at the paging level like every OS post 1980s, we don't need segmentation. However, x86 being x86, we can't turn off segmentation. We have to define a GDT that describes segments spanning the whole address space with no offset (base=0x00000000, limit=0xffffffff) to simulate the effect of having no segmentation.

So that's what we do, we define 3 kernel segments (code, data, stack) and 3 userspace segments (code, data, stack), all following the flat memory model.

stack

We define our stack as a big global array that lives in the .bss. The first thing we do when starting KFS is to make esp point to this memory.

We added the print_stack function that takes the current esp and dumps the stack to the screen in a xxd fashion.

screen scrolling

The logger now supports scrolling. When reaching the last line of the screen, the logger will copy the whole content of the video buffer one line up, and finally keep writing to the last line.

panic

Added a kernel panic function. This is the function that will be called when rust detects an overflow, an OOB access, or fails an assert! check.

It will print a distinctive header, and a small message giving more context to the panic.

How to build

This release is pretty outdated, so we provide a roadmap on how to build it.

reveal build instructions

First, make sure to to checkout the kfs2 tag.

git switch --detach kfs2

Xargo

You need to downgrade your version of xargo to v0.3.11. You must do this outside of your repo as the rust-toolchain file would try to compile it with a non-working rustc.

cd ~
cargo install xargo --force --version 0.3.11
cd -

Cargo.lock and Cargo.toml

Because sunrise developers were so young and innocent at that time, they didn't fully realize the crucial importance of a Cargo.lock in a published project. So they didn't think about including it, and it remained that way for quite a while, until they once tried to checkout an old tag and spent half a day just trying to compile it.

Unfortunately history only goes in one way, and commit logs too ( 👀 ), so we provide a patched Cargo.toml as well as its Cargo.lock as DLC in the release files.

Copy them at the root of your repository, and cargo will now fetch the right versions of our dependencies.

grub-mkrescue

Because sunrise developers were lazy-ass monkeys, busy being high on smoking rolled-up intel-specs pretending it's some kind of psychotrope, they used to let all the hard work of creating an iso image be done by grub-mkrescue, while taking all the credit.

This means that depending on your distro, you might need to install:

• grub
• mtools

This went on until Thog joined them and created mkisofs-rs, and effectively rescued them, before they fall victim to 8042 withdrawal, and OD trying to snort an A20 line, while attempting to relieve the crave.

nigthly version

Using rust nightly but not specifying which version was a stupid idea in retrospective. Fix that:

echo "nightly-2018-03-15" > rust-toolchain

Building

Now you should finally be able to build and create the boot iso:

cargo make iso

that was easy ...

KFS-1

Introducing KFS

This is the first release of KFS, a soon-to-be glorious micro-kernel based on the designs of HOS/NX.

Features

Currently doesn't do much, beside printing a hello world to the screen.

The printing is done by writing to the VGA compatible text mode memory. The logger supports colors and line-wrapping. That's pretty much it.

Boot

We use grub as our bootloader for KFS. The Makefile.toml will generate an iso image that you can directly burn onto a CD / USB stick, and boot from. Or it pass to qemu, cause nobody's got time for that.

How to build

This release is pretty outdated, so we provide a roadmap on how to build it.

reveal build instructions

First, make sure to to checkout the kfs1 tag.

git switch --detach kfs1

Xargo

You need to downgrade your version of xargo to v0.3.11. You must do this outside of your repo as the rust-toolchain file would try to compile it with a non-working rustc.

cd ~
cargo install xargo --force --version 0.3.11
cd -

Cargo.lock and Cargo.toml

Because sunrise developers were so young and innocent at that time, they didn't fully realize the crucial importance of a Cargo.lock in a published project. So they didn't think about including it, and it remained that way for quite a while, until they once tried to checkout an old tag and spent half a day just trying to compile it.

Unfortunately history only goes in one way, and commit logs too ( 👀 ), so we provide a patched Cargo.toml as well as its Cargo.lock as DLC in the release files.

Copy them at the root of your repository, and cargo will now fetch the right versions of our dependencies.

grub-mkrescue

Because sunrise developers were lazy-ass monkeys, busy being high on smoking rolled-up intel-specs pretending it's some kind of psychotrope, they used to let all the hard work of creating an iso image be done by grub-mkrescue, while taking all the credit.

This means that depending on your distro, you might need to install:

• grub
• mtools

This went on until Thog joined them and created mkisofs-rs, and effectively rescued them, before they fall victim to 8042 withdrawal, and OD trying to snort an A20 line, while attempting to relieve the crave.

nigthly version

Using rust nightly but not specifying which version was a stupid idea in retrospective. Fix that:

echo "nightly-2018-03-15" > rust-toolchain

Building

Now you should finally be able to build and create the boot iso:

cargo make iso

that was easy ...