What are
/r/osdev's
favorite Products & Services?
From 3.5 billion Reddit comments
The most popular Products mentioned in /r/osdev:
The most popular Services mentioned in /r/osdev:
Linux kernel
Hacker News
ReactOS
OSNews
OLinuXino
bochs
Qubes OS
hastebin
LWN.net
Wayback Machine
elementary OS
AnonFiles.com
D (Programming Language)
Homebrew
Orange Pi
The most popular reviews in /r/osdev:
The xinu book. Build an OS from scratch. Easy enough to understand. C and assembly code that works on real hardware. Not an affiliate link ;-)
CC u/Hydra1721
The Linux kernel does NOT use Multiboot. It actually uses the Linux Boot Protocol.
I can see where the confusion comes from, since GRUB uses Multiboot and GRUB is often used to boot Linux. But when you boot Linux in GRUB you actually do it differently, using the linux command instead of multiboot.
Again, don't try to implement Multiboot. It's useless. If you want to boot Linux implement the Linux protocol.
Agreed. There was a good comment along those lines in the Hacker News thread:
> I'm a huge fan of Terry's work, because he works hard, has a clear vision for what he wants to do, and makes it happen. That's more than can be said of many self-professed hackers who never see a project to completion or are motivated solely by peer recognition.
> You may disagree with the logical coherency of his goals - I for one think the "temple for God" thing is pure kookery - but he is a master craftsman, and in this context it's all that really matters. There are people who devote their entire lives to pointless things. There are people who work on supposedly pointless things, which later turn out to not be so pointless. There are people who are supposedly working on world-changing things, which really are completely pointless. In the end, you can't predict the future. Just do honest work that you feel is worthwhile, and see what happens.
You might want to look into supporting USB legacy mode first (http://superuser.com/questions/303365/what-does-legacy-usb-mouse-support-in-a-bios-mean), which will give you a simulated PS2 mouse and keyboard. The protocol is much simpler (e.g. http://www.computer-engineering.org/ps2mouse/)
Don't trust tutorials. They're all full of bugs.
First, determine if you actually do need an ext4 driver in your bootloader. Depending on the OS, it might be possible for the kernel and necessary support files to be stored in the EFI system partition alongside the bootloader.
If you really do need an ext4 driver, start by looking at the ext4 specification and figuring out what information your driver will need in order to turn "read this file" into "read these sectors". If this is your first time writing a filesystem driver, you should probably stop and pick an easier filesystem. You can always come back to ext4 once you're more familiar with what goes into a filesystem.
When you've had enough theory and want to start writing some real code, you'll need to look at the UEFI specification. Unfortunately, the UEFI specification gives almost no practical information; it tells you what every function does, but not how to put those functions together to locate the appropriate block I/O protocol to access the partition. You might have to look at how other bootloaders do it. (I'd love to tell you how to do it, but I've only written code to load files from the ESP.)
ReactOS is a re-implementation of Windows with the goal of being able to run Windows programs. It started in 1998 (20 years ago), has had over 100 contributors consistently contribute over those 20 years (20 years * 100 people = how many years for 1 person?), and has over 9 million lines of code. That is just 1 aspect of the project as you've described it. Does that put things into perspective?
I'm considering moving away from the RPi to something from OLIMEX such as this:
https://www.olimex.com/Products/OLinuXino/A10/A10-OLinuXino-LIME/open-source-hardware
It's similarly priced to the RPi with better specs as well as libre/open hardware and software.
Here's a guide to get going with osdev on such a device:
I'm currently using the beagleboard and it's been working great. There's plenty of documentation about the board (http://www.ti.com/lit/pdf/spruf98, 3500 pages). Plus the ARMv7 spec from ARM is a good 2700 pages. That should be enough documentation for you :).
Why do you need open source firmware?
You're probably going to want to call ExitBootServices fairly early in the execution of your kernel, otherwise you don't get to manage memory, set up a syscall handler, start timer interrupts, etc. (As long as boot services are available, UEFI owns the state of the machine, not you.) And everything relevant like reading files from disk is all in boot services—which makes sense, it requires a filesystem driver and a disk driver and memory allocation and all that.
I think it's a reasonable approach to use PE/COFF as your linkage format for drivers, but you're completely on your own for finding the driver on disk, loading it into memory, and processing the relocations. This isn't weird: it's basically what NT itself does, and except for the drivers being ELF instead of PE, it's what Linux does. If you boot NT or Linux directly from UEFI, the kernel is a UEFI application. (See the docs on the Linux EFI boot stub.) And both NT and Linux use the same object file format for userspace libraries and kernel drivers.
Correct. PRS will be as real-time as the OS's scheduler allows. However, you can get decent latency using the Linux kernel's "isolcpus" command line parameter, which can ensure that (almost) nothing runs on the specified CPU cores. Then you can set the PRS threads' affinity to these cores. You can also use kernel configuration flags such as NO_HZ_FULL to reduce odds of being preempted.
The preemption issue is also why I used lock-free data structures throughout PRS. For example, if one of those preempted threads would happen to be holding a mutex, then it would incidentally block other threads on other cores which are waiting for that mutex. Unfortunately, lock-free data structures can definitely have a significant impact on performance.
To me, it sounds like you are unhappy with all of the three and want to build a new one according to your wishes—a sentiment I'm quite familiar with. Unfortunately, you can't do it yourself because you are not a programmer, but you are willing to pay others. Now, since it was already established that a new operating system would be unpayable, your next option would be funding an existing OS or related projects that fit your imagination. According to your description, such an OS would have the polish of macOS, on all hardware configurations, and run all the Windows programs you want. While we aren't there just yet, we are steadily coming closer with Valve supporting the Wine project which allows you to execute Windows binaries under Mac and Linux. Take a look at Ubuntu or elementary (if you want a more Mac-like user interface). They are both very stable, support a wide range of hardware, have a well-polished UI and, thanks to Wine, many Windows programs can run on them.
Thanks for these classifications ! ( thanks to /u/klange as well)
So I kept on digging… i realised that PCIe actually only supports MSI, no dedicated IRQ lines. Turns out my PCI device is connected to the CPU via a PCI-PCIe bridge, as the CPU only has a mini PCIe slot… this probably explains why the irq number is always 0.
this page from the kernel doc . States that the kernel should be compiled with X in order to support MSI. Do you guys know how I can easily check whether the kernel was compiled with that option? I currently don’t have access to the sources used to compile that kernel.
I imagine you would need to implement some/all of a framework/model/api that those drivers use or interface with, e.g. the Windows Driver Framework or the Linux Driver Model.
Yeah, function pointers are like regular pointers. They just store a memory address of where the function actually is, so it doesn't care what the original name was.
When the user space does a write, it executes a syscall. The way syscalls work depends on the CPU. 32 bit x86 used an interrupt, 64 bit has a special instruction.
When user space calls the write() syscall, it eventually goes to the write syscall handler, which is ksys_write:
https://elixir.bootlin.com/linux/latest/source/fs/read_write.c#L620
ksys_write verifies that the fd # the user passed in is valid, and then calls into the vfs layer to handle the actual write. vfs_write is the function that actually calls f->f_op->write
Here's some documentation on the VFS layer:
Linux uses its own "boot protocol" (see https://www.kernel.org/doc/Documentation/x86/boot.txt ), does not support booting from UEFI itself (e.g. relies on a boot loader like GRUB2 or ELILO to handle booting from UEFI on its behalf).
UEFI's "startImage" function can't be used to start Linux. Linux doesn't even use the PE32+ file format that UEFI expects.
While I agree that a bootsector is very limited in size, this example actually challenges everybody's mental health: https://lobste.rs/s/nzop7p/bootos_is_monolithic_operating_system.
tl;dr: it's actually possible to do a superminimalistic set of useful tools in a bootsector alone.
I'm currently working through linux from scratch. Granted I'm barely at the point of compiling all the source software needed for the OS. GCC at the moment. Seems like a logical starting point though.
I don't know QEMU, so I don't know how to handle its emulated disks. But for code on your OS, I'd say start with interrupt 13h, which is the old ROMBIOS routine for accessing disks. You can start with the CHS methods, and then move on to LBA. Ultimately, you'd be using the I/O ports on the disk controllers to bypass the ROMBIOS, which would mean IN and OUT statements to the disk controllers and likely DMA controllers. (I admit, I haven't learned that far yet...) From the osdev-wiki article below and the Bochs team's handy I/O port list, it looks like the ATA controllers are in the port ranges of 0x01F0-0x01F7
It is definitely not legal; a few weeks ago a user on #reactos (Freenode IRC channel), a user was complaining that his/her JIRA account was disabled. Turns out s/he contributed code with a Microsoft Corporation copyright header that was not GPL compatible. In fact, is was illegal obtained and speculated from OpenNT.
This is definitely not allowed by a huge margin in the legal realm, but also in ReactOS's code base. Those guys cannot risk illegal contamination of their code otherwise they risk jeopardizing the project. (See https://www.reactos.org/user-faq and https://en.wikipedia.org/wiki/ReactOS#Internal_audit for more info.)
Oh, at some point after my previous answer I actually read through the Linux boot protocol docs, which has a nice diagram of physical memory layout and some discussion of what physical addresses it picks. Might be worth reading through quickly.
At the beginning, my background in Operating Systems development was mostly around Linux kernel ecosystem due to my jobs. For me, the most difficult part was to understand how task switching works. I started by digging first versions of linux (https://www.kernel.org/pub/linux/kernel/Historic/old-versions/), and some ARM kernel on github. Also, you need to read ARM documentations of your core (Cortex-A or Cortex-M), to understand how your core can switch between operating modes, this is one the of key mechanism of an operating system.
I am sorry, to not giving you much informations, but it's difficult to remember resources which helped me, etc... I mainly learn by trying to reimplement things that other people have done, in order to understand how things work.
I've been using Vagrant. It basically allows you to run a VM using any virtualization technique, and you can use it to build your stuff.
I made a so-called Vagrantfile that, when it provisions a VM, automatically compiles a GCC cross-compiler for me. Quite nifty.
There is already described how CPU works. OTOH there is some other aspect to describe.
First, there are kernel virtual space and user process virtual base. In a most typical setup (like for x86), they share a common space but kernel one is at top half of the whole space. (In x86-32 in a special mode before mass migration to 64 bit, it could be top 1/4 or even 1/8 of the whole space.) This space maps kernel code and data in a rather free way, it can allocate and free any space portion. But, it has to keep some data structures that track use of the whole physical address space (free/occupied, who occupies).
OTOH, specific platform issues add some more restrictions. For example, it is highly suggested that when DAT is turned on or off on x86, this is done from an equal-mapped page (virtual address is equal to physical one). With typical Wintel platform, this means it is most handy to put the code for this switching and some page catalogs to first 1MB (and make it unavailable for userland - that's why often start address of Linux process is 4MB).
Also, each userland process has its own virtual space structure (its part, complementary to kernel one) which are described in OS-specific structures. At Linux, /proc/$pid/maps describes them. A book like this describe them in details.
Also check out this book! Amazon.com/ButHowDoITKnow
This one teaches you cpu and memory architecture on the lowest level possible in a wonderfully intuitive way.
Understanding enough to write an OS is awesome but please don't get discouraged if you aren't ready in the next 5 years.
Based on your post, i have one of the best books to recommend to you
Code: The Hidden Language of Computer Hardware and Software 1, Charles, Petzold, eBook - Amazon.com
This is a very well written book and eases you into concepts. It covers everything from morse code from braille, to building circuits. This is one of my favorite books of all time and can be picked up from any skill level. It's not about OS development but gives you an extremely solid foundation to learn from. OS dev books/tutorials will be a lot easier to read after this.
I expect he means Unix: a History and a memoir. https://www.amazon.com/UNIX-History-Memoir-Brian-Kernighan/dp/1695978552/ref=mp_s_a_1_1?dchild=1&keywords=unix+a+history+and+a+memoir&qid=1605569339&sprefix=unix+a+&sr=8-1
The Design of the Unix Operating System is a classic. It's from the 80's, but still plenty relevant. It's very well written, with plenty of diagrams to help you along.
It doesn't quite start from the very beginning. If you're looking for information on how to start with absolutely nothing (ie, write a bootloader, implement basic device drivers, etc), then you'll need to supplement with other sources. It does, however, do a really great job of explaining things like processes, threads, memory management, and other basic concepts. It doesn't give you source code (though it contains a bit of pseudocode), but it explains in succinct, legible prose, the data structures and algorithms that drive core functionality. Again, it's an old book - $6.00 plus shipping used. Can't really go wrong.
Operating Systems Design and Implementation covers basically the same ground. I prefer the former, as it treats you a little more like an adult and skips straight to explaining how concepts are implemented (and the cover art is just so undeniably classic).