While I like the idea of starting a hobby OS, but the most despairing thing about it that to be useful it would always have to implement 'legacy' interfaces of other OSes. That is, it cannot stand on its own and create a new ecosystem, instead it has to interface to the world and implement TCP, POSIX, know formats widely used files and such.
You end up with an OS kernel that talks Linux/Win32 and takes on a lot of compat code, protocols, and other paradigms.
I wonder what a hobby OS would have looked like it if it assumed nothing, that is, as a thought experiment, as if aliens on another planet invented computing and started writing OSes from scratch. Imagine we discover software from another planet that would not even work with 8-bit bytes, for instance.
(If you want to communicate with other computers) You probably do want to speak IP, but if you want to do all the work, you don't need to use BSD sockets.
POSIX and other such things will let you use more of other people's software... but that's a choice.
You can also do a lot of reuse if you just want to focus on some parts. There's no need to write a bootloader unless you want to. You can pull in lwIP for the IP stack. You can do a lot of interesting (I think) things with barely any userland if you want.
If it's supposed to be a general purpose OS, you end up looking a lot like existing general purpose OSes though, cause it's hard to build everything and the closer you get to existing OSes, the more existing software you can leverage.
If you only want to run a subset of existing software, you might not need to cover that many of the syscalls from whatever OS it was targetted at. My hobby OS runs one specific FreeBSD executable, and it doesn't take that many syscalls to do it.
> Imagine we discover software from another planet that would not even work with 8-bit bytes, for instance.
8-bit bytes didn't become fully dominant until maybe the 1970s. It's a bit hard to find software older than that, but it's around somewhere.
I’d love to see a new operating system that explores radically different APIs for applications. The trouble is writing an operating system is a large effort. Barring market effects, OS has to be heads-and-shoulders better than existing ones in order to convince application developers to write software for it. Windows, macOS, and Linux are good enough for most people, even techies. Additionally, it is often easier to modify an existing operating system such as Linux than to go through the trouble of writing a brand new operating system.
It's really hard for new designs to thrive outside hobby/research usage.
Linux/BSD have massive hardware support, where a new OS needs to start from scratch. The same is true of software support: a new OS that truly has new primitives either adds a POSIX layer (and all POSIX-like software can't benefit from many of its improvements).
Even once you get past hardware and software support (both herculean tasks) performance needs years of hard work to be comparable to mainstream OSs.
And don't even get me started on the ever-growing complexity in standard protocols: HTTP2/3? QUIC? TLS? All "doable", but also huge sinks for time and effort when you already have a huge list of things to deal with.
For a hobby OS .. nobody is stopping you. I have my own hobby OS where everything is custom, I don't even have a standard C library, I made my own, I don't copy posix conventions, everything is just implemented as to what makes sense to me. I never understood the people that go through all the effort to work on their own OS, and all their work is to just create a linux/windows clone from scratch. Of course even accomplishing that is a huge achievement, but if you're going through all the effort, why not do something different?
I get that its a hobby, but are you able to do anything with it? is it purely a learning sandbox or are there some use-cases that you are able to use your OS for that it is a cleaner or more performant solution than existing OS?
> I wonder what a hobby OS would have looked like it if it assumed nothing
There's lots of stuff that was around in the 1960s that became dead-ends because everybody piled in on another way of doing something, and then things got optimised for that way, then every other way was less performant, and soon the lucky one way became the only way.
I've never really had much time to explore these early ideas from well before I was born whilst working, but definitely think it'd be a fun way to spend my time when I eventually retire - to try to recreate some of the stuff from that era that got forgotten and see what it could be developed into nowadays.
For example, just watch Douglas Engelbart's 1968 demo [1] for some ideas - some of the things presented in that took decades before it was rediscovered and implemented on systems available outside of research environments.
I've felt that AI has made this issue a lot less painful. I too have dabbled in hobby OSs a lot, but dropped them around getting drivers and the basic userspace setup, because implementing all the legacy interfaces just kills the fun and the alternative is being unable to do anything meaningful with the OS. Porting the GCC stack to your OS is very rewarding initially, but then it's just reading musl/newlib source code and making your syscalls emulate what they expect, which is far less rewarding.
But, since you can delegate a lot of the grunt work to AI now, I think it becomes a lot easier to have it either, write the legacy stuff for you, or port software over. That way you still have the ability to design a system with the things you think would be interesting, while still having some possibility of running "real" software under it.
Getting someone else's preexisting code to meaningfully run on a thing you made from scratch is quite rewarding. I haven't made an OS but I did make a Game Boy emulator. It runs most of my childhood games just fine, which made it totally worth it for me.
If you only want to please yourself, you can dispense with all that legacy stuff. You can stay busy and enjoy yourself in your own world.
Some of the hobby OS listed here look like they might be standalone worlds:
If you have a goal of making your OS usable, you have to design it with this in mind. Build your kernel and driver subsystem in a way that legacy OS interfaces can be implemented on top of them. Find ways to repurpose existing infrastructure so you don't have to do all the work yourself.
See, I think this is what makes it cool as heck. Write your own OS from scratch, implement the right interfaces, and bam, all this existing software just works.
I love the idea of Wine acting as the universal interface that lets you run windows apps wherever the heck you want.
And of course, this is just an escape hatch. You can support Wine and also support "native" apps which are optimized for your custom OS.
Impressive! I like the goal of making a hobby OS viable as a daily driver.
I've been working on my own hobby OS for half a decade. It does a lot less, but it has helped me realize that we can remove much of the complexity of a generic mainstream OS while still meeting our personal computing needs. I know I'm just poorly reinventing something between DOS and Unix/Plan 9 in an extremely limited fashion, but it's absolutely perfect for experimentation!
I remember all the things I needed to implement before reaching userspace, it's a lot indeed. After that it gets easier because every new change is motivated by a specific smaller goal like supporting the network card of a new laptop, or adding sounds to the emulator, or adding one more syscall or device file. You get rewarded by new capabilities after each addition.
I don't think it is supported anywhere else currently, but Batocera has a simple wsquashfs format. It is basically the game compressed into a squahsfs. Then it can be managed as simply as other traditional ROMs:
No officially-released version of DOS contains generic CD-ROM drivers; the last few DOSes (MS-DOS 6.x, DR-DOS 6/7) contain the redirector (`MSCDEX.EXE` or equivalent) but not the device driver.
From 1992, Windows 3.1 rapidly took over.
The first mass-market CD-ROM drives were the mid-1990s, and ATAPI CDs that could attach to an EIDE interface only became common in the latter half of the 1990s, when Windows 95 was the default PC OS.
Microsoft never shipped any version of Windows 3.x on CD; I used to have a Gateway 2000 WfWg3.11 CD in my travelling toolkit, just in case. Windows 95 originally shipped on 13 1.4MB floppies.
An installer extracts and moves a bunch of files from a blob to the filesystem, writes registry values (on Windows anyway), sets environment variables, sets up services, permissions and a bunch of other things, does it not?
A ROM is a binary blob that some executable accepts as input and interprets, the executable being the emulator, which simulates the target hardware well enough that the ROM's machine code runs as if it were on the original silicon.
It's kind of crazy that we have a hobby OS that supports Linux infrastructure so that can support Wine infrastructure to support a Windows application. Very impressive!
I remember Wine being extremely interwined with XLib not just for the OpenGL rendering backend but also for the windowing and widget/UI rendering interfaces. Is this port running natively without using an X server / emulator?
No? The article extensively discusses the use of XOrg. With that said, the Wine graphics abstractions are a lot better now and there are many working backends (winex11, winemac, winewayland, winehaiku, etc.)
Honestly the most interesting thing to me would be how they got GPU drivers working for a hobby OS. I suspect this was difficult. Unfortunately the blog makes no mention of it.
The source, conventions are quite similar to Linux. So, probably some level of POSIX compliance + reusing standard libraries (such as mlibc) helped a lot.
While I like the idea of starting a hobby OS, but the most despairing thing about it that to be useful it would always have to implement 'legacy' interfaces of other OSes. That is, it cannot stand on its own and create a new ecosystem, instead it has to interface to the world and implement TCP, POSIX, know formats widely used files and such.
You end up with an OS kernel that talks Linux/Win32 and takes on a lot of compat code, protocols, and other paradigms.
I wonder what a hobby OS would have looked like it if it assumed nothing, that is, as a thought experiment, as if aliens on another planet invented computing and started writing OSes from scratch. Imagine we discover software from another planet that would not even work with 8-bit bytes, for instance.
Does a hobby OS need to be useful?
(If you want to communicate with other computers) You probably do want to speak IP, but if you want to do all the work, you don't need to use BSD sockets.
POSIX and other such things will let you use more of other people's software... but that's a choice.
You can also do a lot of reuse if you just want to focus on some parts. There's no need to write a bootloader unless you want to. You can pull in lwIP for the IP stack. You can do a lot of interesting (I think) things with barely any userland if you want.
If it's supposed to be a general purpose OS, you end up looking a lot like existing general purpose OSes though, cause it's hard to build everything and the closer you get to existing OSes, the more existing software you can leverage.
If you only want to run a subset of existing software, you might not need to cover that many of the syscalls from whatever OS it was targetted at. My hobby OS runs one specific FreeBSD executable, and it doesn't take that many syscalls to do it.
> Imagine we discover software from another planet that would not even work with 8-bit bytes, for instance.
8-bit bytes didn't become fully dominant until maybe the 1970s. It's a bit hard to find software older than that, but it's around somewhere.
>You probably do want to speak IP, but if you want to do all the work, you don't need to use BSD sockets.
You could use STREAMS!
Rob Pike of Plan 9 and Go fame lamented this back in 2000 in his “Systems Software Research is Irrelevant” talk:
http://herpolhode.com/rob/utah2000.pdf
I’d love to see a new operating system that explores radically different APIs for applications. The trouble is writing an operating system is a large effort. Barring market effects, OS has to be heads-and-shoulders better than existing ones in order to convince application developers to write software for it. Windows, macOS, and Linux are good enough for most people, even techies. Additionally, it is often easier to modify an existing operating system such as Linux than to go through the trouble of writing a brand new operating system.
It's really hard for new designs to thrive outside hobby/research usage.
Linux/BSD have massive hardware support, where a new OS needs to start from scratch. The same is true of software support: a new OS that truly has new primitives either adds a POSIX layer (and all POSIX-like software can't benefit from many of its improvements).
Even once you get past hardware and software support (both herculean tasks) performance needs years of hard work to be comparable to mainstream OSs.
And don't even get me started on the ever-growing complexity in standard protocols: HTTP2/3? QUIC? TLS? All "doable", but also huge sinks for time and effort when you already have a huge list of things to deal with.
For a hobby OS .. nobody is stopping you. I have my own hobby OS where everything is custom, I don't even have a standard C library, I made my own, I don't copy posix conventions, everything is just implemented as to what makes sense to me. I never understood the people that go through all the effort to work on their own OS, and all their work is to just create a linux/windows clone from scratch. Of course even accomplishing that is a huge achievement, but if you're going through all the effort, why not do something different?
I get that its a hobby, but are you able to do anything with it? is it purely a learning sandbox or are there some use-cases that you are able to use your OS for that it is a cleaner or more performant solution than existing OS?
> I wonder what a hobby OS would have looked like it if it assumed nothing
There's lots of stuff that was around in the 1960s that became dead-ends because everybody piled in on another way of doing something, and then things got optimised for that way, then every other way was less performant, and soon the lucky one way became the only way.
I've never really had much time to explore these early ideas from well before I was born whilst working, but definitely think it'd be a fun way to spend my time when I eventually retire - to try to recreate some of the stuff from that era that got forgotten and see what it could be developed into nowadays.
For example, just watch Douglas Engelbart's 1968 demo [1] for some ideas - some of the things presented in that took decades before it was rediscovered and implemented on systems available outside of research environments.
[1] https://www.youtube.com/watch?v=yJDv-zdhzMY
I've felt that AI has made this issue a lot less painful. I too have dabbled in hobby OSs a lot, but dropped them around getting drivers and the basic userspace setup, because implementing all the legacy interfaces just kills the fun and the alternative is being unable to do anything meaningful with the OS. Porting the GCC stack to your OS is very rewarding initially, but then it's just reading musl/newlib source code and making your syscalls emulate what they expect, which is far less rewarding.
But, since you can delegate a lot of the grunt work to AI now, I think it becomes a lot easier to have it either, write the legacy stuff for you, or port software over. That way you still have the ability to design a system with the things you think would be interesting, while still having some possibility of running "real" software under it.
Getting someone else's preexisting code to meaningfully run on a thing you made from scratch is quite rewarding. I haven't made an OS but I did make a Game Boy emulator. It runs most of my childhood games just fine, which made it totally worth it for me.
I wonder what a hobby OS would have looked like it if it assumed nothing, that is, as a thought experiment ...
There are some hobby OS projects that do this. The best known example is Terry Davis' TempleOS.
https://en.wikipedia.org/wiki/TempleOS
If you only want to please yourself, you can dispense with all that legacy stuff. You can stay busy and enjoy yourself in your own world. Some of the hobby OS listed here look like they might be standalone worlds:
https://github.com/jubalh/awesome-os
It would probably look like the Third Temple.
If you have a goal of making your OS usable, you have to design it with this in mind. Build your kernel and driver subsystem in a way that legacy OS interfaces can be implemented on top of them. Find ways to repurpose existing infrastructure so you don't have to do all the work yourself.
See, I think this is what makes it cool as heck. Write your own OS from scratch, implement the right interfaces, and bam, all this existing software just works.
I love the idea of Wine acting as the universal interface that lets you run windows apps wherever the heck you want.
And of course, this is just an escape hatch. You can support Wine and also support "native" apps which are optimized for your custom OS.
Impressive! I like the goal of making a hobby OS viable as a daily driver.
I've been working on my own hobby OS for half a decade. It does a lot less, but it has helped me realize that we can remove much of the complexity of a generic mainstream OS while still meeting our personal computing needs. I know I'm just poorly reinventing something between DOS and Unix/Plan 9 in an extremely limited fashion, but it's absolutely perfect for experimentation!
This, I assume?: https://github.com/vinc/moros
Pretty impressive, you've gotten much farther than I ever did (I didn't have the patience to implement all the borderline boilerplate an OS needs).
Yes thanks!
I remember all the things I needed to implement before reaching userspace, it's a lot indeed. After that it gets easier because every new change is motivated by a specific smaller goal like supporting the network card of a new laptop, or adding sounds to the emulator, or adding one more syscall or device file. You get rewarded by new capabilities after each addition.
I wish people would start with a "rom format" for windows games. There's so many old games and no standardized way to run them.
I don't think it is supported anywhere else currently, but Batocera has a simple wsquashfs format. It is basically the game compressed into a squahsfs. Then it can be managed as simply as other traditional ROMs:
https://wiki.batocera.org/systems:windows#folder_compression
You mean ISOs? ;)
I think he means where the DOS/config is all set up too. Sort of like a really small VM.
It was a joke, because ISO is a file copy of a CD-ROM disc. ;)
DOS mostly predates CD-ROM.
The DOS heyday was the 1980s.
No officially-released version of DOS contains generic CD-ROM drivers; the last few DOSes (MS-DOS 6.x, DR-DOS 6/7) contain the redirector (`MSCDEX.EXE` or equivalent) but not the device driver.
From 1992, Windows 3.1 rapidly took over.
The first mass-market CD-ROM drives were the mid-1990s, and ATAPI CDs that could attach to an EIDE interface only became common in the latter half of the 1990s, when Windows 95 was the default PC OS.
Microsoft never shipped any version of Windows 3.x on CD; I used to have a Gateway 2000 WfWg3.11 CD in my travelling toolkit, just in case. Windows 95 originally shipped on 13 1.4MB floppies.
https://archive.org/download/microsoft-windows-95_202404
The original comment was regarding Windows not specifically DOS, someone else inferred DOS from the comment.
On that note, FreeDOS does support CD-ROM.
I know. (I've reviewed it and am on its mailing lists; I've contributed a tiny handful of fixes to FreeGEM.)
So does the Win98 emergency boot diskette.
But both post-date Win95.
You mean a packaged file that one simply runs? Isn't that the installer?
An installer extracts and moves a bunch of files from a blob to the filesystem, writes registry values (on Windows anyway), sets environment variables, sets up services, permissions and a bunch of other things, does it not?
A ROM is a binary blob that some executable accepts as input and interprets, the executable being the emulator, which simulates the target hardware well enough that the ROM's machine code runs as if it were on the original silicon.
I was being facetious captain obvious.
ScummVM has effectively done this.
It's kind of crazy that we have a hobby OS that supports Linux infrastructure so that can support Wine infrastructure to support a Windows application. Very impressive!
I remember Wine being extremely interwined with XLib not just for the OpenGL rendering backend but also for the windowing and widget/UI rendering interfaces. Is this port running natively without using an X server / emulator?
No? The article extensively discusses the use of XOrg. With that said, the Wine graphics abstractions are a lot better now and there are many working backends (winex11, winemac, winewayland, winehaiku, etc.)
The screenshot shows an Xterm, I'd assume there's an X server running?
I remember reading about the minecraft port earlier, very impressive work ; those writeups are a great read too.
Honestly the most interesting thing to me would be how they got GPU drivers working for a hobby OS. I suspect this was difficult. Unfortunately the blog makes no mention of it.
The source, conventions are quite similar to Linux. So, probably some level of POSIX compliance + reusing standard libraries (such as mlibc) helped a lot.
I bet it's just running llvm-pipe to render in software.