This is a personal project that im open-sourcing. Its one of those projects-that-should-exist-but-nobody-wants-to-kill-their-business.
It takes ur standard docker compose file and runs it transparently in kubernetes (k3s actually). So ur devs don't have cognitive dissonance between testing ur stack locally on ur laptop and making it work on kubernetes in production.
It is primarily meant as a dev tool on ur laptop, and as a replacement for docker compose.
Could I use this for running the same docker compose stack multiple times in parallel? I wrote a lot of bash glue code to make this happen (without kubernetes) for integration and acceptance testing on a single server. Managing envs and networking was a pain, but mostly, I struggle to keep it up to date with infrastructure changes in my platform.
Tilt is great but it doesn’t solve the problem you’re asking about. This project more directly addresses that. Fundamentally the problem is that you want to maintain the lifecycle of several services during an ephemeral ci run and tear them down when you are done. As you mentioned it gets unwieldy and annoying to try to run all of these on a single machine and doubly so when you have a lot of services/containers. Kubedock is more like what you are looking for, it translates compose calls to Kube equivalents and each service in the compose file is it’s one kube pod with its own lifecycle. It should be possible under that to do what you are saying, spawn multiple docker composes from a single run.
It is worth noting that Kubedock has some really annoying limitations, part of it is that it’s one person the other part is that some concepts don’t translate to kube very well. So make sure that whatever you will be doing fits into those constraints before you try it
With a single Tilt file combined with a docker compose file, almost all of the infrastructure you need is configured on a local machine. It also supports running kubernetes (most of the docs are around this), but you do not necessarily need to it it.My goto when I have more then 2 docker containers/services I want to keep changing code for. Some teams I work with usually have 20 such containers for local dev.
And yes, you can even nest Tilt files and even write normal python if you want to mix things up.
Also, btw - kappal is architected AI first. Meaning:
1. we have a beautiful skill that can be used by Claude
2. The help command generates output that is useful for claude
3. we have a "kappal inspect" command that generates live output of the stack that can be used by Claude.
hope you have a great time using it. Please file bugs!
Just like KIND runs containerd inside docker, you can also run dockerd inside containerd backed pods.
Start a privileged pod with the dind image, copy or mount your compose.yaml inside and you should be able to docker compose up and down, all without mounting a socket (that won't exist anyway on containerd CRI nodes)
To go even further, kubevirt runs on kind, launch a VM with your compose file passed in via cloud-init.
At no point, have I invented a new/better method. Perhaps your way is better.
I just recognise that Docker Compose is loved by most open source developers. And invariably any project you touch will have a docker compose setup by default. And it isnt going away, no matter hard anyone tries to kill. Some things are just too well designed. Docker Compose is one of those things.
I'm just making it possible to run those on kubernetes seamlessly.
I've just moved on from docker compose. Instead I have a K8s like yaml file and use podman kube play. The learning curve is pretty small in my opinion and at least it is a little closer to production.
Great if you’re you, but try getting AverageSWE a local kube setup and see how quickly they ramp up on it
In my ideal world everyone would use kubernetes, it is the hammer and everything is a nail, but we must recognize that it is difficult for a lot of people to pick up.
That being said, if you’re deploying on kube in production, use kube locally. But if you’re not, dont
Using kube in production but really, even if I wasn't, I would still use the podman play kube approach. It isn't hard (at all) and isn't kubernetes, just kubernetes yaml. I actually find docker / compose a bit harder sometimes with the daemon running in the background.
I'm not arguing for the relative superiority of jsonnet vs yaml vs anything else. I just recognise that Docker Compose is loved by most open source developers. And invariably any project you touch will have a docker compose setup by default.
I'm just making it possible to run those on kubernetes seamlessly.
I'm not quite sure what level of testing this facilitates. If you're testing as close to production as possible, you probably want templated k8s config that scales down to a k8s in CI (e.g. Helm with variables applied that make it minimal). If you just want a local stack to test components and not the k8s config, why not just use docker compose itself?
docker compose is beautiful because it uses a simple elegant compose yml file - this is now an open standard. https://www.compose-spec.io/
the standard does not make it mandatory that underlying system should be docker compose (the reference server). it can be anything.
IMHO - kappal is the first project that takes your compose yml file and transparently/drop-in runs it on kubernetes. there is nothing extra you need. It is useful for people who want to maintain their stack as close to production as possible (kubernetes).
If that's not a big goal for you, then this is not very useful for you. But I'd argue ...why do you care if the compose yml is the only think you are using. you get all of kubernetes.
I was just telling some ex coworker friends that there was a great need for a compose frontend to more powerful infra backends, and this feels like the answer.
Once I get working on it I’ll try to add health check support. That is crucial for a lot of what we’re working on.
So this uses k3s underneath. IMO any local kubernetes distribution is a big resource hog over plain docker. Anyone have ideas for something that is less resource intensive but easier to orchestrate than docker-compose?
This is a personal project that im open-sourcing. Its one of those projects-that-should-exist-but-nobody-wants-to-kill-their-business.
It takes ur standard docker compose file and runs it transparently in kubernetes (k3s actually). So ur devs don't have cognitive dissonance between testing ur stack locally on ur laptop and making it work on kubernetes in production.
It is primarily meant as a dev tool on ur laptop, and as a replacement for docker compose.
Compare/contrast with kubedock? It is almost exactly same project as your description from what I can tell
kubedock is its own command set. it doesnt work with compose.yml . This project specifically starts with a compose file.
NOTE: kubedock has some automatic compatibility with compose, this is not their goal - https://github.com/joyrex2001/kubedock/blob/e4c8540f63d65e33...
If you are not on the compose ecosystem, this is isnt useful to you.
Just fyi - the hottest thing in the world right now is OpenClaw. This is OpenClaw docker compose yml - https://github.com/openclaw/openclaw/blob/main/docker-compos...
even if you dont do docker compose....everyone else does it anyway.
Could I use this for running the same docker compose stack multiple times in parallel? I wrote a lot of bash glue code to make this happen (without kubernetes) for integration and acceptance testing on a single server. Managing envs and networking was a pain, but mostly, I struggle to keep it up to date with infrastructure changes in my platform.
Have you tried Tilt? https://tilt.dev/
No, I haven't. Can you elaborate how you think it could help me? Couldn't figure it out from reading the docs, tbh.
Tilt is great but it doesn’t solve the problem you’re asking about. This project more directly addresses that. Fundamentally the problem is that you want to maintain the lifecycle of several services during an ephemeral ci run and tear them down when you are done. As you mentioned it gets unwieldy and annoying to try to run all of these on a single machine and doubly so when you have a lot of services/containers. Kubedock is more like what you are looking for, it translates compose calls to Kube equivalents and each service in the compose file is it’s one kube pod with its own lifecycle. It should be possible under that to do what you are saying, spawn multiple docker composes from a single run.
It is worth noting that Kubedock has some really annoying limitations, part of it is that it’s one person the other part is that some concepts don’t translate to kube very well. So make sure that whatever you will be doing fits into those constraints before you try it
I love Tilt.
With a single Tilt file combined with a docker compose file, almost all of the infrastructure you need is configured on a local machine. It also supports running kubernetes (most of the docs are around this), but you do not necessarily need to it it.My goto when I have more then 2 docker containers/services I want to keep changing code for. Some teams I work with usually have 20 such containers for local dev.
And yes, you can even nest Tilt files and even write normal python if you want to mix things up.
yes. fixing this right now. you will have it in a day or two.
Would certainly try this out!
this is done. Please feel free to try that.
Also, btw - kappal is architected AI first. Meaning: 1. we have a beautiful skill that can be used by Claude 2. The help command generates output that is useful for claude 3. we have a "kappal inspect" command that generates live output of the stack that can be used by Claude.
hope you have a great time using it. Please file bugs!
Might take a couple of days until I have time to try, but certainly will!
Just like KIND runs containerd inside docker, you can also run dockerd inside containerd backed pods.
Start a privileged pod with the dind image, copy or mount your compose.yaml inside and you should be able to docker compose up and down, all without mounting a socket (that won't exist anyway on containerd CRI nodes)
To go even further, kubevirt runs on kind, launch a VM with your compose file passed in via cloud-init.
this is OpenClaw's docker compose yml - https://github.com/openclaw/openclaw/blob/main/docker-compos... . Arguably the hottest thing in the world right now. Mac Minis are out of stock because of this.
At no point, have I invented a new/better method. Perhaps your way is better.
I just recognise that Docker Compose is loved by most open source developers. And invariably any project you touch will have a docker compose setup by default. And it isnt going away, no matter hard anyone tries to kill. Some things are just too well designed. Docker Compose is one of those things.
I'm just making it possible to run those on kubernetes seamlessly.
I've just moved on from docker compose. Instead I have a K8s like yaml file and use podman kube play. The learning curve is pretty small in my opinion and at least it is a little closer to production.
Great if you’re you, but try getting AverageSWE a local kube setup and see how quickly they ramp up on it
In my ideal world everyone would use kubernetes, it is the hammer and everything is a nail, but we must recognize that it is difficult for a lot of people to pick up.
That being said, if you’re deploying on kube in production, use kube locally. But if you’re not, dont
Using kube in production but really, even if I wasn't, I would still use the podman play kube approach. It isn't hard (at all) and isn't kubernetes, just kubernetes yaml. I actually find docker / compose a bit harder sometimes with the daemon running in the background.
this is OpenClaw's docker compose yml - https://github.com/openclaw/openclaw/blob/main/docker-compos...
I'm not arguing for the relative superiority of jsonnet vs yaml vs anything else. I just recognise that Docker Compose is loved by most open source developers. And invariably any project you touch will have a docker compose setup by default.
I'm just making it possible to run those on kubernetes seamlessly.
fair. however, i do genuinely find docker compose yml and dev-experience to be much more pleasant and intuitive.
if you ever wanna try it again - use kappal. you will get a full k8s but with the UX of docker compose.
I'm not quite sure what level of testing this facilitates. If you're testing as close to production as possible, you probably want templated k8s config that scales down to a k8s in CI (e.g. Helm with variables applied that make it minimal). If you just want a local stack to test components and not the k8s config, why not just use docker compose itself?
docker compose is beautiful because it uses a simple elegant compose yml file - this is now an open standard. https://www.compose-spec.io/
the standard does not make it mandatory that underlying system should be docker compose (the reference server). it can be anything.
IMHO - kappal is the first project that takes your compose yml file and transparently/drop-in runs it on kubernetes. there is nothing extra you need. It is useful for people who want to maintain their stack as close to production as possible (kubernetes).
If that's not a big goal for you, then this is not very useful for you. But I'd argue ...why do you care if the compose yml is the only think you are using. you get all of kubernetes.
@sandGorgon8, thank you.
I was just telling some ex coworker friends that there was a great need for a compose frontend to more powerful infra backends, and this feels like the answer.
Once I get working on it I’ll try to add health check support. That is crucial for a lot of what we’re working on.
hey thanks so much !
just curious - i actually didnt focus on healthcheck because i wasnt sure if anyone was using it. are you using it for local dev ?
So this uses k3s underneath. IMO any local kubernetes distribution is a big resource hog over plain docker. Anyone have ideas for something that is less resource intensive but easier to orchestrate than docker-compose?
Podman supports just the pod spec also used by kubernetes. How much orchestration do you need on a local machine?
docker != docker compose.
docker compose is just as heavy. there's not a significant difference between k3s and docker compose.
Looks promising and really interesting to see and it's a very good idea. But when I saw the test folder however, it is completely empty. [0]
So is any of this tested?
[0] https://github.com/sandys/kappal/tree/main/test
the test folder is for a large real project that i test on (and actually use in real life myself). it has a deliberate gitignore. plenty of tests in https://github.com/sandys/kappal/tree/main/testdata and https://github.com/sandys/kappal/tree/main/scripts