I'm a big fan of SQLite embedded nature, which allows for chaining multiple SQL calls with near-zero latency.
I'm currently building a personal knowledge graph server a mix of Notion's custom entities via JSON schema and Obsidian markdown+backlinked references. It's working well, but I suspect your product might be a better fit.
I do have one question regarding permissions: how would you recommend modeling a hierarchical access system in a graph database? Specifically, if a user is granted access to a document, they should automatically have access to all its child documents within that workspace. Is there a standard way to model this 'subtree' permission logic, or perhaps a more efficient approach you'd suggest?
Really impressed with the product good luck with it!
The permissions question is interesting. I think the answer depends on context. One approach would be to create some edge types `hasAccessTo` and `accessibleBy` that connect a user to a node. Then I'd create an edge type `childOf`. The rest is business logic. Permission checks can just traverse up to the first root node with permissions. The downside is this is all business logic, so can't really look at the database and understand this is how it works. Depending on the database you could create a function that returns permissions for any node, that encapsulates this logic.
Anyways, thanks for checking it out! Really appreciate it. Good luck with your project!
Nice work. How do you handle concurrent writers on a single file? That's where embedded databases usually get tricky, and the graph model makes locking even more interesting.
Yes! This is something I've been thinking about quite a bit the past few weeks. We are going in this direction at work and I think graph storage is a natural way to think about this.
Does it have something like litestream to backit up for specific production usecases (i.e. a single webserver is enough and downtime of a few mins is tolerable)?
I used claude extensively (as well as codex, I finding myself switching between the two every few months). How I used claude varied by the stage. I spent a lot of time initially going back and forth with claude on the idea, figuring out what exists, what would make this interesting, key features I wanted as a user and how to build something around that that made sense as a product.
The initial phases of building I would build out piece by piece. For example, building out the file system interactions I would have claude build a feature and explain how it worked in an educational manner (e.g., like it was a section in a book on latticedb internals). I would then read through the code. This was a great way to learn and build, simultaneously.
In the later stages, where the features and work was more complex, I would spend more time discussing, instructing, and verifying, but less time understanding the actual implementation. I'll give you an example. It's been a long time since I've handwritten SIMD code. I could try and review claudes output, but I am certain I'd miss any subtle bugs that may exist. I found it more productive to assume the code was right and focus on thinking about how I would verify that. Benchmarking, playing with latticedb, etc. were my primary tools for verifying the work. I could run a benchmark and see performance was great. Then I'd explore the test vectors and realize they were trivial, completely invalidating the benchmark results. So we would go back to the drawing board, create a new benchmark set, see results weren't great, and evaluate what was wrong with the implementation. Sometimes features would take days to get out just because of the iteration loop.
duckpgq is great. I'd say the tl;dr is duckpgq if you have tables you want to traverse like a graph sometimes, latticedb when graph traversal is the primary mechanism of querying.
One of the motivating use cases for me was experimenting with agentic memory. I use latticedb as the backing data store. Finding related memories is traversing the graph (kind of like graph RAG).
Great question! Two reasons. One, I wanted to build something on my own from the ground up rather than contribute to an already established large project. For me, it's a better way to learn. Hopefully people find it cool and want to use it. But if the best that happens is it's just a fun project I built then that is just fine for me. Secondly, the data layouts are different. They are very similar in the single-file, graph db respect. But lattice is transactional and row oriented while kuzu is columnar.
I did some perf benchmarking with 1M nodes but I'm mostly using it at smaller scales for another project exploring agentic memory.
Most interesting part is a tough one. From a learning perspective the beginning was incredibly interesting because I was spending a lot of time learning about how other DBs work. Even something as relatively simple as writing to disk had a lot more complexity to it than I initially anticipated.
I used LLMs extensively in building this, and the other interesting part was seeing how they failed. I've always been a proponent that tests are no guarantee of quality code, and working with LLMs has only reinforced it. They often write superficial tests. Sometimes a suite of tests would pass, but when I would actually play around with the feature it was clearly broken. LLMs certainly enabled me to build something of this scope, but it was far from "build a graph DB and notify me when you are done"
I'm a big fan of SQLite embedded nature, which allows for chaining multiple SQL calls with near-zero latency.
I'm currently building a personal knowledge graph server a mix of Notion's custom entities via JSON schema and Obsidian markdown+backlinked references. It's working well, but I suspect your product might be a better fit.
I do have one question regarding permissions: how would you recommend modeling a hierarchical access system in a graph database? Specifically, if a user is granted access to a document, they should automatically have access to all its child documents within that workspace. Is there a standard way to model this 'subtree' permission logic, or perhaps a more efficient approach you'd suggest?
Really impressed with the product good luck with it!
The permissions question is interesting. I think the answer depends on context. One approach would be to create some edge types `hasAccessTo` and `accessibleBy` that connect a user to a node. Then I'd create an edge type `childOf`. The rest is business logic. Permission checks can just traverse up to the first root node with permissions. The downside is this is all business logic, so can't really look at the database and understand this is how it works. Depending on the database you could create a function that returns permissions for any node, that encapsulates this logic.
Anyways, thanks for checking it out! Really appreciate it. Good luck with your project!
Nice, I’ll get it added to gdb-engines.com
very cool site!
Nice work. How do you handle concurrent writers on a single file? That's where embedded databases usually get tricky, and the graph model makes locking even more interesting.
Congrats this is really cool and love the examples
I wonder about mapping RDF data (like Wikidata) to this. I guess the RDF predicate becomes the edge in your node-edge style of graph.
Yes! This is something I've been thinking about quite a bit the past few weeks. We are going in this direction at work and I think graph storage is a natural way to think about this.
If you're going to support RDF, please also consider supporting SPARQL for querying the data.
Does it have something like litestream to backit up for specific production usecases (i.e. a single webserver is enough and downtime of a few mins is tolerable)?
I see "Claude" listed as a contributor. Could you describe how and how much? I'm keen to see how this looks in practise.
As for the tool, it scratches an itch I've been having, I'll give it a go soon.
I used claude extensively (as well as codex, I finding myself switching between the two every few months). How I used claude varied by the stage. I spent a lot of time initially going back and forth with claude on the idea, figuring out what exists, what would make this interesting, key features I wanted as a user and how to build something around that that made sense as a product.
The initial phases of building I would build out piece by piece. For example, building out the file system interactions I would have claude build a feature and explain how it worked in an educational manner (e.g., like it was a section in a book on latticedb internals). I would then read through the code. This was a great way to learn and build, simultaneously.
In the later stages, where the features and work was more complex, I would spend more time discussing, instructing, and verifying, but less time understanding the actual implementation. I'll give you an example. It's been a long time since I've handwritten SIMD code. I could try and review claudes output, but I am certain I'd miss any subtle bugs that may exist. I found it more productive to assume the code was right and focus on thinking about how I would verify that. Benchmarking, playing with latticedb, etc. were my primary tools for verifying the work. I could run a benchmark and see performance was great. Then I'd explore the test vectors and realize they were trivial, completely invalidating the benchmark results. So we would go back to the drawing board, create a new benchmark set, see results weren't great, and evaluate what was wrong with the implementation. Sometimes features would take days to get out just because of the iteration loop.
LatticeDB looks good, just curious how useful is duckpgq
https://duckdb.org/community_extensions/extensions/duckpgq
duckpgq is great. I'd say the tl;dr is duckpgq if you have tables you want to traverse like a graph sometimes, latticedb when graph traversal is the primary mechanism of querying.
One of the motivating use cases for me was experimenting with agentic memory. I use latticedb as the backing data store. Finding related memories is traversing the graph (kind of like graph RAG).
Just read through the README on GitHub and this looks impressive! Kudos
Thank you!
Out of curiosity, why did you not fork and build on Kuzu?
Great question! Two reasons. One, I wanted to build something on my own from the ground up rather than contribute to an already established large project. For me, it's a better way to learn. Hopefully people find it cool and want to use it. But if the best that happens is it's just a fun project I built then that is just fine for me. Secondly, the data layouts are different. They are very similar in the single-file, graph db respect. But lattice is transactional and row oriented while kuzu is columnar.
Kuzu appears to be a retired effort.
Sure, but it’s a solid foundation and available to fork, which is why I asked.
ladybugdb ( https://github.com/LadybugDB/ladybug ) appears to be an open source successor to Kuzu fwiw
Thank you for sharing. I think the sqlite-esque local file approach makes sense for a lot of use cases.
What are some of the scales of the data you've been able to test this design on so far?
What was the most interesting part of designing it for you?
I did some perf benchmarking with 1M nodes but I'm mostly using it at smaller scales for another project exploring agentic memory.
Most interesting part is a tough one. From a learning perspective the beginning was incredibly interesting because I was spending a lot of time learning about how other DBs work. Even something as relatively simple as writing to disk had a lot more complexity to it than I initially anticipated.
I used LLMs extensively in building this, and the other interesting part was seeing how they failed. I've always been a proponent that tests are no guarantee of quality code, and working with LLMs has only reinforced it. They often write superficial tests. Sometimes a suite of tests would pass, but when I would actually play around with the feature it was clearly broken. LLMs certainly enabled me to build something of this scope, but it was far from "build a graph DB and notify me when you are done"
See also: https://ladybugdb.com/ "DuckDB for graphs"
Ladybug looks promising after KuzuDB was sunset (acquired by Apple).
However, I really miss the content posted by the KuzuDB team on their YouTube channel.
...turns out they are compared here: https://docs.latticedb.org/comparisons/vs-kuzu
I like the aesthetics of this page. Very clean and visually appealing