Why I'm Building a Database Engine in C#

(nockawa.github.io)

47 points | by vyrotek 4 hours ago ago

20 comments

  • benterix 3 hours ago

    > JIT warmup is real. The first call to any method pays the compilation cost. In a database engine, the first transaction after startup shouldn’t be 100x slower than the steady state.

    Correct me if I'm wrong but isn't it what aot was supposed to solve?

    • landl0rd an hour ago

      AOT is a little fussy in real-world usage particularly for things like reflection. You can probably force it to work but it may make your code much uglier.

      Span<T> is more important for performance TBH JIT warmup isn't a huge issue for a long-running process

      • CharlieDigital an hour ago

            > ...but it may make your code much uglier
        
        Flip side is that if you use more source generation, it may end up making the code more terse/"prettier" where it matters and avoid the reflection hit.

        AI agents seem fairly good at generating source generators so there doesn't seem to be a reason to not use them.

      • orphea 16 minutes ago

        What's fussy about AOT and reflection?

      • sebazzz 32 minutes ago

        With UnsafeAccessor you can often avoid reflection.

  • tracker1 3 hours ago

    Also worth mentioning are VeloxDB and RavenDB, both written in C#. TBF, I haven't used any of them... but aware they exist.

    C# is pretty powerful and capable of lower level usage, such as in the examples given... not to mention a pretty nice interop with C-style libraries. It looks like the intent here might be a custom database engine for service integrations... not necessarily a full rdbms in and of itself.

  • nitwit005 an hour ago

    I would be less worried about the GC pause, than the need to reserve some memory for garbage collection. Any reduction in available memory is going to tend to mean a hit to performance.

  • zerr an hour ago

    I'd say modern C++ is high level and ergonomic enough to stop considering any language with a GC.

    • drfloyd51 an hour ago

      Does C++ have build in memory management now?

  • alex7o an hour ago

    That seems great, I have seen a few similar dbs written in java that say the same thing, that when written correctly you can get the perf very close to C, but at that point you are just writing C with a different syntax. You don't win on any in the security guarantees, so at that point can we just not build everything in wasm and then we can interface it from both dotnet and the jvm?

  • kerblang 19 minutes ago

    Am amused that someone feels compelled to justify writing a db in C#. Such conscientiousness!

    I'm not sure authors of Cassandra, ElasticSearch, MongoDB (and more...?) ever had the slightest twinge of uncertainty about whether a managed memory env would cause far more problems than it fixed, even with less native tooling than in C#. Java bros DGAF

  • DeathArrow 2 hours ago

    I wonder why he didn't use AOT compiling if he's worried about JIT warmup.

    • topspin an hour ago

      It wasn't discussed, so we're left to speculate. If I had to guess, I imagine that the .NET JIT has actual benefits: the variety of architectures has gotten enormous and JIT is likely a performance win after warmup.

  • jaen 3 hours ago

    C# is a great language with almost unlimited power and great ergonomics (as the article shows), but the .NET CLR (runtime) is a bit overcomplicated with a distinct "Java smell", and packaging and distribution is still meh.

    If they could make the developer experience similar to Go, it would rule the world...

    • Rohansi 2 hours ago

      > If they could make the developer experience similar to Go, it would rule the world...

      You can already AOT compile .NET software to an executable to run on whichever platform you need, just like Go.

      Libraries need to be published into a package manager (NuGet) which is more friction than just importing from Git repos but it's not that bad.

      • hnrodey 2 hours ago

        AOT is not a panacea and comes with some restrictions/trade-offs that need understood before depending on it in production.

    • landl0rd an hour ago

      I actually really like the CLR developer experience next to java ngl. I reach for C# in lieu of java (less J2EE SingletonBeanFactoryManagerInstance slop) but particularly F# is pretty nice to use. Haskell has bad tooling, OCaml is getting better thanks to JaneStreet (and if OxCaml gets wide adoption unboxed types are a big perf win) but if nothing else lack of a Rider-esque debugger is just a big time sink.

    • karmakaze 3 hours ago

      I drank the Go kool-aid, then tried to do some high performance things the Go way: didn't work (channels are slow) and I got over it. Still think Go is great for web backends and the like with production grade stdlib.

      Great post with details, not a I'm vibe coding...

      • benterix 3 hours ago

        > tried to do some high performance things the Go way: didn't work (channels are slow) and I got over it.

        What did you choose instead?

  • achillesheels an hour ago

    You do you.