Turso's Extreme Reliability with Glauber Costa

Episode Description

Turso founder Glauber Costa explains his path from Linux to databases, Turso’s reliability testing, SQLite rewrite, and Postgres-compatible future.

Episode Summary

Glauber Costa traces his career from early Linux kernel contributions and virtualization work at Red Hat through ScyllaDB, Datadog, ChiselStrike, and the creation of Turso. He explains why Turso moved from a SQLite fork to a Rust rewrite built around reliability as a core feature, despite limited runway and considerable technical risk. The team uses deterministic simulation testing, fault injection, differential testing, Antithesis, formal methods, unit tests, and weekend “Carnage Mode” runs to discover reproducible failures and validate database behavior. Costa then introduces pgMicro, an experiment that brought PostgreSQL syntax into Turso by compiling different query languages into a shared AST and bytecode virtual machine. A browser-based Doom port demonstrates the VM’s generality. The conversation closes with Turso’s ambition to become an “LLVM of databases,” supporting PostgreSQL, possible community-built MySQL or Redis frontends, massive database-per-agent deployments, and concurrent cloud writes.

Speakers

• Anthony Campolo
• Glauber Costa
• Dev Agrawal

Chapters

00:00:02 - From Early AI Experiments to the Linux Kernel

Anthony welcomes Turso founder Glauber Costa and recalls their conversation at RemixConf before coding assistants became central to software development. Glauber and Anthony compare their early experiences generating code through Claude and ChatGPT, including a small vanilla JavaScript game built to demonstrate what AI could already accomplish.

The conversation turns to Glauber’s open-source background. He dates his first Linux kernel contributions to roughly 2003 or 2004 and recounts submitting a two-line ext2 patch that introduced two new bugs. Although the maintainer’s response was harsh, Glauber continued contributing, worked on the Xen hypervisor, and eventually attracted the attention of Red Hat.

00:04:49 - How Red Hat Supplied Focus and Direction

Glauber explains that employment at Red Hat transformed his output not by changing his innate ability, but by giving him focus, experienced colleagues, and concrete problems. As an independent contributor, he had wandered through millions of lines of kernel code without knowing which improvements mattered. Red Hat’s Bugzilla queue at least offered specific failures to reproduce and fix.

Reproducing bugs could take days in an era without coding assistants, yet each solved issue led to more complex assignments. Glauber describes how a defined problem, even without close supervision, accelerated his growth. This experience established a recurring lesson for his later startup work: talented engineers become far more effective when their effort is concentrated on a clear and valuable objective.

00:09:00 - Kernel Memory, Containers, and a Move Toward Databases

Glauber describes his work on control groups, particularly tracking and limiting kernel memory consumed on behalf of containers. User-space pages were relatively straightforward to count, but filesystem caches, process creation, and other kernel allocations could exhaust memory without being attributed cleanly to one process. Solving that problem required close work with XFS, ext4, and virtual filesystem infrastructure.

After roughly a decade in the kernel world, he joined a startup founded by key KVM contributors. Its initial unikernel project arrived too early and failed to gain traction, prompting a pivot into databases. Glauber nearly left because he cared only about kernels, but databases ultimately appealed to him as machine-adjacent software with room for richer algorithms and independent goals.

00:13:55 - ScyllaDB, Datadog, and the Desire to Found a Company

The database startup became ScyllaDB, where many former kernel engineers built a low-level NoSQL system for petabyte-scale workloads. Glauber recalls frequently tracing database problems into the operating system and patching the kernel directly. Nearly another decade at Scylla turned his initial indifference toward databases into lasting enthusiasm and expertise.

When Anthony asks whether Turso followed immediately, Glauber explains that Datadog came between them. He had long wanted to start a company with Pekka Enberg, partly because of their close and teasing friendship. A humorous workplace reprimand convinced him that if he wanted to keep working with Pekka on equal terms, Pekka needed to become his co-founder rather than his employee.

00:18:00 - Leaving Stability During a Year of Upheaval

Glauber recounts why he did not launch a startup immediately after deciding to leave ScyllaDB in 2020. He had a new baby, COVID had disrupted ordinary life, and he and his wife were working from a small Toronto apartment. They moved to London, Ontario, bought a house, and faced enough uncertainty that founding a venture-backed company felt unnecessarily risky.

He also knew little about fundraising and did not realize that 2020 was relatively favorable for raising capital. Glauber joined Datadog instead, staying slightly over a year. When Pekka began interviewing there too, Glauber decided he would rather accept the uncertainty than repeat their previous employment arrangement, so the pair finally left to build a company together.

00:23:14 - ChiselStrike Becomes Turso

Their first product, ChiselStrike, is retrospectively described as “Convex for SQLite.” It attracted a respectable community, but users consistently cared more about its SQLite deployment capabilities than its higher-level API model. The founders also recognized that Convex possessed greater experience, resources, and conviction around that broader vision, while their own strongest belief centered specifically on SQLite.

They stripped away the parts they could not defend through years of difficulty and focused on turning a small, portable database into something deployable. Glauber stresses that startups demand extraordinary conviction because founders must continue while others doubt the premise. The company survived by narrowing its mission to the area where the team was prepared to withstand setbacks, then adopted the Turso identity.

00:27:15 - The Rewrite They Initially Feared

After discussing Turso’s Finnish mythological name and its unavailable dot-com domain, Dev recalls the community’s excitement about the company’s use of Rust. Glauber then introduces a pivotal lesson: the founders originally wanted to rewrite SQLite but chose not to. They considered a full rewrite the boldest and most aligned option, yet feared it would consume more time and runway than the company possessed.

Instead, they forked SQLite and named the fork libSQL, while Turso referred to the hosted cloud service. Glauber does not reject that path because it produced valuable learning and may have kept the company alive. Still, he characterizes the decision as a compromise shaped by market conditions, staffing costs, fundraising concerns, and insufficient confidence to pursue the larger technical swing.

00:32:00 - The Limits of libSQL and a Low-Value Market Position

The Turso cloud service gained users, but libSQL did not become the local SQLite replacement the team envisioned. Customers generally continued using SQLite locally and adopted Turso as a convenient deployment destination. The naming also caused confusion because people often encountered libSQL without realizing Turso created it, weakening the connection between the open-source engine and commercial product.

More importantly, Turso became associated with cheap starter projects rather than demanding production systems. Even praise from influential developers reinforced the idea that it was useful before migrating elsewhere. Glauber explains that such positioning traps a company in a low-value market with poor contract sizes and weak pricing power. Healthy top-line adoption metrics could not answer the strategic question of how Turso would grow beyond being a secondary option.

00:37:00 - Why Forking SQLite Could Not Fulfill the Vision

Glauber rejects the idea that Turso’s natural role was limited to embedded or on-device SQLite. From the beginning, the founders wanted to transform SQLite into a full deployable database. Achieving that through a fork proved difficult because SQLite’s architecture was shaped largely by a tiny core team, offered limited separation of concerns, and made deep modifications risky and expensive to maintain.

C introduced additional danger because a mistake in an outer layer could corrupt memory used by fundamental structures such as the B-tree. Meanwhile, SQLite’s celebrated comprehensive test suite was proprietary, preventing the Turso team from running it against major changes. Continually rebasing a modified fork onto new SQLite releases left their hands tied and made the ambitious improvements they wanted increasingly impractical.

00:41:19 - Limbo Emerges as an Experimental Rewrite

The hosts discuss why an open-source project might keep its tests proprietary, including modern fears that AI agents could recreate software from its behavioral specifications. Glauber does not claim to know SQLite’s motivation, but emphasizes the practical consequence: Turso could modify the code without gaining access to the full validation system needed to trust invasive changes.

Pekka eventually began a private experiment to discover whether a rewrite could use a fundamentally different architecture. The first version was about 90 percent simulator and only 10 percent database, capable of reading an entire table but not filtering or writing. Named Limbo and hosted on Pekka’s personal GitHub account, the prototype was initially treated as a distraction rather than an official Turso initiative.

00:46:00 - Community Demand Forces a Strategic Decision

Despite receiving little promotion, Limbo attracted strong contributors, including experienced storage engineers whom Turso later hired. The company mentioned the experiment publicly in December 2024, expecting it to function mainly as a future reference, community seed, or recruiting tool. Instead, Glauber returned from the holidays to roughly 9,000 GitHub stars, dozens of contributors, and unexpected participation from highly capable engineers.

The response validated Turso’s belief that developers wanted a better SQLite, but it recreated the founders’ earlier dilemma. They still lacked abundant time, money, and runway. This time they recognized that refusing the opportunity would repeat the decision they already regretted. Within days, Glauber and Pekka committed to the rewrite and began determining how to preserve the existing cloud business while funding the larger technical mission.

00:51:00 - Going All In While Preserving the Cloud

Turso could not simply abandon its hosted product because real customers depended on it, and withdrawing it would destroy the trust required to introduce a replacement later. The founders instead reduced investment in the existing service, cut roughly half the company, and assigned most engineering resources to the rewrite. Revenue remained stable, which reassured Glauber that the cloud product delivered genuine value even without rapid feature expansion.

The company publicly announced that it was going all in. Limbo would become Turso once the first alpha was credible, while the hosted offering would be called Turso Cloud. This temporarily created a confusing collection of names—Limbo, libSQL, Turso, and Turso Cloud—but the founders accepted short-term ambiguity in exchange for a simpler long-term identity centered on the new engine.

00:55:48 - Reliability as SQLite’s Essential Feature

After some humorous discussion of online criticism, Glauber addresses a substantive objection to rewriting SQLite: people value it because it is reliable and battle-tested, not merely because of its feature list. Critics argued that a new implementation would require a decade of testing before it could approach SQLite’s dependability. Glauber agrees with the premise that reliability is indispensable but rejects the conclusion that only age can produce it.

Reliability was also what Turso’s founders admired most about SQLite, and their concern about matching it had contributed to the earlier decision not to rewrite. Pekka’s simulator-heavy prototype was specifically designed to answer whether modern testing techniques could produce comparable or better confidence without manually writing years’ worth of example-based tests.

01:00:00 - Properties, Generators, and Automatic Test Creation

Glauber explains deterministic simulation testing by contrasting it with ordinary unit tests. A unit test specifies particular inputs and expected outputs, but it covers only a tiny portion of possible behavior. Turso instead encodes system properties, such as requiring an integrity check to pass after arbitrary mutations or requiring indexed and non-indexed versions of a query to return equivalent results.

Test generators then explore combinations of writes, updates, deletes, queries, and failures automatically. Early in development, Turso even used SQLite’s integrity checker against compatible files because its own checker was not yet trusted. The approach does not require knowing every exact output in advance; it identifies contradictions and violated invariants that prove at least one execution path or internal component is wrong.

01:04:04 - Making Every Nondeterministic Input Reproducible

Anthony reads a concise definition of deterministic simulation testing, and Glauber expands on its implementation. Turso routes I/O, clocks, randomness, and other environment-dependent operations through controlled interfaces. In production those interfaces reach the operating system, while in simulation they use a deterministic model. The engine avoids uncontrolled threading and carefully audits dependencies that might read a clock or create hidden nondeterminism.

Given the same random seed, every timestamp, interleaving, event, and failure occurs identically on every rerun. This provides two benefits: the simulator explores a vast space of generated behavior, and any discovered bug becomes reliably reproducible. Turso also replaces or controls allocators to simulate out-of-memory conditions, showing that deterministic design does not require rejecting Rust’s standard library wholesale.

01:09:00 - Expanding Simulations to Find Missing Classes of Bugs

Glauber stresses that simulation is not magic; it only explores behaviors represented by its generators and models. At one stage the simulator never created indexes, so it missed index-related defects until the team taught it to generate them. Another bug appeared only after a database exceeded one gigabyte, where SQLite expects a special marker byte that Turso had not written.

Aggressive crash injection prevented simulated databases from growing large enough to cross that threshold. The team responded by separating long-running, fault-free property checks from fault-heavy runs and later built specialized tools for concurrent access. Turso now combines its deterministic simulator with differential testing against SQLite, dedicated concurrency testing, Antithesis, and extended weekend stress runs, allowing each method to cover gaps left by the others.

01:13:11 - Antithesis, Carnage Mode, and Formal Verification

Antithesis is described as an expensive but valuable service built around a deterministic hypervisor. Because it controls the system beneath the database, it can reproduce whole-machine behavior and inject aggressive faults that Turso’s application-level simulator may miss. The team runs shorter Antithesis checks regularly and a two-day weekend configuration called “Carnage Mode” to search deeper execution spaces.

Turso is also adding formal methods, which use mathematical models to specify and verify system behavior without waiting for costly runtime exploration. An intern, Pavan Nambi, used this work to identify ten bugs in SQLite while checking whether his model was correct. Turso partners with aretta.ai to make these difficult, bespoke proofs more approachable through AI-assisted tooling.

01:17:59 - Reliability Is a Product Feature, Not a Reputation

Glauber frames reliability as one of SQLite’s defining features and therefore a mandatory part of any faithful reimplementation. Turso supplements simulation and formal verification with familiar unit, integration, and compatibility tests. No system is literally bug-free, but the team believes its layered methodology provides a strong ability to expose defects early and reproduce them precisely.

When Turso’s methods uncover SQLite issues, the team reports them upstream, although it generally does not attempt to contribute code because SQLite is difficult for outsiders to modify directly. The discussion then connects this reliability work to Dev’s Specter project, where structured specifications and deterministic checks can constrain AI-generated implementations. Both efforts distinguish declared behavior from a much larger space of generated scenarios.

01:22:39 - Testing AI-Written Code at the Right Cost

Glauber explains that testing rigor should reflect the consequences of failure. Turso does not validate dashboard React code with the same machinery used for durable database storage. The team uses AI extensively, often generating code, reviewing it, requesting refactors, and having Claude Code and Codex critique one another, but humans still remain involved in reviewing core engine changes.

Different checks also belong at different points in development. Fast happy-path, adversarial, unit, and integration tests guide the immediate coding loop, while deterministic simulation runs before commits. Daily Antithesis jobs, weekend Carnage Mode, and asynchronous formal verification operate later because they cost more or take longer. AI makes generating conventional tests inexpensive, but speed remains essential when tests must provide continuous feedback to an agent.

01:27:03 - A Testing Pyramid for Database Engineering

Dev compares Turso’s reliability system to the traditional testing pyramid: many cheap unit tests at the base, fewer integration checks above them, and expensive end-to-end validation at the top. Each layer catches different failures and must run at a cadence appropriate to its cost. Glauber confirms that every released Turso version must ultimately pass the full collection of checks.

Carnage Mode is Turso’s internal name rather than an industry standard. Unlike a continuously operating tool such as Netflix’s Chaos Monkey, it is a deliberate weekend switch that unleashes a more resource-intensive Antithesis configuration. Although such runs can expose serious problems, earlier layers catch most issues first. If the final stage routinely found large numbers of bugs, that would signal inadequate lower-level tests.

01:32:05 - pgMicro and the Postgres Translation Problem

Glauber introduces pgMicro through a conversation with a CTO who had used AI to translate PostgreSQL queries into SQLite queries. Glauber argues that query translation alone does not create PostgreSQL compatibility because the systems differ in types, materialized views, semantics, and numerous edge cases. Compressing PostgreSQL behavior into SQLite’s model and expanding it again inevitably loses information.

A better approach emerged from translating PostgreSQL syntax into an abstract syntax tree rather than another SQL dialect. Once parsed, operations such as selecting, joining, and accessing tables can be represented independently of their original textual syntax. This insight suggested that PostgreSQL could target the same intermediate execution architecture as SQLite, provided Turso extended the lower layers whenever it encountered a real semantic mismatch.

01:37:00 - Compiling PostgreSQL Into Turso Bytecode

SQLite is not merely a parser connected directly to storage; it compiles queries into instructions for a bytecode virtual machine. Those instructions resemble a database-oriented counterpart to operations in the JVM or V8, covering actions such as opening and traversing B-trees, creating indexes, loading values, and manipulating records.

Glauber’s pgMicro experiment tested whether PostgreSQL syntax could compile into Turso’s corresponding bytecode. Type handling was the first major gap because SQLite allows flexible values where PostgreSQL enforces richer types, so Turso gained experimental typed behavior. The prototype worked well enough to merge into the main Turso project. PostgreSQL support remains partial, but it starts from an already functional and extensively tested database rather than a blank implementation.

01:42:33 - Running Doom Inside a Browser Database

To demonstrate that Turso’s virtual machine is general enough for unexpected workloads, Glauber ported Doom to its bytecode. The database compiles to WebAssembly and runs inside the browser, with each rendered frame produced by a select query. Game memory is stored as chunked blobs in a table, and player interactions update that state while the query continues yielding frames.

Anthony launches the game during the stream and finds it responsive enough to play despite slight lag. Glauber explains that he built a C-to-database-bytecode compiler and did not need special Doom-specific instructions beyond capabilities Turso already possessed. The stunt functions as a playful generality test: if the VM can execute Doom, its architecture should be broad enough to support PostgreSQL and other database interfaces.

01:47:47 - Building the LLVM of Databases

The hosts refine Turso’s architectural analogy to LLVM. Traditional compilers once needed separate implementations for each source language and processor target, while LLVM introduced an intermediate representation that allowed new languages to reuse shared optimization and backend infrastructure. Turso similarly aims to let SQLite, PostgreSQL, and future query languages compile into a common AST and bytecode engine.

Glauber is most excited not by faithfully reproducing today’s databases, but by lowering the cost of inventing new ones. A developer could create a frontend and parser for a custom interface without rebuilding storage, execution, portability, and reliability machinery. He contrasts this ambition with pg_lite, which runs PostgreSQL in WebAssembly but inherits architectural constraints from PostgreSQL’s process-based design and therefore must leave significant capabilities behind.

01:52:03 - Redis Possibilities, Concurrent Writes, and What Comes Next

Dev asks how far Turso can move beyond SQL-like relational systems. Glauber says the shared AST resembles a structured representation of relational algebra, while the bytecode can be expanded for other models. Redis support, for example, may require only a few new instructions for features such as time-to-live behavior; community contributors have already experimented with MySQL, though Turso itself is prioritizing PostgreSQL.

The episode closes with Turso Cloud’s rollout of concurrent writes, marking the new Turso engine’s arrival in the hosted service after private testing. Users must initially enable it explicitly while the company observes production-scale behavior. Glauber expects major PostgreSQL progress within about six months and reiterates plans for cheaper, faster, developer-friendly infrastructure capable of supporting enormous database-per-agent deployments. The conversation ends at 01:56:45, the total duration.

Resources and Links

• x.com/glcst
• turso.tech
• penberg.org
• github.com/cetic/unikernels
• scylladb.com
• en.wikipedia.org/wiki/Iku-Turso_(creature)
• turso.tech/blog/introducing-limbo-a-complete-rewrite-of-sqlite-in-rust
• github.com/tursodatabase/libsql
• en.wikipedia.org/wiki/D._Richard_Hipp
• turso.tech/blog/working-on-databases-from-prison
• youtu.be/V_qzqY1bb7I
• turso.tech/blog/the-final-boss-of-reliability
• github.com/glommer/pgmicro
• turso.tech/blog/running-unmodified-doom-in-the-sqlite-bytecode-language

Transcript

00:00:02 - Anthony Campolo

All right, we're live. Welcome back everyone to AJC and the Web Devs. We got a new guest to the stream, but someone we all know very well, probably Glauber Costa, the founder of Turso, a distributed SQLite thing. I don't know if that's necessarily correct or not, but it's a SQLite database that is super cool. Uh, we had a great chat 3 years ago at Remix Conf. Um, so yeah, uh, welcome to the show. How you doing today?

00:00:31 - Glauber Costa

Thank you. Uh, remember you very well. It was a great time chatting. I was— and we were, uh, reminiscing on this before it started. The RemixConf, uh, was one of the times right before AI started, right? So we could still code like cavemen. Uh, great times, great times. We actually wrote code by by hand. Can you believe that?

00:00:58 - Anthony Campolo

I was, I was, you know, getting code out of ChatGPT a little bit. I was just, you know, just going into the chatgpt.com, like, write me some code.

00:01:08 - Glauber Costa

Yeah, my first, uh, wow moment with AI actually was in the Claude web interface, way before Claude Code. I mean, I just— let me try this one thing, and then I pasted It works. It was great.

00:01:25 - Anthony Campolo

Yeah, yeah, my buddy and I built like a just, you know, straight vanilla JavaScript little game because he was like, what's the deal with this like AI stuff? I'm like, well, let me show you. Let's sit down, let's build a game. Yeah, all right, we already got some people in the chat here.

00:01:39 - Glauber Costa

All right, Shreyas, nice to see you. Shreyas has been a buddy of ours, he's always around. Great, great fan of Turso. And of course, I am a fan of everybody who's a fan of Turso, so he has my admiration. Amazing.

00:01:55 - Anthony Campolo

Awesome. So, um, yeah, we wanted to just get a little bit on your background. Um, I think you have such a cool background, especially in open source. Uh, you did work on the Linux kernel. Uh, what was the first year that you, um, what year was it that you first started doing that?

00:02:11 - Glauber Costa

It's fuzzy because it's like the fall of the Roman Empire. By the way, if you haven't thought about the Roman Empire up until this point. I ruined it for you. Yeah, it happens with me every day. Daily dose of the Roman Empire. And nobody kind of knows when the Roman Empire fell. It just kind of put that date and say, hey, we're going to choose this. You know, there is a clear before, there's a clear after, there's a fuzzy thing in the middle, and then you kind of pick something in that middle to be like the official date. So I was around the Linux kernel starting in 2003, 2004. So somewhere around that point, I could start calling myself, I suppose, a contributor. My first attempt to send code to the Linux kernel, probably around late 2003, I sent a patch to, That's how we called back then. Today you would call it a PR. Just, they call it, sent a patch to the ext2 file system. And you have fairly old file system. And I thought that I had found the bug. And then I send it. And as a response, I heard, bro, yeah, I don't, it's all good, man. Shreyas was 4 years old. So I was a lot older than 4. And then I send a—

00:03:43 - Anthony Campolo

I was 13.

00:03:44 - Glauber Costa

Yeah, I send the patch and Alvaro, the maintainer of the VFS layer, the virtual file system layer, he responds that he— I somehow managed to introduce 2 new bugs in 2 lines of code. And that was unbelievable. And then people like me should never be allowed to get close to a keyboard again. So that was my introduction to Linux. I did not listen to him, although it wasn't necessarily pleasant. And around the end of 2004, I was contributing, again, still fairly— it's the nature of exponentials, right? Still fairly— I was pretty proud of the output, but compared to what came later, like a dropping nothing, but I, I made a couple of interesting contributions to the Xen hypervisor, and then I was hired by Red Hat. And then, and then I think my career kind of started, because before that, I mean, it was more like an amateur kind of thing.

00:04:49 - Anthony Campolo

Did the— so the Red Hat thing, that, that probably led on from the open source contribution?

00:04:54 - Glauber Costa

Yeah, yeah. But, but the thing is that whatever I've done before was nothing, right, after what I compared to what I've done after, uh, because then I finally had time and, and people around me to help me and, and focus. I think the thing that, the thing that changed the most for me, because I mean, it's the same person with the same ability, and, and I had people around me, that's fine, but I also had access to people on IRC and mailing lists, so It's okay. And obviously at Red Hat, I had a lot more direct access to more experienced people. But I think the thing that changed the most definitely was focus. Because when I was just like in love with the Linux kernel, I was like looking around this massive codebase, looking for stuff to do. And I didn't even know if the things that I was trying to do were worth doing or not. It was just like, what can I do to improve this thing that has millions of lines of code? Like, how do I even start? So again, the thing at Red Hat that I benefited the most was having focus, having somebody tell me, this is what we need to fix. Even though the focus was not that focused when I joined Red Hat, it was right when I was fairly young. They were about to release Red Hat Enterprise Linux 5, and virtualization was going to be one of the key aspects of Red Hat Enterprise Linux 5. Everybody was busy. Nobody had the time for me. So they kind of said, go into the bug tracker. It was Bugzilla at the time. Just go into the bug tracker, whatever you find, fix it. And if you don't find any, you know, just if you find any other bug, open more bugs. So I mean, it's not like I had a fairly well-defined thing. But at least, again, at least I had some focus. I had it there. Hey, there's this bug here that I can go and fix, right? I know what the bug is. Somebody reported the bug. Again, reproducing bugs back then was very hard. You couldn't just ask Claude for directions. You have to go figure out like what to do. It would take days to reproduce one bug. But again, with that, and, and I started doing more and more, and then more complex projects after more complex projects. And, and then, and then at that point, it was no longer the Roman Empire, it was the Dark Ages. Like what, what came after was the Byzantium. Beginning is a lot of processes is one big unit. And then you can isolate resources between those various cgroups. And resources could be CPU time, it could be an amount of memory. And the particular problem that I was trying to solve was how to limit and build— track and limit the amount of kernel memory used for a cgroup. And again, because you can very trivially account the amount of memory that a cgroup uses, because as your program allocates pages, you just counted this page here, this other page there. But there's a lot of things that you do that allocate kernel memory. Like for example, if you open a file, do a traversal on a file system, and that leaves a bunch of memory behind because you have all of those caches inside the Linux kernel. And at the time, nobody kind of knew how should we account for that thing. And if you don't account for that, you can kind of blow up the container because you just go, like you create a fork bomb, for example, and you start creating more and more processes. And each of those processes uses up a little bit of kernel memory. And then at some point, the cgroup allocates all of the kernel memory. And then you think you have memory, but you don't. So the problems that we were trying to solve at the time was is how to account and limit kernel memory. And again, it's very hard because kernel memory doesn't exist on behalf of any specific processes, kernel memory. And a lot of that memory usage came from file systems. So I worked very closely at the time with XFS, ext4, and the virtual file system infrastructure in general. So that's, again, that's where I met Pekka. I didn't necessarily have any interest in databases. In fact, I had no interest in any computer program that was not the kernel. For me, the kernel was just the only thing that I really liked. There was always this joke, like, whatever people are talking about, that's just user space stuff. We don't care about that. It's just like, who cares, right? But then I joined a startup in 2013. So again, around 10 years after the beginning of my period, not my period, you know what I mean, like the period in which I was in the Linux kernel. And that startup was created by 2 key people that created the KVM hypervisor. So I was very good, not was, I am very good friends with them. Although we don't chat that often anymore, just out of, you know, life and situations. But, uh, it was, again, was a startup in, in the, in the OS world. They were trying to do something called uni— unikernels, which at the time didn't work. Now with AI is getting a little bit of a revival. There's some companies doing unikernels extremely well. And, but we tried to do it back then. I think it was too early. And it didn't work. But then the company pivoted into writing a database, and I almost left because I said I have no interest in writing a database. But at the same time, I think I was kind of tired of working in the kernel, and I didn't want to go back, and there were not that many other things for me to do, and I decided to give it a chance. I said, hey, let me at least see how this plays out. And I ended up liking it, and I liked it because In the kernel, you are fairly limited in around what you can do because the idea of the Linux kernel is that the Linux kernel or any other operating system for that matter can't really use a lot of CPU time, can't really use a lot of resources because the kernel doesn't have an existence on its own, right? So you start the Linux kernel What for? What do you do with this? I mean, there's no point in starting the Linux kernel. You start the Linux kernel to run something. So, you know, the kernel exists to give other processes an interface. So every time you go into the kernel, the goal is to get out of it as soon as possible. I mean, you go into the kernel because somebody called the kernel to do something. So you want to get out of the way and, and just go, uh, go back as soon as possible. So the algorithms, they tend to be a lot simpler. Uh, for example, I don't recall ever seeing any recursive algorithm ever in the Linux kernel, just because again, it can blow up your stack quite easily. Uh, and, and, you know, everything is very simple. I mean, it, it's the kind of stuff I like because it's very physical. That's what I like about computers. I mean, Again, unsurprisingly, I totally suck at front-end development. I can't center a div, although these days I probably can, uh, and, and none of that. So I always like to be very, very close to, uh, the systems. And databases are like that. I mean, databases, as far as applications go, they're probably one of the closest things you can get to the machine. But because they are something that has a goal and has a reason to exist by itself, the algorithms get a lot more complex. I mean, you have a lot more things that you can do. And then I ended up falling in love with it. And that's how— and then I stayed in this database company for almost another decade. Database company was called ScyllaDB, for those of you who are in the know. It's a NoSQL database in the petabyte-scale workloads. And that's how I, you know, got acquainted with databases. The folks at Scylla were almost all former Linux kernels, so we would patch the Linux kernel all the time. Like, we found the bug, turns out the issue is in the kernel, and then we're going to go investigate and find out. It was a very interesting thing because, I mean, we really kept it very low level. Uh, Turso is nothing like that. I mean, Turso, I don't think I I ever touched the kernel working on Turso.

00:13:55 - Anthony Campolo

Yeah, ScyllaDB was one of those projects that I'd always hear about on, um, Software Engineering Daily. I don't know if you knew that, um, podcast back in the day. Jeff, the late great Jeff, um, he would bring on these database people, talk about stuff like that, and I would— I was totally new to the industry and I'll listen to it and I would just be like, wow, this sounds so interesting and I have no idea what they're talking about.

00:14:19 - Glauber Costa

In the beginning, neither did I. As I said, I almost left the company, and I'm happy I didn't, but, uh, because I truly had no prior experience or interest in databases. But it grew on me, uh, and today I suppose I do.

00:14:33 - Anthony Campolo

So that fed into Turso then? Did you go from Scylla to Turso? Was there anything in between?

00:14:39 - Glauber Costa

No, there was, there was, there was something between. Yeah, what's up, Dev? That, that was—

00:14:45 - Dev Agrawal

yeah, ChiselStrike. I remember that.

00:14:47 - Glauber Costa

No, I remember that. That's, that's Turso. No, that, that's okay. So because the same company, right? It's the same, it's the same way Scylla— Scylla didn't start as Scylla, uh, that was a pivot. And ChiselStrike was— I actually can't explain ChiselStrike today. That back then was one of the, uh, one of the hardest things for us was like, how do I even explain what this thing does and what this thing is? Today I can, and I'll tell you later, but the thing that I had in the middle was Datadog, because the thing is that when I was ready to leave ScyllaDB, I always wanted to start a company, and I always wanted to start a company with Pekka, because there was this day that we were all working on our Slack channel. Slack already existed back at that point. And I was always dunking on Pekka. Like everything Pekka would say, I'd say, you're so stupid, Pekka. And then the founder of Scylla scolded me, saying, you can't say those things because Pekka might get offended. And then I said, but Pekka doesn't get offended and he's my friend. That's just how he does. That's just how we do. And then he got truly mad at me and I remember at the time it was like, oh, but you know, it's unprofessional. And other people see this and then, and all the, and at that moment I realized that if I wanted to spend my life dunking on Pekka, I would have to start my own company. And I also realized at the same time because of the power imbalance that he couldn't be my employee, 'cause then, right? So he had to be my co-founder and that's how I decided to find— to found the company. But again, at the end of my tenure, That was in 2020. So 2020 was the year that I just could not keep going with Scylla for a variety of reasons. Um, you know, one of them is just that I think I was— I thought I was just doing this for too long. I wasn't really aligned with a strategic direction of the company, so I figured, hey, I would— I should— maybe not the time. But you might remember that 2020 was a very rough year.

00:16:55 - Anthony Campolo

I feel like it was a transition year for so many people. It's like there's like because things were changing so much in the world, it was almost like it kind of compelled a lot of people to make some big changes, like it did with me, like kind of in some way coincidentally, but still it just like, it was kind of in the air.

00:17:14 - Glauber Costa

Yeah. And the thing is, again, maybe it was, again, I don't know if this is PG-13 or R. You can say whatever you want. But, you know, perhaps I was a little bit of a pussy back then, but I wanted to do it and I think I was ready ready for the change, but it didn't feel— there was so much, there was so much already changing in my life. I mean, because I just had a kid, COVID started, right? And, and like, I was— and then I decided to move to a different city because I was living in Toronto at the time.

00:17:50 - Anthony Campolo

You went to Austin, right?

00:17:51 - Glauber Costa

No, no, I, I moved to London, Ontario, small town in Canada, 2 hours away from Toronto, because the problem was like both my wife and I working from home with the baby crying all the time in a shoebox apartment because Toronto real estate is quite expensive. So I just can't do this anymore. I have to, I have to get out of here. And then like I bought a house in, in, in this new town. I didn't really know anybody there and I had just bought a house. I mean, it just didn't feel like starting a company at the time was the smartest idea. Um, also I knew nothing about venture. Today, I still don't, but I think I don't know a little bit more than I used to back then. And I just, for me, it was like, I wanna start a company, but I knew nothing about startups, nothing. And turns out 2020 was actually a pretty easy year to raise money, but I did not know. So for me, it was all like, I don't wanna take the risk and I don't even know where to start. And it feels like a very stupid idea. So then I joined Datadog, but after a year at Datadog, then again, then it was in 2020, COVID was still there, but it became a part of life. It became a little bit of a non-factor. And then Datadog, Packer started interviewing at Datadog, and then I said, oh man, fuck this shit. Let's just start a company together. I'm not doing this again. I'm not working with you again. And then we started, and then we decided to start the company.

00:19:25 - Anthony Campolo

Where were you in 2020, Dev? What were you doing?

00:19:29 - Dev Agrawal

2020, I think I was, uh, that was the first time I was starting to react. No, it was my sec— uh, third year, or I think it was my— yeah, second year, yeah, third year of college. Um, I just started at a contracting firm. I was like, that was my first project. You— and, uh, yeah, I, I was, I was migrating a huge Angular 1 app to React. That's what I was doing. And, uh, then everything went remote and the world kind of ended almost. Yeah, I don't know, for me the world got better because I got to stay at home for the whole, the whole time.

00:20:06 - Glauber Costa

Yeah, so I stay, I stay over a little bit over a year, Datadog, and then we founded what was called at the time ChiselStrike. So now for Dev, because that, that's when I met, that's around the time I met Dev. ChiselStrike was Convex for SQLite. That's what it was. Uh, but nobody understood what that was, and, and Convex wasn't doing great either at the time. Uh, and everybody who came to try it— I mean, we did, we did create like a quite respectable, for that day's standards, Discord community, and people came chat about it. Uh, you know, there was some interest. I mean, we got to like 1,000 GitHub stars, so I mean, there was some level of— it wasn't like zero. But everybody who came to, to our community came because of SQLite, because they saw this as an easier and more powerful way to deploy SQLite. So they were always asking us about SQLite. Nobody seemed to care about all the API things that we were doing. It was almost like something that stood in the way. Um, again, we kind of became a little bit jaded with, we can't explain this thing to anybody. And, and, and it seems like Convex has more money and they're more— because they raise a big Series A at the time from Andreessen. Because they really, you know, they really, they were doing things like that before. I mean, Convex evolved from something that James was already doing. And ChiselStrike for us was just, we were not doing it. I mean, we had the idea and we said, hey, there is a place for this, but we didn't have anywhere close to the level of experience and commitment that they had. And then it just felt to us like it's pointless to keep insisting on this because I don't think we have, and this is something very important in startups. Yeah, Pocketbase was around the same time as well. Um, you truly need— I mean, I don't think I understood that before. I understand now. You truly need an unbelievable amount of conviction to do a startup like this because you're gonna have— you're gonna go through periods in which everybody thinks you're crazy and, and things take time to catch on. And then we didn't As we looked at ourselves, we did not have that conviction, right, that Convex had. And we thought that they were doing it better anyway. They had more structure, more everything. And then we said, why insist? And the stuff that people seem to like about what we're doing is you can distill it down to SQLite. And in SQLite, we actually have that amount of conviction. So for us, it was like, let's— if we remove the parts that we don't have the conviction around and are just there because it's us playing and seeing where things go, right?

00:23:14 - Dev Agrawal

And if you—

00:23:15 - Glauber Costa

is there anything in this? Because again, if it isn't, then we're going to go do something else completely different. But if there is, then we can perhaps save some of it, which we didn't really save any code or anything like that. We, we found out that we did, we did have the, we did have the very strong conviction that there was something around SQLite that we wanted to do. Because again, the reason we did the, let's call it like Convex for SQLite, is that we truly like SQLite. We, we love the idea of like, hey, there's this super small database, it can go anywhere. And then, you know, our goal is like, how do we turn this into like a deployable database? And, and, and bring the good things about this to people. And again, after some soul-searching, we found out that we, we had the necessary conviction to eat whatever shit the world threw at us, but it had to be only on that box. We did not have the conviction on, on, on the extended vision, and then we got rid of it. And, and that's how we renamed the company and so on and so forth.

00:24:24 - Anthony Campolo

I don't know if I've ever asked this before. Why Turso? That name?

00:24:28 - Glauber Costa

Why the name? Because Pekka is Finnish, and apparently there is a character in Finnish mythology called Iku-Turso. The way he's stupid, where he says Turso. And then one day, apparently, apparently this is famous enough in Finland that there are all sorts of beer brands and submarines with, with that name. And there he was having an Iku-Turso IPA, and he named the repository because we had this experiment, like, let's try to launch something, right? And then he had this experiment, he was drinking the IPA and named it Iku-Turso, and we all kind of liked it. But, but the thing is that Iku-Turso is just Iku sounds Japanese and it's very simple, and so we're going to have all sorts of problems getting domains for this. But I liked it because it was short. But at the same time, I mean, the disadvantage of it being short is that domains are going to be a big problem. Uh, and again, it sounds very Japanese, and nothing wrong with that, but that's just not— I happen to love Japan, uh, but, um, that's not where we were coming from. Uh, and then Turso sounded like a cool, powerful name, you know, just has— it's short enough.

00:25:49 - Anthony Campolo

And turso.tech is your domain now, which is—

00:25:52 - Glauber Costa

yeah, because there's an accountant in Florida that owns turso.com. We tried to buy from them, but we couldn't, so whatever, doesn't truly matter.

00:26:01 - Anthony Campolo

That's super interesting. I love hearing about where names come from. That's— I'm glad I asked that. That's really, really cool. Cool. So yeah, I remember you telling me that's story at some point.

00:26:11 - Dev Agrawal

I think we met at— the first time we met was one of the conferences in Salt Lake City. It was not Remix Conf because I did not go to the Remix Conf.

00:26:23 - Glauber Costa

I don't remember.

00:26:24 - Dev Agrawal

It was Kent's conference. The one that Kent C. Dodds hosted. Might have been that one.

00:26:29 - Anthony Campolo

Epic Conference. Epic Conf.

00:26:31 - Dev Agrawal

Epic Conf, yes. I think it was probably that one. Yeah, but yeah, I, yeah, I think that's a really fun story. I think the, the part about Turso that I think got, um, caught a lot of attention from the community was your focus on Rust. Like way before the SQLite Rust rewrite, uh, you, you were— the entire infrastructure of Turso was built around Rust. And I think I remember Primagen being super excited about that and telling a bunch of people. I think he even went to Vercel and when they launched their database product, and, uh, or at least when they partnered with Neon, and Primagen was in their faces like, Neon is the wrong choice, should be going with Tursell.

00:27:15 - Glauber Costa

No, yeah, Tursell was a very different thing than it is today though, and, and lots of lessons there, uh, and one, one of the lessons I don't know, this is— I don't know how much you want to go because we want to talk into reliability there, but like, wherever you want to go, don't worry. One of the lessons, again, you know, if you— it feels like all so very obvious because that's the thing, that's the things that people talk about startups all the time anyway. And but, you know, my favorite movie, The Matrix, right? There's a, there's a difference between—

00:27:54 - Anthony Campolo

that's my favorite movie. My man. Yes, absolutely.

00:27:58 - Glauber Costa

Yeah, not the 5th or 6th or however.

00:28:00 - Anthony Campolo

I haven't even seen the 4th. It's just gonna disappoint me.

00:28:03 - Glauber Costa

You shouldn't, by the way. Although, although I have this theory that they made it bad on purpose. But yes, that's gonna be— yeah, you know what I'm coming from.

00:28:12 - Dev Agrawal

I align with that.

00:28:14 - Glauber Costa

They have all those internal references about how the studio made us do it in the middle of the movie. So for me, it felt a little bit like they probably had to do it, but then they inserted the jokes that they had to do it in the movie because The Matrix 4 has this meta narrative in which, right, go, go watch it or not.

00:28:31 - Dev Agrawal

Again, if you don't, it's a fun watch.

00:28:34 - Glauber Costa

It's not really. I mean, super terrible, man. But, uh, any— anyhow, if you're one day bored and want to watch it. But then there's the, there's a difference between walking, knowing the path and walking the path, right? And I, and to be clear, we regret nothing. Because everything is a learning— something good came out of everything, all the things we did. But at the time, we looked at SQLite and said, hmm, we just love this thing, but this sounds ripe for a rewrite. We should rewrite it. And we decided not to do it. And again, I want to be very fair to the path that took us where we are, because I think if we had rewritten SQLite back then, it would have turned out very differently, and perhaps it would not have succeeded to the level that we did. Perhaps the company wouldn't be here. So I want to be fair to, to the path we took. But it, you know, we're here sharing stories. We wanted to do it. Like, our, our desire was to do it, and the reason we didn't do it was can boil down— I mean, I can explain the Proximal.

00:29:50 - Anthony Campolo

Just real quick, just so I understand, you're saying you want to do it, then decide not to, but then did you decide to do it later? Because I thought much later.

00:29:58 - Glauber Costa

Yeah, Thursday, today. Yeah, yeah. So today we have this rewrite of SQLite from scratch.

00:30:04 - Anthony Campolo

Yeah, but thank you. Like, the journey of how that came about. Yeah, that's really interesting to me.

00:30:09 - Glauber Costa

So we decided, we decided to do it. We, we wanted to do it And but when we, when we looked— and again, the thing is that startups are all about taking the crazy big swings.

00:30:22 - Anthony Campolo

Yeah.

00:30:23 - Glauber Costa

So from the perspective of taking the crazy big swings, we should have done it, right? Because that is the crazy big swing. You get SQLite and, and rewrite it. But yeah, I'll, I'll talk about it, but what Shreyas is asking, but What we decided to do instead was fork it. And again, it's not that we necessarily thought that forking was a bad idea. We figured we can build something cool with this. But for us, it was a compromise. Because for us, what we wanted to do was rewrite it. And the reason we didn't do it boils down to fear. We're not going to have time. It's going to take too long. And we ended up raising a lot of money. Well, not a lot, a lot. Like, for example, Convex raised a lot more, but we raised a pretty decent seed with ChiselStrike. We did spend the year with ChiselStrike. It didn't go anywhere, but because it was getting early signs of traction, we hired a bunch of people. And what we thought is like, hey, with the burn that we have, we have take that many months to do this, and, and there's no way this will work. And, and again, I remember after the boom, like in 2021, 2022, there was like a big bust. So we looked— interest rates went through the roof, so the market was complete— in complete disarray. And again, in hindsight, I should have cared about none of that. I, I should have decided this is what we want to do, this is what we believe should be done. Now let's go find a way and do it. But then again, I wasn't that person back then, so we wouldn't have done it well. But then we decided that there is a plan here that works, and the plan that works is that we fork SQLite and then we create a product based on that in months. And that's how Turso was born. So Turso was born as, again, this cloud offering based on a fork of SQLite that we called libSQL. And, and we call libSQL because first of all, the fork came first. First we forked it, and, and then we said, now that we forked it, let's think about how, how do we monetize it. And we want to go monetize this in, in months, like 1-2 months. Um, and, and then we called it— the cloud service was, was called Turso, right? So that, that's the thing. And then, and then the, the fork of SQLite was called libSQL. And what happened with that is that again, we had it— we had this desire that people would use libSQL instead of SQLite, and then they would deploy this to Turso, which was only, you know, was exclusively the cloud product. And, and again, the first problem with that is that the naming got really confusing. Like, people didn't necessarily know— some people heard of libSQL, but they did not know it was even created by Turso. So, you know, There was a lot of confusion there. And second, our cloud did pretty well. So our cloud was growing and we were growing at a good pace. Again, wouldn't be called a good pace today, but it was a good pace back then. And, but nobody was using libSQL, right, as a replacement for SQLite. This was a little bit sad for us because part of the reason why we wanted to do this is that, hey, we want to replace SQLite. And we think that there is a better— again, nothing wrong with— we do love SQLite, but in the same verge as ScyllaDB, ScyllaDB was a reimplementation of something else as well. We figured we can do better. But nobody used it as a purely local database. People would come to the Tercel Cloud as a way to deploy SQLite. And then things in the fork were technically responsible for what we could do in the cloud, like the whole virtualization thing that we did. There were many things that our cloud wouldn't be able to run without. But at the end of the day, this was still SQLite. And people would use SQLite locally and then And then nobody was using Leap SQL for a use case that didn't involve— they could even perhaps even use it as a staging area to the cloud. Those were all the use cases. And then what happened is that at some point, I mean, we kept asking ourselves, why didn't we go rewrite this? You know, there's always something. We started, we started seeing our cloud grow, but as we, we were, we were kind of cornered in, in a place of the market that we didn't like. And because a lot of people saw Turso at the time, and actually, I actually heard that from, uh, okay, if it makes you feel any good, I still have many Lipsycho instances from that time to production today. And I hope you have it for a long time. We're not going to deprecate it, by the way. We truly believe in doing this. And like, you're running your production, we should disrupt you as little as possible. Sometimes there are no choices, but it is what it is. But the thing, like, even one day, I don't— again, I don't remember which con. I think it was a Netlify conference. I met Theo, and Theo was telling me that he was recommending Turso to a lot of people, which should have made me happy. Uh, but then as, as he kept telling me why, it was all because I think Turso is a great place for you to have your initial projects, right? And, and I'm, I'm sure he, he meant no harm. On the contrary, I think— I'm sure, you know, he wanted me to be happy about it, but that's just not what I wanted to do. And it was just like, I, I don't want to create a database that is a toy just for people to get started, the database for people to do simple things. That's not what we wanted to do. And there's more than that. It's also a terrible way to build a company because you're trapped in a very low-value place.

00:37:00 - Anthony Campolo

Yeah, people use you because you're cheap and then they plan on migrating off.

00:37:03 - Glauber Costa

Yeah, you're always going to be the second option. You're always like, this is what I truly want, but then I'm gonna go use this other thing, right? So it's very hard to command pricing. Our ACV was just terrible. I mean, and it was funny because if you were looking at the metrics, everything was going well, but as we projected this into the future, it's like, how do we get out of this? And we didn't see a way out. And Shrey is just saying like the Neon of SQLite. But the problem is that there is a market, there is a gigantic market for Postgres, and there isn't a gigantic market for SQLite, right? Uh, so, and what we were trying to do with Turso, which again, but at the time a lot of people did criticize us for it and, and said that we had no idea what we were talking about and blah blah blah, because that's not the strength of SQLite. But our vision from the start, remember, even with ChiselStrike, is like transform SQLite into a deployable, like, real-life database. So this, this, this thing that people came up with was just like, oh, you guys don't understand the power of SQLite. What SQLite is really good at is the device and, and etc. But like, we had no interest in, in doing this. That's not— it's just not what we— this— that was not our thesis. That was not our passion. Like, we wanted to transform SQLite and into a real database, so to speak. And this was not happening with the fork of SQLite. And it wasn't happening because making extreme deep changes to the fork was very hard. It was very hard for a variety of reasons. It's very hard because SQLite wasn't designed to be changed in the sense that, I mean, it's— and Once more, nothing wrong with that. I, I respect SQLite deeply, but SQLite is a one-man show essentially, right? There are more people who contribute to that, but it's essentially Richard Hipp and some others. There's no large community around it. And again, even the core devs, I think they're 2 or 3 in number. So it's not an architecture that is very modular, and, you know, things are not very well Not a lot of separation of concerns. So every kind of change that you make, you're potentially changing everything. It's also written in C, which means that one mistake in the outer layer of the database corrupts memory that is used by the B-tree, and then you corrupt the data, and there you go. And the legendary SQLite test suite. Which should help us do this because, you know, SQLite has this legendary test suite. And I think now we're getting closer to the topic that inspired your invitation. It's proprietary. So we just couldn't, you know, we would make changes, but we couldn't really test the changes. We don't hate C. All my career was done writing C, but it's just not what I would reach out today for anything. It had its time. And I think knowing C, it's very— all the ABIs, everything is still very, you know, very C-based. I think understanding C is crucial for you to understand how systems work. But it has— it had its time. It's very hard to work safely with C. And again, I know because I wrote my share of memory corruption bugs and I debugged my share of memory corruption bugs. It's truly, truly hard, right?

00:40:48 - Anthony Campolo

So no jokes allowed on this podcast. What are you talking about?

00:40:52 - Glauber Costa

We don't do the thing. The thing is that tomorrow is on the news. Turso hates— have you seen Twitter?

00:40:59 - Anthony Campolo

I wanna, I wanna dial back real quick. You're talking about the proprietary test suite. That's really interesting to me because I know this is the thing that has started to happen more now, like the AI age, because people are worried about people porting their product and they think the test suite is kind of like a really important part of that. But it sounds like this was something that probably predated AI.

00:41:19 - Glauber Costa

Yeah, predated AI. SQLite always done this, this way. It's really interesting.

00:41:25 - Dev Agrawal

Yeah. Although it is funny that, that there have been some recent changes where open source projects make their test suites proprietary because they're scared of people with AI agents, fleets of AI agents rewriting their entire project somehow.

00:41:42 - Glauber Costa

And I have no idea what was their motivation to do it. Perhaps it was something similar because SQLite is a relatively simple project, right? So just, uh, again, it is not really, but in comparison it is. So perhaps they had, uh, this fear that some— somebody's got— I truly don't know, but it is, it is how it is. It is proprietary. So again, making deep changes, and, and you're always having to rebase your changes all the time because SQLite releases new versions. You have to rebase your changes on top of that. And then again, you're— so at the end of the day, our changes were not incredibly intrusive, and we knew the things that we needed to do to turn Turso, right, LibSQL, however you call it, into a really good database. But we just, we just couldn't do it. Like, we felt that our hands were tied. And, and then what happened is that one day we kind of decided— and the full sequence of events is that Becca started experimenting with a rewrite and say, how, how would that look like? And the first version of this rewrite was 90% the simulator code, 10% database code. Then we're gonna very soon we get into the meat of it, but 90% the simulator code, 10% the database code. The database code couldn't do— could only do a select query but without filters, so you could just select an entire table and you couldn't write to it. So it looks very basic, right? It's truly just a proof of concept. And again, 90% simulator code. All he wanted to answer to himself was like, I have this idea, if we were to rewrite SQLite, I think it should be done with this architecture, this way, would that work? And he did it on his GitHub. Turso never talked about it. I told him that Turso would never talk about it. So like, you wanna do it, fine. I told him like, at first, don't do it. And again, don't do it because of the distraction.

00:43:56 - Dev Agrawal

Right? Hey, let—

00:43:59 - Glauber Costa

there's the product. But, uh, he decided to do it anyway, and, and he put this on his GitHub account, and people would find it and, and share with others. And, and it got you almost 1,000— more than 1,000 GitHub stars.

00:44:14 - Anthony Campolo

Is this Limbo that you're talking about?

00:44:15 - Glauber Costa

This is called— and he named the project Limbo, right? Great. And, and so What happened is that we hired 2 people from this community because they were truly good. So think about it, there's this rewrite of SQLite on this dude's GitHub account, right? And top-notch people, people that were working at Red Hat at the time, was one of our first hires, like on the GlusterFS thing. Oh, he was working on Ceph, I think. Super high-level people start coming and contributing to it, to the point that we hire 2 of them to go work on, on Turso, which was then a cloud service. And then that's when we realized, you know what, perhaps we should just openly talk about it and, and We, we don't have the resources now to do this. It's crazy. And, and again, we are fighting for our lives in the company because again, the company was growing, but almost all the, almost all the other database companies were growing a lot faster. And, and, you know, we, we still have to focus very deeply into what we're doing, so we don't have time to do this. But perhaps if Turso talks about it It can serve as a future reference for what we're doing. The community can grow a little bit more. Maybe this catches on as an open source project on its own. And worst case scenario, this serves as a recruiting honeypot, right? Because good people are coming to this. So let's just talk about it. We also had a slow month in December. We didn't really have a lot that we released to talk about in December. So then that serves as a goodbye message to the year. And then we posted this thing in December 2024, um, the 16th of December 2024. We post this and I disappear to go celebrate the birth of our Lord. When I come back, this thing had 9,000 GitHub stars. It was blowing up. 64 contributors. This guy from Snowflake started contributing to it. Like this other dude that was in prison starts contributing to it. And I was like, how the fuck can you even do this? Like, do you have internet in prison? And this is Preston, of course, who is still today a free man, lives in San Francisco. And I was mesmerized with the amount of attention that this got. Like, it truly seemed that our thesis that the world wants a better version of SQL Lite was fundamentally right. We just, we just didn't quite get there, right, with the, with the— and, and then we had this choice ahead of us. Pekka and I were like, look, man, we wanted to do this a long time ago. The reason we didn't do it is because we thought that we had no money, no runway, not enough runway, the company's too big, and the company— and now we announced this rewrite of SQLite, which is experimental. And we say it in the post, by the way, if you read the post, that we don't know what we want to do with this. I mean, this is perhaps— like, we were very honest. We're very honest people. Like, we have no idea where this is going to go. Don't come asking for more of this. Like, we may or may not work on this. We might put some couple of people in the coming years. And the interest was tremendous. And then we said, look, why wouldn't we do this? And the reasons we wouldn't do this is because we have no time, we have no runway, we have no money, just the same reasons as before, right? And it feels to me that we are given an opportunity to, like, we regret not having the courage to do this before. And now we are given the opportunity to do the same thing, but the irony is that the circumstances are exactly the same. All the reasons that we would have not to do this are the same, right? And so, are we doing this or not? And then at that point, we decided that we had to do this, but we had to do this very carefully. So we said, look, we are going to do this, and now we're going to find the way to do it. And unlike ChiselStrike, Turso was a real company, and we had real users, and we cared deeply about our users. And more than that, that's how— that's the company that we wanted to build. I mean, well, the thing that we always wanted to build, like SQLite as a real database. And in our mind, it's like, look, we can't really pull the plug on this product because that, that's it. I mean, nobody's going to trust us ever again. Neither should they. Uh, it makes no sense to bring— take a product out of the market just to bring the same product based on a different engine a couple years later. Uh, it will completely flop. So we, we have to keep that. We have to keep our cloud, but we can't We can't keep the investment on it. So what we decided to do was that we fire half the company. And again, we reached this decision in 2 days, 'cause for Pekka and I, it was like, look, man, if we don't do this now, we'll never forgive ourselves, right? This is what we always wanted to do. We didn't have the courage to do it the first time. We are given the chance with the same circumstances. How do we make this happen? And first thing, we, again, we need to buy ourselves, uh, time to— because we understood that like the company will be very weird for investors, for anybody in, in the meantime. Our revenue wouldn't grow. Uh, I was actually very pleased that it didn't decrease either. It showed me already a first hint that Thorsell was a really good product, right? Because I imagined that once we announced to our community that, hey, we're gonna go put all, all of our efforts into rewriting SQLite I expected a lot of people would just bail and it did not happen. So that was the first thing that got us very happy. Hey, turns out we have a truly good product here. Because sometimes it's hard to appreciate how good your things are because you see all the problems and whatnot. But then we wrote this blog post. We were writing SQLite and we're going all in. That was a fast follow-up to the limbo post that they put on stream, essentially saying, we need to do this. We're going all in. It's clear that the world needs a better SQLite and we are the ones that need to do it. It has to be us. It's this thing about people want it, people are clamoring for it, and we can't not do it anymore. But unfortunately, to do this, again, we're going to have to decrease the investments to the Terso Cloud substantially. Um, the, the Turso Cloud is— and by the way, I'm saying Turso Cloud because at that point we said this is also an opportunity for us to get rid of the confusion in, in, in names. So we decided to— and Limbo was just a code name anyway. So at that, at that point, we decided to call the rewriter SQLite Turso, uh, and then the Turso Cloud, and then the cloud became the Turso Cloud. And that actually made things more confusing Because now we had Limbo, libSQL, Turso, and Turso Cloud. But we thought we can live with that because this is temporary confusion, because once Turso is ready, then there's no more confusion. And so we decided that we were okay with that confusion, and we would just explain to people, and people would say, I didn't understand any of that, are you guys crazy? As I said, perhaps. But it was a confusion that seemed to us temporary, so we decided to just bite it. Uh, and but we also didn't want to rename Thirstle— Limbo to Thirstle right away because this doesn't work, uh, and, and this will harm the brand. Uh, so we figured when we release our first alpha, at that point we're going to rename it to Thirstle, which is what we did. Uh, and now you had a rewrite of SQLite, and that took almost all of our engineer resources, uh, Just got an email from Glover about concurrent writes. Is this recording or live? Oh no, it's because I'm using Resend and, and it— this is live. But, uh, the way, the way it works is that you've, you've fired the newsletter and we have like 300 people, 300,000 people, so they don't send all 300,000 emails at once. The newsletter is sent over— it might even take many days to send all of them.

00:53:28 - Anthony Campolo

Yeah, I also just saved for another day.

00:53:31 - Dev Agrawal

Yeah, yeah, I also just received the newsletter. Here's, uh, here's my cheers for SQLite Busy. I'll, I'll never miss you.

00:53:39 - Glauber Costa

There you go, there you go. But that's, but that's just because it's going to be sent. I mean, my, my personal accounts received it right after I sent it, but, uh, some people are going to receive it tomorrow. Uh, it's just the way, because they have to send it like slowly. Email kind of sucks, man. Like, email kind of sucks. The infrastructure, like, why can't you just send 300,000 emails in 5 minutes? Like, you should, you should, right? It's just the way the whole thing is designed at the infrastructure level. There's nothing to do with the senders. It's truly like the email infrastructure and the clients. But anyway, that's not a problem that I'm in the position to solve. So as I was saying, like, and at the point we said, hey, we're gonna go, we're gonna go all in on rewriting this. Once we release the first alpha, we're gonna rename this to Turso, uh, and then we're just gonna have this database that is like SQLite but better and gets deployed to the cloud. That's it. But people kept accusing us Because that's what the internet does best, right? What, what the database—

00:54:54 - Anthony Campolo

database people get really riled up, I've noticed.

00:54:58 - Glauber Costa

I don't, man. I, I, I'm— I try to be the chillest person.

00:55:02 - Anthony Campolo

Yeah, you're the chill database CEO. That was what I actually wanted to talk about at some point, is why database CEOs are so aggro.

00:55:08 - Dev Agrawal

Yeah, because no one else thinks about the Roman Empire.

00:55:11 - Glauber Costa

Yeah, I, I don't know, and I, I can't speak for them, but I, I truly I really don't like it. I think it's just very sad that people fight that much. Anyway, that actually makes me so sad that I don't want to talk about it. But the accusation— I mean, the wildest accusation that I received that I'll never forget in the process of rewriting Sicko Light is that I was trying to erase Jesus.

00:55:48 - Dev Agrawal

What?

00:55:48 - Glauber Costa

I don't know.

00:55:51 - Anthony Campolo

If there was someone who was Orthodox, it was really bad because you're a Catholic.

00:55:56 - Glauber Costa

It's, it's, um, I, I can't, I can't even understand why. I, I, my best guess is, is because that, you know, Sicily had, had the code of conduct for— and, and at the time I was speaking not criticizing, but just saying we don't do it like this, and, and this is why we don't do it like this. And we don't believe— we believe a technical community should have a technical direction, uh, but they do whatever they want. And in fact, I said in the tweet, live your life like that, you're gonna, you're gonna have a great life. But it's just not how we believe a community should, should be oriented, right? Because it, you know, pack It's just not the way. And then, and then at that point, I was accused of trying to erase Jesus and etc. And I was like, what are you talking about? But it's funny. The internet is a funny place, man. It's so funny.

00:57:01 - Anthony Campolo

Yeah. So we got someone in the chat here bringing us to our topic that we're going to talk about anyway. PG. Yeah. Let's talk about PG Micro. What is PG Micro?

00:57:10 - Glauber Costa

Yeah, so PG Micro was my turn to do what Pecker did. And, and I— let me just talk about the accusation first, because the, the one that I was getting to is that the, the other more, more technical accusation, uh, that we received is that we had no idea why people liked SQLite. It was not about the features. And it was all because SQLite was very reliable and battle-tested, and SQLite never breaks, and SQLite is perfect. And by rewriting SQLite, we were obviously making it worse because you would have to write 10 years of tests to get to the level of reliability that SQLite has. Therefore, we would fail. And, and, and then this other person said that he never met— he never had any other company that he rooted more for the company to fail. Uh, super fun. It's always spikes my cortisol to hear it. But we, we kind of, we kind of did. It was not Lunduk, but that we, we kind of did. Understand that pretty well. Because again, what those people were missing is that we were very, very well aware of the fact that reliability was the linchpin of SQLite and that reliability was the thing that people liked the most about SQLite. Because guess what? Reliability was the thing that we liked the most about SQLite. And when we decided— and in fact, this is exactly what I said in the beginning. For those of you who are joining now, it, it might be— it might have been for the best that we didn't rewrite SQLite right away because we learned so much in this period. And one of the, one of the reasons we didn't start with the rewrite of SQLite is— was exactly this. I said we, we can't really— it's going to be truly hard to meet the reliability standards of SQLite. But when Pekka came up with this experiment that became the Limbo project that later became Turso. That was exactly what he was trying to do. Can we do it? I mean, is there a way to write the software that would lead to that level of reliability, if not more, without paying the penalty or the capital investment of writing tests for 10 years? Because the SQLite test suite is reported or rumored to have 10 times more lines of code than SQLite itself. It's truly like a lot of work. And we wrote it, we wrote it with deterministic simulation testing in mind. And deterministic simulation testing is truly interesting because it's really hard to plug into existing systems. And we were— so you kind of have to write your system with that in mind. Because, for example, everything that every— for deterministic simulation testing, at the end of the day, is just a way for you to generate tests automatically. So what you do is that instead of writing tests, like imagine a unit test, unit testing codes the the behavior of, of, of the output. What is the output given, uh, at 3 to 6 times, not 10? Don't— again, don't trust my numbers. Uh, it— first of all, for me it's a rumor. So, and, and second, I'm terrible with, uh, with exact quantities. So if Hip says 3 to 6, of course he's right. But, uh, deterministic simulation testing, when you write unit tests You encode, those are the inputs, those are the outputs that I expect out of those inputs, and then you run it. But the problem is, what is the output that you expect out of all the other possible inputs? And there's so many more possible inputs that you could put in your tests. So what deterministic simulation testing does is that The first step, and again, I'm simplifying this quite a bit, but the first thing that you do is that when you're writing tests, you don't write unit tests anymore. And again, it's not that you never write unit tests because we test by layers. So we do have unit tests, but you encode properties of the system, right? So examples of a property, if you write, if you at the end of a battery of writes or mutations, let's call them mutations, writes, deletes, updates. At the end of a sequence of mutations, if you run an integrity test in the database, the integrity test has to pass. So what you are encoding with this property is that you're not going to corrupt the database. And it's funny because in the beginning, because we didn't trust our code for integrity test, we would run SQLite's integrity test because the file format is the same. And so we would just— you close the— we can't do this anymore all the time because we have features that SQLite doesn't support. But now we trust. But the scaffolding of this is that run a sequence of things and then run the integrity test with SQLite. And if the integrity test doesn't pass, that means that a property was violated. Other examples of properties are things like if you query something using an index versus not using an index, you have to have the same response. Because if you have an index or you don't have an index and the query differs, one of them is wrong. Now you might not know which, but one of them is wrong. Or if you write some— if the database committed, a write, and if you read that after, you have to write what the database was committed. So the asset properties. So you essentially encode properties. And then instead of writing tests, you write test generators, right, that fuzz the space of possibilities. And at this point, this could be just like fuzzing plus property testing. But then the cool part of that is that you replace every single operation that happens in the database that could have a non-deterministic behavior with, with, with, with a deterministic version of it.

01:04:04 - Anthony Campolo

Yeah, I'm gonna pop it up in here real quick because I'm a total noob to this stuff. So this was the Google AI overview that it gave, and you can let me know if this makes any sense or not. Um, deterministic simulation testing is a software testing method that runs code inside a fully controlled simulated environment by controlling non-deterministic variables like clocks, randomness, and thread scheduling using random seeds for exact reproducibility and pairing with fault injection.

01:04:32 - Glauber Costa

Yeah, so for example, our, our simulator doesn't use threads because every time you use threads, you may have scheduling issues like this thread in this one run. It pretty much, I mean, at some level is the way it works in Tursil is that you have every I/O goes through a virtualized I/O. And then when the I/O runs for real, it runs against the system. But when the I/O runs in the simulator, the simulator replaces that I/O with with the simulator's view of the I/O of the system. And it's pretty hard to use dependencies because you can use a dependency that itself calls a thread. So we don't have a no-dependency policy, but we do have to scan the dependencies that we use and be very careful with them and audit what they're doing. And for example, it's enough for the dependency to take a clock measure for the system to become non-deterministic already. Because if you read a clock through the I/O interface, if you don't, now your system is deterministic. So we have to be very careful with the dependencies that we pull. We do pull some that we trust. And then you write a simulator. So I mean, our— we don't have like 3 to 6 times more lines of code than Trussell. But we do have this simulator, and what the simulator will try to do— do you not— no, we do use the standard library, but again, there's enough— there is— allocations are deterministic in a way from the point of view of this. But it's— we do fault injection, so we do fault injection in which we simulate out-of-memory conditions. But you can just replace the allocator to do this. You don't have to have a policy and you don't use the Rust standard library. So, Vercel runs in the simulator and then we're always creating new tests randomly. And that's the beauty of it. When a test fails, then you have— because everything is deterministic, The system is behaving as if, hey, give— what time is it? And give me a timestamp, and you get a timestamp. But if you run the simulator with the same seed again, that timestamp will 100% of the time always be the exact same. All interleaving of all events will always be the exact same. And then what happens is that if you find a bug, it is very easy to reproduce the bug. So Deterministic simulation testing is a double whammy because first of all, it finds all the freaking bugs. And second, it— and again, it's not magic, so there are some bugs that it doesn't find. But then in our experience, the reason it doesn't find the bug is that the space of possibilities wasn't explored enough. And then you make a tweak to the simulator, then now it is. And like, for example, we had a We had 2 interesting scenarios in which we didn't find classes of bugs. One of them is that the simulator— it was very simple. The simulator just wasn't creating indexes. So bugs related to indexes were not caught. And then we just taught the simulator to also create indexes randomly. And now we found a bunch of other bugs. And the one that was amusing the most And that's actually interesting because that one led us to rethink our testing strategy and evolve our testing strategy to be what it is today. That was a bug that only happened in databases that were larger than a gigabyte. And we wrote a blog post about it. I'm not going to go into details about it. It was a very fun discovery. And every time the database crossed a gigabyte, if you will run the Turso, integrity check, it will pass. If you run the SQLite integrity check, it would fail. And we had no idea why. We spent a couple of days trying to figure out why. And then when we looked at the database, we could query just fine. The data was all there. It didn't look corrupted. But SQLite would report it as corrupted. And this happened because there's a special byte that SQLite, every time the database crosses a gigabyte, SQLite adds a special byte to the next page that comes after the first gigabyte, and Turso did not. So by not— but, but not finding the byte, the structure of the file was same, but that one page at the gigabyte marker didn't have that special 1 gigabyte marker, which we didn't really know was a thing. And then SQLite would fail the integrity check. And, and the reason we had that is that we were so aggressive by injecting faults that none of our databases actually got to a gigabyte because they would fail before that. Like, the, the failure— the simulator would inject failures, uh, and, and stop the— so you, you can— you would, you would, you would start adding data, adding data, adding data, and then the simulator comes and crashes you and like simulates a crash. Uh, so in none of our runs that it run— it ran for long enough to, to get to that point. So we started, we started having specialized versions of our test suite. So for example, there are entire runs in which we just check properties without any fault injection because that allows the database to run longer. And then there's a separate run that adds fault injection. And then later we found out that the simulator was pretty hard to find issues with concurrent access. Uh, so, and, and, and then we wrote another simulator that only stresses concurrent access, right? So today, Turso is tested with the Turso deterministic simulation testing. Turso is tested with a concurrent tester. Uh, Turso is tested, uh, with differentiate— differential testing. In which we randomly generate queries, compare the result that SQLite and Turso will give, and see if they're the same. We also use Antithesis, which is a very interesting tool, which is a deterministic hypervisor. So in the deterministic hypervisor, you can— because it sits at the layer below, you can put any software there, and they're a lot more aggressive in finding issues. So whatever bugs Scaper Simulator, Antithesis usually finds. And then we have Carnage Mode, which is a special run of Antithesis that runs for 2 days in a row during the weekend. So every weekend we run Carnage Mode, and that is like trying to obliterate the database to oblivion. And it finds a bunch of bugs. And now we are adding formal methods to the test suite. Right, so, uh, and by the way, uh, that there is this guy, we actually brought him in as an intern. He came from an open source community, lives in Bangalore, and using formal methods, he found 10 bugs in SQLite itself. So what, what he did is that he, he used formal methods to verify Turso, but then just so he could figure out if the model was correct or not. He said, let me run against SQLite as a test run. And then the model disagreed with SQLite. I mean, and, and as we started investigating, those bugs were reported to SQLite and, and fixed. So there were— that was, uh, Pavan Nambi. He, he's not on Exalate, but he's on our GitHub community a lot. Um, so again, we test Tursil, man. We— this, this thing about like, we— those guys are not going to build a reliable system because only SQLite can be a reliable system. It's just not true. Uh, you— it is correct that we're not going to have the time and the capital, uh, to invest in, in a 10-year run, uh, of writing 6 times more lines of code than Turso. But the, the, the truth of the matter is that the testing and the reliability community evolved tremendously in the past 10 years, uh, and there are a lot of those techniques that we can use, and we do use them. So again, formal methods and Antithesis, Carnage Mode.

01:13:11 - Anthony Campolo

Could you talk a little about what is Antithesis? It sounds like it's a company, but it's also like a testing method. So just go into that.

01:13:19 - Glauber Costa

Antithesis is a company, very expensive product, worth it. But Antithesis is essentially deterministic simulation testing on steroids, if we can. I'll put it in very simple terms. And because they sit underneath the database, underneath the system, the hypervisor itself, which is the thing that schedules virtual machines, is deterministic. So everything you do in that system is deterministic, and they inject a Tremendous amount of faults. They can find bugs that our simulator cannot find. They can find bugs in our simulator. They can find bugs in like—

01:14:10 - Anthony Campolo

this brings me back actually. I see a blog post right here, Finding Bugs in Raft Implementations.

01:14:16 - Glauber Costa

Yes, they find bugs. If the bug is there, they'll find it. And the thing is that an antithesis can be tuned. You run for longer and then it's way more expensive. It uses a lot more resources, but it finds more bugs, or you can run just for a couple of hours and then you find some bugs, but you don't find a lot of bugs. Because the more it runs, the more it stresses things and the more it explores. So our carnage mode is essentially a weekend run of Antithesis with all faults that we can possibly inject. And it finds a lot of bugs. It finds a lot of bugs. And in fact, again, it also found the bug in SQLite. We were running something and I posted about it on X the other day about how our carnage mode found a bug in SQLite. That's essentially that. So we can— we get a rough estimate of how much it costs. Hundreds of thousands of dollars for the most basic things. It's quite expensive. Totally worth it. My best vendor. Because they find all the bugs, man. Now, the reason we started adding formal methods is that Exactly, because synthesis is very expensive. So, uh, formal methods are much harder to do, but they catch the bugs a lot— they catch the bugs a lot earlier because you don't need to run anything in, in formal methods. It's a mathematical thing, right?

01:15:43 - Anthony Campolo

Yeah, I was gonna say, we haven't really defined formal methods, but it's basically like using a math proof to try and find a bug, essentially.

01:15:50 - Glauber Costa

Essentially, right?

01:15:51 - Anthony Campolo

So, so if the bug is found at that level Do you have a guitar under you?

01:15:57 - Glauber Costa

A guitar? No, this is just my microphone because I was just sipping some tea. So formal methods are amazing. And if a bug is found at that level, you just— it's much cheaper. It's just way cheaper than letting Antithesis find it, right? So we want— and again, we also do integration tests, we also do unit tests. We do all sorts of things, and, and they all serve a purpose in the Turso testing world. But man, if— again, everything has bugs. SQLite has bugs. Turso has bugs. Everything has bugs. But we are very confident in, in our ability to find them. And, uh, you know, between— how does one define a formal method? I'm not familiar So formal methods essentially have a mathematical model of your software, and then essentially you can prove mathematically that it either works or it doesn't.

01:16:57 - Anthony Campolo

Yeah, I'll go back to my, my AI overview. Formal methods are mathematically rigorous techniques used to specify, develop, and verify software and hardware systems. You have formal verification, formal specification, and automated synthesis.

01:17:12 - Glauber Costa

That's pretty much what it is. Yeah, we also have a comp— we also have a partner that helps us with formal methods called, uh, aretta.ai. So R-A-R-E-T-T-A.ai. Uh, and what their company's formal methods are incredibly hard to do, uh, very bespoke, because again, it's essentially a mathematic— there's—

01:17:35 - Anthony Campolo

there—

01:17:35 - Glauber Costa

it's very hard to generalize them because it's essentially a proof of that system. So Aretha is trying to use AI to make formal methods a little bit more approachable, and it's been, it's been working fairly well for us.

01:17:51 - Anthony Campolo

I really like your guys' blog post, by the way. You write a lot of really great technical blog posts, so big ups for that.

01:17:59 - Glauber Costa

It's just, uh, you know, it's what we're doing, and then, and then we post about it. But look, everything we do in Turso is done with the idea that we, we want to build the most reliable database that we can. Uh, we consider, we consider reliability a feature of SQLite, and if we want to reimplement all features of SQLite, it's not done if that feature is not there. Right, so we truly see— again, we truly see reliability as a feature. We truly see it as a must-have for, for every system, and we take it incredibly seriously. Incredibly serious. I think, I mean, I think the fact that our testing methodology found so many bugs in SQLite already—

01:18:52 - Anthony Campolo

and then were you able to actually upstream those, those bug reports in a meaningful way?

01:18:56 - Glauber Costa

The bug reports Yes, but again, it's very hard to get your code into SQLite. We don't even try. Yeah, uh, but we reported those bugs, and to the best of my knowledge, some if not all were fixed fairly fast. And so again, it's a— Richard Hipp is a tremendous, monstrous individual.

01:19:17 - Anthony Campolo

Yeah, and a very, very interesting fellow. Dev, what do you know about all this type of testing? Have you ever used any of this stuff?

01:19:24 - Dev Agrawal

Not much. So I, I have been recently exploring more like different kinds of, uh, testing because of my project Specter, which is essentially, um, a way to define some, some sort of structured specifications up front and have AI generate like the implementation and like kind of have, have some deterministic verification of the AI-generated code. So, I have been looking a lot at, like, what are some different levels of kind of testing that you can perform. And one of them that I kind of ran into was something similar to what— actually, a couple of them. So, one was kind of trying to take some test scenarios and generate a lot more test scenarios. To kind of explore a wide variety of kind of like inputs and outputs and like basically problem spaces that you might not have declared upfront. Another one was like Gaurav was mentioning, like what if every single dependency of the code that's being tested was somehow mocked or you gave it— so, you run it twice, it'll give you the same result because you gave it the same seed. So, um, and then like a lot of other ways. So yeah, it seems like there's like a bunch of different layers. It's kind of like a cake. Like, I would always— go ahead.

01:20:57 - Anthony Campolo

I was gonna say, I just want to step back and ask a super duper noob question. So when it comes to these type of testing methods, are you still testing like happy paths versus edge cases?

01:21:06 - Glauber Costa

We do, because do and the reason we do it is exactly because every single one of those testing methodologies have a completely different, uh, cost-performance curve. So when, when, when you have— like, again, we do use a lot of AI to code. We still review all of the code, and we don't believe we should stop, uh, perhaps one day. But, uh, I think, again, and this is not a general statement because people love to make general statements Right? But, but the, the reality is that every class of problems is very different, and, and you have to make the choice. Like, I don't test, uh, our dashboard, man, the same way we test. And, and we, we use AI. We don't read necessarily— we read through it, we skim. And, and remember, I don't actually have a front-end developer because we, as I said, we kind of got rid of them Uh, when, when we started. And we did not— we— this is the time in which we kind of had to hire them back, and soon we will. But we kind of say, hey, look, I mean, we're going to start making improvements to our dashboard because it was really quite bad. And this was just the price that we're okay paying for allowing ourselves the time and, and, and the runway to rewrite SQLite. But, you know, what business do I have reading React code, I'm not going to understand it anyway. And like, what's the worst that can happen?

01:22:39 - Anthony Campolo

Famous last words.

01:22:40 - Glauber Costa

Yeah, you know, so it does happen that sometimes I push some code and then like it's wrong and somebody complains that it doesn't load on mobile, you know, but you know, it's very different than HR data in a database.

01:22:55 - Dev Agrawal

So you might have a useEffect DDoS your own service. But then again, it happened to Cloudflare when people were still reading the code.

01:23:02 - Glauber Costa

So not to discount it, right? But different domains have different requirements. And then we use a lot of AI to write versions of the code, or again, we read it. And the way we've been doing is that we, most of us, we go in a loop in which we're still fairly in the loop. We write code and then we read the code and then we tell the agent to change this, change that, rewrite this, refactor that. Uh, feels like reading patches in the Linux kernel, to be honest. Uh, yeah, I mean, the Christmas theme is the clear example of the kind of stuff that we don't have the bandwidth to do while we were writing SQLite. It was super cool though. I mean, I, I agree with you. Maybe you should try to see if Claude Code can come up with a Christmas theme. But that was actually—

01:23:49 - Anthony Campolo

I was just about to ask that, whether you guys are Claude Code, Codex, what do kind of models you're all using?

01:23:53 - Glauber Costa

I, I use Claude Code, but, uh, we don't, we don't mandate people use— I, I actually use both, but I like to use Claude Code and Codex, and then one reviews the code that the other wrote.

01:24:03 - Anthony Campolo

I do that too.

01:24:04 - Glauber Costa

Yeah, yeah, yeah. So this is a technique that seems to, to work pretty well, although a lot of this is quite frankly just black magic. I mean, nobody knows if it, if it really works or not, but it feels like it works, so we'll do it. Uh, and you don't run Carnage Mode in the loop of the AI agent because Carnage Mode takes a weekend, right? So it's, uh, uh, you run the unit tests and, and the even like antithesis for antithesis for us is a daily thing. Uh, we run antithesis every night for a certain amount of time. You don't put this in. So every, everything like, uh, formal methods are completely— it's a completely different thing Like it's its own thing. And we send it to the system and sometimes something pops back. I mean, it's a completely asynchronous thing from our point of view. Antithesis Carnage Mode and our DST are, you know, the DST is a lot faster. So Antithesis Carnage Mode, there are like post-commit things. We'll run it every day or every weekend. The DST we run when we are about to commit. So we're about to commit something, like we've been developing for a day or two. Now it's time to run the simulator, and then it will find bugs. But during that loop with the agent, right, the best thing is happy paths and adversarial examples. Because those are very easy. To run, right? And you use the agent itself to generate them. So it's fine. And the big problem with that is that you end up having to have a lot of tests to cover a meaningful part of the space. But again, this is not a problem anymore because it's so cheap to generate them. So we do have a lot of unit tests and a lot of integration tests, and they're there forever, right? Once the feature is done, they're always going to be there. Um, but, but they serve it at— they're, they're still the best guardrails for the agent because they're very fast to run, right?

01:26:16 - Anthony Campolo

And then did you answer this question, um, how do you categorize severity of each issue?

01:26:21 - Glauber Costa

Oh, they're all severe. If, if there is a bug, like databases shouldn't have bugs.

01:26:25 - Anthony Campolo

Yeah, I was trying to find some information about Carnage Mode. I couldn't find it.

01:26:30 - Glauber Costa

So no, this is our own nomenclature.

01:26:32 - Dev Agrawal

It's like an internal thing.

01:26:33 - Glauber Costa

Yeah, Carnage Mode is just like saying, like, now I have the whole weekend, man, we can unleash all resources on that, then let's bring mayhem. Uh, it's not, it's not established.

01:26:43 - Anthony Campolo

Search it now, they're gonna find this stream.

01:26:45 - Dev Agrawal

So I think maybe, uh, Netflix Chaos Monkey might be like a similar-ish, uh, reference point.

01:26:53 - Glauber Costa

Similar, but the thing about the Chaos Monkey is that it's always running. Like, for us, Carnage Mode mode is a flip. Like, now we're gonna go carnage mode, like run the, run the whole weekend and find a bunch of stuff.

01:27:03 - Dev Agrawal

Yeah, I think to put in, uh, to put the testing stuff in perspective, there's like the test pyramid that most people are familiar with where like you have unit tests at the bottom where you, you, you're supposed to have like the most number of unit tests. They're the cheapest, fastest to run, and they run, they test like individual things. Then you have integration tests, and then you have the end-to-end tests. So those are like the 3 layers with, like you were saying before, different cost to performance kind of ratio. Like they're testing different things. You still want all of them. They all kind of catch different things.

01:27:35 - Glauber Costa

And because of that, you kind of have to run them at different parts of the process.

01:27:39 - Dev Agrawal

Right. That's the whole thing.

01:27:40 - Glauber Costa

But we never release a version of Turso that do not pass the whole thing, right?

01:27:45 - Dev Agrawal

Of course. Yeah. Yeah. So I think what I was saying earlier is that I'd like to see some a sort of a diagram like that for all the kind of kinds of testing that you guys do. I think you probably have a blog post where you describe— or you probably have—

01:27:58 - Glauber Costa

we don't. I think, uh, somebody on X the other day said that we should write a blog post with, with the whole thing because we've written about all this but scattered around, right? So here, here's how we do this, here's how we do that. And, uh, it's not, it's not a bad idea, it's just that I haven't had the time to, to, to do it.

01:28:15 - Anthony Campolo

So what's the compute cost?

01:28:16 - Glauber Costa

No, it is It's not that much, but it's fairly expensive. It's just, it's not, it's not $100K compute per week. What I said, it was a couple of hundred thousands and something. So just one hundred thousands, many hundred thousands, but that is our yearly spend with, with synthesis.

01:28:34 - Anthony Campolo

How many RPS does Turso make as of now?

01:28:37 - Glauber Costa

Requests per second? I don't know. It depends. But I, I do, I do know that this person, uh, asked for LPG Micro and we went into the testing. So let me, let me talk. Yeah, let me talk about that now. And, and the, the one day I am, uh, talking to the CTO of a relatively big company and he's been asking me why, why can't we just do what you guys are doing for Postgres. And then he even say, I vibe-coded, uh, for— let me just pause here because with so many issues, how do you have the bandwidth to make the number of decisions on how to handle different carnage mode issues? Again, there's not that many issues. That's also, that's also the thing, right? So just after, after all of this, uh, there's not that many issues. So, so it, it's fair because I, I said this thing, which is a great soundbite, uh, we don't— everything's a priority because databases should have no bugs, which is true. But, you know, by the time we get to that weekly run, we caught a lot already, right? So, so, so the goal—

01:29:52 - Dev Agrawal

it's a similar thing with the testing pyramid. Like, if most of your bugs are caught in the end-to-end phase, and that probably means you don't have good unit tests.

01:30:00 - Glauber Costa

Exactly. Yeah. Uh, and, and so It does find issues, right? Uh, so, so it's not that, that, that it never finds anything. I mean, and closer, closer to the release dates, it finds hopefully fewer and fewer, but not always. Sometimes you put some stuff at the late in the game and then, and then it finds a bunch of issues. But, uh, it's not that many. Uh, it's not, it's not that many. But, but, but when we, when we do find them, it does mean that it escaped all layers. So it's probably— we don't really prioritize what we see there, just whatever needs to be fixed. On the other hand, it is very rare. Again, it does happen because remember, none of this is magic, right? You have to write the simulators, you have to write the test generators, you have— it does happen, but it's usually not the case that there is a bug that this testing methodology finds that is in a released version of Turso. So it's usually in the next release. So on the other, you know, we also have the ability to just say if it takes too long to fix them, we just delay the release, right? Prioritization works when you have a deadline and say, hey, we, we have to release by, by that date. Um, I can give an example of a unicorn company I work for Or just an example, they sent throw exceptions for their JavaScript app to Claude and return the summary to the user. Interesting uses of AI. It's a, you know, you have to get creative in the age of AI. I love saying the phrase the age of AI. It makes me sound so visionary. But anyway, just to, we are, I'm chatting with this guy and he He tells me, I vibe-coded a translation layer that translates Postgres queries to SQLite queries, executes them on SQLite.

01:32:05 - Anthony Campolo

Boom.

01:32:06 - Glauber Costa

Now I have Postgres on SQLite. I said, no, you don't. And the reason you don't— and I started explaining to him— the reason you don't is that there's a tremendous impedance mismatch between SQLite and Postgres. Uh, you're gonna try to translate this to Postgres, uh, you know, even to Trussell, like, how do you translate the Postgres types? What you're going to try to do is you have to compress the Postgres types into SQLite types and back. You're going to have all sorts of, uh, edge cases with that. SQLite will not support materialized views. So how do you do that, right? And so you can fool yourself thinking that this worked, but it truly didn't work. But then as I started thinking about it, I had an idea. I said, hold on, I think I know a way to make it work. And the realization that underpins this— and then I ran an experiment, and then I ran an experiment, and the experiment went did quite well. And I'll tell you in details because I think your audience is going to appreciate that, how exactly the experiment was. But my experiment was this, instead of doing this translation layer and translating queries, like get a— yeah, so we'll get to Shreya's question, but instead of getting the queries, Postgres query translate to a SQLite query. I have a better idea. What I wanted to do was like, can I translate the Postgres queries into the AST, which is the intermediate representation, like, like an abstract syntax tree? So every, every language of any kind gets parsed to AST, be it JavaScript or Postgres or SQLite. And once I have the AST, it doesn't truly matter if it's Postgres or SQLite anymore. I mean, everything that was special about Postgres is lost at the AST layer. The AST is really truly an intermediate representation. The AST has things like join, table A, table B, select, like, as a tree. And a lot of people do not know, a lot of people do not know that SQLite is a very special database. I did not know that, just so you understand. I'm not saying, oh, it's very widespread that some people may not know. Like, when we started working with SQLite, like the ChiselStrike days. I did not know that. But SQLite is a VM. SQLite is a bytecode VM. So SQLite, what it does is that it has the SQLite dialect and it compiles the SQLite dialect to a VM like the JVM or .NET or JavaScript V8, right? Just, uh, so all SQLite is doing is compiling the language to an AST and then running, and then you have another step that compiles that AST to this VM, and then it executes the code in the VM. And in V8, you're going to find things like load an integer, copy a string, things like that. And in the SQLite VM, you're going to find things like open a B-tree, transverse a B-tree, create an index, but still a bytecode language. At the end of the day, it's just that. My thinking with pgMicro was, can I compile the Postgres language to the SQLite bytecode? If I find a mismatch, can I then improve the bytecode and at that point do the translation. Because if I can do that, then there's no impedance mismatch. I, you know, if there is anything that the SQLite bytecode doesn't do, I'll just expand it. And the first— so I run a prototype, we say let me do like the most basic stuff, and the first big thing was exactly types. So again, SQLite doesn't have types. SQLite is just, as they call it, the JavaScript of databases. You create a table and you say this column is an integer, and then you write the string Anthony. Congratulations, Anthony is now an integer. And, you know, it is the way it works. So what I did is that I added types to Turso. So Trussell has now experimental support to types. And if you go look at the date that I added that, that was the date that we kind of decided if this works, we're going to go write Postgres. Now it takes time, right? Because— and the reason this is not incredibly crazy is that we're not starting from scratch. We're starting from a working database, a database that's tested to this is like crazy and already has, you know, it can already do select, inserts, updates, indexes, all of that. And all I'm doing is, is translating this to the bytecode. And then the hard part is when you find the impedance, when you find the stuff that SQLite doesn't do, then, then we have to extend. Uh, so I— we added experimental support to, to Turso. Uh, it is not, uh, production ready yet. It will be. But the goal that we had was exactly like, if this works, we can rewrite Postgres using this. And this is incredibly powerful. And this was the pgmicro experiment. Again, I think it worked tremendously well. And then we merged it to Turso. So now pgmicro is not a thing anymore. All of that code was merged to Turso. So now Turso supports Postgres. Now, it supports Postgres as well as we supported SQLite a year ago.

01:38:33 - Anthony Campolo

Here's the historical—

01:38:37 - Glauber Costa

I have to go update the README, by the way. I keep forgetting, but I have to go update the README and say, hey, this is now part of Turso in case people find it. But we support Postgres the same way we supported SQLite a year ago, which means we support like 30% of it, 40% perhaps, against Postgres is much larger than SQLite. But our idea is that, look, if we can do this, first of all, we can have a much more modern version of SQLite that doesn't have like process per connection. We can really re-architect the thing. We can have a version of Postgres that runs natively in the browser. We can have a version of Postgres that runs embedded on devices and runs it as a file, that runs it in memory, that does— because the same way I'm expanding SQLite with the good things of Postgres as I do this, it goes in, in, in the other direction too. Uh, and Turso, the cloud, the thing that started defining us, especially with AI, is that it is truly the best. We, we have customers on, on the Turso Cloud today, they're, they're deploying more than a million databases, like a single customer. And with agents, it, it, it became a thing. Like, we had before, if you've talked about like, hey, somebody deployed a million databases, you go But why? Now it's because you have those agents coming online all the time and the agent decides to create a database, they create a database. There's nothing that prevents them from doing this. With agents, the idea of a database per agent became very attractive and the idea is by supporting Postgres, we're going to be able to offer a version of Postgres that you can deploy by the billions as well. That's essentially in a nutshell what that is. Do they have enough UUIDs? They do. Like, you need more than a couple million to exhaust the number of UUIDs. But again, the goal there is that everything that we're going to do is going to inherit the reliability of Turso. So we want to write this in the open like we did Turso. So everybody's welcome to come and contribute. We have almost 300 contributors at the moment, and we were hoping to get even more. But the goal is that you would have a database. By the way, we just don't talk about it, but somebody put MySQL support already because it's not that hard. MySQL is actually simpler than Postgres. It's somewhere in the middle. Why not? As a company, we're not going to invest in it, but if somebody comes in the open source community and wants to improve MySQL support, by all means. Redis was floated as well. I think that Redis is a great thing for us to do. Uh, and, and there are a couple of bytecode instructions we will have to add to support Redis, but it's 2 or 3. And now the, the most interesting part: how do you know if, if a VM, uh, is powerful enough and general purpose enough to support any workload? And the way you prove this is you run Doom. So I ported Doom to Turso. And because Turso runs in the browser, you have a Turso database embedded in the browser running the game Doom. And essentially the render loop of the game is a select query. So every— there is a select query that yields. So the select query, it's a select query that generates infinite amount of rows. And every time it yields, That's a frame of the game, uh, and, and then you keep playing the game, right? Uh, and do you have that on GitHub? I do. And if we can do screen sharing, I think Anthony should go play Doom. Uh, and, and the goal there is that like devs never played it. The memory, the memory, uh, and no bytecode instruction, no extra, no extra bytecode instructions were needed to use to play the game in Turso. Like, because we, we did use some instructions that SQLite doesn't have.

01:42:33 - Anthony Campolo

We got that, we got the black.

01:42:34 - Glauber Costa

There you go.

01:42:35 - Anthony Campolo

Here, how do I get to the actual game?

01:42:38 - Glauber Costa

It's right there.

01:42:42 - Anthony Campolo

Oh, there.

01:42:42 - Glauber Costa

Okay, you press Esc and Enter to start, arrows to move, Control to fire, Space to use an item. Esc, you select, you select the, the window and then Esc and Enter.

01:42:54 - Anthony Campolo

Oh, Escape then Enter.

01:42:55 - Glauber Costa

Yeah, Escape. Yeah.

01:43:00 - Anthony Campolo

Okay, I think I'm I think I'm in there now, so enter to start. Which episode? We'll go easy mode now. Well, there we go.

01:43:13 - Glauber Costa

So this is every frame is a select query and then we filter the results of the select query, get the, get the input that you did.

01:43:21 - Anthony Campolo

There we go. All right, let's fucking go. Wow, this is dope.

01:43:27 - Glauber Costa

Yeah, and then I wrote a C2 SQLite VDB compiler. VDB is the bytecode instruction, uh, and there's—

01:43:34 - Anthony Campolo

so remember where the armor is.

01:43:36 - Dev Agrawal

Yeah. So is every interaction like an update query?

01:43:42 - Glauber Costa

Yeah, and everything, it updates a blob. So the memory is a blob in, in a table, uh, and, and it's like chunks of 64 bytes, 64 kilobytes, and then you chunk it and then you have those blobs. This is running in, in database code. Yeah, again, the render is a select query.

01:44:02 - Anthony Campolo

You can feel there's a slight amount of lag, but it feels pretty responsive. Yeah, for the most part, like I'm able to play it like it's a real game, which is just absolutely incredible.

01:44:13 - Dev Agrawal

If someone on YouTube is asking, wait, you're actually playing a game? And yes, this is Doom running in Terso, running inside the browser. Which is— it's running in Wasm?

01:44:26 - Glauber Costa

Yes, uh, Turso compiles natively to Wasm. And then, oh man, your ammo's doing— and by the way, that's what I write in the blog. If you want to keep playing, it's— it's this— you're just reading a technical blog and, and studying. This is work time for you, okay? It's all good.

01:44:45 - Anthony Campolo

That's true. Yeah, that's really cool.

01:44:48 - Dev Agrawal

We're on the clock. This is doing research for the company. That's amazing. Okay, so basically, if you can run Boom, you can run Postgres.

01:44:59 - Glauber Costa

That's, that's what we say, right?

01:45:01 - Anthony Campolo

Actually playing, right?

01:45:03 - Dev Agrawal

So, so, so the database, it's the same database that, that people, that your customers have been using for a while, but that same database can now accept queries that are in Postgres dialect, because in between you have a compiler layer that can compile Postgres dialect queries into an AST that Turso understands, which is also the same thing that it's doing.

01:45:30 - Glauber Costa

That's a parser really, right? It's not the entire compiler, but like that's what Postgres would do. Like that's what any database would do. Like the database gets the query, compiles it to AST, and then do something with that. And the only difference is that we just compile Postgres to the same AST that we're already using for SQLite. So when— and at that point, the AST is already— the AST to bytecode is already hooked up, right? So at some point, very early in the process even, the database doesn't even know if it's running SQLite or Postgres. It's just running the bytecode. Uh, and there's the same bytecode that just ran Doom, but running Postgres, running SQLite, running— it could be, it could be anything.

01:46:16 - Dev Agrawal

So that's where the LLVM analogy comes in.

01:46:18 - Glauber Costa

That's right.

01:46:19 - Dev Agrawal

As in, like, that's right, the front end, the front end of the database is different.

01:46:23 - Glauber Costa

That's right.

01:46:24 - Dev Agrawal

Like with the LLVM compiler, you can have a different front end, but yeah, the compiler—

01:46:29 - Glauber Costa

and the analogy breaks because again, you're right. I mean, we started framing towards And I think it was very well received by internet standards. So the amount of hate was around like 5%, which is great.

01:46:42 - Dev Agrawal

Did he post it on Hacker News yet?

01:46:44 - Glauber Costa

Oh, of course. Okay. And again, 5% is a great result. It means people really love it. And the idea of an LLVM of databases resonated fairly well. And that's exactly that. If you're not familiar with LLVM, the compiler, Before LLVM, you have like a C compiler that has C to, and then all the architectures. And then you had the C++ compiler for all the backend architectures. And then you had whatever. And what LLVM did is like, we're going to be an intermediate representation layer. And then you can just write frontends. You're going to have the C frontend, the Java frontend. Again, you can, there are compiled versions of Java and And they tried to do this in the LLVM project. Then Rust came later and it became— by the way, the database that I like the most is not even SQLite and Postgres written in our bytecode. The database I'm excited the most is whatever you're going to write in the future.

01:47:47 - Anthony Campolo

So the database is the friends you made along the way?

01:47:50 - Glauber Costa

No, because if you think about Rust and Zig, etc., etc., is not that they would not have existed without LLVM, but LLVM made the act of creating a programming language so much easier because now again you didn't have to write all that stuff. You could write just the front end and, and everything else was, was already there for you because it was the same infrastructure. So that is exactly what we're trying to do. Uh, our idea is that if by turning Turso into the LLVM of databases— SQLite was fairly hard, Postgres will still be fairly hard, uh, but after those 2 are done, quite honestly, if you want to come and create like your crazy experimental database, it's going to be super simple. Uh, so why not go, go, go nuts, you know? So I'm very excited. The same way that Rust was way more interesting than all the languages that LLVM, uh, supported in the beginning, right? I think I would have— that's what I want to see.

01:48:43 - Anthony Campolo

Okay, I want to just make sure all these questions before we Go to the next thing. What are your thoughts on pg_light?

01:48:50 - Glauber Costa

Uh, again, pg_light is great. I think, I think it shows the— uh, I, I'm good friends with, uh, the creator. Um, it shows that the, that the demand for running Postgres in a file is tremendous, but it's not the right architecture. It's not, it's not the right architecture because it's super hard to, to do. Like, you have to leave a lot of things in the table. Like, Wasm doesn't support threads. Postgres is incredibly threaded. In fact, it's more than threaded, it's process-based. So it, it, it has limited applicability. Uh, a lot of the things that people say that our implementation won't be able to do, uh, they can't do either, which, for example, like loading arbitrary Postgres extensions. Uh, so at the end of the day, I think they have more limitations in, in a sense than, than what we're going to come up with, and our performance is going to be a lot better. Uh, so this is a, you know, In some sense, pgLite reminds me of the fork of SQLite. I mean, the right idea, but not enough ambition. And I think what we're doing here is the right amount of ambition.

01:49:55 - Anthony Campolo

What was your question, Dev, that you had?

01:49:59 - Dev Agrawal

Yeah, I was just saying, so like when you were saying that you can write your own database, like technically speaking, you would be writing a frontend, like the query language or the interface of how you want people to use the database. And then instead of building the entire database itself, I just need to write the parser layer from my custom query language to, um, to Turso's AST.

01:50:25 - Glauber Costa

That's right.

01:50:26 - Dev Agrawal

Okay, so, uh, so I guess what does Turso's AST look like and how different exactly is it from like SQL? Like, uh, like what's the general idea of, uh, what, what, what makes up the Turso AST? Like, how do you— what does the backend look like?

01:50:44 - Glauber Costa

So again, the AST is just a tree of statements, uh, and then you're gonna have things— it, it's essentially a superset of relational algebra, right? You're gonna have joins, selects, and then, and then inside the select you have the parameters. Uh, if you, if you think about How would you express a SQL-like or any SQL query in JSON? You're going to end up with something very similar to what the AST is. It's a structured tree-like element, amount of elements. So you're going to have a table, then you nest another table in there, and then you have attributes to that table and you create a structure. That's the AST. The name AST means abstract sentence tree. The whole point is to try to get the thing that is your language and transform that to a representation of that that is abstract. It doesn't have the details, it doesn't have the syntactical things, but all the elements are represented in the AST, the attributes, the nodes, each Each node is the operation, and then in each operation, you have the attributes like which table, which columns, and all of that, right?

01:52:03 - Dev Agrawal

Right. Yeah. And kind of going off Asher's point here that you can go farther than just a SQL-like thing. So, I guess, yeah, in that kind of vein—

01:52:14 - Glauber Costa

You can. Yes.

01:52:15 - Dev Agrawal

If I try to imagine a database that's very, very different from what SQL looks like, Yeah, uh, like the relational algebra, like how much of— how much work would that be to implement? Like, are there things within Turso? Within—

01:52:29 - Glauber Costa

no, it's not a lot of work. In fact, as I said, uh, one, one thing that will be super cool will be Redis. Uh, and the— I, from my understanding, which could have gaps because, you know, I'm a person The only thing, the only bytecode instruction you will need to add is something to deal with TTLs. Because Redis has TTLs, SQLite doesn't have TTLs, also Postgres, my understanding is that it doesn't have Redis-style TTLs at least. So you would have to add a bytecode to handle that. Once that's in, It's all like translating the Redis language to opening files, closing files, finding something in a file, all of that.

01:53:28 - Anthony Campolo

Awesome. We can probably start winding it down here. We almost made it to 2 hours. We had an awesome group in the chat today. Really appreciate everyone being here asking questions. Um, last things you want to end us with, Glauber? Where Turso's going, what you're working on now, any calls to action?

01:53:43 - Glauber Costa

Yeah, so Turso is, uh, we just announced concurrent writes on the cloud, uh, which has some of you, uh, as some of you were, uh, remarking, some of you are receiving the email right now, but I, I started the newsletter around 4 hours ago. Uh, so we announced, we announced the concurrent writes in the cloud, which nothing more. At the end of the day, it's just like Turso is finally being executed on the cloud. Our cloud was still running SQLite up until today. We've been running this in private beta for a while, and, and now it's public. You still have to, you still have to flip a switch because it, you know, it's a production thing. So we, we want time for people to report issues, and the kind of issues that you don't find at the private beta because you don't have scale. So I would like to We're going to run this. We're not going to default to Turso, and you're going to have to flip a switch saying that you are aware of the consequences of your actions for a couple of months. We don't expect this to pick up tremendously, but some people are— I see in the Slack channel the people who are enabling this already and it's been going great. A lot of people truly want concurrent writes in the cloud. Turso is going great. We're going to keep doubling down on making our cloud even better. Uh, we're gonna keep doubling down, making our cloud more accessible, cheaper, faster. Uh, again, unlimited databases. There, there are a lot more ideas that we have to make it even more like incredibly developer-friendly, which has been the core of our brand since, since the beginning. Uh, and, and look, the, the Postgres thing is, is not a year-long project. This is something that we believe we can tackle in perhaps 6 months. Uh, so in 6 months we can come back here and, and, and talk about like how we have—

01:55:30 - Anthony Campolo

we'll pay you to run the database cheaper.

01:55:37 - Glauber Costa

Many things are cheaper than free, Shreyas, because free comes with a lot of problems, man. Free is always like— our free tier is severely overcommitted, so our queries are not that fast, uh, right? It's, uh, free is best.

01:55:51 - Anthony Campolo

Yeah, amazing.

01:55:52 - Glauber Costa

But our free tier is awesome. People love it, and we're gonna We can do it because, you know, so the Crystal Cloud is so efficient.

01:56:01 - Anthony Campolo

Well, thank you so much, Glauber, for being here. This is an awesome discussion. We hit so many different topics. So, um, and again, thank you everyone for being out here. Um, Dev, we don't have anything lined up for next week, so maybe just you and I, but, um, I got all sorts of stuff we can show. We can show my new drum app. That'll be fun.

01:56:22 - Dev Agrawal

Definitely.

01:56:23 - Anthony Campolo

Yeah. All right, Eric, I don't know if you're still watching, but you should come on the stream. Hopefully SQLD will—

01:56:29 - Glauber Costa

yeah, SQLD is the first version, like, on Leap SQL. We're gonna, we, we're gonna keep there, but our investments are— modern investments are all on, on Turso, of course, and the infrastructure.

01:56:42 - Anthony Campolo

All right, all right, we'll wrap it up here. We'll catch you all.

01:56:45 - Glauber Costa

See you all. It was a pleasure very much.