Deno Crate Organization - Paris Deno

Episode Description

Two talks from the Paristino Meetup explore Deno's Rust crate organization and internal architecture, covering its design philosophy and toolchain.

Episode Summary

This Paristino Meetup features two complementary talks about Deno's architecture. Anthony Campolo opens by tracing Deno's origins back to Ryan Dahl's famous "10 Things I Regret About Node" talk, explaining how issues with promises, security, and the module system motivated a ground-up rewrite. He walks through Deno's four main Rust crates — the CLI executable, Deno Core, Rusty V8, and the TypeScript package — showing how each serves a distinct role in the runtime's modular design. Bartek Iwanczuk then goes deeper into the internals, describing Deno less as a runtime and more as a full JavaScript/TypeScript toolchain with bundling, linting, formatting, and documentation built in. He explains how the TypeScript compiler runs inside V8 using snapshots for speed, how the SWC project replaced slow JS-based dependency analysis with fast Rust-based parsing, and how tools like dprint and the custom linter were adopted or built to replace slower JavaScript equivalents like Prettier and ESLint. Both speakers emphasize that Deno's crate-based architecture enables independent development, easier testing, and the possibility of composing custom runtimes from individual pieces, while also lowering the barrier for open-source contributors.

Chapters

00:00:00 - Meetup Introduction and Housekeeping

The session opens with the Paristino Meetup's automated introduction, explaining the community's mission as the first French and Parisian meetup dedicated to Deno. Details are shared about the meetup's cadence, formats ranging from lightning talks to hour-long workshops, and how to find the group on LinkedIn, Discord, and YouTube.

Host Sylvain Pontoreau then welcomes the audience and outlines the evening's agenda: two talks, one from Anthony Campolo on Deno's crate organization and one from Bartek Iwanczuk on Deno's internals. He explains how to submit questions via the YouTube chat using a hashtag, setting the stage for the presentations to follow.

00:08:40 - Deno's Origins and Ryan Dahl's Regrets About Node

Anthony Campolo begins his talk by explaining that Deno is an anagram of Node and recounting the naming debate between "Deno" and "Dino." He then directs the audience to Ryan Dahl's influential talk on his regrets about Node, highlighting how Node's lack of native promise support, insufficient security defaults, and a tangled module/package system all motivated the creation of a new runtime.

Anthony distills these regrets into three core problems: Node diverged from modern JavaScript standardization after ES2015, it grants too much system access by default, and its module packaging system is overly complex. These issues collectively explain why a complete rethink was necessary rather than incremental fixes, and they set the foundation for understanding Deno's design principles around security, TypeScript support, and a single executable distribution model.

00:15:52 - Deno's Design Principles and the Move from Go to Rust

Anthony outlines Deno's core principles — secure by default, native TypeScript support, a single executable, built-in tooling, and a guaranteed standard module system. He then traces the project's early history back to 2015 when Ryan Dahl first conceived a V8 binding project using Go, explaining how concerns about competing garbage collectors in Go and JavaScript led to the decision to use Rust instead.

This section also covers why Deno avoids monolithic design. By decomposing the runtime into separate crates, different teams can independently test and develop subsystems. Anthony explains how this modularity enables Deno to serve not just as a traditional server runtime but also as an embeddable runtime for serverless functions, MapReduce operations, and GUI applications like Electron, broadening its potential use cases significantly.

00:22:19 - Deno's Four Main Rust Crates

Anthony walks through Deno's four primary Rust crates. Rusty V8 provides zero-cost bindings between Rust and V8's massive C codebase, leveraging Rust's type system for safety and using GitHub Actions for pre-built binaries. Deno Core introduces the ops and resources abstractions that allow JavaScript to call native Rust functions through a single, optimizable binding interface, integrated with the Tokio event loop for async operations.

The Deno executable crate itself is described as surprisingly minimal — it consumes Deno Core to provide the user-facing CLI and aims for browser compatibility through ES modules. Finally, the Deno TypeScript crate handles type checking and stripping but is noted as a work in progress with ongoing performance challenges, as the team works to shift compiler responsibilities into Rust.

00:35:13 - Bartek's Overview of Deno as a Toolchain

Bartek Iwanczuk opens his talk by reframing Deno not merely as a runtime but as a complete JavaScript and TypeScript toolchain, noting its built-in bundler, dependency inspector, documentation generator, formatter, linter, and test runner. He shares that Deno 1.0 was strategically released exactly two years after the first commit, and that the team follows a rapid release schedule with weekly patches.

He then presents the architecture diagram, identifying four main components of the runtime: V8 with its Rust bindings, the Tokio event loop, the TypeScript compiler running via TSC, and the module graph system responsible for dependency collection and code caching. This framing sets up his deeper exploration of each component and how they interact within the binary.

00:40:46 - Rusty V8, Deno Core, and the Module Graph

Bartek explains how Deno evolved from an early foreign function interface called Libdeno to the cleaner Rusty V8 crate, which provides a purely Rust interface over V8's C bindings. He describes Deno Core's JS runtime abstraction, which wraps a V8 isolate and handles asynchronous ES module loading — a non-trivial challenge given top-level await semantics.

The module graph system is then explored in detail. It collects the full dependency tree before execution by fetching sources, parsing imports, and recursively resolving them. Deno caches downloaded code globally on the machine rather than per-project like node_modules, and also caches transpiled JavaScript output so TypeScript files don't need repeated compilation, making the developer experience seamless.

00:47:58 - TypeScript Compilation, V8 Snapshots, and SWC

Bartek discusses how Deno runs the large TypeScript compiler source directly inside V8, and how V8 snapshots solve the resulting performance problem by serializing the compiler's heap state into the binary so it doesn't need to be re-parsed on every invocation. He also addresses the pain caused by TSC's fully synchronous API, which made asynchronous dependency fetching extremely difficult.

The introduction of SWC — a Rust-based JavaScript and TypeScript compiler — proved to be a turning point. By moving dependency analysis from TypeScript into Rust via SWC, Deno could perform full upfront analysis and feed all sources to TSC in a single pass. The team is moving toward using SWC for all transpilation, reserving TSC solely for type checking, resulting in dramatically improved performance.

00:55:17 - Built-in Formatting, Linting, and Documentation Tools

Bartek covers Deno's built-in formatter, which originally wrapped Prettier but was replaced by dprint — a Rust-based drop-in replacement that runs almost instantaneously. He notes that Deno takes an opinionated, no-configuration approach to formatting, similar to Go's gofmt philosophy. The linter follows a similar story: rather than bundling ESLint, the team wrote a custom Rust-based linter that implements recommended rules from ESLint and its TypeScript plugin.

The documentation generator, denodoc, is highlighted as another standalone Rust crate that parses JSDoc comments and can output to terminal or web interfaces. Bartek emphasizes that all these tools exist as separate crates with clear boundaries, meaning each tool's integration into the Deno binary requires only 50 to 80 lines of code, and contributors can work on individual tools without needing to understand the entire Deno codebase.

01:02:19 - Composable Runtimes, Contributing, and Closing

Bartek describes the long-term vision of extracting more functionality into independent crates — browser-compatible APIs, the Fetch implementation, and Deno namespace APIs — so that advanced users could theoretically compose custom runtimes by selecting only the pieces they need. While this workflow isn't fully polished yet, the architectural groundwork is being laid with each release.

He closes with an encouraging invitation to contribute, noting that many newcomers after 1.0 found the Rust codebase surprisingly approachable, and that substantial portions of Deno are written in JavaScript and TypeScript as well. Both speakers thank the audience, and the host wraps up the meetup, noting the next session would likely be held online in December.

Transcript

00:00:00 - Bartek Iwanczuk

[unclear]

00:01:25 - Anthony Campolo

[unclear]

00:03:21 - Bartek Iwanczuk

[unclear]

00:05:29 - Denoemie robot intro VO

Welcome to the YouTube channel of the Paristino Meetup. My name is Dino Amy. I'm the robot voice of Paristino Paris. Deno is the first French and Parisian meetup focusing on Deno technology. The meetup is organized about every two months at the beginning of the month, and we usually plan the event one to two weeks before. The format can be regular or physical, like any classic meetup, but we also sometimes do an online format with a stream on YouTube. For more information, you can find us on the following social platforms: LinkedIn, Discord, Meetup.com, and of course YouTube. You can also contact us directly on Twitter or at the email address [unclear]. We will be happy to answer your questions. Of course, we are constantly looking for speakers. The proposed topics can be of any kind, such as personal or professional feedback, the presentation of a library or a framework, the use of Deno-specific features, and so on. The meetup and talks are inclusive, beginner-friendly, and if you've never had the opportunity to give your first presentation, you can still get started — it's even highly recommended. Don't hesitate. Apart from the classic 20-to-30-minute talks, the length and format can vary if you want, ranging from five-minute lightning talks to one-hour workshop sessions.

00:07:11 - Denoemie robot intro VO

Finally, we have a GitHub repository where you can submit your abstracts. Even if my voice sounds emotionless, we would also like to warmly thank Paris TypeScript, our meetup partner. If you are interested in TypeScript beyond Deno, go to TypeScript Paris to find an active community on the subject. Sorry about my robotic presentation, but you'd prefer my accent, believe me. So, once again, on behalf of the Paristino team, welcome to the channel.

00:07:45 - Bartek Iwanczuk

Hello guys.

00:07:47 - Sylvain Pontoreau / meetup host

Welcome to this new meetup. Tonight we have two talks. The first one is by Anthony. It's about the crate organization — probably quite a rusty talk, maybe. We'll see. For the second talk we have Bartek, and he will talk about Deno internals and the architecture around Deno, all the updates around that. If you have any questions, you can use the hashtag "question" at the beginning of your sentence inside the YouTube chat, and I will ask each question at the end of each talk. So it's time to start. Anthony, it's your turn.

00:08:39 - Bartek Iwanczuk

All right.

00:08:40 - Anthony Campolo

Bonjour! That is the extent of my French that you will get to hear. I am very happy to be here to talk with you all about Deno. Specifically, I'll be talking about Deno crate organization. Let me go ahead and share my screen.

00:09:00 - Bartek Iwanczuk

There we go.

00:09:02 - Anthony Campolo

As I said, my name is Anthony Campolo. If you want to find me online, you can find me at ajcwebdev. That's my Twitter handle, my Dev.to handle, and also my GitHub handle. Anywhere where developers congregate, you can find me there. I want to give a shout out to Craig Morton. He's actually the one who retweeted this event, which is why I'm speaking here. He's written a fantastic suite of articles about Deno over on Dev.to. What's shown on the left is just a selection — he's written about twice as many articles as that. Really great stuff. I highly recommend checking it out. So, what is Deno? Deno is an anagram. An anagram is a word, phrase, or name formed by rearranging the letters of

00:09:58 - Bartek Iwanczuk

[unclear]

00:10:00 - Sylvain Pontoreau / meetup host

Can you hear me?

00:10:01 - Anthony Campolo

Yes. Can you hear me?

00:10:04 - Sylvain Pontoreau / meetup host

We are not seeing your slides.

00:10:06 - Anthony Campolo

You're not seeing my slides. Okay.

00:10:07 - Bartek Iwanczuk

Yes.

00:10:10 - Sylvain Pontoreau / meetup host

Classic issue.

00:10:12 - Anthony Campolo

Yeah, sure.

00:10:15 - Sylvain Pontoreau / meetup host

Someone on the chat was asking.

00:10:16 - Bartek Iwanczuk

Yeah.

00:10:19 - Sylvain Pontoreau / meetup host

It's good.

00:10:21 - Anthony Campolo

There we go. Right?

00:10:22 - Sylvain Pontoreau / meetup host

Yep, it's good. Thanks.

00:10:24 - Anthony Campolo

Cool.

00:10:25 - Bartek Iwanczuk

All right.

00:10:25 - Anthony Campolo

Sorry about that. So, Craig Morton — here is his information again. And I was talking about Deno being an anagram. An anagram is a word, phrase, or name formed by rearranging the letters of another, such as "cinema" formed from "iceman." So "Node" — N, O, D, E — is rearranged to "Deno" — D, E, N, O. A lot of people wonder whether it's Deno or Dino. The story behind that is it was originally "Deno," because Ryan didn't really think about it too much and just called it that. And then at a certain point, we introduced a dinosaur creature as the mascot. And then people were like, well, he's a dinosaur, so Dino the dinosaur, obviously. And now everyone calls it Dino. So what was Deno originally became Dino — that's why people will probably continue to argue about that forever.

But what actually is Deno? It's a secure runtime for JavaScript and TypeScript. This has to do with Node as it was originally structured. I highly recommend everyone check out a talk from Ryan Dahl. Here's a quote from it: "JavaScript has changed quite a bit in the last 10 years since I originally sat down to do Node."

00:11:46 - Anthony Campolo

Ryan Dahl created Node. He's also the creator of Deno, and he gave a talk called "10 Things I Regret About Node." I've watched this talk multiple times. For the life of me, I can only find eight things that he regrets. If he wants to let me know what the other two are, I would love to know — I'm very curious. But here is a selection of the things he regrets. I really recommend you check out this talk, because Node itself was actually launched with a talk. You can go back and watch Ryan's original talk about Node.js on YouTube from 2009. I think it's really cool that you can have this project that is so massive — so many people use it, so many people depend on it, so many people have written Node code — and at any point in time over the history of Node, anyone could have gone online and found an hour-long talk by the creator of Node explaining exactly why he created it and exactly why he structured it the way he did. I think that's really valuable, and we should take advantage of that resource.

00:12:56 - Anthony Campolo

I highly recommend everyone check out this talk. It's probably referenced more than it's actually watched, which is unfortunate but understandable, because it's a very dense, very technical talk — and that's kind of how it has to be to explain everything he wants to explain.

So the first point is not sticking with promises. Promises didn't exist when Node was created because JavaScript itself has evolved over the years, especially with ECMAScript. Since promises didn't exist, there were different implementations in different libraries that competed with each other, and they diverged from Node. So part of making Deno is about taking not only promises but also async/await and baking all of that into the runtime.

The next big point is security. Node is not very secure, and you want a system to be secure by default, so that you have to grant it permissions. It doesn't start with all permissions — you need to do it the other way around. Now, if you look at the rest of this list, it basically covers everything involved with writing Node: you have package.json, you have index.js, you have your node_modules, and then the whole require syntax.

00:14:27 - Anthony Campolo

The goal is to take a lot of the basic fundamental pieces of Node and completely rewrite them, and completely change how we think about them. This list of items will look fairly incomprehensible to anyone who hasn't written Node code. If you've written Node, then it should all look fairly familiar. But for people who are fairly new to Node or haven't written much Node, you can boil these down to three core points.

First, Node is fundamentally divergent from the standardization of JavaScript since ES2015, so we need to rewrite it because JavaScript itself has changed so much with all these ECMAScript revisions. Second, Node has too much access to the computer it's running on — we need permissions, and you have to explicitly grant them instead of having them there from the start. Third, the module and package system is a bit of a nightmare. This is where a lot of the changes come into play in terms of how we want to break apart Node and make the different pieces available to import.

So seeing all these problems — this is where you stand in the rain at night, stoically facing the dark battle that is software engineering.

00:15:52 - Anthony Campolo

So what do we do? We create Deno. Deno's principles are: it needs to be secure by default — no file, network, or environment access unless explicitly enabled. We want to support TypeScript out of the box, so TypeScript is a core component for anyone who wants to write Deno code. We want to ship only a single executable file. We want to have built-in utilities such as a dependency inspector and code formatter, and we want to have a standard module system that is audited and guaranteed to work.

00:16:28 - Bartek Iwanczuk

Mm.

00:16:29 - Anthony Campolo

And we have that at Deno Land. What's interesting is that a lot of the history of Deno is influenced by Go, even though it's written in Rust. I'm going to talk about how those two forces have influenced where the codebase is now and how it got here.

Back in 2015, this is when Ryan first started thinking about this project and thinking about the issues he saw with Node and how he could possibly address them. He was thinking of a project called V8 Worker — a binding to V8 using Go. V8, for anyone who's not familiar, is the core engine running JavaScript both inside the browser and inside Node. It was created by Google to be the JavaScript runtime inside Chrome, and then Ryan extracted it and adapted it so it could work on the server. We want to use V8 because it's just such an incredible piece of technology. The question is, what do you use to wrap it? Originally he wanted to use Go, and the idea was that V8's APIs wouldn't be allowed to just create random JavaScript objects.

00:17:57 - Anthony Campolo

This is about the idea of permissions and how you could lock the runtime down. Instead, he wanted to do a message-passing system that would pass messages between Go and V8. They decided not to do this because they were worried about the garbage collector. Go has a garbage collector and Rust does not — that's one of the core differences between the two languages. Since Go has a garbage collector and JavaScript also has a garbage collector, they were worried the two would get into conflict and that there'd be race conditions between them. By using Rust, which is more like C, those conflicts are avoided. So while Node is a C runtime for JavaScript, Deno is a Rust runtime for JavaScript and TypeScript. That's kind of how you can think about the differences between the two projects — what they're made of and what they're made to do.

So why do we want to avoid monolithic design? This is a really core piece of how Deno is structured and organized with crates. We don't want a monolithic design because we want to be able to separate concerns inside the CLI.

00:19:20 - Anthony Campolo

Because Node is this one big monolithic project, it can be really hard to test it, to test different components of it, and to break it apart so that different teams can work on it. By having different pieces decomposed, you can test them independently, and you can have more people working on the overall project by having them work on subsystems they can own themselves. This is why we want a more decoupled design instead of a monolithic one. It also gives us a principled binding layer for the CLI.

The other big thing is that we want to be able to use Deno as an embedded runtime in lots of different environments. We don't just want a single monolithic server running on EC2, always on, always serving requests. We want to be able to do other things, like using it as a serverless runtime. For anyone who's not familiar with serverless, it's a term that refers to the paradigm introduced by AWS Lambda — a function-as-a-service idea where instead of paying for a server to always be up and accepting requests, you just have individual functions that respond to incoming events, such as a user clicking your page, trying to make a purchase, or trying to upload an image.

00:20:55 - Anthony Campolo

Any of those can be a function owned by a serverless runtime. Serverless runtimes started with Node, actually — the idea was to take all the Node code you have and break it apart into small functions and just run those. That's where AWS Lambda came from. Over the years they've introduced other languages and runtimes like Ruby or Python. Deno is really fascinating from a serverless perspective, and this is one of the reasons I'm really interested in it in terms of how it can cross-pollinate with Redwood.js, which is a project I've spent a lot of time working on. But I digress.

The other big things: you have databases doing MapReduce. This is a classic data-science problem where you have a whole bunch of data and you want to break it apart onto different clusters and then do different operations on it. By having an easily embeddable runtime, you could parallelize that task very easily. And then you also have GUI applications like Electron — which is also built on Chromium. So all of these projects are using similar tech.

00:22:19 - Anthony Campolo

Deno is aiming at a lot of these similar use cases. A Rust crate is like a Ruby gem or a package on npm — it's just a way to take a bunch of code and package it together so that it's easily shareable and distributable. So a Rust crate is how Deno is organized into individual pieces. We have four main Rust crates. There's actually a whole lot more than this if you look at Ryan's crate profile, but this covers the main pieces. This is based on a talk Ryan gave for Deno Israel a couple of months ago, and all of this is in motion — that's partly what the next talk will be about. But as of now, this is a fairly good way to look at the overall project.

00:23:32 - Anthony Campolo

You can go to any of these crates online, find them, inspect them, and take a look. We've got the main Deno executable — that's the main thing most people will be using when they download and use Deno. Then there is the Deno TypeScript package, for type checking and type stripping at compile time. For anyone who hasn't used TypeScript, the idea is that you add types to your JavaScript code — explicitly declaring whether something is a string or a number. This enables a lot of extra tooling like autocomplete. What happens is all the types are run through a compiler and stripped out, and you get JavaScript code at the end. Even though you write in TypeScript, what the compiler produces is just regular JavaScript. That's what the type checking and type stripping is about. We also have the Deno Core package, which contains the ops and resources — Deno concepts we'll talk about later.

00:24:54 - Anthony Campolo

And then there's the Rusty V8 crate, which provides the bindings between V8 and Rust — V8 being the runtime, and Rust being what we use to talk to V8. We'll start with Rusty V8 because it's the most complicated and has the most going on. V8 itself is just an absolutely massive C library — it has about 800,000 lines of code. Rusty V8 is a framework that adds bindings to it so you can write Rust code instead of having to write C code. This gives you an interface between Rust and V8 and provides safe bindings, because V8 itself has a lot of security baked into it. It's sandboxed — the browser is one of the most hostile environments in all of computing, and a ton of work has gone into that security. Rusty V8 tries to leverage all of that security work. You can also recompile it with different compilation settings.

So if we look at each of these in a bit more detail: the zero-cost interface means that objects can be manipulated in Rust exactly the way they're manipulated in C.

00:26:30 - Anthony Campolo

So there's a guarantee that you know exactly what you're putting in and what you're getting out. The type system is basically what forces the safety. Because Rust is based on the functional paradigm — even though it's not really thought of as a functional language but as a systems language — it is actually typed, I think, in the ML family. I may be wrong about that, but I'm pretty sure that's right. And that's why it's very useful for enforcing typing. With recompiling, you have pre-built binaries that can be created through GitHub Actions. GitHub Actions is a somewhat newer piece of GitHub that can be used for continuous integration. Then the V8 source code is distributed inside the crate file itself.

Now let's talk about Deno Core. Deno Core is meant to address problems with Node, as we discussed at the beginning with Ryan's regrets. There was no centralized binding interface in Node, which made it really hard to get global metrics, monitor things, do security checks, and understand the overall state of your Node process.

00:28:13 - Anthony Campolo

This led to lots of problems with big companies like Walmart, who would have their Node process blow up and then didn't really know how to inspect it. They had to introduce other tools to look into Node and get stack traces. So there were a lot of problems. You'd have callbacks issued from C without being requested, and you'd give users the opportunity to create code without any back pressure, which is an issue.

With Deno, they introduced native bindings through the concept of ops. Ops are just native functions called from JavaScript — everything in the core executable is built on ops, and since it's all built on a single abstraction, they can optimize it and get better performance. You have just a single executable binary. This is really nice because you can use array buffers as your parameters and return values — basically a generic fixed-length raw binary data buffer. A lot of this is just about knowing exactly what you're putting in and what you're getting out, and being able to inspect it easily with the array buffers.

00:29:39 - Anthony Campolo

Then you can pass pointers back and forth between JS and Rust, and it's really easy to know what you're doing with those pointers. It's also integrated with the event loop. Deno uses a Rust event loop called Tokio, which I don't know a whole lot about but sounds super interesting — you should look into it if you're curious. An op will either complete synchronously with a result or return a promise corresponding to a Rust future. Rust futures are used for async data. If you're interested in the async stuff, that's another big feature that could be a whole talk in itself.

So we have ops, and then we also have resources. These are the core ideas we're building on in Deno. A resource is a lot like a file descriptor in POSIX — a specification for operating systems in C, like Linux. It's an integer identifier given to JavaScript to reference an object allocated in Rust. If you look on the right, that's a diagram showing how the Rust and V8 bindings work.

00:31:00 - Anthony Campolo

This was from Ryan's talk in May, so I'm not sure how accurate it still is, but that's where it originally came from. And then, Deno itself — as I said, this is the main thing you're going to be interacting with. I find it hilarious that this is the entire crate page: "This provides the actual Deno executable and the user-facing APIs. The Deno crate uses Deno Core to provide the executable." There you go. If you click the repository link, it will take you to their main GitHub, which has a lot more information. But this is the whole shebang when it comes to Deno. This is what hit version 1.0 a few months ago, when everything blew up and a lot of people were talking about it, and we're now in the 1.4.x range.

The goals of the main Deno are to have a single executable, to provide secure defaults — which is what we've been talking about throughout this talk — and to be browser-compatible, using things like ES modules.

00:32:21 - Anthony Campolo

So we have changes in the language not just for promises and async, but also for the module system itself. Node originally used CommonJS, which used the require syntax — it was different from the way ES modules work, and there's been a lot of confusion there. Everyone is now converging around ES modules. All of this also gives you better built-in tooling for unit testing, code formatting, and linting. They also don't want to leak any V8 concepts into userland, so separating out Rusty V8 from the Deno Core executable helps a lot with that. And they want to be able to serve HTTP efficiently.

Finally, we have the Deno TypeScript crate. This one is a work in progress — it's where the compiler is incorporated, and the compiler was causing performance issues. They've had to strip away most of the TypeScript compiler and move that work into Rust. This should not be used in production. If you're interested, you can talk to the core team and they'll let you know the state of things. But this is still a work in progress. That's the whole talk. Thank you for listening.

00:33:47 - Sylvain Pontoreau / meetup host

Okay, thanks Anthony. Great talk — it was super clear. I didn't see any questions on the chat. Maybe we'll wait a couple of seconds just in case, because there's a delay from the stream.

00:34:02 - Anthony Campolo

Sure, yeah. Feel free to send me messages on Twitter if you're curious about anything. I'm always open to talk.

00:34:10 - Sylvain Pontoreau / meetup host

At the beginning of your talk you mentioned serverless — do you know [unclear]? He creates a lot of things around that, especially for AWS, including support for Serverless and DynamoDB. And if I remember correctly, there's also someone at Microsoft who implemented Deno for Azure Functions.

00:34:41 - Bartek Iwanczuk

I remember.

00:34:42 - Anthony Campolo

Cool. Yeah, I'm very interested in different deploy targets for it. It's a really cool space to get into right now.

00:34:50 - Sylvain Pontoreau / meetup host

I don't know about GCP — I don't know if they've done anything on it.

00:34:55 - Anthony Campolo

They'll get there eventually. Someone will do it.

00:35:00 - Sylvain Pontoreau / meetup host

Okay, it seems we don't have any questions, so maybe we can just pass the mic to Bartek for the second talk. Thank you, Anthony.

00:35:09 - Anthony Campolo

Thank you.

00:35:13 - Bartek Iwanczuk

Hello everyone. Thanks for having me here today. I'm going to be talking about some of the Deno internals. As it turns out, a lot of this stuff has already been described by Anthony before my talk, which I think is really good, because I'm going to focus on some more complex stuff. Let's get right into it.

So briefly — what's the goal of this talk? I want to give you a high-level overview of the Deno binary, the Deno CLI: this thing that you download to run your code. I'd like to describe some of the most significant parts of the runtime, and I don't have to explain anymore why Deno is not a monolith, but I will go a bit more in depth on why that's the case.

Let's start with something a little bit controversial. Deno presents itself as a runtime for JavaScript and TypeScript. However, I feel that calling it a JavaScript and TypeScript toolchain is a bit more concrete, because not only can we run our code — we've got a plethora of other tools built into the binary itself. If you want to run code, you use the runtime itself: deno run or deno repl.

00:37:08 - Bartek Iwanczuk

You can bundle your code using the built-in bundler. We've got a dependency inspector, a documentation generator, a formatter, a linter, a test runner, and more. A bit of history first: the first commit of Deno was made on May 13, 2018, and Ryan's talk about Deno happened on May 13, 2020, which is not a coincidence. We thought it would be cute to release it exactly two years later. Move fast and break things — that's kind of what we're doing right now.

1.0 was released in May and we're already at 1.4.4 as of last Sunday, but it will change — sorry — by Friday. We release on a weekly basis with patch releases, so we try to fix bugs as they come in and release often. We do minor releases every month — or rather, we did. Now we're switching to a six-week cadence for minor releases as well. To be determined. There is no plan for 2.0 yet. Maybe there is, maybe there isn't — can't confirm, can't deny. Let's jump into the actual architecture. Deno is not a monolith — that we already know.

00:38:56 - Bartek Iwanczuk

Instead, it's a collection of Rust crates. What does that mean exactly? Here's my somewhat clunky diagram of how the Deno binary itself looks, and I'll try to walk you through each of the pieces one by one. The main runtime — the thing that actually runs your code — consists mainly of four pieces: the actual JavaScript engine, V8, and the layers built on top of V8 to make it usable from Rust; an event loop (in Deno's case, Tokio; in Node's case, libuv); a TypeScript compiler — we use TSC, the same TypeScript compiler you have in Node, but it's not used the same way, which I'll describe shortly; and one big [unclear] that I've just called the module graph, which is a system for code caching and analysis.

Let's start with the most essential part: the engine. As we already know, Deno runs on V8, the JavaScript engine used in Chrome, Chromium, and all the related projects. Super fast, super optimized, super complex. Deno is very tightly integrated with V8 — it's not something we can swap out for another engine easily.

00:40:46 - Bartek Iwanczuk

In the early days of Deno, we didn't have this crate called Rusty V8 that Anthony told you about — the layer of bindings from V8 to Rust. We had something called Libdeno, which was essentially a collection of a few files that formed a foreign function interface between Rust and C. It worked and got us a very long way. However, it was very hard to maintain and reason about, and the boundary where we go from V8 to Rust and back again was very unclear. That's how Rusty V8 came to life. I'm not going to go into details about Rusty V8 because Anthony already did that. Suffice to say, it's just a Rust crate that handles the foreign function interface with C, but exposes a purely Rust interface to end users. We use this Rusty V8 crate in Deno Core.

Deno Core is also a low-level crate, but it's a bit higher level than Rusty V8, so you don't have to interact with V8 directly when you use Deno Core. Instead, we provide a few abstractions that make your life easier. One of the abstractions already mentioned today is ops and resources.

00:42:28 - Bartek Iwanczuk

Ops and resources are the mechanism that allows you to call Rust functions from JS and provide native bindings like file reading or opening a network socket. The rest of Deno Core is basically infrastructure around that. We provide a single structure called JsRuntime, which is hopefully self-explanatory. It gives you a single instance of a V8 isolate — the actual thing that executes your JavaScript code — and wraps it in Rust APIs that allow you to do things like module loading.

ES module loading is quite different from using CommonJS. ES modules, especially with top-level await, are fully asynchronous. So loading and execution are done asynchronously, and it's highly non-trivial to connect all the dots from V8 and expose an interface that allows you to load modules asynchronously.

One thing to keep in mind: Deno Core has no concept of TypeScript or any other fancy tools. It's a bare-bones JS runtime, and if you want to use it, you're going to have to get your hands dirty. Let's go over to the module graph and module caching, because before we can execute anything, we first need to obtain the source code.

00:44:21 - Bartek Iwanczuk

The module graph is a piece of logic in Deno that is responsible for collecting the whole dependency tree of the module you want to execute. If we start with a single entry point — as you can see in the screenshot in the corner — the module graph will take it and perform some actions to collect everything you need to actually execute the code: fetching the source, parsing it, figuring out what the imports and re-exports in that file are, obtaining those, and repeating until you have everything. Once you've analyzed all the files and see that there are no new imports, you can start assembling those files into the graph structure.

An important detail about Deno is that it can execute code from the internet directly without installing it. So it would be very silly if you had to download the code over and over again on every run. Deno doesn't do that. Instead, the code is cached on your machine — similar to node_modules, but node_modules are per-project, so everywhere you have a package.json you'll have a node_modules folder, unless you configure something like PnP or a shared folder.

00:46:10 - Bartek Iwanczuk

In Deno, the cache is global for your machine. So if different projects use the same sources, they can use the same cache — which just means less stuff to download from the internet. And importantly, it's not only the sources obtained from the internet that we cache; we also cache transpiled sources. Deno is a TypeScript runtime, but obviously you can't execute TypeScript directly — it has to be transpiled to JavaScript. In Node, you set up a build pipeline to transpile your sources from TS to JS. In Deno, this is all done automatically for you. Your experience is pretty seamless: you just run your TS file and all the magic happens behind the scenes.

Okay, so let's talk about TypeScript in a bit more detail. TypeScript is awesome — let that be said — but there are some challenges with it too, or at least some hoops we had to jump through. Deno has a custom implementation of the TypeScript compiler host. When you download typescript.js, this huge file that's almost 10 megabytes, you get all the infrastructure you need to build your custom host.

00:47:58 - Bartek Iwanczuk

If you've ever seen an LSP or TypeScript plugin, those plugins implement their own hosts as well. In Deno's case, since we already talked about the JS runtime provided in Deno Core, it's no surprise that Deno's TypeScript is actually implemented using that JsRuntime. Basically, we get this huge TypeScript source code and run it directly in V8. However, there are some problems with this approach. Since this typescript.js is so big, it's also pretty slow to execute. So if we were to parse, compile, and execute the same code over and over again every time we invoked the TS compiler, it'd be dead slow.

One solution incorporated into Deno is V8 snapshots. The V8 engine provides an ability to execute some JS code and then snapshot it — V8 just takes a chunk of the heap allocated to the running isolate and serializes it into a binary format. We use this technique to snapshot the TS compiler and write it into the final binary. So when we invoke the TS compiler, we don't have to go through the whole parse-compile-execute process.

00:49:45 - Bartek Iwanczuk

We just restore this serialized memory into V8 and we're up and running. And by the way, you can check the time it takes to invoke the TS compiler in Deno on our benchmarks page — we do keep track of it.

The other problem with TSC is that its API is fully synchronous, and this caused us many, many headaches along the way. Mainly, if you want to execute TS code you first need to transpile it, and since the TS APIs are synchronous, we had to block every time we wanted to obtain new source code. TSC would first parse your file, check for import/export statements, give them back to you, and then you'd have to fetch them and feed them into the compiler again. Since it's synchronous, that's a no-go if you want to download files from the internet.

So what we did first was use some of the synchronous TS APIs and hacked it together with some Rust magic to get a sort of asynchronous dependency analysis. However, the whole setup was extremely cumbersome and error-prone, so we had to look for a better solution — and we'll get into that solution in just a moment.

00:51:33 - Bartek Iwanczuk

In the early days of Deno, say a year ago, Deno was very tightly integrated with TSC. We used it not only for type checking, which is the main use case, but also for module loading and transpiling sources. However, TSC is quite slow — parsing files and emitting code is just slower than in Rust. So we went looking for a way to move all of this into Rust, and we found SWC. This SWC project has been a game changer for us and allowed us to move more and more of our infrastructure from JS into Rust.

So let's look at SWC. SWC stands for Speedy Web Compiler. It's a super-fast JavaScript and TypeScript compiler. Why do we use it? First, it's crazy fast — an order of magnitude faster than, for example, Babel or TypeScript. Second, it's written in Rust. Our general consensus is that if something becomes hard, slow, or cumbersome to do in JS, just move it to Rust. And that's exactly what we did.

00:53:22 - Bartek Iwanczuk

We introduced SWC into our codebase and started using it for something super simple: taking the dependency analysis out of TypeScript and putting it in Rust. So instead of feeding source files one by one into TypeScript and obtaining imports as we go, we opted for full upfront analysis and then feeding all the sources to TypeScript in one go. Nowadays SWC is used for a lot of heavy lifting in Deno. The heaviest task is obviously this dependency analysis, but we're moving in a direction where all transpilation and compilation of your sources happens in Rust, and TSC is used only for type checking.

Moving on, let's take a look at dprint. Deno has a built-in code formatter. If you type deno fmt, it's going to format your whole current directory — or rather, the JS and TS files in the directory. Deno fmt started out as a Prettier formatter: we had this subcommand, but underneath it just downloaded and bundled Prettier and ran it on your codebase. The problem was exactly the same as with TSC.

00:55:17 - Bartek Iwanczuk

It was slow. And besides that, it was kind of silly to provide a subcommand in the binary and then require the user to download more code just to run it. So dprint came along, and dprint is basically a drop-in replacement for Prettier. If you use Prettier today, you can just switch to dprint and it should work exactly the same — just faster, a lot faster. The other advantage is that it's written in Rust, so integrating it into Deno was a breeze. The experience with dprint is just phenomenal: you type deno fmt and it's done before you lift your finger off the Enter key.

There was one minor downside with this approach, which was that Prettier was configurable — you could tweak all kinds of knobs and switches. And dprint allows you to do the same. However, in Deno we opted for an opinionated configuration for this formatter and we don't allow you to customize it. It's probably one of those holy wars about whether you should configure it or not. But I think it's similar to gofmt — it's no one's favorite, but everyone uses it and everyone's happy. That's the direction deno fmt is going.

00:57:07 - Bartek Iwanczuk

Okay, moving on — let's take a quick look at deno lint. Deno also provides a lint subcommand that runs static analysis on your code and shows you common problems, much like ESLint. The initial idea for implementing this functionality was the same as with the formatter: take ESLint, bundle it for the browser, and then run it inside Deno. However, we actually never got to a place where we were able to run ESLint that way. And ESLint is also slow — it takes a second or two just to run. So we decided to just write our own linter. I have to warn you, it's still a work in progress — that's why it's hidden behind the unstable flag. But it's working. We've got recommended rules from both ESLint and ESLint TypeScript implemented in it, so you can just run deno lint --unstable. It's similar to the formatter in that it's very fast — orders of magnitude faster than ESLint. And again, no config — we're pretty opinionated about it.

Let's also visit denodoc, which is our documentation generator. Denodoc is yet another Rust crate that is not part of Deno itself.

00:59:02 - Bartek Iwanczuk

It's a separate project. You can go and work only on that project. You can also use it in your other projects, or compile it to WASM and run it from Node. Its purpose is simple: just parse a file and show me the JSDoc that is defined in it. We provide two ways to see this documentation: one is in your terminal, and the other is a nice web interface that consumes the output from denodoc and presents it in a web UI. By the way, you can also write your own interface — one contributor did exactly that and made an offline version you can run directly from your desktop.

So there are a lot more parts in the Deno binary, but we don't have time to go into all of them, and they get even more complex than what we've covered. But I wanted to give you an idea of why these tools are written in such a way that they aren't actually part of Deno itself — why you have to work on a separate library just to use it in Deno later.

01:00:32 - Bartek Iwanczuk

The reason is, again, what Anthony said in his talk: you encapsulate this complexity in separate libraries and draw clear boundaries between those interfaces. So the implementation of the linter, formatter, or doc generator in the Deno executable is like 50 to 80 lines each, because we just consume external crates. And what's great about this approach is that you can scope your interest. If you're interested in, say, Deno's doc generation, you don't have to learn the whole Deno codebase, how to build it, how to debug it locally. You can just focus on a single repository — in this example, denodoc — which is way simpler to develop and contribute to than Deno itself.

The whole idea with our recent work starting from 1.0 is to factor out as many parts of the runtime into separate crates or separate libraries as possible. For example, in Deno we have this browser-compatibility layer — quite a lot of APIs available in the browser that are also available in Deno, like TextEncoder and TextDecoder. We managed to pull all of those APIs out into a separate crate.

01:02:19 - Bartek Iwanczuk

And then we have an implementation of the Fetch API, which is also a separate crate, and we plan to extract more and more functionality. Think about all the APIs available on the Deno namespace eventually becoming separate crates as well. So one can imagine a future where you need Deno but not all parts of it — you'd just grab the web APIs crate, and if you only need network access, you'd grab a Deno networking crate, and compile your own runtime with just those two pieces. This is obviously for the most advanced users. Most of the time you don't want to compile your own JS runtime. But if you do, there is a way — it's not painful, but it's also not a breeze right now. Stay tuned; we're working towards that.

One more thing I wanted to talk about is contributing to Deno. All of this infrastructure can be overwhelming and hard to get into. But if you want to, don't be discouraged — just come by our GitHub or our Discord and try to contribute.

01:03:56 - Bartek Iwanczuk

There's been a ton of new people who started contributing after 1.0 was released, and the general feeling I get from them is that it's not that hard to start contributing to Deno. Even though it's written in Rust, a lot of the stuff just makes sense if you look at it. Of course, you might be fighting the borrow checker or some other Rust quirks for a while, but in general you can dive into some module and get an idea of what it's doing without much trouble. And by the way, it's not only Rust — we've got a ton of JS code and even more TS code. The whole standard library we provide is written in TypeScript. So if you were thinking about contributing to Deno but haven't gotten to it, please do drop by our Discord or our GitHub and let's see what we can do together. Thank you. A bit fast, but hopefully I talked about some of the interesting bits that aren't so obvious when you look at Deno itself. If you want to message me or talk about some project, just drop me an email, and I'll be happy to take your questions if there are any.

01:05:42 - Sylvain Pontoreau / meetup host

Thanks for two great talks tonight. Thank you, thank you for that. We don't have any questions at the moment, sorry.

01:05:53 - Bartek Iwanczuk

Thank you for having me.

01:05:55 - Sylvain Pontoreau / meetup host

You're welcome. It's our pleasure. We don't have any questions right now, so let's wait a couple of seconds and

01:06:06 - Bartek Iwanczuk

we will see. If you're too shy to ask in the chat, just drop me a message or an email.

01:06:14 - Sylvain Pontoreau / meetup host

Yeah, it seems people are shy tonight. No questions.

01:06:18 - Anthony Campolo

Yeah.

01:06:19 - Bartek Iwanczuk

Not many people, suddenly. So we don't have any other questions.

01:06:28 - Sylvain Pontoreau / meetup host

Okay. So if nobody has a question, we're gonna end the meetup. Anthony and Bartek, thank you very much. We need to wrap things up.

01:06:41 - Denoemie robot intro VO

[unclear]

01:06:41 - Sylvain Pontoreau / meetup host

Yes, thank you for that. If you want to say something.

01:06:45 - Anthony Campolo

Yes, thank you.

01:06:46 - Bartek Iwanczuk

That was really nice. And

01:06:51 - Anthony Campolo

as always, the meetup is recorded,

01:06:54 - Bartek Iwanczuk

so if anyone who wasn't there wants to see it, they can.

01:07:00 - Anthony Campolo

You can use it to share it

01:07:04 - Bartek Iwanczuk

with some people for your talk.

01:07:06 - Anthony Campolo

No problem. So yeah, again, thank you everyone. The next meetup will be in two months — I hope we can do it in a physical way, but I think the next meetup will be online as well.

01:07:26 - Bartek Iwanczuk

Yep.

01:07:26 - Sylvain Pontoreau / meetup host

Maybe in December.

01:07:27 - Bartek Iwanczuk

Yeah, maybe in December, at the beginning of December.