Astro with Fred Schott

Episode Description

Fred Schott discusses ESM's impact on web development, the evolution from Webpack to Snowpack, and how Astro's HTML-first approach changes site building.

Episode Summary

Fred Schott joins the show to trace the arc of his work from the Pica project through Snowpack and into Astro, all unified by a bet on ESM (ECMAScript Modules) reshaping web development. He explains how ESM gave browsers native module-loading capabilities, making traditional bundlers like Webpack less necessary during development. This insight drove Snowpack's core innovation: instead of bundling everything upfront, it builds files on demand and lets the browser handle module loading, resulting in near-instant dev server startups regardless of project size. The conversation covers the broader ecosystem migration from CommonJS to ESM, the rise of native compiled tools like Esbuild, and the risks of over-customizing build configurations that lock developers into specific toolchains. The discussion then shifts to Astro, which builds on Snowpack's foundation to offer an HTML-first framework where pages are rendered as static HTML by default, with JavaScript sent only for interactive components through partial hydration and Islands architecture. Fred contrasts this with JavaScript-heavy frameworks like Next.js and Gatsby, arguing that the majority of the web is static content that doesn't need heavy client-side frameworks. The episode closes with a look at Astro's path toward V1, its growing ecosystem of translations and community contributions, and an invitation to join the project's Discord.

Chapters

00:00:00 - The Origins of Pica and ESM's Promise

Fred Schott recounts how his work on Google's Polymer project in 2015–2016 coincided with ESM being ratified, giving browsers a native module system for the first time. He explains that the npm package ecosystem had accumulated significant technical debt, and ESM presented an opportunity to rethink how packages were authored, distributed, and consumed.

The conversation establishes the central thesis running through all of Fred's projects: when the browser gains native capabilities that were previously handled by tooling, everything downstream changes. Anthony notes that ESM was a sea change in web development, and Fred acknowledges that while developers have been writing ESM for years, the ecosystem is only now beginning to ship it natively to browsers at scale.

00:03:00 - The Painful Migration from CommonJS to ESM

Fred frames the CommonJS-to-ESM transition as one of the largest software migrations happening quietly across the JavaScript ecosystem, comparable to Python 2 to Python 3. Chris shares his frustration with packages that have moved to ESM-only while tools like Webpack and Next.js still struggle with compatibility, illustrating the real-world friction developers face.

The discussion highlights how competing ideas about file extensions, folder resolution, and package entry points have created confusion, with build tools like Snowpack, Rollup, and Webpack each interpreting package configuration differently. Fred points out that npm hasn't provided clear leadership on standards, leaving individual tool authors to make inconsistent decisions that compound the migration pain for end users.

00:07:04 - Snowpack's Unbundled Development Model

Chris notes that despite Snowpack's advantages, major frameworks still rely on Webpack, though Vercel's hiring of the SWC maintainer signals awareness of the performance gap. Fred explains Snowpack's conceptual breakthrough: by not bundling during development, it leverages the browser's native ESM loading to achieve near-zero startup times, building files only as they're requested rather than scanning and bundling the entire project upfront.

Fred contrasts this with Webpack's model, where startup time scales with project size because bundling must happen before anything can be served. In Snowpack's approach, pages you never visit during development never get built, making it scale elegantly for large projects. The trade-off is a slightly slower first page load compared to a pre-bundled setup, but the overall developer experience improves dramatically.

00:11:17 - Bundling as a Production Optimization

Anthony asks why bundling is still needed in production, and Fred reframes bundling as what it always was before the Webpack era: an optimization step rather than a foundational requirement. Shipping thousands of raw source files forces the browser into cascading load-parse-load cycles that are slower than receiving pre-bundled assets, even with HTTP/2 and HTTP/3 improvements.

Fred recounts Snowpack's evolution from shipping without a built-in bundler to adding Esbuild-powered bundling in V2, acknowledging that the original bet against production bundling was somewhat proven wrong. He explains that Snowpack still supports plugging in Webpack or Rollup for users who need their features, and that by handling all file transformations before handing off to a bundler, Snowpack strips away much of the configuration burden.

00:15:07 - Esbuild, Babel, and the Risks of Over-Customization

The conversation turns to how Esbuild blurs the line between bundler and transpiler, performing one-to-one file transformations that previously required Babel while being orders of magnitude faster. Fred notes the irony that Esbuild's creator, Evan Wallace, originally framed it as a proof of concept, raising questions about long-term commitment despite widespread adoption.

Fred warns that heavy customization through Babel and Webpack plugins creates lock-in that makes migration to newer tools painful. He argues that the simplicity of Snowpack and Vite's one-file-in, one-file-out model makes integrations far easier to reason about, and encourages developers to resist adding build-step complexity for trivial preferences, saving customization for genuinely valuable use cases.

00:19:02 - Skypack and the Return of CDN-Based Code Loading

Fred introduces Skypack as the CDN-focused exploration of ESM's impact on code distribution, explaining how ESM's URL-based imports revive the old model of loading libraries like jQuery from shared CDNs. The potential benefit is that users visiting multiple sites using the same CDN-hosted dependencies could skip downloads entirely thanks to warm browser caches.

Chris shares his own experience hosting third-party scripts across multiple providers, highlighting how confusing modern script distribution remains. Fred acknowledges that while Webpack and npm eliminated the patchwork of tools like Gulp, Grunt, and Bower, they also centralized everything into bundling, phasing out the CDN model that Skypack aims to bring back with ESM's native capabilities.

00:21:22 - The Cycle of JavaScript Tooling and Default Choices

Chris and Fred reflect on the recurring pattern of JavaScript developers cycling through tool preferences every few years, from the Grunt-versus-Gulp debates to the current Webpack-versus-Vite transition. Fred highlights that Esbuild's intentional lack of configurability may be a feature rather than a limitation, preventing the kind of plugin sprawl that made Webpack migrations so difficult.

Chris argues that JavaScript has historically lacked strong default options, pointing to Snowpack's built-in TypeScript and Tailwind support as examples of sensible defaults that reduce complexity. Fred expresses hope that the next generation of tooling will be built on simpler foundations, noting that when the base tool is complex, developers naturally reach for more complexity to solve problems that the tool itself introduced.

00:25:49 - Introducing Astro and HTML-First Development

Anthony transitions to Astro, framing it as the framework that demonstrates what becomes possible when you build on Snowpack's foundation. Fred describes Astro as an HTML-first tool where every page is written to static HTML rather than being rendered from a JavaScript application, resulting in sites that are fast by default because they ship minimal JavaScript.

Fred contrasts Astro with Next.js and similar frameworks, explaining that while the developer experience feels similar to writing JSX, the output is fundamentally different. Instead of optimizing a large JavaScript application after the fact, Astro starts from static HTML and only adds JavaScript where explicitly needed, making it structurally difficult to build a slow site.

00:29:18 - Partial Hydration and Islands Architecture

Chris describes his experience adding a single React component to an Astro project and being amazed that it worked without the typical React boilerplate. Fred explains partial hydration as the practice of shipping HTML for the entire page and sending JavaScript only for interactive components, like an image carousel, while leaving everything else static.

Fred credits Jason Miller with coining the Islands architecture concept, where interactive components are treated as isolated islands in a sea of static HTML. He explains advanced features like directive-based lazy hydration, where components below the fold aren't loaded until they become visible, dramatically improving Lighthouse scores compared to traditional single-bundle approaches.

00:31:27 - Data Fetching, CMS Integration, and the Path to V1

Chris asks about data fetching in Astro, and Fred explains the dual model: data loaded in Astro components fetches at build time and is rendered into static HTML, while interactive components handle their own browser-side data loading through standard fetch APIs. He notes that headless CMS integration is fully supported, citing examples of Shopify and WordPress-backed Astro sites already in production.

Fred discusses Astro's path toward V1, emphasizing that the milestone is less about a date and more about stabilizing syntax and behavior so an ecosystem of components, themes, and translations can confidently be built on top. He envisions an HTML-first component ecosystem that doesn't force developers to opt into JavaScript prematurely, filling a gap that JavaScript-heavy frameworks have left unaddressed for most of the web's static content.

00:38:12 - Web Components, Community, and Closing Thoughts

Anthony asks about the connection between Astro and web components given Fred's Polymer background, and Fred notes that while web components haven't won the developer mindshare battle, they have broader real-world usage than most people realize. The conversation then shifts to how developers can contribute to Astro through good-first-issue tags, Discord support, translations, and community events.

Chris encourages listeners to simply run an Astro build with a React component to experience the paradigm shift firsthand, and Fred reinforces that 85% of the web is static content that has been poorly served by JavaScript-heavy frameworks. The episode closes with Fred expressing gratitude for the conversational format and reflecting on how the changes he anticipated years ago in the ESM space are now actively unfolding across the ecosystem.

Transcript

00:00:00 - Anthony Campolo

We should be good.

00:00:01 - Christopher Burns

I think I can hear a radio, Anthony. Yeah, it's someone playing an FM radio in your house.

00:00:08 - Anthony Campolo

No.

00:00:09 - Fred Schott

I do hear that, too. You're not crazy.

00:00:22 - Anthony Campolo

Fred Schott, welcome to the show.

00:00:24 - Fred Schott

Thanks for having me.

00:00:25 - Anthony Campolo

Really happy to have you here. We've been fortunate enough to have a lot of really great, accomplished guests on this show throughout the year. We've been doing it, but I really think your work is in a class of its own in terms of its impact on the web development space.

So we have so many cool things to talk about. To set the stage here, there's a range of projects you've been doing, and I think all of it kind of goes back to this Pica project that birthed a lot of it. I actually have the first README here, which was published in January of 2019. It was talking about publishing packages. It says authoring good packages in 2013 was simple, right? JavaScript and npm publish. What changed that made that more complicated?

00:01:10 - Fred Schott

That's such a nice little flashback. Oh, God. It's been a while. This project has really evolved over the years. I've been very lucky to work on a lot of fun things out of this project. You're totally right, it all came out of Pica. That really came out of some work I was doing at Google at the time. I had been on the Polymer project, which was under the Chrome umbrella. This was 2015, 2016, right as ESM was basically being ratified. That's the ability to use import and export in your JavaScript and essentially give the browser a way to load code that was much more in line with how modules were working in Node and all this history that we'd gained from Node, but built into the browser.

So the whole project was really like, what changes when the browser, and when JavaScript itself, has a module system? Packages were kind of just the natural first place to look, seeing all the things that npm had to do to get around limitations and just how npm had grown. Kind of just grown for the sake of growth. There were a lot of decisions made to get numbers up. Some things were just a little bit more relaxed than they probably should be for a module system of a million packages.

So there's new technology, a lot of old debt, and the package ecosystem was really where I first wanted to take a look in this space.

00:02:17 - Anthony Campolo

The ESM stuff is what really attracted me to a lot of this work in the first place. It's been great because you are a prolific podcast guest, and I love listening to lots and lots of podcasts, so I've been able to follow this work that you've been doing very closely for probably two years now. And as more of these projects have started coming out, like Vite and other things that are very heavily based around ESM, it gave me the context to kind of understand it.

It was a sea change in how web development works that we have been living through and will continue to live through for a long time. So I'm curious, having been through this journey, do you feel like ESM has kind of gone mainstream yet, or do you feel like we still haven't reached that tipping point yet?

00:03:00 - Fred Schott

This moment right now is essentially what that packages problem was built around. The idea was that Node was going to have to make this switch and it was going to be really hard. Now, if you're working in Node today, it's common to be like, oh, this package only works in the old system. It doesn't work in ESM, but I need the old system because my company hasn't upgraded yet. There's this idea that it's a huge software migration that we're all undertaking, kind of quietly without realizing it.

So that was the idea behind that. A lot of what I'm lucky enough to get to work on is looking ahead a couple of years and trying to guess either where the puck is going, or where I want it to go, or where I think it should go, and trying to build around that future.

We started Snowpack and Evan had started Vite around the same time, but both looking at what happens when, okay, take away the packaging question, just like a build tool for web developers that isn't trying to do all the heavy lifting of Webpack because actually a lot of that is based on limitations. If you give a bundler the ESM language, it can actually do a lot more stuff close to the browser. It can kind of offload a lot of responsibility, like module loading and hot updates and all this stuff around how your dev environment works. That not only is easier as a tool to build, but it's also faster.

Vite and Snowpack were both kind of out of that. Okay, let's look at another problem with this technology. That was the nice thing about Pica. It had this guiding mentality of the technology is going to change a lot. We're just really early. I think it's hit the mainstream in that we've all been writing this code ourselves for five years now, people at different stages. It's been out in the world being written for a long time. It's only now that we're starting to actually ship it to browsers, now that they've caught up.

00:04:33 - Christopher Burns

I am going to hold up my hands and say I'm not really much of an expert on module systems, so I know what they are, and ESM is this thing that we all should be using. As you said, it's a transition.

My biggest gripe I've had so far with it is actually really, really recent. Sindre has updated all of his core modules to only support ESM. So obviously, if you're a maniac that likes to always have all of the version numbers up to date and you use something like Next.js or Webpack, you'll update it, you'll go to run it, and it goes, Webpack just doesn't know what to do, because I don't think it compiles ESM. Obviously you're much more of an expert on this than me, but I just went, what do I need to do? It looks too complex. I'll downgrade for now. I'm waiting for someone else to fix my ESM problems.

00:05:27 - Fred Schott

It's the migration that I think needs to be given the most credit. Take away that the language looks like this and the old system looks like that. Even from that, it's like we are moving a million packages quietly from a totally lower level. I can't think of one that's been done, or maybe like Python 2 to Python 3. There are horror stories of people trying this.

I think giving the people a little bit of credit that it hasn't exploded, but it is still really painful. I've had to upgrade one or two things where it's like, well, this package needs this, but this other one needs that. I can't make both of them happy at the same time. I see a lot of people on the Node team and the people who are trying to push this migration are also trying to limit some things in the new system, trying to fix old mistakes, things that they feel like they shouldn't have allowed in the older require/module.exports world, like importing by a file path without an extension.

I can't remember where that landed, but in some projects you need an extension. There's no smart folder lookups. There's no like, if you miss the extension, we'll add it back for you. TypeScript is like, no. Everyone's just kind of in this weird muck of competing ideas because so much has changed at a lower level. Everyone's trying to figure it out. Webpack does support ESM, but again, at that package level, if the package isn't set up to point to it, the package author had to do the right thing, and Webpack had to do the right thing.

Not to go on a total tangent, but as someone building a build tool, right? Snowpack and Rollup and Webpack, where it really gets tricky is they all have different ideas of what a package should look like. Snowpack says module, right? That's ESM module, that's the place. But Webpack will say, well no, that's for Node-specific module, you really want to put your like... Everyone has a different idea of where that should go. And there isn't a right answer because there's no leadership there from npm.

00:07:04 - Christopher Burns

I've quickly played around with Snowpack before, not too much on it because it's something I'm sure we want to talk about, but I think the biggest thing with it is that it's so good. But Next.js and all these big frameworks still use Webpack, so you're kind of stuck.

00:07:19 - Fred Schott

I think Vercel just very loudly hired the lead maintainer of SWC, which is a Rust-powered compiler. I wouldn't count them out there. They are aware of how they're falling behind. So Snowpack, one of the things that we're able to take advantage of is this idea of a native compiled compiler, Esbuild being the hottest one right now, where instead of a Babel or JavaScript build tool that might be a little slower, you get this Go-powered binary that's just going to, in a millisecond, build your files.

So there was this kind of, there's a name for that, where it's such a seismic shift that all of a sudden build tools that use that are able to ship 10x or 100x faster builds, and tools that don't are kind of being left behind a bit. Next.js has so many users. If anyone's going to be left behind, it's not them. But they clearly see, like, oh no, there's something big happening. We need to play catch up.

00:08:05 - Christopher Burns

And then there's going to be a big migration path.

00:08:08 - Fred Schott

The user is always the one who has to pay the price of upgrading. I know Webpack is trying to do it. Their answer is like, if you're using Webpack-specific, or sorry, Next.js, if you're using Webpack-specific configuration, start to not do that. That isn't much of an answer. I'm sure they'll have to figure it out. But again, it's the user who kind of has to pay the migration cost always.

00:08:27 - Christopher Burns

Why are we talking about things like Webpack? I think we should really dig into what Snowpack is because a lot of frameworks, as we said, still use Webpack. But Snowpack is this really, really cool thing. And if you've been adventurous to try something very low down, Snowpack is really, really cool.

I've had a little bit of usage, but I'm sure you two have had a lot more. Obviously, let's talk about the big benefits over something like Webpack, just straight out of the box. How is Snowpack better? Zero configuration changes.

00:08:59 - Fred Schott

Out of the box. We can even just talk about it conceptually a bit, and then we'll kind of reel it back. The conceptual difference is the main difference, and everything flows out of that. In Webpack, Webpack is a bundler. The flaw of building everything for the last decade on top of bundlers means that you're stuck in the way of a bundler. You're thinking in bundlers. You're bundling during development, in production, always bundling, bundling, bundling.

The problem there is as your project grows, you're now having to, even just to start up the dev server, do some bundling. So you're actually scanning your files. You're doing this work that's based on how big your project gets. So as it gets bigger, your startup times get slower and your dev environment gets slower.

Snowpack, by not having to do so much bundling, we're able to leverage that ESM module thing where it's like the browser doesn't need me to bundle anymore. I can kind of just ship this code and it actually knows what to do. It'll load things. It'll fetch things from my build. It lets the browser do a lot of that loading.

It's not that we're a better bundler. We just actually don't do bundling during development because now the browser has the tools to load code on its own. So you start to see bundling more as it was, a workaround for a limitation where the browser couldn't load code. So let's bundle it and send it to the browser, and the browser will just get these really dumb files. Webpack is telling the browser what to do. Now it's kind of a symbiotic relationship where the browser will load the page and then Snowpack will give it back its HTML and some of the JavaScript files. The browser will see those JavaScript files, see what they need to load, and then it'll ask for files, load them, see what it needs, load more, and stream basically down what it needs to load the page.

The result of that is zero startup cost. It's essentially an instant startup because it's that first page load that is going to kick off the building and loading only of what's needed. That first page load is definitely slower than Webpack because you didn't just spend a minute starting up. But the idea is that no matter how big your project is, you're still only, on that first page load, going to build the files you need. So you will essentially just never touch those files on that 404 page or that about page that you don't really use unless you're building it and seeing it and trying to work on that.

In development, it's only once you go to build that you're actually going to touch that file and actually build it, because obviously in production you'll need it. But during development you get this, it scales beautifully with any size project versus the existing ecosystem will really slow down as you grow, which is one of those problems. It's really hard to see at the start. And then two years down the road, you've kind of dug yourself into a hole.

00:11:17 - Christopher Burns

It seems like a really oversimplification, but when we talk about bundling and compiling, to me, I always imagine it as the end product cannot understand the beginning product, and it needs to slowly go down the pipe to understand it by adding things, then translating things. And then when we talk about things like Snowpack, it's just making that a lot smaller.

00:11:40 - Anthony Campolo

Should call it a game of telephone. Is that the term you're looking for there, Chris?

00:11:44 - Christopher Burns

There we go. Maybe.

00:11:47 - Fred Schott

I'm sorry. I'm not sure what the question was. I missed the question there.

00:11:50 - Anthony Campolo

You had mentioned that you still need to bundle in production. And so for me, the way I think about this is you want to minimize the amount of requests that have to go over the wire, because latency and TLS handshakes and all this stuff. You don't want to have to make a request for every single module, because that would just be ridiculous.

So is that kind of why you still need to bundle for production, or are there other reasons as well? And just what considerations go into bundling for production versus development?

00:12:18 - Fred Schott

That's where, in this new world, a new way of thinking, you start to think of bundling more as kind of what it always was originally, before the Webpack and npm explosion, which was it's an optimization. So yeah, if you ship your raw source code to the browser, especially on large projects, that's going to be thousands of files. The browser is able to kind of parallel load them, but it's still like instead of getting everything at once, loading it and parsing it and being done, it's reading, loading, reading again, loading again, and juggling all this. It's slower than if you just bundle a lot of it.

Snowpack was trying to figure out, do we still need bundling? And the end result was if you can do it in a way where the user doesn't have to worry about it so much, now that it's a production optimization, it's a little bit more invisible. It's a little bit less complexity that the user needs to understand. In general, it's still the right call. It's nice that we get to push it down to the end versus building everything on top of a bundler.

But even with the advances in HTTP/2 and HTTP/3, these newer network protocols that let you do more loading in parallel, it's still a good idea to bundle even simply to optimize for that user's experience.

00:13:23 - Anthony Campolo

Yeah, I keep hearing about HTTP/2 and how it'll fix all this, but everyone's still on HTTP/1.1. For the most part, the internet is always slowly migrating in different pieces. Slowly but surely.

00:13:35 - Fred Schott

H2 is supposed to fix everything and H3 was supposed to fix everything. I'm sure H4 will also try to fix everything. It does keep getting better. But yeah, that was a lot of my hope with Snowpack. My bet was that this isn't required anymore, and I was more or less proven wrong. It's still nice to have at the very least.

00:13:50 - Anthony Campolo

And I know that Vite brings in Rollup to do the production build, and I believe Snowpack, you are still using Esbuild to do that, is that correct?

00:13:59 - Fred Schott

Snowpack started with no bundler built in because that was the bet. We were an experimental tool. We weren't even V1 yet. Let's push the boundaries a bit. We originally were like, you shouldn't, if you want to bundle, you can, but again, it's a production optimization. You do the work. We're not going to do that.

It sounded really cool at the time. Even now, a couple of years later, it's like, yeah, that's a little alienating to people. You do the work. Because what was the bet of this won't be needed? Actually, it's kind of needed.

Around V2, we shipped an Esbuild-powered compiler and bundler, getting all the benefits of a literal Go binary doing your bundling for you instead of Webpack. It's really fast. It's really nice. But again, it doesn't have all the bells and whistles of Webpack. Esbuild is a much, much, much newer project.

So if you still want to use Webpack, you want to use Rollup, we do give you hooks to essentially bring in a plugin that will connect that bundler for you. So you do get a little bit more of a mix and match. Snowpack is doing all the work of building your site so that by the time the bundler gets it, it's just JavaScript and CSS and HTML, so you don't need all the configuration. We do kind of strip that out from the bundling stuff.

00:14:55 - Anthony Campolo

Yeah, it's hard with these tools because as we're moving from the old group of technologies to this newer one, it's hard to know which things are being replaced by which others. And so, as you say, Esbuild is not really a replacement for Webpack. I would usually tell people that Snowpack and Vite are meant to be replacements for Webpack. Then you have things like SWC as well, which is kind of almost like a replacement for Babel. And then again, the question becomes, okay, well, what is Babel versus Webpack? Most people use Babel and Webpack together and don't even realize where the boundaries are between those two tools. So where does the Babel stuff come into this story?

00:16:07 - Fred Schott

There's one answer there, like the lines have been blurred again, so we're kind of trying to figure that out. Esbuild, the way that we use it most, is as a one-to-one file transformation. So give us your, you know, like App.jsx, right? And it's a React file. It's a JSX file. And Esbuild is just not even going to bundle. It's just going to transform that to JavaScript.

Esbuild is a bundler, but it's also doing this thing that you would have used Babel for. At the same time, it doesn't want to be as configurable as Babel. What do you call that? I don't know, it's a really cool tool for sure. It's so fast that everyone has just kind of adopted it because it's so fast.

But to make things even more complicated, Evan, the CTO of Figma, who actually built Esbuild, is very explicitly like, I want to prove that this is possible. I don't know if I'm going to work on this forever. So at the same time you're like, how do I bet on this in the future? It's a proof of concept by the author's own admission.

00:16:40 - Anthony Campolo

That's interesting. Yeah. Evan Wallace is his name. I hadn't heard that, actually. That's good to know.

00:16:43 - Fred Schott

I probably am overstating. I think that's how he started. Like, how do you, as a CTO of a major company, rationalize working on a bundler?

00:16:50 - Anthony Campolo

People probably want to give him a bunch of money to work on Esbuild now, but who knows? Open source is so, so weird. And you are building kind of a company around this stuff as well. And I know that Skypack is a big part of this story as well.

So where does the non-open source parts of this come into play? Because Skypack is a CDN, right? And so I always kind of thought that that was meant to be the connection point between a lot of this stuff, because once you can leverage ESM, you can import URLs straight into the browser. So is that also the idea here?

00:17:25 - Fred Schott

Skypack was all coming back to this idea of Pica, looking at the ESM technology and like what changes packages were. Some of the first exploration Snowpack was like looking at build tools. Skypack was looking at what code loading and code distribution look like in a world of ESM.

The nice thing about the ESM syntax is the browser loves URLs. It loves loading things from URLs. So there's actually no difference between a relative, like, hey, load this JavaScript file that I've hosted and shipped with my website, versus, hey, load this totally external file from a third-party CDN.

It's a very old-school callback to when you load jQuery from a CDN. If you ever worked at that time, there used to be this idea of third-party CDNs actually hosting a lot of the code you would need. You'd get this really nice, like, they had done the work of scalability and distributing this thing. And then there was this kind of unproven but still conceptually sound theory of like, my user will probably have seen this URL before somewhere. Some other site has loaded jQuery, and some other site is probably loading jQuery from Google's CDN or jQuery CDN. So if I load it from there and the browser's already seen it before, then my user doesn't even have to hit that request. It can skip that part of the site.

Imagine how much faster my site would be if all my dependencies didn't have to get loaded at all because the user had been visiting different websites, seeing these files, and now was coming to me with a warm cache. That is an older model that is really phased out in the world of npm and Webpack, where everyone bundles their own stuff. The bet with Skypack was that's going to come back if we can distribute your dependencies via that sort of import-by-URL, and we'll do all the work of optimizing them.

You'll also get that extra benefit of if I load this from my site, there's a chance that user has already warmed their cache from some other site. The more people use it, the faster the whole network gets. That was the big bet with Skypack.

00:19:02 - Christopher Burns

Using script tags was how I started coding when I did web dev. If you've been around long enough, you're like, oh, I pull my jQuery, my Bootstrap CSS. You put all your files in the header and it'll all be fine. Everything will load. Or you need to use a carousel, find the carousel CDN, stick that in, and then it just kind of disappeared.

It was this whole different way of thinking. You'd kind of use Sass, so you already knew about some pre-compiling to get all these benefits. And then it took it to the next level by compiling it completely.

I think the only place it still really exists, if you're in this modern JavaScript world, is if you're at a company that needs to compile some kind of script that then needs to be loaded on a third-party website. Normally that script's in CommonJS. This is a really interesting part of the internet that caught me by surprise. You have to make a script for your own company. I need to install that script onto any website and it needs to do X functionality. We see a lot of chatbots and stuff like that. Where do you actually host that script?

So you've made your CommonJS script. Where do you host it? And I've done this. I've hosted on four different providers because it's super confusing. The best provider, surprisingly, was Vercel. This script that I have for my company that goes on to all these other websites, the best place to host it was Vercel because of the cache management, and it just did all of that. But it's super confusing. That area of the internet of like, I need to make a script that runs on every other person's website.

00:20:38 - Fred Schott

I feel like I'm always in this position where so much of what Snowpack is doing is trying to fix problems in Webpack, but you can't do that without acknowledging everyone in the world uses Webpack or some other thing trying to be Webpack. It got rid of that exact problem where it's like, I just run it all through this one tool and it's going to do everything for me.

Gulp and Grunt, I used to spend so much time on the Bower ecosystem. There's all these pieces that I had to fit together with tools that never really worked, or they were written in Java, so I couldn't debug them. And all of a sudden, npm, Webpack, Babel, this class of tools that now is seen as maybe slower, older school, but they solved a lot of problems even in that story.

I don't know if web developers are masochistic. We like doing things that are horrible because what you've just described, I have flashbacks to stuff I've done that's not too different.

00:21:22 - Christopher Burns

I do remember the battles of developers, Grunt or Gulp. You're like, well, I only use Gulp. Well, I only use Grunt. And it's normally you need a certain thing, like uglification. Okay, uglification Gulp, uglification Grunt. You move to a different agency and they only use Grunt and you're like, I've got to learn Grunt. We just moved away from that.

It's so funny because I think we tend to do it every five years. This next cycle of now, this is it, then.

00:21:50 - Fred Schott

This one thing I like about Esbuild is that it's really like, and again, maybe it's because of some opinions by its creator, Evan, or maybe it's just like they're trying to fix something that they saw as a problem in Babel, but it really doesn't let you configure much. You can tell it how a JavaScript file should be loaded, or a JSX file, or even a Svelte or Vue component. But if you're like, oh, I want to add this thing so they don't have to do this. Babel was so customizable that we got this world where it's like, and we saw this as Snowpack. How do I move my site to Snowpack? The answer was, it totally depends what you've done to your site. That's custom because we're built on Esbuild. We're not going to support your Webpack plugins. We're not going to support that Babel transform. Or if you are, it's a couple extra steps.

It was really like the more customization you did, which I don't think was ever seen as a bad thing, I feel like is becoming more of a bad thing where if you're doing it just because of personal preference, I think we now see, and I hope we're talking about this enough, you're kind of digging yourself this hole where you're locking yourself into that one ecosystem.

When Webpack was the only thing out there, fine, who cares? But now, in this world where we don't know what the next tool is going to be, will it be Snowpack or Vite or something totally different? The more you customize for one system, the more you lock yourself into that system. And I think that's a risk going into the next couple of years.

So if there's a good reason, like some CSS or JS library that you really want and it needs a Babel plugin, go for it. Totally fine. But if there's some personal thing where it's like, oh, I just don't like to do dot, dot, dot, and instead of changing my opinion about this totally non-consequential thing, I'm going to write a Babel plugin or Webpack plugin, that's where I would definitely recommend people hold off, just because you don't want to lock yourself in for a bad reason. For a good reason, totally.

But we have seen a lot of people struggle to adopt, whether it's Snowpack or Vite, because they're so ingrained in Webpack configuration world that they've dug this hole that they can't get out of.

00:23:31 - Christopher Burns

A quick side point is, I think JavaScript's lacked strong default options for a very long time. And that's what you just said with Esbuild. It doesn't let you customize much.

If we look at things like Apple with privacy filters, I think it was like 80% of people choose not to be tracked by default. Obviously that has caused carnage elsewhere. But it's this thing of how many developers really need to understand everything happening underneath, how they need to customize it, or do they just want to load TypeScript or Tailwind?

I know the two big things are things that Snowpack has built in out of the box, so no customization needed. You just start using it.

00:24:13 - Fred Schott

If I have any hope for this next generation of tooling, I think it's going to be based on less complexity, so you can do a little bit more. I just said don't customize too much, but I think adding support for X or Y... Now I'm about to contradict myself. I don't know what to say about that. I realize I'm about to say customize away. It's so much easier. But that's the exact problem.

What we didn't understand about building on top of Webpack was that we were building on top of a very complicated tool. If you're going to build Create React App or even Next.js, when your foundation is this level of complexity, you've kind of already swallowed that pill. So you're going to reach for things like, how do I fix this? Oh, I'll add something to my bundler. I'll add complexity at the build step because it's already so complex. I have to use it all the time. I have to use it during development. I have to use it in production.

A lot of libraries in the last couple of years came out at that time where, oh, I have to add a plugin for Webpack. Okay, fine. I don't care. I know a lot of CSS and JavaScript libraries do that. But once you take away that foundation, really the whole thing just gets so much more simple. What we're able to do, again, we don't have to bundle your whole project ahead of time. When you load a file, we build it and we serve it. It's such a simple model versus like everything's getting bundled and you don't know what Webpack is doing. It's this nest that, every time they upgrade, it takes them a while because there's so much complexity there.

But Snowpack and Vite have this really clean model of build a file, get a result, ship it to the user. It's a one-to-one file transform, which is just so much easier to reason about. So to add something like Sass support or Tailwind support, you're just not worried about all these side effects and what might go wrong when I build this file, what should the result be? If I can think that way across every file, that's a really simple model that is only going to benefit these integrations and these tooling authors.

00:25:49 - Anthony Campolo

And now that we've set the context with Snowpack, I'd like to get into Astro here because Astro to me is a tool that's trying to demonstrate what you can do with Snowpack. When you start building on this new sort of tool, what sort of frameworks or libraries can we create with it?

So I think Astro will be considered a framework in your eyes. It's not really that important whether it's a library or framework, but it's a way to take things like JSX and transform them into HTML. And I know partial hydration is a really big part of it. So what is your high-level pitch for Astro? How does it compare to other things in the ecosystem?

00:26:28 - Fred Schott

I mean, it's definitely what I'm most excited about right now. I love this format because I feel like we've said a lot of foundational language. I can now totally blow by this. I think you put it exactly right. That was very intuitive. When you build all this foundation-setting and you have your Snowpack and you have ESM module loading, Astro is like, what is possible now? It's probably the least ESM exploration of anything we've done, and it's much more like, now that I have Snowpack, what can I do with it?

So it is essentially a way to build websites that treats every asset as a top-level asset. That's like the Snowpack. There's a lot more going on there, but the Snowpack angle is like a bundler. Everything is JavaScript, right? So even that CSS file, you're going to have to bundle as JavaScript somehow or throw it off as an asset to come back later. Bundlers are really JavaScript first.

What's awesome about Astro is that we're actually HTML first. Instead of building everything to JavaScript and bundling, and that's your build tool, we actually take your site and essentially every page of it and write it to HTML. So that sounds a lot like what Next.js is doing. But what's really nice about this is the actual foundation of what you're building isn't a React or a JavaScript app. It's much more like an old school templating language where the actual foundation is the HTML instead of like, okay, how do I optimize this big JavaScript application that I built? It feels a lot like a JSX, but it's actually writing HTML out.

So the end result is much more like we'll strip away anything that's not needed. We'll optimize it around HTML as the base level, really static, really fast language. You get a built-in solution, just conceptually, to the problem of I'm shipping all this JavaScript and why? What's going on? My site's slow. I don't understand it. It's really hard to build a slow site with Astro, and that's because of that conceptual HTML-first identity.

00:28:11 - Christopher Burns

I watched one of your talks where you said that Astro is HTML but with superpowers instead of JavaScript with superpowers. It was really, I think it was the React Conf talk. I gave it a go. What really blew me away was not just using HTML, but when, obviously, in your demo you show adding a React component and having every bit of React that you might need to make something functional in this encapsulated file without running anything else. It just completely blew my mind.

That's me being 100% honest. I'm just writing HTML. I need to write this one price slider. You don't need to install React. You install, say, the React package. You need, say, React Ranger, right? The slider, and it's done. It just works. Honestly, I know I said it, but it completely blew my mind. That was it. That was just React without React. As you also say in that talk, how does it even work?

Truly.

00:29:18 - Fred Schott

It blows my mind. We're building our doc site in Astro. We're building our website. I use Astro a lot now and it constantly blows me away, which is a cool feeling.

What you're describing is partial hydration. That's the idea that essentially the whole site doesn't have to be JavaScript. We can actually ship HTML and then only hydrate the parts of your site that need it. So like a React component, right? Most of your site being static HTML. You got your layout. You got your header, your body. But then all of a sudden you have an image carousel that obviously needs interactivity. So instead of shipping JavaScript for everything and then only using a small bit of it, the idea is ship HTML for everything and then only send JavaScript for the components that need it. Only send JavaScript for that carousel and think of it as this isolated component on the page. With a developer experience, it still feels like you're building a full kind of application.

00:30:02 - Christopher Burns

And this is a concept that you call Islands architecture.

00:30:07 - Fred Schott

Yeah. So if you take that to the extreme, that's Islands architecture. And I actually didn't coin this. I think Jason Miller did, and he might have actually stolen it from someone else. He gives credit, I think, in his blog post or on Twitter somewhere, but not important. The idea is take that idea to the extreme. Build your site on top of this idea and think of your interactive components as islands on the page in a sea of static HTML, basically.

So again, instead of a world where you've bundled everything, the bundler being the most important thing, we live in this world where it's actually HTML being written, components being injected instead of having a top-level app that's JavaScript. Each one is an isolated island. Let's say you have a really lightweight header and a really heavy component that's like a carousel. These are all going to load and render in isolation. So that header will kick in much, much quicker while not being blocked by this image carousel that might be heavier and have a lot more weight to it.

You can also do some really cool things. If that image carousel isn't even above the fold, you can actually give it a special directive that will basically say only hydrate this once it becomes visible. So if it's below the page, we aren't even going to kick that off. That'll just, when you scroll there, if you scroll there, we'll load it. But that Lighthouse score that's based on first page load, it won't even, it'll be like it doesn't even exist. And that's very different from the current way of doing things, where your whole page is thought of as one bundle. So no matter where it is on the page, you're going to load that for the user, and it's going to block that thing that you actually need. That's critical because it's all in one bundle.

00:31:27 - Christopher Burns

My biggest question after seeing that initial counter, and building my own counter and seeing my own React component, was: what about data fetching?

00:31:45 - Fred Schott

And I think this is where we get into we're still pre-V1. It's still a beta. We're still figuring this stuff out ourselves. So to any listeners, if this sounds interesting, come join us. It's a really interesting problem. Right now we lean a lot on it depends, which is what we're trying to get rid of for V1 and actually give some opinions about it.

But right now, if you do it within an Astro kind of HTML component, it's going to load it at build time. So it's a data fetch that your user will never see. It'll get loaded. We support fetch and async/await, all these nice things in your build step so that by the time it gets built, it's already basically rendered into the template. That works for things that are generally useful because it's being shipped to everyone.

But if they're saying user-specific, like a user avatar in the header, then you're going to need to send it as a React component, and that React component is going to have to do some data loading. So that's where it's just basically, and we support Preact, Vue, Svelte, it's basically just however that component ecosystem does data loading. It's in the browser, fetch is pretty much standardly available now. We kind of lean on that a lot. It depends is kind of the answer there. But it's all possible. It's just on you, basically, understanding is this interactive or isn't it. And if it's interactive, I need to think of it as running in the browser and rendering in the browser.

00:32:52 - Christopher Burns

The demos you used in the talk was that you showed fetching Pokemon.

00:32:57 - Fred Schott

Yeah, a lot of our demos are Pokemon-based.

00:32:59 - Christopher Burns

I was going to say, in the Pokemon demos, you're fetching data after client load. My big question was, when I last checked, I didn't know if you could use some kind of headless CMS with Astro. Is that something that is supported, or going to be? Or maybe not? Because I think that's the interesting question. If someone asked me right now, I just want to build a marketing website super, super fast with my HTML knowledge and some React, I'll say Astro is probably what you want.

And then if they say, oh, it also needs a CMS. What kind of CMS? Is it a markdown CMS? Yes, Astro again. Is it headless CMS? I don't know if it is supported so far.

00:33:48 - Fred Schott

That's one of those things that I'm most excited about, is more examples, more tutorials. But the answer to all those questions is yes, Astro can do it. Cassidy of Netlify has been doing a lot of Shopify e-commerce sites with Astro. Chris Coyier built basically a CSS-Tricks in Astro that's talking to a WordPress CMS.

So yeah, it's one of those things where it's like, if it's being loaded as a part of, like, what pages do I want to generate? How does my site, like, I actually want to talk to CMS and generate pages from that CMS. That is totally within the world of Astro, fetching that data and creating pages from it. So definitely a top-level thing we want to support going into V1. There are a couple of examples. We just haven't brought them all together yet, which is definitely on us.

00:34:26 - Christopher Burns

It sounds stupid, but a lot of marketing websites are built in Gatsby and Next when 90% of the features are not needed. It sounds like Astro, once all the documentation's together, may really be a good candidate for that if you want to use React but also just build a simple marketing website, not a dashboard.

00:34:49 - Anthony Campolo

This is a good segue. I want to get into the comparison page you have on your docs, which is really fantastic. I wish that every project had this, which is Astro versus X. You have comparisons with Jekyll and Hugo, so old-school static site generators. You have stuff like Docusaurus and VuePress, and you have Nuxt and Next, Eleventy, and SvelteKit.

Now the one that I do think is missing here that should be here is ElderJS, because ElderJS is, I think, one of the only other ones out there that actually does do partial hydration. But the main takeaway when you read through this is that you are just going to get huge performance hits for things like Docusaurus and VuePress compared to these more partial-hydration-based models. What do you think? Is anything else that might be comparable to it? The only other one that I know would be Slinkity, which is very, very new, but is taking Eleventy and adding Vite to try and create a similar experience to Astro.

So do you see other projects that are sort of taking influence from this idea?

00:35:47 - Fred Schott

Yeah, Slinkity is awesome, actually.

00:35:50 - Anthony Campolo

Ben Holmes.

00:35:51 - Fred Schott

Yeah, that's a great project. It's very aligned with what we're doing. I think it uses Snowpack. Maybe they switched to Vite.

00:35:56 - Anthony Campolo

He did. Yeah, like just a week ago. I'm kind of working with him a little bit. I just wrote the first blog post about it yesterday.

00:36:02 - Fred Schott

Oh, nice. Ben's great. The launch video for that I think was the most creative, unique thing I've ever seen, where it's like him in front of a whiteboard. No flashy animations. I love that. Ben's great.

Yeah, that's totally aligned. It's that same idea. I think it's Islands architecture. He might even call it that. You're using Eleventy, so you're getting that HTML generation and then it's the components on the page. I think he uses shortcodes, so he hasn't gone as far as we did into like, no, we want you to use something that feels like JSX, but that same idea of you're thinking in HTML first and then adding hydration, adding interactivity where you need it.

I don't think that's public yet. I'm waiting to play with it.

00:36:35 - Anthony Campolo

So technically it's supposed to be released tomorrow. But since I wrote the blog post, it kind of was like a little bit of an early launch, so I kind of jumped the gun a bit. I have a bad habit of doing this. I get too excited and then, like, ship it, just ship it.

00:36:48 - Fred Schott

Yeah. No, I think that's totally, it's exciting to see more things that just aren't JavaScript as the default built-in thing. I think once you start thinking in Vite or Snowpack, it opens up your mind a bit to seeing how this would work.

00:36:59 - Anthony Campolo

Where do you think Astro is going in the future? You've mentioned V1 a couple times. Is there any sort of very general timeline? We work a lot on Redwood here and our V1 story has taken a very long time as well. So we're very sympathetic to that.

But I'd be kind of curious how you're thinking about this progression and when you're going to think of it as kind of being something you want to really advertise as, like, this is production ready. We want this to be like a V1 type thing.

00:37:22 - Fred Schott

Yeah. I don't even want to commit to a date. We're trying to get there soon. I think what we're really looking at is things that would break an ecosystem. Because I think what we're most excited about is what components would you build for Astro? What themes? Our V1 is really limited to what is something we want to commit to as a stable syntax, a stable behavior, so that you can then confidently build on top of it. Because that's what I'm most excited about.

The ecosystem is always the most important thing in a project. The React ecosystem is great. Vue's ecosystem is great. They kind of define the project. So I think this idea of what if there was an ecosystem that was HTML-first, I can't think of anything that exists like that today where we let you use React components, we let you use Vue or Svelte or anything, but at the same time, what if there were these HTML-first components so that you weren't opting into JavaScript before you needed to? You weren't having to make that decision too early. You could just use something that was designed for HTML.

00:38:12 - Anthony Campolo

It's so funny that you were originally working on the Polymer project. That's kind of how this episode started, because why wasn't this web component?

00:38:21 - Fred Schott

Yeah, I've gotten that before. It's like, it sounds like you're just trying to do web components, but different. It's like, no, that's not what I'm trying to do.

00:38:27 - Anthony Campolo

Like, why did web components fail? Why did it not solve problems? Why did it not become adopted? I have no real context there because I didn't kind of live through it, but I'm very curious about that from a historical angle.

00:38:38 - Fred Schott

That could be a whole episode of its own. You should talk to some people who worked on it. I worked on that a while.

00:38:42 - Anthony Campolo

Give me your hottest hot takes.

00:38:44 - Fred Schott

I would say, I don't think they think that it's failed. If you look at how many sites have a web component on the page today, it's a higher number than you'd think.

00:38:51 - Anthony Campolo

I'll have to look into that then.

00:38:52 - Fred Schott

But as a PR kind of fight, it's hard to say they won that fight, but it's out there for sure.

00:38:57 - Anthony Campolo

You got any more questions, Chris, before we start closing it out here?

00:39:00 - Christopher Burns

I think you answered my question about the headless CMS situation, because that was my biggest head-scratcher with Astro. If that box was ticked, I could see using it for literally 85% of my React projects right out the gate. As in, this is pretty much everything I need right out the gate.

My final question: we do have a lot of listeners that do like to sometimes help out. What are Astro's current pain points? What could you need help with? What potential areas could anyone help out with? Because I know it's very collaborative, especially on your Discord.

00:39:35 - Fred Schott

The nice part about having done this a couple of times, like starting with Snowpack and Skypack, I feel like this is the first time I felt we got our Discord right. There's an amazing community of developers there, kind of helping out on all sorts of things.

We actually just rewrote our how-to-get-involved section, so this is actually the perfect time to ask me that. There's a couple of issues that are tagged as good first issue. We're super supportive of people just jumping in and getting involved that way. Our support channel is getting active as more people use it, so helping out, just being a part of the Discord. Translations are actually something we've been really focusing on. We kind of opened it up for one or two languages and all of a sudden people start submitting more and more translations. So if there's a language that you speak that no one else does, we're kind of open to translating anything at this point, not trying to lock it down too much, and have just seen a ton of excitement in that space. Something that also, as we get closer to V1, we're going to really commit to.

But yeah, I mean, even just to come and hang out. We do events, we do office hours, we do RFC calls that we stream on Discord. We're playing Jackbox games on Fridays. It's a really fun community that I'm really proud of. astro.build/chat is the URL I should give there.

00:40:37 - Christopher Burns

If there's one thing you've heard from this episode that I mean as something you should do, it's just run Astro build and make a React component. It will blow your mind. Trust me. A React component in Astro with none of the wrappers of React. Even using things like useState is mind-blowing. Trust me, you have to see it to believe it.

00:40:58 - Fred Schott

There was a point that I think you made that is really kind of key here, which is for most of the web, it's not like dashboards and chat apps and email inboxes or Facebook. A lot of the web is just static content.

So you said, oh, for 85%, this is what I would use. That's I think exactly the point here is that the 85% of the web, or whatever that number is, hasn't been best served by a really JavaScript-heavy framework. I think they have been waiting for something that's much more HTML-first because most content on the web, e-commerce, a blog, a content site, it's all about getting content, getting posts that you've written, images for a product you might want to buy. You don't need all of that heavy lifting of a heavy JavaScript framework.

You can start from first principles of HTML. I'm just excited by the idea that there hasn't really been anything like this, and that has been an unanswered question for a long time. Slinkity as well, I think, is a great way to look at this, but Astro being a way to solve that problem and build something that's static-first, HTML-first is really exciting to me.

00:41:54 - Anthony Campolo

And we'll include links to all those things we were just talking about in the show notes. And then for people who want to get in touch with you, where can they follow you? Where can they contact you?

00:42:03 - Fred Schott

So I'm hanging out on Discord all the time. Astro Chat, most days I'm in there. But I'm also on Twitter at FredKSchott. That's just my name spelled out. I'm on Twitter pretty actively as well.

00:42:13 - Anthony Campolo

Thank you so much, Fred. It's a real pleasure getting to speak with you. And thank you so much for all this stuff you've built. That's super cool and helping to push the web forward.

I've really enjoyed getting to follow this work for my own development in terms of learning how things are changing, how they're going to be in the future. I find that even if you're not going to use some of these things in production, it's very useful to be aware of them and to know why they are making the choices they do, because it gives you a better understanding of how the web works, what the pain points of the web are, how you should be addressing them in your own apps.

And then maybe after you try fixing that problem yourself, you realize, okay, I'm pretty sure someone already said there's a framework for this. Maybe I should just go use that framework.

00:42:53 - Fred Schott

Yeah, it's really fun. I've been lucky enough to have worked kind of in this space for a while. So it feels like these things that are happening now are stuff that I was like, maybe that'll happen one day. And now, yeah, we're in it. It's a lot of change happening. That's scary, but also exciting.

Thank you guys for having me. This has been a great interview. Thank you. I really enjoyed it. Genuinely.

00:43:10 - Christopher Burns

My very final comment. I actually learned how to color your console logs from the Astro build website, because obviously when you open the console, it's got a nice little message that's also in a bright color. And I was like, how do you even do that? I had to Google it and I was like, you can do that.

00:43:29 - Fred Schott

I think that was Nate Moore who did that. He's also part of the project and has a nice little flair for that sort of thing.

00:43:35 - Christopher Burns

Exactly. It's like, do you even know what console.table does?

00:43:39 - Fred Schott

Yeah.

00:44:11 - Fred Schott

Yeah. Thank you guys for having me. I really genuinely meant that. It was a lot of fun. A lot of interviews are like, all right, so, like, a spreadsheet of just questions. It was fun going down memory lane a bit. And I think it's all good context.

00:44:20 - Anthony Campolo

We're really scratching our own itch with a lot of these. We find that, because our listeners are other developers...