Sync Engines with Dev Agrawal

Episode Description

Anthony Campolo and Dev Agrawal discuss sync engines, PowerSync's approach to client-server data synchronization, and Solid 2.0's beta release changes.

Episode Summary

Anthony Campolo and Dev Agrawal reconnect for a late-night stream, sharing career updates before diving into the world of sync engines. Dev, now in his first DevRel role at PowerSync, explains how sync engines differ from local-first and offline-first concepts, positioning them as tools that eliminate the "network tarpit" of client-server state management. He walks through PowerSync's architecture, showing how it keeps a client-side SQLite database synchronized with backend databases like Postgres or MySQL, offering reactive queries and automatic optimistic updates. The conversation covers trade-offs like initial data sync times and authorization complexity, comparisons with tools like Convex and Apollo, and how sync engines reduce accidental complexity in application code. They then shift to Solid 2.0's beta release, highlighting the removal of Create Async in favor of naturally async primitives, a new two-function Create Effect signature, optimistic stores, and action entanglement. Both topics converge on a shared theme: simpler programming models benefit AI-assisted development by reducing the surface area for mistakes, making these tools particularly well-suited for the age of AI coding.

Chapters

00:00:00 - Catching Up and Career Updates

Anthony and Dev open the stream by workshopping a name for their recurring late-night show format. Anthony shares details about his new role at Atmosera, where he's contracted to teach enterprise developers at a major payroll company how to use AI tools like GitHub Copilot and Amazon Q for tasks such as writing tests, updating documentation, and analyzing technical debt. He mentions the shift to working India time zone hours.

Dev talks about joining PowerSync in December as their first DevRel hire, explaining how his conference talk on sync engines and fine-grained rendering at Local First Conf in Berlin led directly to the opportunity. They discuss the nature of DevRel roles, common skepticism around the position, and how Dev's role reports directly to the CEO rather than sitting under marketing, product, or engineering, giving him autonomy to shape the community effort.

00:05:07 - Sync Engines and Local First Fundamentals

Dev provides a foundational explanation of the sync engine landscape, tracing the lineage from Meteor's real-time data model through Apollo and Relay's client-side caching to modern sync engines. He distinguishes between key terms: local-first refers to a philosophy where user-owned data on the device is the source of truth, offline-first means apps continue working without connectivity, and sync engines are the technology layer that keeps client and server databases synchronized.

The discussion clarifies that sync engines have evolved beyond the local-first niche, with recent efforts focused on making them useful for general application development. Dev explains how client-side normalized caching in tools like Apollo was a step toward having a proper database on the client, and sync engines represent the next evolution where the primary data interaction happens entirely on a local database with automatic synchronization to the backend.

00:11:27 - The Network Tarpit and Why Sync Engines Matter

Dev explains what PowerSync calls the "network tarpit," the accumulation of glue code required to manage API calls, cache invalidation, retries, serialization, and state synchronization in traditional web applications. He argues this accidental complexity exists in every app that talks to a server, not just local-first or offline-first applications, making sync engines broadly applicable.

The conversation connects this to the broader history of web development, with Anthony drawing parallels to his experience with Redwood and Apollo's declarative query model. Dev emphasizes that sync engines provide optimistic updates out of the box, consistent client-side state across multiple queries, and eliminate the need for manual cache invalidation, reducing application code to just UI components and SQL queries against a local database.

00:17:44 - PowerSync Architecture and Code Walkthrough

Dev shares his screen to walk through PowerSync's documentation and code examples. He shows how PowerSync supports multiple backend databases including Postgres, MySQL, MongoDB, and Microsoft SQL Server, with SQLite used on the client side. The setup involves defining a client-side schema, connecting to the PowerSync service, and implementing an upload handler that pushes local mutations to the developer's own backend for validation and authorization.

The walkthrough covers reactive query hooks for frameworks like React, demonstrating how developers simply write SQL queries that automatically rerun when underlying data changes. Dev contrasts this with traditional approaches where developers must manually invalidate caches and refetch data after mutations. He also explains PowerSync's architecture, where a separate sync service acts as a replication node listening to the backend database and streaming changes to connected clients.

00:23:47 - Comparing Sync Engines and Trade-offs

Dev compares PowerSync with other players in the space, explaining how Convex functions more as a real-time server-side database without a full client-side database or built-in optimistic updates. He discusses how PowerSync differentiates by supporting any existing backend database and giving developers control over their authorization and validation logic through their own backend endpoints, rather than owning the write path.

The conversation turns to honest trade-offs of sync engines: initial data synchronization can cause long loading times as client-side databases get populated, authorization becomes more complex when clients can make local mutations freely, and partial sync strategies are needed to avoid downloading entire datasets. Dev acknowledges these challenges while noting that the key question is how well a given sync engine helps developers manage them.

01:04:09 - Solid 2.0 Beta Release Deep Dive

Dev walks through the major changes in Solid 2.0's beta release. The biggest shift is that Create Async has been removed entirely, with Create Memo now naturally handling promises and async data fetching. He shows code examples where fetching API data is done inside a regular Create Memo, and the new Loading component replaces Suspense with different behavior and naming to differentiate from React.

They cover Create Optimistic Store, a new top-level utility that pairs with actions to provide automatic optimistic updates with rollback. Dev explains action entanglement, where multiple actions affecting the same UI area get merged to prevent flickering intermediate states. The breaking change to Create Effect, now requiring two functions to separate dependency tracking from side effects, sparks discussion about how most existing Create Effect usage will be replaced by simpler built-in primitives.

01:17:17 - AI, Frameworks, and Developer Tooling

Dev explains his claim that Solid will be the best framework for both humans and AI. The argument centers on reducing the surface area for AI mistakes by encoding more behavior into the framework itself, so developers and AI agents write less complex code that naturally handles edge cases. He references Ryan Carniato's recent streams demonstrating how Solid 2.0's approach produces simpler, more robust code compared to manually handling async edge cases.

Anthony and Dev discuss practical concerns around AI tooling for frameworks, including the need for copy-as-markdown buttons on documentation sites and whether MCP servers or Skills files provide more value. They briefly preview topics for their next stream, touching on agent orchestration via Kanban boards, Anthony's Autoshow project expanding into multimodal content generation, and Dev's experiments with using Trello to manage AI-assisted DevRel workflows.

Transcript

00:00:03 - Anthony Campolo

Welcome back, everyone, to a special edition. What I call the Web Devs at Night, I think, is what I called it. Yeah. Late Night Web Devs, Evening with the Web Devs. We're still workshopping the name, but how you been, Dev?

00:00:20 - Dev Agrawal

I've been pretty good. It's nice to be back on here on your channels and, I guess, now my channel as well. Yeah, a good name. Workshopping in progress. Devs at Night. I don't know, something like that.

00:00:36 - Anthony Campolo

Devs at Night. Yeah, that's a good one. Yeah, because I haven't talked, I did a stream with Nicky T where I mentioned this a bit, but I haven't mentioned this on my stream yet. I have a new role that I started probably right around the time we did our last stream. Let me see, what was the date that we did that?

00:00:57 - Dev Agrawal

Probably December.

00:01:01 - Anthony Campolo

So I want to get the exact date. It was September 19th.

00:01:05 - Dev Agrawal

Yeah. Oh, September. Wow.

00:01:08 - Anthony Campolo

Yeah, right. Longer ago than I thought. Yeah, so I started in November and I'm working for a company called Atmosera and I'm basically being contracted out to a large payroll company that everyone knows, but I technically can't say the name of, to teach their enterprise devs how to use AI with their enterprise code. So nice. Yeah, it's fun. I think once I kind of finish the contract out I'll have a lot to say about it. Right now all I can really say is that I'm just leading these two-week-long workshops where we walk them through how to use Copilot and Amazon Q and stuff to do honestly really basic stuff, just like write tests and update your docs and analyze technical debt and stuff like that. So yeah, that's been really fun. But I did that for a couple months and then they asked me to do the Indian time zone because they're a worldwide company. They have like four or five India offices. So yeah, now I work from like 11 PM to 6 AM every day, which is pretty wild, but I've always been kind of a night owl, so it works out.

00:02:26 - Anthony Campolo

So yeah, like I just woke up a couple hours ago and I figured if I still want to stream, this would be the time to do it. Kind of still catch people from the Americas as they're kind of finishing up their day, and then yeah, so that's where I'm at with my work stuff. I'm also starting to write blog posts for a kind of buddy of mine who works for a database company that's starting like an AI publication, kind of like how JavaScript Jam was owned by Edgio but wasn't really branded by Edgio. It was kind of treated as an independent property. They're doing a similar thing. So I'm about to start publishing some new AI blog posts, which will be really fun. I'm working on one right now about OpenClaw versus Hermes, which is really, really interesting actually.

00:03:18 - Dev Agrawal

Nice. Yeah, I saw something about Hermes. I didn't read too deep into it, but just on the surface it looked very interesting.

00:03:28 - Anthony Campolo

Yeah. So you have a new job now, right? How long have you been at PowerSync now?

00:03:32 - Dev Agrawal

Yeah, so I technically started with PowerSync in December, so that's when I switched from my old job. But I actually started working in late January. So I took about a month of a break to go spend some time with family back in India. And so yeah, I think it's like March 23rd, so it's been two months almost exactly now that I've spent at PowerSync, and yeah, it's pretty fun. So a little bit of background there. I went to the Local First Conf in Berlin last May to talk about how signals and fine-grained rendering and sync engines are a really nice pair for each other. And it's like the whole talk is on YouTube. It's pretty fun, but it's not a super new concept. It's something that Ryan Carniato has kind of been talking about on his stream almost forever. That's most of what my conference talks are. I take stuff that Ryan talks about and turn it into conference talks. Not a lot of original content of my own or original ideas.

00:04:52 - Anthony Campolo

Is this talk, is the video on this.media? Is that the one you're talking about?

00:04:57 - Dev Agrawal

No. If you just search Local First Conf and my name,

00:05:04 - Anthony Campolo

it should be Sync Engine's Best Friend.

00:05:07 - Dev Agrawal

Yes, Sync Engine's Best Friend: Fine-Grained Rendering. I was trying a bit of a play on dogs. A dog is a human's best friend or man's best friend, something like that. Yeah. But so basically the local-first space, the sync engine space has been getting really, really interesting in the last couple years. There's a lot of players, and PowerSync is obviously one of the many players. They saw my talk. They had been looking to start a dedicated DevRel and developer community education effort for a while now. We started talking a bunch last year around July, and then in November there was SyncConf in San Francisco, which I wasn't planning on going to, but they were one of the sponsors, so they invited me out there. I spent some time with their team. Obviously had a bunch of fun at SyncConf itself in San Francisco. It's always nice to be in SF and try to walk down the extremely uphill and downhill streets there. That's a fun experience. Yeah. So I met the team there. I think we all clicked pretty, pretty fast. And then right after that they were like, okay, we want to bring you on board.

00:06:38 - Dev Agrawal

And they were also very excited about all the work going on in Solid. They wanted me to build a Solid adapter pretty early on, but I just kind of never really got around to it, which I almost feel ashamed about. I've been at PowerSync for two months now, and there's still no official Solid adapter. Yeah, but that's kind of a little bit of the background there. I was very skeptical going in, but throughout the conversation there were a lot of green flags that I saw within PowerSync, the product, the company, and everything, and I was happy to jump in and get back into DevRel professionally.

00:07:28 - Anthony Campolo

What were you skeptical about?

00:07:33 - Dev Agrawal

The usual stuff. You know, DevRel's getting kicked out as soon as something happens within the company. I'm sure no one there in the DevRel position has experienced that before, slash us.

00:07:45 - Anthony Campolo

And certainly not in the last two weeks, right?

00:07:46 - Dev Agrawal

Right, yeah. I mean, I just think most people have either a misunderstanding of DevRel or they don't know what they want out of DevRel, or it's just like, yeah, they're just trying to like...

00:08:05 - Anthony Campolo

We want Dax is what we want out of DevRel, right?

00:08:07 - Dev Agrawal

Right.

00:08:07 - Anthony Campolo

We just want Dax.

00:08:09 - Dev Agrawal

Yeah. Some people say we want Dax, some people say we want Lee Robb. There's like a whole variety everywhere.

00:08:16 - Anthony Campolo

They are the perfect inversions of each other.

00:08:21 - Dev Agrawal

Pretty much. Yeah. Dax spends all of his time taking shots, and Lee spends all of his time cleaning up after someone has taken shots.

00:08:33 - Anthony Campolo

So you did DevRel at Clerk, right?

00:08:35 - Dev Agrawal

Yes.

00:08:38 - Anthony Campolo

Yeah, so it's not your first DevRel role. And is this the first DevRel role at PowerSync?

00:08:46 - Dev Agrawal

Yes. PowerSync currently has a couple product people, an engineering team, and a product success team, which is a group of highly technical people that provide support and are kind of on the borderline of solutions engineering. And yes, I am the first hire in the DevRel space, which usually means fill in the blanks of the entire kind of product-community lifecycle.

00:09:16 - Anthony Campolo

Which department are you under?

00:09:20 - Dev Agrawal

It's independent. I'm not under either of them. I just directly talk to the CEO and the chief product person pretty much a couple times every week, which is very different from how many organizations do it. Usually it's under marketing, if not under product or under support.

00:09:49 - Anthony Campolo

Engineering is usually the third one. The question is always, is it under marketing, product, or engineering?

00:09:57 - Dev Agrawal

Yeah. And I think the DevRel role kind of really works best, especially in these smaller teams and when there's one or two people in DevRel, when it's kind of given the autonomy to do things the way that people see fit. Because usually a lot of people who don't spend too much time with the community don't have a great vantage point of what are the things that we should be doing. And it's really nice that I'm getting some of that here. And it's also that the only time I've done DevRel professionally was at Clerk, but after Clerk I've spent a lot of time with Solid, TanStack, and I've gotten open

00:10:53 - Anthony Campolo

source DevRel at this point.

00:10:55 - Dev Agrawal

Yeah, so a lot of open source DevRel, like going to conferences and stuff. So that's kind of contributed to this point, because obviously they didn't reach out to me after they saw my work on Clerk. They reached out to me after they saw my conference talk about sync engines and reactivity, which was really nice. And I think I can now confidently say that the last two or three jobs that I've had have been through conference talks.

00:11:27 - Anthony Campolo

No, that's really cool. I don't know if that's like rare or not, but definitely shows a lot of initiative. So how did you end up, what inspired you to do the sync engine talk in the first place?

00:11:43 - Dev Agrawal

Ryan, like usual. But yeah, I mean it's everything that happens in the React-versus-fine-grained-reactive-frameworks world. The kind of motivation of that talk was that sync engines bring in a lot of new capabilities, or capabilities that were not possible before, not just in terms of DX but also in terms of efficiency and performance. And Solid kind of has a very similar feel over a lot of, obviously React, but also other frameworks in that space that are now moving to signal-based solutions. So I just wanted to show that, hey, these two things are very similar to each other and you can put them together very nicely. Yeah, so there wasn't really anything too deep there. I think another point there was that I wanted to talk to people in the local-first and sync engine space and figure out what's their view of client-side frameworks, and what does it look like from their perspective to be dealing with so many JavaScript frameworks, and which ones tend to be more difficult to work with or easier to work with.

00:13:13 - Dev Agrawal

And throughout my entire time in Berlin, the only thing I kept hearing from everyone who builds sync engines is that React is the worst. It's the most complicated to build integrations with React that actually work, and that don't completely blow up your application with infinite re-renders, especially with all the new async and new transition stuff in React.

00:13:39 - Anthony Campolo

Let's step back, and before we do the kind of sync engine 101, an interesting vantage point is when you first started talking about these sync engine things. We were actually going to do a stream with Peter about this, and then we couldn't get it together. I kept hearing you talk about sync engines, and at first I was really confused about the term. I was like, what is this? What is he talking about? And then it was one of those things where after a while I was like, wait a second, I know what this is. I just know it by a different term. Some people have called this local-first, and I'm sure there's a slight difference between the two. But for me, the reason why I first heard about this is this was a big conversation around 2020, 2021, because you had Apollo with React and they had some sort of caching mechanism that you could kind of do this with, but it was really crappy. And then you had Amplify DataStore, which I remember Swyx was always really into, which is apparently a kind of nicer version of that.

00:14:43 - Anthony Campolo

And then people would say those were kind of similar to, I think it was called MiniMongo. There was some sort of Mongo thing, and this relates to Meteor. You probably know a lot more about all of this than I do. So I remember hearing about all this stuff back in 2021, and it never really made a whole lot of sense to me. But then when you started talking about sync engines, I'm like, oh, this is what all those people back then were talking about. So how do all these terms relate to each other? Was there a switch between local-first and sync engines?

00:15:12 - Dev Agrawal

Yeah, there's a lot of kind of weird history behind it, and it's always kind of fun to dig into, because Meteor was the second framework that I learned and started using, but it was the first full-stack framework for me. And this was back in 2017, after Meteor had already kind of died out of the hype. But I think I had found some tutorials on it, and it just seemed really nice to build things with. It just felt really simple at that time, and I didn't quite get it because it was the first time I was actually building full-stack apps, and I had never really done it any other way. I used Meteor for a few years, and then I experienced the MEAN, MERN stack of like a SQL database and ORM and API Express server, and then you had

00:16:17 - Anthony Campolo

a similar experience that I had with Redwood, because I learned Redwood first, and then it was like, oh, Redwood was very inspired by Meteor. There's a lot of similarities there.

00:16:27 - Dev Agrawal

Yeah, exactly. And I think the key difference there, actually, no, Redwood did use Apollo or something like Apollo from the beginning. So there was a client-side caching mechanism built into it. So if you think of stuff like Apollo and maybe even Relay or in some ways just TanStack Query, which are basically ways to cache some data that you have fetched from the server on the client side, that's pretty much it. You can determine how long to keep this cache alive and things like that. You can make changes to the cache data on the client side. But in a sense, it's just a way to cache things. Apollo and Relay kind of take it a bit further where they don't just cache data that's coming out of a query, they can also kind of match and normalize it across queries so that if you have two queries that are looking at the same data, you don't have the same data cached twice. You just have the data cached once and then the queries are looking at it. So in that sense you have moved a bit closer to a proper database setup on the client instead of just simple key-value stores.

00:17:44 - Dev Agrawal

So a sync engine, you can almost think of it as an evolution of that, where the primary way of interacting with data is completely on the client side. You have a database on the client, you write queries against it, you write select queries, you write update and insert queries, and all the changes happen on the local database. Then a sync engine's responsibility is to somehow keep it synchronized with your actual backend database, whatever that ends up being. So a sync engine is the piece of technology that you would use to synchronize the data between client and server. Local-first is not a technology. It's just a set of ideas of what apps should look like, or what certain apps should look like. I think another analogy here may be building cloud-first or cloud-native applications, which is a set of ideas of how we should build applications, and infrastructure-as-code tools like Terraform or AWS CDK, which are tools that you would use to build applications on the cloud. One is a tool and one is a set of ideas on how to build apps.

00:19:12 - Dev Agrawal

That's a similar distinction between a sync engine and local-first apps. That makes sense. Yeah, you can use sync engines to build apps that are not local-first. In fact, a lot of the work in the last two years in these sync engines has been to pull them out of the local-first niche and apply them more generally, like, hey, you can build any sort of app with a sync engine. That's kind of been the push in the last year or the last two years.

00:19:48 - Anthony Campolo

Interesting. I want to get into that. But yeah, so what I always heard was local-first. The point, or one of the big draws, is that the app would then work when it's offline. So if you are, you know, walking around with your phone and you lose your Wi-Fi signal, you can keep doing stuff on your app. It doesn't just break. So is that kind of the idea?

00:20:10 - Dev Agrawal

Sort of. So what you're describing is an offline-first app, not a local-first app. I can keep drawing Venn diagrams of all of these. But yeah, an offline-first app is where even if you go offline, most of the app should still continue to work. Local-first is like an even smaller circle inside offline-first, which mainly says that the data that you have on your device is the source of truth. As far as the application is concerned, you own your data. It's less about the app working offline and more about data sovereignty.

00:20:51 - Anthony Campolo

So it's like iMessages, like if you turn off iCloud.

00:20:53 - Dev Agrawal

Yeah, it's more like any sort of connection to the cloud is just for keeping a backup or keeping my devices synchronized or collaborating with someone else. But the data lives on my device. I own it. The application doesn't own it. The developer, the company, none of them own the data. And even if the application kind of goes out of existence, the company gets bankrupt, the data still lives on. So the data has a longer shelf life, or a longer lifetime, than the app itself, which is very different from how you might build apps normally. So that's what local-first is. Obviously, to be local-first, you also have to be offline-first, so it's a subset of offline-first. And then you can use a lot of sync engines to build offline-first and local-first apps. Not all of them. Some sync engines explicitly disable that, saying that, hey, we are only for the online use cases, not offline or local-first use cases. There are other sync engines that say that we are the most optimized for local-first, and you don't want to use the sync engine for the other use cases.

00:22:15 - Dev Agrawal

There's all the variety.

00:22:17 - Anthony Campolo

Yeah, this is getting expansive. So let's hone in. What is PowerSync designed for out of that whole spiel you just gave?

00:22:26 - Dev Agrawal

The way that I'm trying to talk about it, and the way that we are trying to position it, is that you can pretty much do everything across the spectrum with PowerSync. Some of our recent features are trying to enable that more. PowerSync is essentially a way to keep any of your backend databases in sync. Currently we support Postgres, MySQL, MongoDB, and Microsoft SQL Server. Not SQLite yet. SQLite is used on the client side. So PowerSync will keep the data between a client-side SQLite database and your backend Postgres, MySQL, whatever you have. It'll keep all the data in sync. So on the client side, you just interact with the SQLite database and all the syncing happens automatically. Obviously you can come into that sync process and define your validation, auth, conflict resolution, whatever you want. You have opportunities to customize all of that, but at the base that's the default experience.

00:23:47 - Anthony Campolo

So how does all this relate to Convex?

00:23:52 - Dev Agrawal

Convex? They try to say that Convex is a sync engine, which is true in some senses, but Convex is more like a real-time database where you write queries against the database and if any of the data changes, your query will rerun automatically and the client will be updated with the latest data. Convex is purely kind of a server-side thing. It doesn't have a full client-side database that you can query. There's no optimistic updates, which is one of the biggest features of a sync engine, which is that you make a mutation on the client side and you see the updated UI immediately rather than waiting for a round trip. That's not true with Convex.

00:24:39 - Anthony Campolo

Everyone was always talking about that back in the Redwood days, was optimistic updates. That was the big thing.

00:24:45 - Dev Agrawal

Yeah, and it's been the big thing with React and the new Solid 2.0 as well, and a decent bit in the new SvelteKit. So now it's something that's being built into the web framework layer. But yeah, sync engines kind of offer that out of the box with a lot more guarantees than what a web framework can, because it's essentially a complete database on the client side. There's another tangent that we can go on here, which is TanStack DB. But yeah, that's almost certainly going to require a whole stream if you want to dig into the whole landscape of sync engines and local-first.

00:25:31 - Anthony Campolo

Yeah, we can expand on that once we narrow down PowerSync first. Sounds like PowerSync is trying to be a fairly generic, general sync engine that can be applied to a lot of these different use cases that you're talking about. You also mentioned that they're trying to expand it out so it could be used for other things that are not in this category of things you just talked about, local-first and offline. So what are the benefits of using a sync engine for those other use cases?

00:26:04 - Dev Agrawal

The main benefit of a sync engine to me is that it eliminates a lot of the complex implementation logic that we have to put in place when it comes to calling APIs and keeping the state on the client side in sync manually. A lot of the reason why things like React Query and then Remix and Next.js simplified how we build apps is by getting rid of the client state and kind of reducing it as like, here's a query, when anything changes, just rerun the query and your UI is up to date, right? But the problem is that, first of all, rerunning so many queries means you have to manage the cache, the invalidation logic, with things like TanStack Query. If you have something like Apollo and Relay, you have a whole custom query layer which informs the Apollo and Relay system how to do it more granularly so you don't have to invalidate things. But if you don't have that, then you have to write a lot of that logic yourself. And then there's a lot of optimization, and just doing things over the network, that are involved. I'm hoping to make a video diving a lot deeper into all the issues that come in this area.

00:27:48 - Dev Agrawal

PowerSync likes to call it the network tarpit. Yeah, network tarpit. That's also the headline on the PowerSync website, "Escape the Network Tarpit." I think I've talked about a similar concept in some of my talks. I just called it glue code or duct tape. But I really like the network tarpit. Yeah, that one. And I think that this sort of logic exists in every web application or every mobile application. Any application, really, that talks to a server over an API has to keep some sort of state in sync. So this is not a problem that only local-first or offline-first apps have to deal with. This is a problem that I have been trying to deal with pretty much on every project since I stopped using Meteor, because Meteor was also a sync engine, and Meteor was for a lot of these real-time app use cases and not for those offline, local-first use cases. So to me, that's the selling point of really using any sync engine, not just PowerSync. Most sync engines have this property that they simplify a lot of the application logic, and it boils down to just writing SQL queries from UI components, or writing database queries from UI components.

00:29:28 - Dev Agrawal

Like if you remember that Sam Selikoff slide that kind of took off at Next.js Conf, it just pretty much boils down to that. Just being able to directly access data on the client side and have all the data synced seamlessly between the client and server. Getting optimistic updates for free, that's another big challenge. Having to write optimistic updates by hand tends to get very tedious, very complicated. And sync engines kind of just eliminate all of that because you just write the mutation once and things get synced automatically.

00:30:09 - Anthony Campolo

Yeah. It's interesting, I'm thinking how this pairs well and brings us kind of full circle to your first episode. The first thing you came on to talk about was your Solid real-time thing. So it's not the same, obviously, but it's kind of that same idea of always wanting a real-time or instantly updating, really interactive type of UI. It's like a running theme, it seems like, for you.

00:30:37 - Dev Agrawal

Yeah, there's definitely been a lot of overlap there, and spending my time building server signals and a Solid Socket library has also kind of influenced a lot of this because when you build that real-time library, a lot of this conflict resolution and offline stuff kind of comes into the picture automatically. Yeah. And that definitely kind of pushed me toward thinking more about sync engines. But I guess I initially started building Solid Socket because I had already used a sync engine before that. So it's all kind of mixed up. But yeah, in general, I've kind of spent a lot of my last few years trying to find ways to remove unnecessary abstractions that I keep running into and try to keep things as simple as they can be. And I think I used to think it was kind of a weird obsession that doesn't have a lot of real-life output. But the last year building with AI has completely convinced me that that was always the right thing to do, because turns out when we get rid of a bunch of complicated logic and make things really simple to author, like just accessing data directly from UI components, that turns out to be really great for AI.

00:32:14 - Dev Agrawal

And it can just continue to write components with data access anywhere. It doesn't have to worry about network calls, doesn't have to worry about cache invalidation or anything like that. This is something that pretty much everyone in the sync engine space has seen, but it's something that we're all trying to better formulate and talk about, that hey, having sync engines makes it really easy for AI to build apps and we just need to better explore that area.

00:32:51 - Anthony Campolo

Cool. Can you pull up the PowerSync docs and show some code examples? We can get something a little more concrete to look at while we're talking about this.

00:33:01 - Dev Agrawal

Yes, absolutely. I should. A lot of this stuff is very difficult to kind of just talk about.

00:33:11 - Anthony Campolo

So far I'm following everything you're saying.

00:33:15 - Dev Agrawal

Share screen. Can I... do I have to do my entire window? Come on. Okay, looks like I can just share this. Okay, you see that? You see the docs page?

00:33:32 - Anthony Campolo

Yeah. Bump it up just a couple.

00:33:37 - Dev Agrawal

Okay, that's fine. We don't need this. Yeah, so let's start with... I mean, the setup guide is a little bit longer, but I think this is a fine place to start. So initially, one of the biggest advantages of PowerSync is that it works with pretty much any database that you might be using, and we're adding more here as well. It supports pretty much any client runtime. If we go down here, Dart, Flutter, JavaScript, Kotlin, Swift, .NET, Rust, everything's supported. This includes React Native and Capacitor stuff as well. So it's one of the very few sync engines that you can use regardless of what you're building. We actually had a category in our hackathon for best submissions using the Rust SDK, and because it's a very small native SDK, you can use it for embedded apps and things like that.

00:34:48 - Anthony Campolo

So you were mentioning how it can connect with all the different front-end frameworks, and I'm not seeing any of that in the docs.

00:34:58 - Dev Agrawal

Give me a second, let me find it. Yeah, so I was talking about support for client-side integration. With any sync engine, you have a database on the client side, which is what your UI is going to directly query and directly make changes to. Here I was talking about what are the different languages that you can build apps in on top of the PowerSync database. Let me come and show code here. I'm going to skip some of the setup here. Source database, sync streams. I'll get back to that later. Client SDK. Yeah, so this is, let's go with the JavaScript web here. Basically it looks like this: you install the SDK on the client side and install SQLite. So SQLite is how all the data is stored on the client side. There's a schema for the client-side SQLite database. So here there's a todos table. Let me switch to web here. Yeah, so there's a todos table. You can put indexes on it, you can define a schema like this, and then we just define your database. So you give it your schema, you tell it where the database is going to be stored on the local file system in the browser.

00:36:37 - Dev Agrawal

This would connect to the PowerSync service, which syncs data from your source database right down to the SQLite database. There's some authentication, so we can give it a token, and this token will be used for authorization. Then there's the upload data. When you make changes to the local SQLite database, all those changes get queued up on something called an upload queue. Here you tell the PowerSync SDK how to take those uploads or changes that have been made locally and push them to your own backend, where then you can write them to your source database. This integrates with whatever backend you have. Ideally, you're going to implement a backend API on your backend server which is going to accept all these local changes and process them, run whatever validation and authorization is required, and then make the changes to the actual database that you have on your backend. Then those changes will get synced down to the client, and the client will always have the updated copy of whatever happened. Does that kind of make sense?

00:37:58 - Anthony Campolo

Yeah, it does. So it sounds like you're not handling as much of the backend stuff as you are the front-end stuff, because like you said, you're kind of implementing how it's going to update on your actual backend. When you first described this, I imagined that would be something a sync engine would do, but I guess you want that to be more general because there are different trade-offs with how you're going to implement that.

00:38:24 - Dev Agrawal

Yeah, so some sync engines do. The main thing here is that a lot of sync engines kind of own the backend database layer as well, so they can do a lot more there. PowerSync currently plugs into whatever database you have and whatever backend you have, which is a nice piece of customizability that most people would need to have. You can have a pretty simple version of this, which is like you just process the writes as they come in, like in the last-writer-wins, without a lot of complicated conflict resolution logic there. But we do want you to be able to put any authorization and validation logic you want. And if the sync engine starts owning that write piece, then it also needs to have opinions about how you implement auth. So for example, Supabase row-level security, right? That's something that you would use when you rely on Supabase to make the changes to the database for you. You can use that here. So instead of having your own endpoint, I have a demo app that does it, where in this upload data I just call Supabase client and make changes.

00:39:53 - Anthony Campolo

PowerSync integration for Supabase?

00:39:56 - Dev Agrawal

Yeah, PowerSync integration. Yeah, it does a lot of this kind of thing out of the box. This is an area that we're looking at to see if there are ways to simplify it further. But basically this is where you would add all your authorization and validation logic. Once you have this here, the rest of the app becomes pretty simple. You have your database, you just run a watch query: select everything from todos where list ID is this list ID. Whenever the list ID changes, or whenever anything in the todos changes, this query reruns and you get the updated todos here. You can run these reactive queries against your local database, and this will always keep your UI, or whatever you have, in sync with whatever happens. And then you can also write the data. So you can just say on the client side database.execute, insert this into todos, and it will happen on the local SQLite table immediately. Your UI will also update immediately, and then it'll go through that upload data function to your backend. If the backend decides to accept that, like it's complete, it's fine, all the authorization and validation pass, then the UI wouldn't change, obviously.

00:41:31 - Dev Agrawal

But if the server decides to reject it for any reason, then it will be rolled back on the client.

00:41:43 - Anthony Campolo

Interesting.

00:41:46 - Dev Agrawal

Yeah. And all of that kind of happens on the client side automatically. So one of the big places where sync engines maybe differentiate from each other, or something that they solve that you wouldn't get in a normal non-sync-engine world, is consistency on the client-side database. I mentioned earlier how you might have two queries that look at the same data, and a system like Apollo and Relay can do smart things to make sure that if two queries are looking at the same piece of data, you have that piece of data in one place and you don't duplicate it. Well, another problem is that if you have two queries looking at the same piece of data and somehow one of the queries reruns and you see the updated data there, you might still be looking at the old data on the other query if for some reason it didn't update or it didn't receive the updated data. So one of the challenges is to make sure once something has been inserted into todos or something has been changed, any place in the application that's querying it should see that update.

00:43:05 - Dev Agrawal

You should never be reading the same data in two places and seeing two different results.

00:43:13 - Anthony Campolo

Yeah,

00:43:18 - Dev Agrawal

Yeah. So that's kind of like the simple version. There's also React hooks and reactive integrations for every framework.

00:43:28 - Anthony Campolo

That's what I was curious about. Can you go to that?

00:43:32 - Dev Agrawal

Yeah. So I guess this was db.

00:43:39 - Anthony Campolo

I see there's React Native, but I haven't seen anything just for React regular.

00:43:44 - Dev Agrawal

Yeah, I mean it would look similar, but instead of a for-await loop, this would just be a useQuery, I think. Yes, so it lives on React Native Expo and, yeah, PowerSync SDK. No, not here. That's the source code.

00:44:10 - Anthony Campolo

Oh, here we go.

00:44:10 - Dev Agrawal

There we go. React Hooks.

00:44:11 - Anthony Campolo

It's under... so there was, there's a thing. In the client SDKs under Node.js there's framework integration. That's where it has React Hooks, Next.js, Vue, Nuxt, TanStack Query, and then Expo Go.

00:44:29 - Dev Agrawal

Okay, so I'm sharing it now. It opened in a new tab, so I had to reshare from StreamYard. Okay, so reactive queries. Yeah, so it really just looks like this. You have a useQuery, you give it a SQL query, you give it the parameters, and what you get is the results. This is one of the coolest parts about sync engines in general, which is that they play really nicely with the declarative and reactive approach to building UIs that we have developed in the last decade. You just give it the query, you give it the parameters, and when anything changes, the sync engine makes sure that you always have the updated data. Again, this also reduces a lot of complexity in building these apps, because you are just writing queries. And whenever you're making a change... I guess that's not going to be here. Let me see, usePowerSync. Yeah, so similar to .getAll here, you just say PowerSync.execute and you run an insert or update query. Once you've made a change to the client-side database, you don't need to do anything else.

00:45:55 - Dev Agrawal

Everything has already been updated, which is very different from what building apps might look like without a sync engine, where you have to do a bunch of query.invalidate, refetch this data, update this piece of cache. None of that exists here.

00:46:12 - Anthony Campolo

Yeah, no, totally. It's so funny because this is exactly what it was like using Redwood, because Apollo had useQuery, and you would have code that looks very, very similar to what we're looking at here in terms of the declarative nature of it. You just have the query, and you didn't run SQL, you would run a GraphQL query, but it would still be kind of the same basic thing. So it's always funny that every couple of years go by and they're like, oh, here's another thing people are trying to figure out now that we had years ago. People just would've used GraphQL.

00:46:47 - Dev Agrawal

Yeah, I think every time there's one or two small improvements. For example, the biggest thing here would be optimistic updates. I don't think you would get that in Redwood out of the box. But a lot of the other things kind of follow that same principle, where you want to be able to co-locate your data dependency as close as possible to your component. And some form of query language is usually very, very helpful. So that's what GraphQL kind of brought, that it was a query language that you can compose and interpret and do smart things with. TanStack Zero and a lot of these sync engines today are building their own query language so that they can do fancy things, and PowerSync is just SQL. It's a SQLite database. No need to reinvent the database and the query layer. SQLite is pretty powerful.

00:47:54 - Anthony Campolo

ORMs too.

00:47:56 - Dev Agrawal

Yeah, that's also another nice thing that comes with it, that you can just use Drizzle, which is like the best ORM that exists today, and you can just use that on the client.

00:48:11 - Anthony Campolo

Nice. So what kind of stuff are you doing in terms of the DevRel? Are you writing blogs, building examples, interfacing with the community, or doing a little bit of everything?

00:48:26 - Dev Agrawal

Yeah, I think it's a little bit of everything for now. I've written two blogs so far. I've made two videos so far. I think this last week or so has been all about the hackathon. "Unleashing the Power of Sync," yeah, that was my first blog post. That title had been stuck in my head pretty much since we started talking last year. I've been trying to use that for a bit. One of the demos that I've built is a chat app and I just called it Power Chat. Anything else I'm building, this is the name template that I'm following for all my internal projects, like Power Something, PowerBot, PowerChat, PowerAgent, Power To-Dos, whatever.

00:49:16 - Anthony Campolo

Power Notes. With Auto, everything starts with Auto now. So you have offline-first apps with TanStack DB and PowerSync. That was your first big tutorial blog post, it looks like.

00:49:29 - Dev Agrawal

Yeah, it didn't quite end up being a tutorial. There's not really a lot of code examples there. It mostly ended up being a conceptual piece, really. It was like a follow-up to some of the discussions that my first blog post induced. Because in my first blog, the "Unleashing the Power of Sync" blog post, I said that, hey, you can build offline-first apps and you can integrate with your existing databases and you can handle conflicts however you want. Some of the other sync engines don't offer that, and they kind of started talking about how this is an impossibility, or how this leads into all sorts of problems that you might get into if you have something like this. So that's kind of what inspired me to write this other blog post, which was kind of me saying that, hey, it's okay, it's completely fine. You actually can build offline-first apps. Here are just a few things that you have to keep in mind when you're doing that. But as long as you're doing that, it's completely fine. It's not something to be scared of, where some people just treat it as a very scary concept.

00:50:58 - Anthony Campolo

Right. Is there kind of a Hello World example that tends to get used, like how Ryan has his Hacker News? Is there a similar type app that people always build with a sync engine?

00:51:09 - Dev Agrawal

Yeah, just a Linear clone. I mean, Linear was kind of the one that inspired, or started, or kicked off a majority of the sync engine space, right?

00:51:25 - Anthony Campolo

Man, I've never used Linear, and people constantly say it's like Linear for blank. I'm just like, I don't know what that means.

00:51:33 - Dev Agrawal

That basically means it's a clone of the app blank, but with much, much better UX.

00:51:43 - Anthony Campolo

Word.

00:51:46 - Dev Agrawal

Linear's whole thing was that you can replace Jira with something that doesn't actually suck.

00:51:52 - Anthony Campolo

Right, yeah, I think I remember hearing that. So if people want to get started with PowerSync, what would be the way to do that? It's an open-source thing, but do you create an account? Is there a service? How do you guys make money?

00:52:08 - Dev Agrawal

So PowerSync currently works by... there's something called the PowerSync service, which you have to deploy. The PowerSync service listens to your actual backend database. It acts like a replication node, which means your backend database will stream any changes that it receives to the PowerSync service, thinking that it's just replicating data to a read-only node. The PowerSync service will receive all these updates, and it'll maintain its own internal store of what changes have been made, what changes should be synced, what clients are connected right now, which clients receive what data, and all that. Then the client SDK connects to this PowerSync service to receive all the updates and update its own local SQLite database. PowerSync currently makes money by obviously deploying this and running this PowerSync service on the cloud so that you don't have to run it yourself. You can run it yourself. It is open source as well. The only thing right now is that it doesn't have the dashboard that we provide on the cloud version. But other than that, you can run it through Docker, you can deploy it wherever you want.

00:53:35 - Anthony Campolo

Kind of like Supabase, how there's a Supabase service but you could run Supabase yourself.

00:53:40 - Dev Agrawal

Exactly. Or you can also run Convex yourself now, but it doesn't have all of the same stuff that the cloud version will. So that's the cloud-hosted offering. There's also enterprise plans for people who are self-hosting too, and some support around that. So that's the cloud version. Obviously you can host it yourself as well. There might be a world where we might not need to deploy a separate service. It might be something you can embed in your backend. This is something that Replicache works that way, but it's also one of the biggest complaints about Replicache, that you have to build that entire backend integration piece yourself. So offloading that to a separate service that handles the syncing kind of really simplifies the whole integration piece.

00:54:49 - Anthony Campolo

Right, right. Okay, let's talk a little bit about the AI thing, and that will transition us into the Solid stuff. So you mentioned briefly that local-first or sync engines are really useful for AI. Can you dig into that a bit more?

00:55:08 - Dev Agrawal

Yeah, so I actually just made a video on this and I put it on PowerSync's Twitter and YouTube. But I'm going to make more videos about it and dig deeper into it. So this is a great opportunity for me to kind of refine how well I can explain this.

00:55:30 - Anthony Campolo

Is this still the Quest app?

00:55:32 - Dev Agrawal

Yes. The main idea is that imagine, like, take any framework or any codebase that you have today and imagine you have to add features to it. You have to build something on top of it. How many things would you have to change, or how many things would you have to add, to do that? The way I think about sync engines is that because it's just your UI components and your reactive queries to read data, and your mutation queries in your events, like your insert, it's literally just UI components talking to a database over SQL. That's all there is to the programming model for the default experience. That's kind of like... how do I explain it? I don't know. I definitely need to do some workshopping here. But it's like what I said earlier: it eliminates a lot of the logic that you don't have to write, or that was the accidental complexity, which is making network calls, or serializing and deserializing data from the network, comparing it against local copies, cache invalidation, retries, a lot of things like that. They just get eliminated, and the only code you have to write is your SQL queries and your UI components.

00:57:06 - Dev Agrawal

In any other kind of stack, you would still have to write your UI components and your queries, but then you would also have a bunch of code in the middle for all sorts of weird state management, cache management, or network bullshit. So a sync engine kind of just eliminates all of that and only needs you to write the stuff that was important, or that was the essential complexity, not the accidental complexity. I don't know how much sense that makes.

00:57:42 - Anthony Campolo

I guess my question is, what is the drawback? Why would you not use a sync engine? Because it sounds like it's just something that, like you said, eliminates a whole bunch of problems. So what's the trade-off? What's the additional complexity that you get from a sync engine?

00:58:00 - Dev Agrawal

The additional complexity is that sync engines so far have not supported all the weird things that you might want to do. Because sync engines kind of come from the local-first, offline-first world, there are certain problems that they introduce. One of the big ones is having to sync large datasets. For example, you open an app, and because a sync engine needs a bunch of data on the client side to show you anything, what a lot of apps will do is that as soon as you open the app, you won't be able to see anything for two minutes. Because the app is not only downloading the sync engine and the client-side database bundle, but then it's also downloading a huge chunk of data so that it can fill the client-side database. Once it's done, you get an instant experience because all the data is there, but it takes forever to get to that point. That's typically been one of the biggest weaknesses of sync-based apps. In some sense you can say that it's a feature, because the whole point is that you get instant interaction all the time, even if you have to wait for the data a little bit.

00:59:28 - Dev Agrawal

But then obviously that limits a lot of use cases for what you can use sync engines for. Obviously the first time you go to an app, having to stare at a loading screen for 10 minutes, that's not a great experience. Single-page-app developers have been scrutinized for this point already for the last 10 years just because of JavaScript bundles. Now, we don't want to recreate that problem with actual data, but that's normally been one of the biggest pain points. The other one that is similar is authorization or validation, where if you're allowing the client or the client logic to make changes to the database wherever it wants, or make whatever changes it wants anywhere in the code, how do you actually validate them? That's an issue. Again, this doesn't exist in the local-first space because the client or the local data is treated as authoritative. The user is actually free to make whatever changes they want. It's not like there can be someone else to say that, no, you cannot make these changes, because the user actually owns that data. But that doesn't work for non-local-first apps because you actually have business logic and authorization rules and policies that you need to enforce.

01:00:53 - Dev Agrawal

So that's something that you need to deal with in this model. It's nice to have the ability to make whatever mutations you want from anywhere. And it's kind of like the Firebase days as well, where Firebase allowed you to just give write access to the client side and run whatever queries it wants. But you still need a way to enforce some sort of authorization and validation on top of it. These are the trade-offs that come with it. I wouldn't necessarily call them trade-offs. It's more like these are things you have to think about when you're using a sync engine for apps. Now the main question is, does the sync engine you're using make it easy to actually deal with these things? How easy does it make it for you to define your authorization rules, or for you to be able to define what we call partial sync, which is syncing only a subset of the data to the user at a time instead of the entire universe?

01:02:11 - Anthony Campolo

I see you have a whole authentication section. So you support authentication clients like Supabase, Firebase, Auth0, Clerk, SuperTokens, stuff like that. Interesting.

01:02:27 - Dev Agrawal

Yeah. And the PowerSync service itself is really only responsible for the read side, or taking changes in your source database and syncing them to your local SQLite database. All the changes go through your backend. So the authorization that you do with the PowerSync service itself is only really concerned with what data can the user see. Your read-side authorization goes to the PowerSync service. And then when it comes to what changes a user can make, that's entirely left up to your backend, because your backend implements that upload data endpoint. That's where you put your authorization and validation logic. Different sync engines have different ways to do it. Zero, for example, has these things called mutators, where they're just JavaScript functions where you give them some context, and within the function you basically have your authorization in normal TypeScript. It's going to run both on the client side and the server side. Then in that function you look at the context, you run your validation. It's a little different from how PowerSync does it, because in PowerSync it's all in your backend. And on the client side you just write a query, which gets a little deep into the differences between these sync engines.

01:04:09 - Anthony Campolo

Yeah, no, that's great. Cool. I think that's a kind of good overview. I feel like I got a lot more clarity now. I'd be curious to kind of try it out. Let's talk a little bit about the Solid 2.0 beta release that's live now, right?

01:04:28 - Dev Agrawal

It is, yes. And we've been getting a lot of discussions, a lot of feedback recently, so it's really fun times and I'm very excited to see where this goes. I think some of the community projects have already started moving over. One of our community members, Brindley, has already moved TanStack Router and a couple other TanStack things to Solid v2. It's fun to see.

01:04:56 - Anthony Campolo

Awesome. Yeah. So is there anything that needs to be updated from our massive stream we did six months ago, or is pretty much all that still holds?

01:05:09 - Dev Agrawal

I think, yeah, almost everything there still holds. I mean, when we had that stream last time, there were some missing parts, or some things that weren't fully clear on how they were going to look in Solid version 2. But obviously now that there's been a beta release, a lot of those things have now been clarified in terms of the features

01:05:43 - Anthony Campolo

and what you're kind of getting, the benefits. That's all the same?

01:05:48 - Dev Agrawal

Yeah, the core proposition has kind of always been the same thing. I think one of the main things that have changed is that at that time we had a primitive called createAsync, which is how Solid handled any sort of async data fetching, or async reactivity.

01:06:15 - Anthony Campolo

Are we getting rid of createAsync?

01:06:17 - Dev Agrawal

Yes.

01:06:21 - Anthony Campolo

I just migrated all my shit to use createAsync. That's what the docs said to do for the last year.

01:06:26 - Dev Agrawal

Yeah, so now createAsync has disappeared and now every reactive primitive is naturally async. So you can move from createAsync to createMemo.

01:06:40 - Anthony Campolo

So is createAsync still a thing?

01:06:43 - Dev Agrawal

No, you just use a regular createMemo. Basically the difference between createAsync and createMemo, really the only difference was that in createAsync you could return a promise and it would handle that promise, whereas createMemo, the normal memoization primitive, didn't do any special handling with a promise. Now it's just that you can return a promise from anywhere, from a createSignal function or from a createMemo function.

01:07:19 - Anthony Campolo

I always thought createAsync, though, was an alternative to createResource, not createMemo, right?

01:07:25 - Dev Agrawal

Yes, it was. But it's more like the way that createResource always had two functions. createAsync streamlined it.

01:07:42 - Anthony Campolo

How do I hit an endpoint? How do I hit an API? What do I use?

01:07:46 - Dev Agrawal

You just call fetch user or fetch API. Just call fetch inside createMemo.

01:07:51 - Anthony Campolo

Interesting.

01:07:53 - Dev Agrawal

Let me find the beta release, and I think there are some code examples in here. I'll share my screen. We can quickly see what this looks like. I love this: the suspense is over. There we go. api.listUsers, you just do it inside createMemo and you have your users in here. You can just pass it into your components and you can render it however you want. This used to be createAsync, now it's just createMemo.

01:08:32 - Anthony Campolo

That's super relevant for all of my apps.

01:08:37 - Dev Agrawal

Yeah, definitely.

01:08:38 - Anthony Campolo

Well, now I know my migration path.

01:08:43 - Dev Agrawal

I guess one of the other things is that this used to be called Suspense, now it's called Loading, because A, we're trying to differentiate from React, and B, this also works very differently from how Suspense works in React and in Solid today. Different behavior, different name.

01:09:09 - Anthony Campolo

And then you got createOptimisticStore. Is that a top-level utility?

01:09:14 - Dev Agrawal

Yes. So this is what I was talking about, that now web frameworks are taking more responsibility for optimistic state rather than your data layer. The idea of an optimistic signal or an optimistic store is that it's something you would pair with an action. An action is kind of like a transition in React or in Solid currently, where inside an action function you would set some optimistic data. So here, for example, in our add todo action, we're going to push this todo into our optimistic todo store. This is going to update the UI immediately. Then we are going to hit our backend endpoint to add it to our actual database. And once it's written to the server, we can just say refresh todos, which is going to rerun this function, basically a way to invalidate or refetch some data. And while this entire action is ongoing, we are going to see the optimistic update that we added here. But once the action completes, this update will automatically be rolled back by Solid. So when we refresh this api.getTodos, if the new todos were in this updated data from the server, that's what we're going to see on the UI and we are not going to see this anymore.

01:11:01 - Dev Agrawal

But if the server decided to reject it for some reason, this api.getTodos, when it reruns, is going to come back without the new todo, and this is going to get rolled back automatically, so you won't see it in the UI anymore.

01:11:15 - Anthony Campolo

That's cool, because that's the thing I always was asking with the optimistic updates. What happens if you update it and then it's wrong because something like he just said happens? That's one of the things I was always wondering about. Optimistic updates. That's good to hear.

01:11:30 - Dev Agrawal

Yeah. Actions and optimistic updates go together, where any optimistic update would be kept alive until the action completes, and then it automatically gets rolled back. There's also a lot of features in here that Ryan calls entanglement. I'm hoping to find a better word, or an easier word, to describe it. But the idea of entanglement is that if you trigger one action and then you trigger another action, and both of those actions affect the same part of the page, the same part of the UI, they're going to get entangled together, which means they're both going to wait for each other to resolve and they're going to basically resolve together. Similarly to how transitions in React get merged, actions in Solid 2.0 can get merged with each other, but only if they affect the same part of the UI so that you don't see intermediate states flicker in.

01:12:51 - Anthony Campolo

I saw there's a breaking changes section.

01:12:55 - Dev Agrawal

Yes, breaking changes.

01:12:57 - Anthony Campolo

So...

01:12:59 - Dev Agrawal

index doesn't exist anymore if you were using it. The big breaking change is probably that createEffect used to accept just... like, createEffect was just an effect function. Now it's two functions, which we can see more here. Share this tab. Let me find an example for createEffect. Yeah, so this is what effects look like now. You have the first function where you actually establish the dependency, or where you read signals, and then the second function where you run your side effect. So instead of just saying console.log(count()), we have the first function where we read the count and return it. And in the second function we receive the count as the value and then we log it. So, small change to effect there. This has been one of the more controversial changes, as you might have imagined. But I think for most people that get angry with this change, my first instinct is to ask, okay, what are you actually using createEffect for? Because a lot of times if you're using createEffect for something like async, or for the ability to override local signals from props, which are the two most common use cases for effects, you actually don't need to use effects for those things anymore because we have those things built into reactivity.

01:14:46 - Dev Agrawal

So what actually happens with 2.0 is that a lot of the usage for createEffect goes away with much simpler things, and you're only left using createEffect for things that it's actually designed to do instead of stuff like async, which we have better primitives for. And that's something that's been haunting the React ecosystem forever. You might've seen a recent article about someone banning useEffect from their codebase or from their team, and it went super viral.

01:15:22 - Anthony Campolo

Yeah. I mean, I had been trying to write my Solid apps in a way where I didn't need to use createEffect. You know, Claude would keep adding it in for whatever reason. I was like, no, don't do that. Use createAsync.

01:15:39 - Dev Agrawal

Yeah, so that's going to be one of the education pieces, both for humans and AI, to learn how to use the new reactivity system efficiently. And I think another update that might be interesting... let me find it from here. I think it was the reactivity. It's probably in the same one. Batching, yes. Let's say that you have a Boolean signal. So we don't have the full example here, but let's say you have a submitted signal that can be true or false. And inside this event handler, we set submitted to be true. And then let's say that right after this setSubmitted, we try to read the value of the signal. Well, in Solid today you would get true, because you just set the signal to true. Now if you read it, the value will be true, right? That's the intuitive and maybe the expected behavior. In Solid 2.0, that's changing in that it's actually still going to be false. And once you call this flush, now you're going to see the updated value. So by default, whenever you set new values into any signal, they're not going to take effect until you call flush.

01:17:17 - Dev Agrawal

So they're all going to get batched, like, okay, these are the new values. They are going to be committed at some point, not right now. And calling flush is like, okay, go and commit those values now. Now everything is up to date.

01:17:39 - Anthony Campolo

Okay, so I see you have this Solid 2.0 documentation page you're looking at. These look super useful. I hadn't seen these.

01:17:50 - Dev Agrawal

Yeah. And thank Ryan's newly discovered love for agentic AI for these docs.

01:18:01 - Anthony Campolo

Yeah, I was gonna say these look pretty clean. Great. Yeah, I'm gonna use these to create a migrate skill.

01:18:09 - Dev Agrawal

Perfect.

01:18:10 - Anthony Campolo

Yeah, yeah, let's talk about that just briefly to kind of set up our next episode. So I find this so funny that Ryan is in love with AI now because I did his stream and I talked with him about AI, this is, I think, six or seven months ago, and he didn't know jack shit. He clearly had not been paying attention to the space at all. And I was like, eventually Ryan's going to get into this because obviously he likes technology, so he's just kind of a late adopter, I think, as he said. And so, yeah, I think it's super interesting. And now he's all in it and he wants to make Solid a really good AI framework. So I'm super hyped because I've been using Solid now with AI for like two years, and I think it definitely has a lot of potential. So you had tweeted saying that Solid is going to be the best framework for humans and AI.

01:19:11 - Dev Agrawal

Yes.

01:19:13 - Anthony Campolo

So what does that mean?

01:19:13 - Dev Agrawal

Which is a very bold claim.

01:19:15 - Anthony Campolo

Yeah. What does that mean to be the best framework for AI?

01:19:19 - Dev Agrawal

Yeah, honestly, it's a lot of the same stuff that I mentioned when I was talking about sync engines and how sync engines are great for AI. The whole point is that AI tends to make a lot of mistakes. And the best way to get reliable outputs is to minimize the surface area for mistakes that AI can make. So naturally, any abstraction that reduces or eliminates certain complexities that increase the chances of mistakes, anything that removes them makes it easier for AI, because now the AI has basically fewer opportunities to do something dangerous or to introduce bugs. So that's kind of the idea behind both PowerSync, or sync engines.

01:20:35 - Anthony Campolo

And people say the same thing for Effect, that Effect is really good for AI.

01:20:41 - Dev Agrawal

In a sense, yes. Effect is not quite about removing the surface area for mistakes, but more about making things like errors and dependencies more explicit at the TypeScript level, like on the LSP or compiler level. So it's more about trying to bring out all the weird inconsistencies or weird places where things can go wrong and make them explicit on the type level so that you have no choice but to handle them. And if you don't handle them, your compiler is going to yell at you that, hey, here's an edge case that you did not handle. That's really good for AI because it always gets immediate feedback of what are the things that can go wrong, and it can decide how to address them. The angle with Solid 2.0 is more that it's less about making things explicit. It's kind of the other way around, where it's encoding more behavior into the system so that a lot of the async behaviors become more implicit, and the code that you have to write at the end kind of handles more edge cases naturally without you having to add a bunch of complexity.

01:22:16 - Dev Agrawal

I think in one of Ryan's recent streams, what he did was basically implement some basic use cases, or some basic scenarios, with async. He used the regular React style, where we just use useEffect for everything. He showed all the different places where this goes wrong. And in the end, he built a version that didn't have a lot of those edge cases, that was very robust in terms of handling a lot of the edge cases properly, but it was very complicated to read and it was even more complicated to write. And that was kind of his demonstration of, see, if you go trying to handle these edge cases by hand, things are going to get really complicated, and you're going to have to hold a lot of things in your head to figure out where things are going wrong or where things can go wrong. And if you try to build something similar with Solid 2.0, the code actually stays pretty clean, pretty minimal, pretty simple. It's the most simple and idiomatic way that you would write your code, but it also has all these edge cases handled by default.

01:23:43 - Dev Agrawal

So what you're getting out of the box is something that's pretty robust, that doesn't need any additional logic to handle edge cases. And yeah, that's just less room for the AI to make mistakes. All of these things that I've said need a lot of code samples to make clear sense.

01:24:09 - Anthony Campolo

That's what I was going to ask. Ensuring the AI actually writes all this correctly, using the conventions correctly, that's where the difficulties can come in. But I'm optimistic that, like you said, you give it good examples, and if it's working with a codebase that is already using all of those, it will continue to use it. The issue is even more so coming into older Solid apps that aren't already doing that. It'll want to keep doing things in this previous way. That's the issue I kept running into, that it kept wanting to use isServer from the Solid web thing, right? I was like, no, don't do that. It kept using createEffect, it kept using onMount, it kept using all these older conventions. That's going to be the biggest thing, is how do you make sure it uses the most up-to-date things?

01:25:08 - Dev Agrawal

Yeah, definitely. And that's going to be probably the main problem with authoring AI instructions, skills, Agents.md, whatever, to try to steer it towards the new patterns. But overall, if you can see what I'm sharing here, this is basically what we have to teach it to write. Like, hey, write a createMemo, fetch your data inside it, and then just render it. Wrap it with Loading wherever you want. And if you want to show some pending indicator for when things are happening in the background, just use isPending and do that. If you're making changes, here's a way to define an optimistic store, define an action, write to the optimistic state, make your server change, refresh data, and that's it. This is what I mean by not having to deal with edge cases, or not having to think about where things are placed. You can keep the loading boundary wherever you want. You can fetch data wherever you want. You don't have to put them in a specific order. You don't have to create a new component just so that you can wrap with Suspense, or once you set an optimistic value, you know that it's going to get automatically cleared at the end of the action at the right time.

01:26:36 - Dev Agrawal

So a lot of small things like that, when they are taken care of, the overall code should be pretty simple. And we just have to convince the AI that, hey, the code that you have to write doesn't have to have 20 createSignals or 20 createEffects. You just write what makes the most sense.

01:27:01 - Anthony Campolo

Yeah, totally. So something that I think if Solid really wants to be the best friend for AI, there's something very, very important it needs, which it does not have: copy markdown buttons. Can I, if I open a PR, can I open a PR for that? Because this is constantly bugging me. I always have to go to the GitHub repo to get the markdown files. So you've got to have a copy markdown button. So I'll open a PR for that. It's been on my mind for months and months.

01:27:33 - Dev Agrawal

Yeah, I haven't used copy as markdown in forever. I think what I tend to do is give my agent the URL or it just browses the docs on its own. And most harnesses...

01:27:48 - Anthony Campolo

I usually don't use copy markdown because most copy markdown buttons have the open-in-markdown-file thing. You can open the markdown page and it will just give you the URL, and they give you the raw markdown as if you're looking at the Git repo. Because what I like doing is I grab all those links for the docs I want, and I have a custom script where it just takes a list of URLs, dumps them all to a single markdown file, and then I feed that to the LLMs so it won't have to go search through a docs repo to find the specific docs it needs. So I like being able to do that, grab a bunch of URLs and stack them all together and print them all out into one file.

01:28:27 - Dev Agrawal

That makes sense. Yeah, I should try that sometimes. But what I've seen is that pretty much every coding harness at this point is never going to dump the entire HTML response if you do a web fetch. It's always going to do some processing to remove all the extra HTML stuff and kind of boil it down to just the actual content. I think they probably use some accessibility stuff that browsers use to remove all the formatting and only have the actual content there. I think in my head that's replaced any agent-specific thing that we might have to do on our servers, because the harnesses take care of it. But having the option there on the docs page, or on the docs server, to serve something just as markdown, that's still helpful for scripts or manual collection things where you don't have that stuff in place. So that's been something I've been thinking about for a while. Also stuff like, should there be an MCP server, or just a docs search endpoint, or a CLI, or Skills, Agents.md... just trying to juggle everything is

01:29:56 - Dev Agrawal

Is very interesting.

01:29:58 - Anthony Campolo

Yeah. I think right now the most bang for your buck you get is from Skills. I think it's the right level of abstraction. It's pretty agnostic towards tools. You can customize them for lots of different use cases. So I think having a couple good SolidJS skills would be way more useful than having a whole MCP server. I'm still not totally sold on MCP for something like that. I think MCP is most useful if you're trying to connect 10 different services and data sources and crap together. Then I think it still makes sense for that. But it's like, what would a Solid MCP server do?

01:30:44 - Dev Agrawal

I know that Svelte has an MCP server that they've been talking about for a bit. Angular has had one, but now they're also investing into Skills. And I think the Svelte MCP server lists sections like get documentation, auto-fixer, and playground link, which are somewhat helpful tools for sure.

01:31:15 - Anthony Campolo

Interesting. Yeah, I'll have to look into that. Okay, cool. I want to kind of table this and have a longer discussion around AI and Solid and the types of stuff that I've been building with it, but this is probably good to start wrapping it up here. We'll schedule something like another week or two. Any last things you want to mention in terms of PowerSync or Solid?

01:31:44 - Dev Agrawal

Well, the hackathon just ended, so I can't ask anyone to participate in the hackathon anymore. I guess, yeah, just check out PowerSync. Play around with this new Solid beta.

01:31:59 - Anthony Campolo

Do you guys do events or meetups or anything like that?

01:32:02 - Dev Agrawal

No, not yet.

01:32:05 - Anthony Campolo

You guys should do Twitter Spaces.

01:32:07 - Dev Agrawal

Twitter Spaces are dead. Ever since they have been buried behind live streams, I don't know if there is value in Twitter Spaces anymore.

01:32:19 - Anthony Campolo

Interesting.

01:32:20 - Dev Agrawal

I hope so. I mean, I think at this point it requires having someone in your Twitter Space with a large following so that there's some chance that someone shows up.

01:32:35 - Anthony Campolo

Don't you have like 10,000 followers?

01:32:39 - Dev Agrawal

10,000? Do I have 10,000? Okay, I have 14,000. Yeah, yeah. I don't consider that to be a pretty big number. And I think a lot of it, more than half of it, came from being in that space with Elon Musk, which was a complete accident.

01:33:00 - Anthony Campolo

Is there a Discord? A PowerSync Discord?

01:33:03 - Dev Agrawal

There is a PowerSync Discord. It's not powersync.discord, it's discord.gg/powersync, or this one. Yeah, you just put it in the chat.

01:33:26 - Anthony Campolo

I just realized someone said hi way back. Who is this? Mercer Case 2. I don't know who that is. Hello, though. Thanks for watching.

01:33:40 - Dev Agrawal

Good crowd. A lot of activity in chat.

01:33:45 - Anthony Campolo

Let me just grab these last couple links.

01:33:49 - Dev Agrawal

Yeah. I am very curious to hear about your role and more about AutoShow, so definitely want to do that on the next one.

01:33:59 - Anthony Campolo

Yeah, yeah. It's multimodal now. It doesn't just create text, it creates images, music, text-to-speech, and video. All sorts of things.

01:34:09 - Dev Agrawal

Lovely. What kind of models do you use under the hood? All of them? Nice.

01:34:22 - Anthony Campolo

Name a model. I probably got it.

01:34:26 - Dev Agrawal

But there isn't a way to bring subsidized tokens from my Cloud Code or Codex into Automaker, right?

01:34:34 - Anthony Campolo

No, no, not right now. It's an interesting idea though. I should look into that.

01:34:41 - Dev Agrawal

Yeah. I mean, even then it wouldn't handle most of it. There's only so much that Claude and GPT can do when there's also text, video, image, and audio generation involved.

01:34:57 - Anthony Campolo

Yeah, certain services do have almost all of them now. Gemini has pretty much every modality. ChatGPT has most of them as well. I'm not sure if you could use your ChatGPT credits to generate Soras. Probably not.

01:35:16 - Dev Agrawal

I don't think so. I think it's only on the API. It's not on the Codex subscription.

01:35:21 - Anthony Campolo

Yeah, yeah. I mean really, AutoShow, the way I'm thinking of it, is for people who are not technical. So someone who has a $200 subscription and is trying to share it with their other apps for token usage, they should already just be using the CLI, you know?

01:35:39 - Dev Agrawal

Yeah, just use FFmpeg and write a bash script or something.

01:35:46 - Anthony Campolo

Yeah, exactly. You could just cobble together your own version of it pretty easily.

01:35:51 - Dev Agrawal

Yeah. What I've been thinking about, or trying to experiment with, is using my Trello board as kind of a way to manage agents. I don't know how far it would go, but yeah, it's mostly because, like, you get those loops

01:36:06 - Anthony Campolo

going for every ticket.

01:36:08 - Dev Agrawal

Exactly. Yeah, there's something there. I think a lot of people have ended up at the Kanban interface for agent orchestration independently.

01:36:23 - Anthony Campolo

Like who? Who are you talking about?

01:36:26 - Dev Agrawal

Like, Conductor, I think, was one of the first big ones. Conductor.build, which is on top of Cloud Code. I mean, most of them are on top of Cloud Code, but yeah, Automaker is one that was built by WebDevCody. There are some people who have added Kanbans to OpenCode and T3 Code. But now I'm thinking that instead of building a Kanban interface, let's just use the one we already have. We already have a very nice one, and let's plug agents into that.

01:37:09 - Anthony Campolo

That makes a lot of sense to me. Yeah, I've vaguely heard of Conductor. I didn't really know what it was. I haven't heard of Automaker at all. I'll have to check these out. These don't look like they're really my style, but they're interesting.

01:37:25 - Dev Agrawal

Yeah, that's fair. But the reason I bring it up is because I'm trying to track all the DevRel stuff in Trello as well. So maybe I can have Trello cards for, like, here's an idea, generate a bunch of tweets, tweet suggestions for this, or generate a video script for this. Or maybe once I've recorded it, I put it in Trello and it automatically creates show notes, creates cuts, and turns them into little bits and pieces. Maybe if Automaker has an API, I can plug that into my Trello, and yeah, that's all stuff for next stream. I guess that's a little bit of a preview of what I'll have in my head when looking at Automaker. Like, can I bring this into my Trello board?

01:38:18 - Anthony Campolo

Awesome, man. Well, cool. Thanks so much, dude. Always appreciate doing these. Sounds like PowerSync is pretty interesting. And yeah, I'm glad you're working on tech that you find interesting and you're excited about, and that you're back in the DevRel world, which is always fun.

01:38:36 - Dev Agrawal

Yeah, definitely. There's never been kind of a better time to be working on sync engines.

01:38:46 - Anthony Campolo

All right, well, with that, we'll close it out. Thank you, everyone who was watching. We'll catch you again in a couple weeks. And that wraps it up for today, so catch you guys next time.