Talking Serverless with Josh Proto
Josh Proto of Serverless Guru discusses cold starts, Lambda best practices, framework choices, and the future of serverless adoption.
Episode Description
Josh Proto of Serverless Guru discusses cold starts, Lambda best practices, framework choices, and the future of serverless adoption.
Episode Summary
In this episode, Anthony Campolo and Christopher Burns sit down with Josh Proto, chief operating officer of Serverless Guru, to explore the current state and future trajectory of serverless computing. Josh shares his unconventional path into the industry, moving from education and biochemistry into marketing and operations for a serverless consultancy. The conversation quickly turns technical as Chris raises the question of whether serverless has reached its potential, prompting a discussion around cold starts, provisioned concurrency, and the practical challenges developers face when migrating traditional Express applications to AWS Lambda. Josh emphasizes the importance of the single responsibility principle—one goal per Lambda function—and warns against the common anti-pattern of forcing an entire monolith into a single function. The group then examines infrastructure-as-code frameworks like Serverless Framework, SAM, CDK, and Pulumi, weighing trade-offs around language support, plugin ecosystems, and first-mover advantage. The episode closes with a forward-looking debate about who will drive the next major leap in serverless: established cloud providers like AWS, abstraction platforms like Netlify and Vercel, or emerging edge-native providers like Cloudflare and Fastly.
Chapters
00:00:00 - Meet Josh Proto and His Road to Serverless
Josh Proto introduces himself as the chief operating officer of Serverless Guru, a consultancy focused on serverless-first cloud infrastructure built predominantly on AWS. He explains how he joined early in the company's life to help with marketing and positioning after a career in education, biochemistry, and personalized tutoring.
The conversation touches on Josh's experience as a career switcher and how his background in learning science and game design gave him transferable skills. He describes tinkering with Alexa skills, C# in Unity, and Swift development, all of which prepared him to engage meaningfully with the serverless community and eventually help shape Serverless Guru's public presence.
00:05:22 - Podcasting, COVID, and Educating the Market
Josh explains how the Talking Serverless podcast was born out of the early COVID-19 lockdown period, when he and founder Ryan Jones saw an ongoing need to explain what serverless actually is and counter skepticism in the market. He frames the podcast as an extension of his education background—building frameworks for teaching complex topics.
The discussion also covers advice for career switchers, including the value of a growth mindset and the ability to bring cross-disciplinary perspectives into tech. Josh notes that while the serverless market is still early, these educational efforts are laying important groundwork for broader adoption.
00:08:47 - Has Serverless Hit Its Gold Rush Yet?
Chris asks whether the serverless industry has found its gold rush moment. Josh references an article exploring whether serverless has peaked in hype, arguing that while buzz may be fading, the real value proposition is well understood—teams just need to implement it correctly. He stresses that you shouldn't run an entire monorepo under one Lambda function.
The group discusses how serverless shines when applied to the right use cases and how enterprise teams are seeing genuine velocity gains. Josh frames the current moment as one where the gold veins have been found but the hard work of extraction—proper implementation at scale—is just beginning.
00:11:51 - Cold Starts, Provisioned Concurrency, and Performance
Chris raises the cold start problem, contrasting a small startup's experience with that of a high-traffic company like Snipcart. Josh explains cold starts in plain terms—the delay before a Lambda function's code is ready to execute—and walks through how warming functions were an early workaround before AWS introduced provisioned concurrency.
The conversation broadens to include Lambda@Edge for regional distribution, the impact of function size on startup time, and how enterprise clients are finding these tools sufficient to overcome cold start issues. Anthony adds that keeping functions under five megabytes is a well-known best practice championed by developers like Brian Leroux.
00:16:55 - Cloud Economics and the True Cost of Functions
Anthony introduces the concept of the cloud economist, referencing Corey Quinn's business built around AWS bill optimization. Josh picks up the thread, describing a future where individual Lambda functions carry assigned monetary values—capturing not just raw compute savings but also productivity gains from replacing broken infrastructure.
The group explores how switching from always-on servers to pay-per-use compute can save significant money, but also how monitoring and tracing become essential to avoid unexpected costs. This frames serverless not just as a technical choice but as a financial strategy requiring ongoing discipline and visibility.
00:20:17 - The Lambda Monolith Anti-Pattern
Chris poses a detailed question about whether complex Lambda functions degrade faster than equivalent logic on an always-on server. Josh confirms that function size and complexity do affect cold start performance, and the conversation turns to the widespread anti-pattern of stuffing an entire Express application into a single Lambda function.
Josh advocates for one goal per Lambda—the single responsibility principle—and shares that many clients have come to Serverless Guru frustrated after attempting exactly this monolith approach. Anthony notes that the first item in AWS's own anti-patterns guide addresses this issue, underscoring that it remains poorly understood despite being well documented.
00:27:25 - Frameworks Abstracting Away Best Practices
Chris asks a provocative question about framework providers that quietly bundle serverless functions into Express-based monoliths behind their abstraction layers. Josh acknowledges this is a real tension: when the abstraction doesn't follow serverless best practices, users discover the limitations only at scale.
The discussion highlights that discipline and commitment to proper architecture matter more than which framework you choose. Josh draws an analogy to powerlifting, arguing that extracting full value from serverless requires daily practice and organizational buy-in rather than a one-time migration decision.
00:32:19 - Navigating the Infrastructure-as-Code Landscape
Anthony shifts the conversation to infrastructure-as-code tools—Serverless Framework, CDK, SAM, Amplify, Terraform, and Pulumi—and asks when each is the right fit. Josh explains that Serverless Guru primarily uses Serverless Framework but evaluates alternatives based on client needs, especially around language support and advanced features.
He highlights the importance of third-party plugin ecosystems, noting that Serverless Framework's first-mover advantage created a rich community of contributions that newer tools have yet to match. The takeaway is that framework selection should be driven by specific workload requirements rather than brand loyalty.
00:37:18 - Who Will Drive the Next Serverless Leap?
Chris's closing question asks whether AWS, platforms like Netlify and Vercel, or frameworks will push serverless to its full potential. Josh points out that major cloud providers have an asymmetric motivation problem—they still profit from virtual machine customers—suggesting innovation may come faster from specialized providers or the framework layer.
Anthony bets on emerging edge-native providers like Cloudflare and Fastly as the source of the true quantum leap, where cold starts are architecturally eliminated. The group closes by discussing the Fastly outage, the risks of depending on newer providers, and where listeners can connect with Josh online and through Serverless Guru.
Transcript
00:00:00 - Anthony Campolo
Chris, this will be your chance to air all of your serverless gripes.
00:00:04 - Josh Proto
Let's do it.
00:00:15 - Anthony Campolo
Josh, welcome to the show.
00:00:17 - Josh Proto
Hey, hey. I'm so happy to be on. I really appreciate it. Thank you, Anthony. Thank you, Chris.
00:00:21 - Anthony Campolo
You are someone who hosts your own podcast, Talking Serverless, which is how we first got to know each other. I've been a two-time guest there, which has been a lot of fun. You also work at Serverless Guru and, I think, are a co-founder there. If you want to talk a little bit about what that is, and your background and how you got into serverless, we'd love to hear that story.
00:00:45 - Josh Proto
Definitely. Right now I'm the chief operating officer of Serverless Guru. Whether I'm a co-founder depends on how you define it. Not in the original sense; I think Ryan is the single founder. But I'm the veteran who knows most of what's gone on because I joined in early 2019. I originally helped Ryan and Serverless Guru define itself in the marketplace. We're a consultancy that provides serverless-first cloud infrastructure and cloud service consulting. I remember when I first met Ryan. He said, I'm doing this thing and it's getting traction, but no one knows about us and I don't know how to solve that problem. Coming from a long career in education and retooling myself into marketing communication, I thought, I am exactly the right person for this.
[00:01:35] That was when I really started to delve into the industry and ask, what is serverless? I knew about the cloud and AWS beforehand. We predominantly work with AWS and AWS Lambda when we talk about serverless at Serverless Guru. That was my first inoculation into the industry. It's been a fantastic ride the entire time. I couldn't be happier with my choice.
00:02:00 - Anthony Campolo
Yeah, that's very cool. And you're a career switcher like I am. You also have an education background, so I'd be curious why you decided to make the switch and how you approached it. Were you a boot camp student, a self-learner, or something else?
00:02:18 - Josh Proto
Sure. One thing I'm always fascinated by in people's stories, including my own, is what stays the same as our lives change, as we grow older, and as we switch careers. I've always been very people-focused and experience-focused. In education I worked in personalized tutoring, helping children and adults who were nontraditional students. Maybe there was a learning difference, ADD, or ADHD, or they were coming back to school after 20 or 30 years. I had to architect situations where learning happens, which is complex because I'm a biochemist by trade. I ran my own chemical company for a while, so I know enough to be dangerous about how memories are formed on a biochemical level. That gives a granular view compared to the human process of learning when we're talking and showing a picture.
[00:03:13] This is a circle. This is the area and circumference of a circle. How do we get someone to memorize that, learn it, and teach it to someone else? I had spent five years getting comfortable with that process. I was also retooling myself to be a game designer because I had played around with it in college: coding in C# on Unity, doing some JavaScript development, and making iOS games with Swift for me and my friends. It was really fun. I was thinking of doing that, but I was reading a Design Principles book and realized I could apply those principles to almost anything. That made me excited. I was playing around with Alexa skills, and I'm a big D&D nerd. I had a campaign running for 11 months, which is a unicorn for campaigns that actually continue to run.
[00:03:59] During that time, I was building Alexa skills and working with AWS in the Portland community to scratch that itch. Those skills gave me the foundation for when I met Ryan. I knew enough to talk about it and to figure out what the market wanted, what excited the market, and what excited participants in this area. Let's start some conversations. There's a tremendous amount of innovation and value when people get together and talk about what interests them, like what we're doing right here. Business is human to human. Unlocking those human skills felt like a big opportunity I saw within the serverless industry.
00:04:43 - Anthony Campolo
Totally. I bet you went down the rabbit hole of spaced repetition and how to optimize learning time, what it takes to internalize something to fully learn it. It's useful to have those prerequisite skills when you want to learn anything, especially coding. You also host a podcast like I do, so I'm curious what drew you to podcasting as a medium, how it interfaces with your business, and where you think the value comes in.
00:05:22 - Josh Proto
Totally. I'll answer that. I just remembered the first part of your first question: any tips for someone who was a code school graduate. First, I think there's a lot of value in a growth mindset when you think about problems and your own skill set. There's always going to be someone who has done it longer, but you can see things with a new mind or apply skills from a different discipline into the new one, which is incredibly valuable. There's also talent discovery that industries may be unaware they have.
As for what inspired me to do podcasts, it really happened around March 2020. The COVID lockdown had recently hit in Oregon.
[00:06:11] Ryan and I were watching everything happen, and we were still seeing a problem in the serverless marketplace. A lot of people were asking us to describe what serverless is or how it is useful, or they weren't sure there was real value. Ryan had been in the weeds building applications and solving problems for companies where it was clearly beneficial. We wanted a platform for describing that, getting the information out there, and engaging with other people who were also doing that. We wanted to be a different voice in the echo chamber of "serverless isn't hitting the mark" or "they still use servers, so how does it call itself serverless?" It's like wireless internet: there's still a wire somewhere; my house still has electrical wiring.
[00:07:01] That's a funny thing I always see. My background in education also pushed me toward it because I wanted to lean into our strengths: we'll sit down with you and teach your team more about serverless. I do that less nowadays at a very granular level, but building frameworks for how we teach something like serverless is what I was interested in solving. The problem is not solved. The market is still very early, but we're in the process of doing that.
00:07:28 - Christopher Burns
I ran serverless functions early in my company's history. We started with serverless functions wrapped in the Serverless Framework, straight to AWS. This was a year and a half ago, maybe two at this point. I found them to be, I wouldn't say lackluster, but I didn't understand the problem they were solving before I tried to use them. At that point I'd never spun up an actual server, configured Nginx, or configured a runtime such as Docker or PM2. The next stage was Vercel functions and Netlify functions, an abstracted layer of AWS. Then I moved on to a full server, hosting PM2 on an Ubuntu droplet. I've seen the whole scale of services at this point. The only side I haven't seen is complete scale to 20,000 servers, but my company isn't big enough to have that problem yet. My biggest question with serverless is how far do you think it's got left to go?
[00:08:47] Like, if you're still searching for gold, have we hit the gold rush yet with serverless?
00:08:53 - Josh Proto
I really like your question. I recently read an article by Mark Hinkle called Has Serverless Jumped the Shark? I liked it because last year I was reading articles about why serverless isn't hitting the mark. This one said serverless is doing pretty well in the sense that it's getting people to think about event-driven architecture and allowing developers to spend less time on infrastructure configuration, which is useful once we get there. They also talked about how trendy serverless is right now, and it's actually dipping. People aren't talking about it as much. It's sort of an old hat, as my grandmother would say. I think there's a lot more gold in the value proposition of serverless. We may understand that now, or we've been able to figure out the use cases where serverless makes sense.
[00:09:44] You don't want one Lambda function supporting your entire monorepo. If you're handling 300,000 requests a minute, maybe don't put that under one Lambda function. But there are instances where it makes sense and it can really increase your team's velocity. We probably understand that now. Now the problem is implementation: being able to implement it when it is the correct use case. I'm also spoiled. I've never had to configure a server or do anything with networking. One of the people on my team, absolutely brilliant, told me that as a consultant they had their own servers and had to move wires around and make sure they weren't overheating. I never thought that would be part of being an engineer or tech consultant, having to do some plumbing and electrical engineering.
[00:10:33] So I think we have found the gold veins. Now we just have to mine it. We know how to find it, and now we need to put in the labor to extract the value.
00:10:45 - Christopher Burns
I've got two contradictory points of view and want your thoughts. This is even controversial for me. The first is Marco Arment. He's a developer who makes Overcast, the podcasting app. He says he runs three servers. He wrote them in PHP because he knew PHP, and he says they're a hassle to get going, but once you do, you just leave them forever and they carry on ticking. Everything is good. The opposite is Snipcart, who did a conference talk where they said, we used to be serverful, and then we snapped a finger and now we're all serverless. The biggest gripe I see and have felt with serverless is cold starts. Are they manageable at this point? Snipcart is a successful company, so if they flipped to serverless functions today, each function would get thousands of hits a minute and cold starts wouldn't be a problem.
[00:11:51] But if you're a tiny startup, cold starts could slow down your app to the point where the service seems really slow, even though you're just trying to grow and get more customers. What do you think the balance is? Do you think cold starts will be a problem that slowly fades away? Do you think we should use hacks like a cold-start pusher to keep the function hot? I'm open to it because I would love to know the answer.
00:12:27 - Josh Proto
I'll do my best to answer. There's a realm of use cases that Serverless Guru doesn't have the strongest grasp of: single founders or very small applications, because that isn't the work we do day in and day out. We're usually handling enterprise production workloads, like an international company with 50,000 people a minute doing something. That's where we sit. That said, the promise of serverless is that everyone benefits from the pay-per-usage model. With AWS's release of provisioned concurrency, I think last year, I really haven't heard cold starts being a practical issue.
00:13:08 - Anthony Campolo
You should dig into what provisioned concurrency is, because that's kind of a niche thing in this realm that our listeners may not know.
00:13:15 - Christopher Burns
What is a cold start, as simply as possible?
00:13:19 - Josh Proto
Sure. I haven't taught on this one in a second. When you execute business logic using a Lambda function, the function needs some time to become active, or the instance where the function is held needs time to become active. A normal request to a server can happen very quickly; I forget if it's 50 milliseconds or less than 10. Different programming languages have different optimized performance in AWS Lambda. If you're using Java, it's going to take longer than Node and JavaScript, for example. That's something we think about because at scale, those millisecond differences add up. In the beginning, to get around cold starts, where there's a gap between when the function code is available to execute and when it actually runs, people used warming functions. That's what you were describing, Chris: pinging the instance to keep the code active so you wouldn't run into a situation where you're trying to pull a request or go to a next page on a website or buy something, and it would take an incredibly long time.
Google has done research on how long is too long for something to load on the internet. Several years ago, if it was 10 seconds or more, you might as well not have a website because the drop-off was around 96%, so functionally no one waits. AWS now has good documentation on provisioned concurrency. One of our team members, Vishnu, did a fantastic talk on load testing for AWS Lambda and dug into the granular details. We had a goal of wanting to break AWS Lambda with requests: how long does it take to start, how many can it process before it shuts off, and how can we play with different dials like provisioned concurrency. High level, I'll describe it as being able to pre-test your Lambda function to be ready for X amount of requests or provision X amount of availability. There's also something interesting we were just talking about: AWS Edge, where you can distribute your functions regionally. I may be off there; I'll get back to that one.
00:15:37 - Anthony Campolo
CloudFront just added functions. That might be what you're thinking of.
00:15:40 - Christopher Burns
It was Lambda@Edge that you're thinking about.
00:15:42 - Josh Proto
Yeah, I'm definitely thinking Lambda@Edge.
00:15:44 - Christopher Burns
If I remember right, when you set up Lambda, you say run in EU West 2. But when you use Lambda@Edge you say run in their closest center.
00:15:56 - Josh Proto
Exactly. You can get marginally more beneficial runtimes, which is great if you're running huge volumes. One reason Serverless Guru is in Portland and AWS/Amazon is in Seattle is because there's a huge regional data center for Amazon, so we get very low latency and fast compute time. But there are places on the map where that isn't possible. With Lambda@Edge, provisioned concurrency, provisioning time, and resources and memory for your Lambda functions to stay awake, we're seeing at an enterprise level that this is fixing it and making it easier to extract value. Then there are interesting conversations around monitoring and tracing, making sure what you spin up isn't costing you a lot or causing unexpected costs.
[00:16:47] That's an interesting situation that comes after that.
00:16:55 - Anthony Campolo
Well, yeah. This is like Corey Quinn, who does the Last Week in AWS and the Screaming in the Cloud podcast. He has an entire business based around optimizing your AWS bill. He calls himself a cloud economist, which is a job that didn't exist ten years ago because people are spending so much and it's so complex with so many different things and places. What I usually hear is the only way to get visibility into all of it is to look at your bill. That's the only place in the AWS console where it's consolidated.
00:17:31 - Josh Proto
I was trying to remember exactly what I was reading last year, but I was doing a lot of research and reading a bunch of AWS Lambda case studies. I read that a future role will be a cloud economist. With AWS Lambda, we'll be able to assign a monetary value to our functions: how much money does this function save us, how much does it cost, what's the true value of this code? That's interesting to me to learn how to figure out and calculate internally. We've done calculations like that with clients. There's a raw dollar value of how much you save switching from servers to pay per compute time; you save maybe 120 hours of a server always being on. But there's also a productivity gain.
[00:18:18] If something was breaking and now we have a working function, it's saving X dollars. That's an interesting place to break down.
00:18:26 - Christopher Burns
To further the question on cold starts, let's take a simple example. We're going to do a Hello World request back. We're thinking in JavaScript here. Serverless is far more than JavaScript, but we'll start with TypeScript. Before you've even sent your Hello World in TypeScript to AWS to compile, you're compiling it down to ES5. I believe they include ES6 now as transpilation. Then that Hello World function sits on AWS as a server and they give you a URL. Correct me if I'm wrong; this is my knowledge from the last time I looked at it. When you ping that URL, you get your function back saying Hello World. The first time you ping it you could get a one-second delay. That's the cold start. Every other time it's 50 milliseconds. You leave it five minutes, ten minutes, and then you do it again, you're going to hit the cold start a second time.
[00:19:30] But my next question, the greatest question of all time. Do your Lambdas slow down faster than an always-online server as a function becomes more complicated? Say you're running the same JavaScript logic: importing five modules of various complexities into a serverless function versus running that function on an always-online server. We're not talking about an eight-core CPU versus 200MB RAM; let's treat them as equals. Would you say performance is the same?
00:20:17 - Josh Proto
Assuming the same amount of memory usage between a server and Lambda?
00:20:22 - Anthony Campolo
Yeah. Does the size of the function itself impact cold start?
00:20:27 - Josh Proto
I believe it does. This is a fantastic question because it's something I haven't thought about much, but I have seen that size of the function and complexity matter. I'm thinking about deployment configuration: how do I package all my code and infrastructure as code so it launches my application, then CI/CD is another part of that. In the past we've optimized by breaking things into small chunks so you're not throwing it all into RAM. There has to be some level where the size of what you're trying to do and how you break it into smaller pieces affects speed, and you get diminishing returns if you try to force it all the way through.
00:21:18 - Anthony Campolo
Yeah. Brian Leroux constantly bangs the drum on this. He's done vigorous research correlating cold start times with the size of your functions. They recommend keeping them under about five megabytes. When you upload to Netlify or Vercel, and probably AWS, there's a size limit where they'll tell you they won't take it, maybe around 50MB. This has happened to people with Redwood apps if they bundle too many dependencies. This is where the term Lambda comes into the conversation. If you build your entire application as something hosted on a single Lambda, the chance it's useful and under that budget size limit is very small. We've trapped ourselves into an unworkable architecture to a degree.
[00:22:22] And this is something I've been thinking about for a while. As Chris and others have hit these issues with serverless functions, I eventually realized it's because we're not doing it right. Plenty of people in the serverless world have been saying this forever, if anyone would pay attention. It's a known problem, but not necessarily understood, because there isn't a lot of great information. They put out a guide recently, I think by the team behind the Serverless Application Model. They do a lot of education. There's an anti-patterns guide, and the first anti-pattern is the lambda. I'm curious: you see lots of these projects and consult for lots of people. Are people building lambda monoliths? Do they know they shouldn't? How is knowledge of this problem in the area?
00:23:22 - Josh Proto
When I started about a year ago, we got a lot of people coming to us saying, "Are you using AWS Lambda? That doesn't work." I said, "Oh, interesting. What happened?" They said, "We tried to put our entire monolith into Lambda and it didn't work." That's the extreme example, as close to Lambda as you can get. There have been instances where, because of client requirements, it's not just about the technology. With infinite time, money, and people, we can have an optimized technological process. When those variables aren't infinite, concessions are made. One role we have as consultants at Serverless Guru is always advocating for technology best practices. We're seeing breakdowns exactly as you said, with no understanding that the sizes of functions matter.
[00:24:16] And it's awesome to have people like Brian. I love Brian. I've talked to him a couple of times on the Talking Serverless podcast. Being in a place where he sees where things break and the data around it is fantastic, because then you can optimize your own projects and services.
00:24:36 - Christopher Burns
I was going to bring in the point where you say this doesn't work. I think 99% of people in a company say, "We've got an Express server. It's worked for five years. Our API is Express." Then they open the first example of the Serverless Framework, that's a Lambda Express, and go, "Okay, easy, copy-paste." The Lambda function now runs the Express app with every single route in it. Then they go, "It's very slow," because it's a completely different experience. They're expecting the function to do route management between 100 functions and then run the function.
What's the best thing to do? One Lambda function per function if possible? Is it better to have ten functions on Lambda but never go over 50? What's the most effective way, without being too hard on DevOps, to distribute an Express application onto a serverless system?
00:25:54 - Josh Proto
I think a good rule of thumb is one function per Lambda, or one executable, one goal per Lambda.
00:26:08 - Anthony Campolo
Single responsibility principle. If you want to get computer sciency about it.
00:26:12 - Josh Proto
Thank you, Anthony. That's perfect because I don't have a traditional computer science background. I would say something like that. That's where it really shines. If you add too many things into Lambda, too many functions, you don't get the true benefit. If it's coupled, it's not ideal. Decoupling the functionality per Lambda function has value. Having that single...
00:26:36 - Anthony Campolo
Single responsibility.
00:26:37 - Josh Proto
Single responsibility. That's where Lambda functions shine. It's hard in the field, though, because there are situations where a company has had an Express server for a while and thinks, "I can do Express and Lambda." Maybe you can, but that doesn't mean it's optimal. In an ideal scenario, you should greenfield it, and use Lambda and API Gateway and other AWS ecosystem tools rather than trying to use a technology that is compatible but not necessarily supportive of a multi-year project with all the problems, use cases, and edge cases that come up.
00:27:25 - Christopher Burns
It's very much that thing: if you look at all the use cases, it doesn't say Coca-Cola smacked their serverless application onto Lambda and it just worked. It's very distinct flows: the Sunday Times uploads an image, then a Lambda function runs to make a mobile version. It's distinct things. My final question: a lot of services in the current ecosystem say serverless is the default option, but behind the scenes they bundle it into an Express app that gets bundled into Lambdas. How can framework providers take the other approach and say, "No, we're not just going to make an Express app behind the scenes. We're going to go single functions per Lambda"? Would that take a lot of work and restructuring? I believe even Next does that by default with their API folder, making Lambda Express the abstracted layer, if that makes sense.
[00:28:33] It's just that they're abstracting it into an Express app.
00:28:36 - Josh Proto
Oh, interesting. So you're saying the abstraction isn't necessarily pure serverless, or it's not using best practices?
00:28:45 - Christopher Burns
Yeah, I definitely think some of them. I'm not going to name names in case I'm wrong.
00:28:49 - Josh Proto
Of course. I'm already thinking of stuff, and I'm not going to name names either. This is a fantastic question. I love it.
00:28:55 - Christopher Burns
Yeah, some of them just abstract it into an Express server, making it a monolith. This isn't against any company or framework, but they start with: "Yep, we've tested serverless functions. They work great. We've written five Hello Worlds in different situations." Then companies use them and want to put 50 functions on it. Things start to slow down. Things don't work like they did when it was empty. The promise of speed and organization gets really hard. How do we get back to the core principles of serverless and make sure we're using it as it was meant to be used?
00:29:40 - Josh Proto
For that last part, how do we make sure we're using it correctly? How do I make sure I'm able to bench 250 now and 300 later? I haven't benched that much in a long time, but I used to be a powerlifter. It's like anything: discipline. The technology is more advanced and allows you to do so much, but only if you fully adopt it and marry yourself a little bit to the process. Part of solving that problem is being committed to solid foundations, whether on your team, organization, or framework. It's something you work on every day and constantly learn. Our consultants are in charge of client work.
[00:30:28] I'm not one of them as much, but we have a Slack channel with AWS updates. People read them, comment, test, tweak, and share problems that are breaking down. There are times when things break and we don't have a solution, but we're trying to solve it. We're committed to the technologies and the paradigm, and that's the only way we make it work. You can't just open a box and say, "Make my organization serverless like Coca-Cola." The case study isn't "We destroyed all our data centers globally and now we just run serverless." That isn't what happens. They could probably never get board approval for that anyway.
00:31:02 - Anthony Campolo
You just hit the big turn off the data centers button, right?
00:31:04 - Josh Proto
Yeah, exactly. I think it looks like a Coke can and they squish it or something. I'm not totally sure. If you want to sponsor me, Coca-Cola, feel free. I think you asked a really amazing question, Chris, because it's about business market strategies around abstraction layers. What happens when your abstraction layers aren't using serverless best practices? Then you get to see at scale how valuable the best practices are. Maybe there's a world where everything could run as Lambda off Express and it's fine, and we're arguing for the purity of the principle. We'll eventually see who wins out and what major problems people are willing to pick service providers for. That's a high-level question most people aren't thinking about at an individual level.
[00:31:53] Maybe an executive team member making the decision about what provider to use or what framework to base a five-year plan on is thinking about it. A good evaluation is whether they're following best practices and what benefits and risks come with that versus someone who isn't. That's important research for people to do.
00:32:19 - Anthony Campolo
This is a good segue to what I want to get into next: abstraction frameworks. Not necessarily abstracting away the Lambda part itself, but abstracting other pieces that hook into this architecture. You mentioned event-driven architecture at the beginning: events coming into your system, like email signups or image resizing, small single-responsibility tasks contained in a single event. Lambda processes the event, but it still has to be sent to a database, or you need a queue before it goes to Lambda. There are lots of pieces that make up larger applications, and we see frameworks optimized for building them: CDK, SAM, or Amplify within AWS. Then there are things like Serverless Framework, which predates those, and Arc. There are also tools that aren't AWS-specific, like Terraform and Pulumi.
[00:33:25] So we get into infrastructure-as-code stuff, which I've been talking about on this podcast for months because I find it interesting and powerful. I'm curious: you guys have been big into Serverless Framework, but you're aware of the entire space. When would you say Serverless Framework is enough, versus when you'd look at other options, or when you should hand-roll everything?
00:33:56 - Josh Proto
There's definitely value to using a framework. At Serverless Guru, we're predominantly using Serverless Framework. We're also partners and enjoy working with the team at Serverless, Inc. We have a relationship with them. As a consultancy, our policy is always to do what's best for the client and solve the problems causing real pain for them and their organizations. Because of that, we've done a lot of research on different frameworks. We've done work with SAM or CDK, and Pulumi. We did exploratory research into Pulumi last year and a couple of others that escape my mind. I was on Slack the other week and saw our teams talking about how certain languages are supported, but if we want advanced features the client needs, we shouldn't use Serverless Framework, we should use something else, or use Serverless Framework because it's optimized for that language.
[00:34:52] There are granular levels where you might not figure it out until you're doing large workloads or heavy lifting to optimize queuing or native invocations. I may have forgotten your full question, so let me make sure I'm answering it.
00:35:07 - Anthony Campolo
Yeah, that was fine. Mostly I was curious how you think about the different options available to you. What I got out of that is language is very important. They don't all support the same languages, and just because they support a language doesn't mean they support all its dependencies. That's interesting because we've been talking about Lambda as if it's just Node. Python has been well supported for a while. Then I think the enterprise demanded Java, so everyone felt the need to put Java into these things, which makes no sense.
00:35:45 - Josh Proto
You can put C# in it too. We've been doing that recently because it's a requirement we have to solve for. Another point on frameworks: it's early. Frameworks are at a place where more people using them will make them better over time. That's the play a lot of them are making, because the more people use it, the more it breaks and gets fixed. Another thing to look at, and a huge reason we were interested in Serverless Framework, is that it has a bunch of third-party plugins people build and fix in the community. That's a great place to be, especially as a later early adopter. Someone may have already found a problem with your application or a certain use case and built a plugin to fix it.
[00:36:34] I'm definitely ignorant about the third-party and open-source support for other frameworks, but I'm sure they're not the only ones. That's another thing to evaluate, because if you have third-party contributions, the ecosystem can grow and get better independent of your team. If you can scale the efficiency of a system outside your own business, time, and human resources, there's extra value in that ecosystem.
00:37:05 - Anthony Campolo
They had the first-mover advantage of getting started. That's why so many things are built for it, and it will be hard for anyone to catch up, though not impossible.
00:37:15 - Josh Proto
And the domain name. They got the SEO: serverless.com.
00:37:18 - Christopher Burns
My final question, and I think this will wrap it. We figured out that the gold is still in the ground and we're just starting to find it. Who will make the first move to truly expand serverless to its full potential? Will it be first-party services like AWS? Third-party ones like Netlify and Vercel? Or a framework like Serverless.com?
00:37:48 - Josh Proto
I haven't done a business model comparison between AWS, Netlify, Vercel, and the framework level. But I think it's going to be there. As far as the cloud provider, there's a world where a new cloud provider specializes in serverless and promotes it. There's asymmetric motivation for AWS, GCP, or Azure to promote serverless because they also have clients paying for virtual machines. There's an interesting battle there that I don't know a ton about, but I can smell the scent a little bit. In the short term it's going to be either at a framework level or that tier layer-two level. That being said, by the time AWS gets around to saying, "We're done with virtual machines; we're going to pump serverless," it's going to behoove people like us who are interested in technology and serverless now to understand patterns and best practices. Once they turn the switch, there will be a higher deluge of organizations and individuals looking to implement it. I have to remind myself of that because I'm in this world day to day, trying to learn and support my team who is doing this. We're still early, really early. Even if the hype goes down to a tenth of what it is now on Google Trends, the conversations I'm having with clients and people on the business side are that it's valuable. They're saying yes. It's popular to them.
[00:39:33] That's where it matters. Having demand from an average person, like a logistics company saying, "We should use serverless," or a furniture company saying the same. There are legacy industries that aren't purely tech-focused that will be interested. That's a hidden area of demand that will keep increasing outside the first three you mentioned.
00:39:58 - Christopher Burns
Like, you go into a sofa shop to order a sofa. You don't care if it doesn't order in milliseconds. You want to know it ordered in a minute, because you're used to a Windows form from 2000. My perspective on the quantum leap for serverless is that it's a mixture of all of them. For raw performance, we have to see that from a provider level, first or second. For integration, the interesting side is between the frameworks and the provider. How close to a tee can they get it? I would love to get to a point where you can open a PR on your React branch and it builds the functions and database required to test that PR inside a tiny environment that you can then tuck away. The industry isn't there yet, but I think in a year or two it will be.
00:40:59 - Anthony Campolo
I think we'll see innovation from layer one, layer two, and frameworks. But if I had to bet on what the quantum leap will be, it's not going to come from any of those. It's going to come from new layer ones. That will change everything: Cloudflare, Fastly, and other clouds building edge-native serverless functions. That's where you avoid cold starts from the beginning and architect it so you don't have to think about that. The question is how you migrate Lambda workloads to this new infrastructure because it won't directly port. The technology will get there, but it won't necessarily be compatible with old serverless technology. That's the complicated thing we have to figure out.
00:41:57 - Josh Proto
I'm also interested in the marginal benefits of shifting from AWS to Fastly. We all saw what happened to Fastly last week or the week before, when things shut down. When you move to someone niche, there are extra risks. The benefits of those new layer ones have to be good, and they have to provide the same level of security and confidence as legacy L1s. They may be more performant, but to take market share you have to play in the same ring, which will be difficult. I think it's possible.
[00:42:40] I think there's enough people who are interested in something like that.
00:42:43 - Christopher Burns
And a side note: with Fastly going down, it was a user changing a config option. They didn't say what user took down Amazon and half the internet, but it was a user changing a config setting. Everything runs on machines abstracted to the nth degree. Your serverless function runner has a runner underneath it, then an OS runner. It goes all the way down.
00:43:15 - Anthony Campolo
Somewhere in that huge stack is a person who can fat-finger a database and drop it. All right. Thank you so much, Josh. This has been a great conversation. We appreciate your perspective as someone in the trenches working on this technology. Where can listeners find you online or get in contact with you?
00:43:36 - Josh Proto
Yeah. I have a Twitter I use for all things serverless, called ServerlessJosh on Twitter. I think my profile picture is an iteration of myself with much shorter hair, but I promise it's me. I do wear the same beanie. I mentioned my lovely pigeon Giovanni Pepperoni. I'm a pigeon dad, so you can find me there. You can also find us at serverlessguru.com. If you send an email to the contact form or info email, it will go to me and I will see it. I'll be one of the people on the chain. Mention that you saw me on the podcast if you want to talk more or get a virtual coffee. I'm always interested in talking about serverless, business, and innovation. Those are some of the most exciting things.
And we're always interested in collaborations as well. If you like building serverless applications, please ring us up. Maybe there's an opportunity for us to work together.
00:44:30 - Anthony Campolo
Awesome. Thanks so much.