The Implications of ChatGPT for DevRel
Published:
An opinion piece about the implications of ChatGPT on the workflows and careers of Developer Advocates and how they can best utilize it.
Outline
Introduction
I haven’t heard one DevRel person speak publicly yet about this ChatGPT fact. In a matter of minutes, ChatGPT can write a typical sample/example app of the kind DevRel’s usually create for new tools, integrations, tutorials, meetup talks, and conferences.
This is a game changer. Despite the relative simplicity of their functionality, these projects can sometimes take days or weeks to build. Between reading unfamiliar, out-of-date docs, combing through old, open GitHub issues, and waiting for responses on Discord, the time adds up quickly.
But not with ChatGPT. It’ll either give you the immediate solution or tell you why something can’t be done in the way you’re trying to do it, followed by an alternative approach. For example, try telling it to write the most ubiquitous example possible.
I would say the most ubiquitous example possible is a React component fetching from a Node API. After telling it to do that, try telling it to write an Express server implementing the API you want. Bam, you’re done. Not only is it fast, if you prompt ChatGPT to do so, it’ll also improve your error handling.
It’ll make your error handling cleaner and more resilient to edge cases than the average DevRel’s, that’s for sure. With one more prompt, ChatGPT will even go as far as doing something I’ve never seen a single DevRel do in person in my entire career in tech. It’ll write a unit test.
What About Hallucinations?
You don’t have to worry that much about hallucinations because there’s so much training data on libraries like React and Express. It’s already seen every permutation of every app you could build.
Open source code has far more training data available than will ever be needed for learning how to generate those types of examples. ChatGPT falls over when you give it internal, highly proprietary code because it lacks context.
You can’t shove your entire work’s monorepo into the model which is what would be needed to actually ask interesting questions and get correct answers back. But a self contained, step-by-step coding workshop?
I can usually go a few hours with ChatGPT and only have to correct 1-2 slightly muddled code suggestion. Usually it generates reams of code flawlessly and instantly with a few tiny bugs that are obvious to fix at the end.
A ChatGPT Challenge for Developer Advocates
If you doubt this is possible, I challenge you to:
-
Write out the steps of a workshop or presentation you plan to create (a technical one, with live coding and a goal).
-
Include information about the tech stack you intend to use for the presentation (which has to be at least 3 years old and have achieved popularity before September 2021).
-
Then before you write any code, ask ChatGPT to create the workshop first and give it a detailed, step-by-step outline of the presentation.
-
The outline can’t be half assed, you need to actually know, and be able to articulate, what you want to build.
-
After doing that, see what it writes and compare its results to what you end up writing.
Compare both the quality of the code but also how fast ChatGPT is able to generate the code and how long it takes you to write the code. Caveat is that to truly test whether this is possible you have to pay the $20 and use GPT-4. If not, then it doesn’t count since the results are significantly better to the point of them being different tools entirely in my mind.
Always use the most up to date models at all times when working with this tech or don’t even bother. If you don’t want to pay for it, that’s fine. But you need to first understand what the state of the art models are actually doing before you can have an informed opinion on whether the money is worth it or not.
I think the widespread use of the free GPT-3 model is why a lot of people have formed incorrect intuitions about how useful, intelligent, knowledgable, or capable ChatGPT is. GPT3.5 versus GPT4 is like the difference between a hundred dollar ChromeBook and a fully maxed out MacBook Pro.
Why is No One in DevRel Talking About This?
I’m unsure whether no one in DevRel is talking about this because either:
-
Very few Developer Advocates have truly attempted to incorporate ChatGPT into their workflow so no one has realized this yet.
-
Everyone has been using it non-stop for the last 2 months and now takes 20 hours off a week but doesn’t want their boss to know.