IPFS and the Decentralized Web with Daniel Norman
Daniel Norman explains IPFS, content addressing, gateways, pinning, DHTs, moderation, and 2022 plans for browsers and developer tooling.
Episode Description
Daniel Norman explains IPFS, content addressing, gateways, pinning, DHTs, moderation, and 2022 plans for browsers and developer tooling.
Episode Summary
Anthony Campolo and Daniel Norman discuss IPFS and the decentralized web in this 2022 QuickNode Twitter Space. Norman traces his path from Bitcoin, Scuttlebutt, and Ethereum to development at Aragon and developer advocacy at Prisma and IPFS, contrasting Web3’s distributed complexity with Web2’s polished tooling. He explains IPFS as a content-addressed alternative to HTTP’s location addressing, using CIDs to identify data independently of where it is stored. The conversation then covers HTTP gateways, pinning services such as Filebase, content discovery and retrieval, provider records, peer IDs, and the Kademlia-based DHT. Norman also addresses permanence and abuse through gateway-level denial lists, before discussing browser integration, low-powered devices, mobile experiments, WebTransport, QUIC, libp2p, and improved developer abstractions. He closes with documentation, ProtoSchool, and community forums as learning resources.
Speakers
- Anthony Campolo
- Daniel Norman
Chapters
00:00:00 - Daniel Norman’s Advocacy Role and IPFS Work
Daniel Norman describes developer advocacy as helping newcomers navigate an ecosystem and become productive. In his 2022 IPFS role, that includes answering forum and chat questions, maintaining ecosystem websites, learning unfamiliar systems, and turning those discoveries into documentation, guides, or videos. He is working on a guide to debugging content routing, including what happens after a file is added to a local IPFS node.
Anthony Campolo establishes their earlier connection through Prisma and Redwood.js, then asks how Norman moved from Web2 development into decentralized technologies. The opening frames IPFS not only as a protocol to explain, but also as an ecosystem whose complexity requires better educational material and developer support.
00:02:02 - From Bitcoin and Ethereum to Aragon
Norman recalls encountering Bitcoin around 2010 or 2011 without yet understanding its cryptographic foundations. Reading Andreas Antonopoulos’s Mastering Bitcoin around 2016–2017 helped him connect elliptic curves, signatures, and public-key cryptography to practical systems. He was also drawn to Scuttlebutt, a peer-to-peer social network, because it challenged the centralized services that had defined his early experience of the internet.
Ethereum expanded that interest by making programmable applications possible through smart contracts. Norman joined Aragon after completing a coding challenge involving Ethereum data and browser rendering. He explains that Aragon used Ethereum for DAO state while IPFS stored larger application artifacts and releases, revealing how the two networks could work together when blockchain storage was expensive.
00:08:59 - Developer Advocacy, Prisma, and Web3 Tooling
After Aragon’s team was dismantled, Norman took a break, worked on a side project, and discovered Prisma while looking for a Node.js database tool. He joined Prisma as a developer advocate, finding a role that balanced technical work with writing, conversation, and community interaction. He credits Prisma and its ecosystem, including Redwood.js, with helping build a large developer community.
The discussion contrasts Web2’s mature productivity tools with Web3’s more complicated distributed systems. Norman argues that open protocols offer inspectability, forking, and experimentation, but their tooling was still evolving in 2022. He compares the situation to JavaScript’s earlier transition from jQuery to modern frameworks, suggesting that developers should expect an unsettled but improving experience.
00:13:52 - IPFS 101: Content Addressing Instead of Location Addressing
Norman introduces IPFS as a protocol for routing data across the internet in a distributed way. HTTP generally uses location addressing: a client requests information from a particular server or URL. IPFS instead uses content addressing, assigning data a cryptographic identifier derived from its contents. If two nodes add the same cat image with the same parameters, they can obtain the same content identifier, or CID.
Because the CID identifies what the data is rather than where it resides, a requester can retrieve the image from any reachable peer that has it. Norman notes that firewalls, NAT traversal, and connectivity complicate the ideal model, but the core promise remains resilience without a single coordination point. Campolo then asks how this model connects to ordinary websites and HTTP.
00:18:25 - HTTP Gateways and Accessing IPFS Content
Norman explains that IPFS gateways make the newer protocol usable through familiar HTTP clients. A gateway is server software that speaks HTTP to an application while also querying the IPFS network. A developer can request a CID in a URL; the gateway checks whether it has the content locally and, if necessary, asks other peers to locate and retrieve it.
This arrangement lets developers build HTTP-based applications while relying underneath on content addressing. Norman distinguishes the gateway experience from the underlying implementation, mentioning the distributed hash table and Bitswap as important IPFS components that most application developers need not handle directly. The gateway therefore acts as a practical bridge between established Web tooling and distributed retrieval.
00:24:30 - Pinning, Storage Redundancy, and Retrieval
The conversation turns to what keeps IPFS data available. Pinning tells a local node to retain particular content rather than remove it during garbage collection. Since a laptop may go offline, users can pin the same CID through multiple nodes or services, such as Filebase or Web3.Storage, increasing redundancy for an image gallery or similar application.
Norman describes a pinning service as an IPFS node that accepts API requests to retain content. He presents Filebase as a 2022 example with an S3-compatible interface, allowing familiar tools to upload data while making it available through IPFS. He then separates content discovery, finding a peer that has the CID, from content retrieval, opening a connection and transferring the bytes.
00:29:31 - Provider Records and the Kademlia DHT
Norman explains how IPFS nodes announce content through a distributed hash table. A node publishes provider records that map a CID to a peer ID, rather than placing the full file in the DHT. Other nodes can query the DHT to discover which peers may provide the requested content. New nodes bootstrap by connecting to known peers, then exchange information and issue get or put operations.
The design uses a key space based on 256-bit identifiers and draws on Kademlia, the DHT design discussed in the conversation. A second record maps a peer ID to an IP address. Keeping identity separate from network location matters because an address can change when a laptop moves or a user changes networks, while the peer ID remains stable.
00:35:45 - Permanence, Abuse, and Gateway-Level Denial Lists
Campolo raises the problem of content that remains available on a distributed network even when people have legitimate reasons to remove it. Norman says IPFS itself is neutral and does not impose a universal content filter, but content addressing enables practical controls around particular CIDs. Public gateways can maintain lists of identifiers associated with malware, copyright violations, or abusive material.
He describes the Bad Bits denylist and related work, current in 2022, toward a more standardized format that different node and gateway operators could use. Norman emphasizes that these lists operate as an additional policy layer rather than changing the underlying protocol. The exchange presents moderation as a matter of gateway and operator coordination within an open network.
00:38:58 - Browser Integration and New IPFS Implementations
Norman outlines several areas he finds promising in 2022, beginning with deeper browser support. Brave already supports IPFS URLs, while the IPFS Companion extension offers related functionality in other browsers. He distinguishes direct ipfs:// handling from gateway URLs, including path and subdomain approaches, as Anthony describes the confusing behavior he has encountered while experimenting with Brave.
The discussion extends to low-powered and satellite-connected devices, mobile applications, and higher-level abstractions that could make IPFS easier to use. Norman mentions Durin as an early mobile proof of concept and describes ongoing research into new environments. These experiments connect his infrastructure interests with the developer-advocacy goal of making distributed applications more approachable.
00:43:07 - WebTransport, QUIC, and libp2p
Norman introduces WebTransport as a browser networking development built on HTTP/3 and QUIC over UDP. He explains why browser-to-node connectivity matters for IPFS and discusses the constraints imposed by conventional TLS certificate workflows. The proposed changes could make it easier for browser applications to connect with distributed peers while retaining encrypted transport.
He connects this work to libp2p, the networking library used by IPFS and other projects, including Ethereum’s proof-of-stake work. He also mentions new implementations such as the Rust-based Iroh project and briefly acknowledges Filecoin as a related topic beyond the session’s scope. These are developments he is following in 2022 as the ecosystem works toward better interoperability and developer tools.
00:45:50 - Learning Resources and Closing Discussion
Norman points listeners to the IPFS documentation and ProtoSchool. The documentation was being improved for newcomers, while ProtoSchool offered lessons on content addressing, Merkle DAGs, data structures, and the fundamentals beneath IPFS. For questions, he recommends the IPFS forum and chat, noting that the community was working to be welcoming to people starting from basic concepts.
Campolo asks how listeners can contact Norman, and Norman says his direct messages, forum activity, and IPFS chat are useful routes. Campolo thanks him for explaining IPFS from its conceptual foundation through its ecosystem and future directions, says they should continue the discussion in a later session, and closes the space. Norman thanks Anthony for the questions and says he looks forward to continuing the conversation.
Episode Links
Transcript
This automatic transcript retains the original timestamps and wording apart from clear name corrections. It may contain transcription errors or turns that combine speakers.
00:00:00 - Daniel Norman
actually go
00:00:01 - Anthony Campolo
quite a while back day down so far right yes we do yes we will get into all of that we will give some people a couple minutes to kind of start trickling in but yeah like what do what are you up to these days i mean what is like your just a quick gimme a slice in the day of daniel your job yeah it really
00:00:22 - Daniel Norman
changes you know as an advocate there's so many sort of aspects to this role but it's really i mean sort of from a high level it's to help developers who are sort of coming new to an ecosystem in my current role it's focused on i p f s sort of the decentralized web and so helping them sort of navigate their way and be successful as developers with this new ecosystem that they're getting introduced to so that means like tending to the forums and looking at the chat maintaining the different sort websites that we have we
00:01:00 - Daniel Norman
have ation ecosystem projects website and of course there's always logistics and actually learning stuff i think as an advocate a lot of the time i find myself learning something figuring something out and then using that learning experience is something that perhaps could be scaled by writing about this learning experience or by improving documentation or writing a guide and so specifically now i'm actually working on making a video guide for debugging content routing in i p f s
00:01:40 - Anthony Campolo
as in like whether it's being routed to a content identifier or like an i p n s hash or like where it's actually routed you to in terms of the content
00:01:51 - Daniel Norman
yeah sort of so i mean when you add a file onto say your local i p f s node there's this whole sort of process that it goes through for others in order
00:02:02 - Anthony Campolo
to be able to yes and this is a huge interesting question and we should set a lot more foundation before we dive into that though so absolutely absolutely yeah this is really really great stuff so yeah so daniel you used to work for prisma and i previously and still today have been on the redwood js core team which is a framework that is entirely built prisma redwood could not exist without prisma so you and i you go way back we've been crossing paths in these two communities for so long so i was very excited to
00:02:33 - Anthony Campolo
see that you had been doing things on i p f s at all 'cause you had made this web two to web three switch and i kind of missed it at the time so how long have you been doing this what was your role um previously to web two and like why did you make the switch i would really love to hear about all that
00:02:50 - Daniel Norman
yeah sure so i sort of started with web three i mean at the time i don't know if web three was really a popular term but around two thousand and sixteen
00:02:59 - Anthony Campolo
seventeen i started i'll say i wrote a blog post literally with web two end the title in twenty eighteen
00:03:07 - Daniel Norman
okay okay yeah i mean i think the web three summit in berlin was one in the web three foundation that was created by gavin wood was like i think one of the very first sort of
00:03:17 - Anthony Campolo
milestones he claims to have invented it i have yet to see an actual attribution to prove that but yet he claims he has been the term
00:03:25 - Daniel Norman
yeah yeah so i mean for me it was really like bitcoin was sort of the starting point where i'd heard about bitcoin back in two thousand eleven i think it was or ten and i had no clue what it was and throughout the years i was trying to really figure it out but just my lack of understanding of like the foundational concepts and sort of the computer science behind it this was even like prior to university for me was like my sort of interest in these like okay so they're applying all of these bitcoin is
00:03:55 - Daniel Norman
applying all of these cryptographic building blocks elliptic curves and and cryptographic signatures and public key ography in a really novel way and around two thousand and sixteen seventeen i was reading andreas anopouolis' book it was called i think mastering bitcoin and that was really like the best breakdown that i had sort of read at that point and then i was like oh okay i get this now i really get this and so i was really excited what was happening there but also in other communities that weren't sort of directly related to cryptocurrencies like scuttlebutt which
00:04:36 - Daniel Norman
is sort of this distributed offline firstrst social network that is also based on sort of peer-to-peer networking and i was always sort of fascinated with this like building these networks that aren't relying on these sort of centralized intermediaries like google and facebook that was sort of the internet that i knew from the nineties when i sort of got into it as as a sort of kid and teenager getting into technology and so i got into bitcoin got into scuttle butt and then ethereum came around and ethereum was sort of starting to really pick up and
00:05:16 - Daniel Norman
you could suddenly build any application because you had this sort of virtual machine that could allow you to really program using smart contract i mean we still have that and that's when i realized okay i really want to get closer to this and i had a friend who was working for a project called aragon which was i think one of the very first dow frameworks for ethereum it still exists you can check it out at aragon was that the betting market one or is that a different no i forgot the name of the you're talking
00:05:52 - Daniel Norman
about the prediction
00:05:54 - Anthony Campolo
yeah as a similar name i would always confuse those two
00:05:58 - Daniel Norman
yeah yeah the name wow it doesn't i know exactly which one you're talking about maybe we can look it up but aragon was my friend was working for agonau aug augger yeah that's the one yes and so my friend was working for aragon and i had left a job in like centralized financial technology it was just this kind of like company doing financial technology i was bored of this and i was looking for something new and interesting and so i reached out to this friend at aragon and he told me hey you know just do
00:06:36 - Daniel Norman
the challenge the coding challenge and the coding challenge was retrieving some block data using i think web three js from the ethereum blockchain and you know doing some rendering in the browser and i into the job i had one week like a trial week in switzerland and that's when i really sort of became familiar with their stack and at aragon they were using ethereum as sort of like the stateholder state machines so to say for like daos so to say so organizations where you you can think of it as kind of like a github organization
00:07:13 - Daniel Norman
even so you can have different people with d different members with different permissions and then you can have different apps like a payroll app or a voting app and essentially you're doing all of this on a experience and they were also using i p f for all of the sort of like stuff that required more space because block space on ethereum blockchain is quite expensive and so i p f s was a really robust solution to store things like first of all versions of the actual application so each time a new version was released it
00:07:46 - Daniel Norman
was published using this paragon package manager onto i p f s with some crypto to cashs and so it f s was used as sort of a block storage so to say whereas ethereum was really holding like the actual dow state and anyone could go and create a dow and know most of the state is stored in ethereum and so the stuff is in it f s and so i sort of really got to experience these two distributed networks ethereum and i p f s of working alongside each other and we had to implement an
00:08:23 - Daniel Norman
immense amount of logic in order to really implement all of these functionalities that in web two was pretty trivial i mean you relied we built this whole layer using r x j s to make it a bit easier to work with events from ethereum and to sort of reduce the state in order to get like the current- that's fairly complicated yeah it was really hell and and after about a year and a bit working there working there sort of team got dismantled the aragon black team and and i sort of left the project and i
00:08:59 - Daniel Norman
was taking a little bit of a break and up to that point i was just working as a software developer but i was very vocal about like all of these different technologies and i loved talking about it with other people and while i was on i was sort of working on a side project with a friend and i was looking know for an o r m or something to interact with a database and i was using no js and up to that point i was-
00:09:25 - Anthony Campolo
wait so you got into prisma via web three yeah that's the irony of it all what
00:09:33 - Daniel Norman
yeah i went back in fourth i had no
00:09:35 - Anthony Campolo
idea they anyone even use prisma for web three
00:09:38 - Daniel Norman
yeah and i mean prisma was also my first role as an advocate so up to that point i was just a developer and i realized okay working remotely full-time as a developer it's a bit difficult for me 'cause i'm somewhat in the middle i'm like an ambivert so i'm not like a full fledge of a who can just be in front of a code editor the whole day i need that kind of like social interaction and to bounce ideas and it was quite hard for me and so after i left aragon i was doing a
00:10:08 - Daniel Norman
bit of writing and and i was working on this side project i discovered prisma i was like wow this is great like the no js ecosystem did not have something like this for a very long time you know i was using sqlize back in the day and next js and i had really pretty much given up on all
00:10:26 - Anthony Campolo
higher level distractions besides i've used type o r m they are an absolute nightmare and prisma has changed the entire javascript ecosystem finally giving people a useful o r m
00:10:40 - Daniel Norman
yes so we share that view and i think i was equally excited the same way that you are and so i applied for this developer advocate role i'd been doing a bit of writing and that was sort of my introduction to developer advocacy and then i realized oh this is a real thing and at prisma i spent about a little over two years and and we really were able to grow thanks to projects like redwood two and all of the other great ecosystem projects and really just a really strong team we were able to grow
00:11:13 - Daniel Norman
this developer community over i think a hundred and fifty thousand developers and it's now the sort of the defaco way to use relational and also document databases with next js redwood and you have also remix remix
00:11:26 - Unidentified speaker
reix
00:11:27 - Anthony Campolo
yep
00:11:27 - Unidentified speaker
mm-hmm
00:11:28 - Daniel Norman
00:12:01 - Daniel Norman
developer tooling space and sort of bring that sleek and developer experience and developer oriented tooling to web three and it's a challenge because well web three is you building these distributed networks that are inherently more complex arguably they're more resilient and interoperable when you're using stripe and all of these great apis that you have you don't really have to think about all of these underlying complexity of like swift banking systems but essentially what a lot of web three is about is just about building these open protocols that hey one can investigate and improve and fork
00:12:48 - Daniel Norman
and experiment with and so i think the next couple of years is still gonna be hairy if you're a programmer sort of working in this space the tooling is gonna change it's almost the evolution of javascript if you remember it from the days of like jquery to where we are now we've sort of gone through these crazy
00:13:07 - Anthony Campolo
i don't remember it actually i learned to react first and that was it i have a bootcamp student that was taught react and nothing else
00:13:14 - Daniel Norman
oh wow wow that's amazing i mean so you probably you didn't miss out on a lot just painful i mean i think yeah reacting in many ways makes things easier i'm nerd easier
00:13:25 - Anthony Campolo
nerd since i'm a history nerd i know what jquery meant and how jquery got worked into into javascript apis itself so like query selector didn't exist until jquery it like so i get that and i like i get to re the rewards of j queryry without having to learn it yeah yeah absolutely cool and also i wanna give a shout out filebase is out there in the crowd if you go to docs filebase dot com you can find a really sweet guide that integrates quicknote and file base together so that's pretty sweet but we've been
00:13:52 - Anthony Campolo
talking about heady concepts here let's dial it back and give the i p f s one oh one for someone who has never even heard about it has no idea what it is no concept of it whatsoever how would you describe ipfs to them
00:14:08 - Daniel Norman
i lost you in the last sentence because i have a time limit on twitter and so my twitter just got cut out for a minute could you repeat the question oh
00:14:15 - Anthony Campolo
sorry i was just saying i was just i p f s one oh one like give the i p f s like brief description from someone never heard about it and needs like the first introduction to it 'cause we haven't given that like what is ipf s
00:14:26 - Daniel Norman
yet yeah so ip f s is a protocol for routing data over the internet in a distributed way many people are probably listening to this are familiar with h tt p hypertext transfer protocol this protocol that's been around for quite a while now since the nineties i believe and ht p is a very very useful protocol for communication between two points right the core idea with h d t p is that you sort of point somewhere you use what is known as location addressing in order to fetch information or data and you can transfer anything
00:15:07 - Daniel Norman
over ht t p any sort of thing that can be serialized as binary information so htt p is very popular but it has sort of one challenge with it that it's very much oriented towards client server communication you have this idea of a server and it's serving your information it's usually constantly connected to the internet and it's serving to you as a client to your browser information that you're interested in and the main point with h dt p is that it uses this notion of location address point to a specific server p f s is
00:15:44 - Daniel Norman
the main difference between say ht t p and ip f s is that ipfs uses this notion of content addressing so instead of finding things like we do in http by where they are by pointing to www dot wikipedia dot com slash whatever you actually address things based on a cryptographic hash which is a signature of what the content is so as a simple example imagine that we have the same image of a cat my machine and on your machine
00:16:24 - Unidentified speaker
and
00:16:25 - Daniel Norman
we add that to i p f s assuming that we have the same parameters the same configuration we're gonna get the same content identified that content identifier is essentially a hash a signature off that cat image it's a very small it's about two hundred and fifty six bit
00:16:43 - Anthony Campolo
by the way i love the example i used to when i was explaining deep learning to people how would explain to them how you can teach a computer to tell a cat from not a cat so i like the cat example
00:16:52 - Daniel Norman
right we always need cats to explain things so okay so we both added this cat onto our i p f s node and we got the same signature now here's the beauty out content addressing and this is why also i p f s is a much more resilient protocol by design it's that when you're trying to fetch that cat image it doesn't matter where it is as long as you can connect to it and it's available on the network which means that both of us are now serving this image if your computer goes down it's
00:17:28 - Daniel Norman
still retrievable because i'm online if i'm offline and you're online it's still retrievable as long as folks can connect to you by the internet so as long as you're really connected to the internet now of course there's a lot of details and firewalls and nat hole punching and a lot of that get in the way of this but fundamentally this idea of i p f s the idea behind i p f s is content addressing now we can talk about what this means in practice how this is implemented but the core idea is that you
00:18:01 - Daniel Norman
can you have a protocol that allows you to retrieve information based on what it is instead of where it and that opens up the whole window to a window of opportunities when it comes to sort of building much more resilient and sort of distributed networks that have no single coordination point so i'll pause there maybe you have some questions and maybe you can guide this in the next
00:18:25 - Anthony Campolo
sort of direction that was all great that' was a really good description for me the things that tripped me up and that i had to learn to like work with this tech was where does h t t p come into the picture that's was like the gateway so if you could explain the gateways and then how do i make this content and h tm l files that i could just exposed so it's like you could just go to this concept and it's a website that's what i find really interesting here yeah no
00:18:55 - Daniel Norman
that's a great sort of leeway to gateways so okay we've built this protocol but what does that mean in practice now like how do we get people to be able use that protocol typically ht t p was introduced to the masses so to say via web browsers so you had mosaic which became netscape and then you had mozilla and you had internet explorer and you had google pro and all these browsers they were essentially different flavors of a web browser that can speak h tt p amongst other things of course knowing how to render h
00:19:36 - Daniel Norman
t m l and c s s and implementing a whole bunch of standards being able to do d n s lookups through the operating system and you know a whole bunch of things but fundamentally browsers were sort of the mainstream software that got h tt p to be adopted now of course it's not only browsers you have also c l i tools like curl and so on that speak the h tt p protocol in fact you have libraries in almost every standard library of a programming language to speak h tt p when it comes to
00:20:14 - Daniel Norman
i p f s i p f s is much younger and so we're not quite at that level of maturity however there are a bunch of ways in which you can use i p f s today and one of them is using what is known as an i p f s gateway so p f s gateway is essentially a server a piece of software that is running on a server that speaks to protocols it speaks ht t p and it speaks the i p f s protocol to be precise here just for the sake of
00:20:48 - Daniel Norman
precision i p f s it' be arguably considered to be two protocols one of them being the d h t protocollia and the other being bitwarf which is the sort
00:21:01 - Anthony Campolo
of bid exchange the binary exchange protocol what does the developer need to know in terms of how those two feeds are distinguished like is that exposed to developer at all or is that just an internal thing
00:21:12 - Daniel Norman
we can get to that but let's just finish up first on the gateway a pin in that yeah put a pin in that yeah let's flag that and we'll get back to it okay so you have this gateway right and a gateway essentially allows you to use content addressing the key concept behind i p f s by speaking only ht t p you don't need to about all of this d h t bitswa all of the implementation details of id f you don't really need to know all of these implementation details what you need to
00:21:43 - Daniel Norman
know is that every piece of content has a unique identifi known as a content id or sid c i d and that can ask the i p f s network using h t t p using an i p f s gateway that speaks h tt p you can ask it for a specific
00:22:09 - Anthony Campolo
this twitter space yeah you could basically reply to it with a tweet and that would be like the closest thing
00:22:16 - Daniel Norman
okay cool so i'll do that in the background but sort of the simplest thing to do in order to sort of use i p f s would be to make an ht t p request to an i p f s gateway now there's many gateways it doesn't matter which gateway you're using for most purposes and you pass along in the u r l the sid that you're asking for once you pass that sid it reaches the gateway the gateway takes that sid and then it checks do i have it locally and if not it will
00:22:49 - Daniel Norman
ask the network hey can you give me this sid does anyone it and it will sort of figure out that whole process of like the distributed fetching that content id and it will return to you in the htt p request that's sort of in the ideal case i'll pause there in case you have any questions and i'll also in the meanwhile get a link to one of those
00:23:12 - Anthony Campolo
yeah i mean i down the i p f s rabbit hole like over six months ago and learned all of what you're just describing already so i'm not the best like new person to come at this like what did or did not make sense to me like that all made perfect sense i've already done all this but i wanna throw out for anyone in the audience who has questions please feel free to just raise your hand at this point we would love to get the up here if they have any questions they wanna ask daniel
00:23:38 - Anthony Campolo
open floor i could ask daniel probably three hours worth of questions so we'll have plenty anyway but if anyone does wanna kind hop up here and get into the conversation please do but i would say continue on from there
00:23:49 - Daniel Norman
dan i wanna say to also the listeners here there is no such thing as a bad question or naive question like really this the simplest question that comes to
00:23:59 - Anthony Campolo
mind yeah and i used to be i used to be a teacher so for me people raising their hand asking questions is quite natural i just
00:24:09 - Daniel Norman
chaired an example gateway link i just replied to the the tweeter space i hope you can see that and um
00:24:21 - Anthony Campolo
yes i do see that that should be good so if aaron wants to see this just go to the original tweet for this space and then we'll find that link there
00:24:30 - Daniel Norman
okay so we spoke about the core idea content addressing we spoke about these s gateways that essentially speak both i p f s and ht t p and then they allow you to build an application that is fully based on ht t p but underlying it uses sort of the underlying notion of content addressing so i think the clearest example for this say you're building an image gallery app and let's use file base as an example because filebase has also recently implemented this thing called the pinning a p i spec which we can get to
00:25:10 - Daniel Norman
so i mentioned that in i p f s you do content addressing but where are your files storage this is where sort of the notion of pinning comes so you might be running an i p f s note on your computer and you might be interested in serving files kind of like how fit torrent used to work right you would download something and then after you uploaded it became available to other peers in the network who would download it from you so i p f s works in a similar fashion in some ways and it
00:25:47 - Daniel Norman
has this notion of pinning so what is pinning pinning is when you add a file to your local i p f s node i p f s then pins it meaning that 'll keep it around and it will not garbage collected so pinning is basically the notion of i want to keep this around now since you can have a single sid stored on multiple locations in i p f s you might actually be interested in pinning it to multiple i p f for example if you're just running a node on your local machine it's probably
00:26:24 - Daniel Norman
gonna go offline at times especially if it's a laptop so you might be interested in using a service like file base or web three storage for that matter in order to tell in order to essentially store multiple copies and increase the redundancy your images in this image gallery that you're building this is why pinning is useful so you can essentially create and ensure that you have multiple copies of your sid on the network and so file base from a high level could be considered and correct me i know you're listening file filebase is essentially a
00:27:03 - Daniel Norman
pinning service i mean filease also supports other networks
00:27:07 - Anthony Campolo
besides ip s but ip f s yeah yeah the question of a pinning service itself would be an interesting one to also pin it and possibly get into later mm-hmmhmm
00:27:18 - Daniel Norman
is just an ip f s node that will accept a p i requests and then in the a p i request you can tell it hey pin thisid for me sometimes you can even just upload the file and it will just upload it will sort of transfer the file and then pin it on your account or make it essentially available to the i p s network and that's all a pinning service is file base is unique in that it gives you an s three interface so you can use all of the different c l i
00:27:52 - Daniel Norman
tools and programming language libraries and sdks that you have for s three so it's it's a web two friendly exactly it's kind of like sneaking the addressing in it gives you an s three a p i but behind the scenes it's making the content that you're uploading available over i p f s so that's what file-based does and you can think of filebased on an abstract level is something similar to what a gateway is and i p f s gateway because an ip ways like saying essentially hey you know you can't speak i p f
00:28:27 - Daniel Norman
s okay let's speak a language that you know h t t p that's great okay let's do ht t p and behind the scenes we'll make the i p f s magic happen for you filebase does a similar thing where he is like okay you don't wanna worry about this whole ping details and figuring out of the details of i p f s will give you an s three compatible a p i and behind the scenes we'll make it available over i p f s and do the content addressing so that's inning now assuming you're
00:28:55 - Daniel Norman
building a gallery app you probably want to make sure that the images in your gallery are available at all times and for that there's sort of two things when you make a request to the i p f s network either directly via node or via an i p f s gateway there are there's sort of a lifecycle that the request goes through and it's it could be sort of abstracted into two steps the first one is content discovery which is how do i find who has it maybe anthony has it maybe filespace has it i
00:29:31 - Daniel Norman
need to find one of these nodes and i need to find their ip address so that i can connect to it and start retrieving that file that's content discovery and then there's content retrieval which is the second step i was able to someone who has it and now i'm actually opening a connection directly to them and retrieving the byes off that file these are the two steps so content discovery and content retrieval the other end of this is when you're adding a file there's content publishing you've got an noun hey this sid is now available
00:30:09 - Daniel Norman
by this peerier or by these two peersers by file base and anthony and each one of you has a unique id that is known as the node i or peer id sorry that's the right word so how does all of this magic happen where this d ht comes in distributed hash table we need a way because i p f s isn't controlled by any single entity we need a way for anyone to be able to productively participate in the network what does that mean it means that well if i'm building this gallery app and bunch
00:30:46 - Daniel Norman
of cat images i need to announce it somehow to the world in a way that anyone can retrieve that record of hey i have it just a record of i have it we're not even talking about the full image because the image is pretty big and this is what this distributed hash table problem solved the distributed hash table is essentially based on the same distributed hash table that is used by fittorent that basically gives you a unique id that is your pure id it's just a random number and that random number is is is sort
00:31:25 - Daniel Norman
of situated in a key space in a range of numbers between zero to two to the power of two fifty six that's because we use two fifty six bits to represent peers and content ids in i p f s and i'm publishing to this gallery all of these new images and i need to publish these records that are called provider records to the d h t how do i do that well i connect to a bunch of other nodes we usually when you start an is no you connect to these strap nodes these are nodes
00:32:09 - Daniel Norman
that we operate at protocol labs but theoretically these can be any node we can even connect to the file bank's i p f s node to bootstrap it and the bootstrapping process is just used in order to exchange information about some peers once the i p f s node establishes to about say twenty peers or so i mean it can be configured in different ways then it can start making queries to this d ht this distributed hash table it can do basically two operations it can do get for a key or a put key and
00:32:46 - Daniel Norman
the value it can either write or and i won't go into the details of how all of this magic actually happens under the hood how all of this distributed hash table actually can work in a way that anyone can use it but sort of the key word here if you're interested in sort of diving deeper into this is look up a paper called delia published by actually someone who's working for i p f s now a researcher was published in two thousand and two and calia is this sort of the implementation that allows us to
00:33:18 - Daniel Norman
do to implement this distributed hash table and so when i add my cat images for the gallery i actually use the put operation i start putting provider records these are records pointing from the unique content id that has been generated from the image like the cryptographic signature pointing to a peer id and the peer id is the unique id that was generated when i created my no a random number that was generated in this keyspace that we mentioned so it's a two hundred fifty six bit number that's my peer id and so these provider records
00:34:01 - Daniel Norman
are basically mapping content to providers and a provider is initially usually just my note but it can also be other nodes for anthony say you become interested in those images you could decide to pin them onto your node and then once you pin them to your node your node will actually start publishing to the d ht provider records pointing from the same sid of the cat image or the multiple cat images which each has sid to your peer id and that's how we achieve redundancy that's how we can achieve these sort of interoperability and sort
00:34:41 - Daniel Norman
of distributed nature then we have a second kind of record and this is an implementation detail that doesn't really matter which maps from your peer id to an ip address now you may ask why not just map from the provider directly to the ip address and the reason is that your ip may change even though your id will not so you might have a might be actually moving with your laptop from one place to another and your ip will change and that's why we have these two kinds of records
00:35:19 - Anthony Campolo
so i'll pause there think [crosstalk] or if for using like a v p n your ip n could be like switching at any point in time or if you're just like on a phone and you're going from like one location to another your ip is changing you now exactly yeah yeah awesome awesome yeah that was an awesome technical deep dive thank you so much for that i'll be curious to zoom out a little bit and talk about some more kinda like high level philosophical questions 'cause i actually did a stream with ben meyerers who does
00:35:45 - Anthony Campolo
a semantic stream where i walked through i p f s and he he brought up a really great question this is a question that i've had myself the question of what happens when people put things on the network that become permanent that others want taken down for legitimate reasons oh that's a great
00:36:03 - Daniel Norman
question so there's a couple of ways to sort of prevent that but by definition the technology i p f s the protocol is does not sort of implement any sort of actual strict denial list or the protocol itself is neutral and it doesn't sort of implement any kind of content filtering however this is the beauty of using content ids right because you have these signatures so there are a couple of approaches that we have and we're actually there's a spec that is kind currently being discussed in the i p f s specs repository about having
00:36:46 - Daniel Norman
a denial list system so we have one website where we publish a list of sids which are consideredconsider inappropriate or use so to say and this could be things like trial pornography and whatnot things that you no one disagrees we don't want to havevailable yeah and
00:37:07 - Anthony Campolo
that's that's when i asked the question like reasons that we all consider legitimate no one wants this content to be out there in the world
00:37:15 - Daniel Norman
yeah we publish what is known as the bad bits denialist and it's basically just a list of sids that have been flagged for various reasons either copyright violation malware or things like pornography and each entry is essentially just a content id that you can sort of connect or you can sort of upload and then you can have that as an additional layer that you sort of block through your public gateway now in addition to this there's also work to actually standardize the format for this and to sort of create an alliance that will allow multiple
00:37:56 - Daniel Norman
is to sort of contribute and to sort of anyone who's operating an i p f s node should be able to plug it into this deny list or that denial list or to have very easy
00:38:05 - Anthony Campolo
sort ofability this is when ben asked this question i said likely people will get together and kind of collectively decide what content they don't want and so that sounds like exactly what you're describing yeah exactly and so
00:38:19 - Daniel Norman
i mean once the standard is in place and i can i'll share a link to this we also have a link we also have an email address and a form for content copyright violations and malwares abuse at ips dot io wait you're saying i can't download movies off of i'm sorry i mean theoretically you can but you might have problems if you're trying to download it from a public gateway
00:38:47 - Anthony Campolo
alright cool cool
00:38:49 - Daniel Norman
so does that answer your question
00:38:51 - Anthony Campolo
that was a fantastic answer yes and that was something i didn't know anything about and have been thinking about myself so yeah i really
00:38:57 - Unidentified speaker
i
00:38:58 - Anthony Campolo
appreciate that answer appreci for sure appreci yeah i'll just open up the floor here like what are some things that you're excited about or wanna talk about or things that are coming up with i p f s like ip is like a massive long-term system you don't work on like the kind of cycles that i think a lot of companies do but i would be curious like now that you're in this like what are you excited about
00:39:18 - Daniel Norman
yeah many things so a couple of things the first is deeper integration into browsers so brave browser already supports i p f s links in the u r l bar
00:39:31 - Anthony Campolo
yeah weird stuff happens with it i use it and i've been trying to figure it out but i like that it at least an attempt to make it work the problem for me is like where do i set the gateway because it's that like de web link and so that doesn't always work it seems because of like there's this difference between the content hash go and like the route or does it go in the sub domainin and like that whole deal
00:39:54 - Daniel Norman
yeah yeah yeah yeah d you're raising all the legitimate sort of like challengeschalleng that are faced by by most people who are new to i p f s you have this subdomain path gateway or a sorry you have a subdomain style resolution and you have a past style resolution and basically this t l d r there is that subdomain is the approach that you want to because of cross origin exactly stuff yes but i'm not even talking about that i'm talking about being able to type in your browser ipfs colon slash slash and then the
00:40:33 - Daniel Norman
sid that is now possible inside brave browser so it's not even using a gateway because what you you were discussing about park style resolution and sub-domain style resolution is a gateway specific feature i'm talking about deep integration into browsers where you can just type in the sid essentially ibfs colon slash slash and then the sid and the browser will figure it out so have that already in brave browser with the full i p f s node in other browsers you can use an extension called i p f s companion which will allow you to do
00:41:14 - Daniel Norman
essentially the same thing so these so getting deeper integration into browsers that's one thing getting on i p f s into low powered environments where you might not be able to afford all of these network connections that's another thing that is really interesting so ipfs in low powered devices and satellite devices that's another thing a third thing is having better higher level abstractions for developers that have that of dx developer experience finesse to them that's another big topic that is being discussed there's a great team at fish called fish they're working on some really really
00:41:55 - Daniel Norman
great stuff and and interesting experiments there so that more developers can sort be onboarded and can start building and be productive building things on top of i p f s and then i think there's still a lot more things and then the final one for me is this sort of new cambrian explosion of i p f s implementations that addresses one of the challenges that i mentioned one of the things i'm excited about which is getting i p f s into new so mobile devices browsers satellites yeah is there an i p f s mobile
00:42:31 - Daniel Norman
app so there is an experiment it's really sort of crude but it is a proof of concept and part of sort this ongoing research and it's called during let me know as soon as i can use it [silence] [silence] you can download it today it's called during which is d u r i n and it's available on the app store and it's really mostly a proof of concept at this point and so i mentioned all of these new i p f s implementations as an i p f s implementation called iro that is rust and
00:43:07 - Daniel Norman
finally there's another sort of exciting thing that is happening in the browser space which is there is this new protocol that is aiming to replace well not replace but to sort of give an upgrade to web sockets called web transportransport and web transport is essentially built on top of http three and which is built on top of quick which is built on top of u d p and there is something really exciting happening with web transport which is you'll be able to do self-signed certificates the way that the browser will accept so it's i don't
00:43:50 - Daniel Norman
wanna get into the hairy details of this but if you just search web transport web transport is this new protocol that is aiming to replace web web sockets and it's already sort of being worked on by the w three c and that will enable a lot more interoperability between browsers using i p f s and i p f s nodes that are running on the server because currently if you want to communicate using a browser with an i p f s node like uber which is the most popular implementation you need to have a t
00:44:30 - Daniel Norman
l s certificate and for t ls certificate you need to get basically through a certificate authority and that really sort of gets in the way of decentralization so being able to initiate a secure and encrypted connection from a browser to any i p f s node in the network would be a huge milestone and this is something that web transport would enable and so a lot of the work on this is being done in this sort of lower level library calledlib peter pink which is essentially the networking stack that is used by i p f
00:45:00 - Daniel Norman
s and of course it's used by a lot of other projects like ethereum two is usinglib e two p and so these are sort of the high level things that i'm excited about you getting i p f s into new environments new runtime a lot of the work done on new implementations new transport protocols and getting i p f s into browsers and i think that's pretty much it i mean there's a whole lot of exciting stuff happening with filecoin and sort of this kind of like an incentivization network for storage that is being built
00:45:35 - Daniel Norman
on top of i p f s called filecoin which is exciting but i think that's just outside the scope of this i'm really like i think there's so many exciting things happening just in the i p f s space that i would stick to that for now
00:45:50 - Anthony Campolo
yeah no it's awesome stuff and i'm gonna get you on fs jamm to do a whole nother hour of this to talk about all this stuff so don't worry about it but yeah i think this has been such a great conversation it was really awesome get a whole overview of i p f s what it is why it's useful the entire ecosystem behind it so the last thing i would just ask is for people who want to get involved and want to learn about i p f s or get started with i p f s
00:46:15 - Anthony Campolo
what would be the best way for them to do that
00:46:18 - Daniel Norman
so two places the i p fs documentation that's one place we're really working hard now to improve the documentation and make it easier for newcomers you can o to the documentation at docs ipfs tech second place is protoschool that's like kind of like a bunch of- i'm
00:46:41 - Anthony Campolo
not familiar with that one actually so i'll have check that
00:46:43 - Daniel Norman
out yeah protoschool check it out it's like we have really like if you wanna understand the underlying fundamentals it'll explained merkel dags and like the underlying data structures and how all of this sort of works under the hood and some of the core concepts behind content addressing these are just sort of two great learning resources beyond that i would highly recommend if you have questions either join us in the chat but even better is if you join in the forum discussed or pfs io or now it's been changed to discuss ipfs tech and yeah i
00:47:17 - Daniel Norman
mean drop your questions we are trying to be as friendly as we can to newcomers to this community and yeah i think those are
00:47:27 - Anthony Campolo
[crosstalk] and then if someone wants to get in touch with you how should they do that
00:47:32 - Daniel Norman
dms are open feel free to slide into dms i'm also very active in the forum and the it f s chat so either one of these places it's a good place to find me but really i check my twitter pretty regularly so i think that's if you really need to reach me
00:47:49 - Anthony Campolo
i can be found there awesome well thank you so much daniel this is a really great conversation and definitely looking forward to chatting more about this but close it out for us and catch us here same time next week for another tour space
00:48:05 - Daniel Norman
anthony this was great thank you so much for having me i really enjoyed your questions and yeah hope to chat soon absolutely