The HTTP/2 Spec

We'll go back everyone. This is the change log on your host Adam Stikoviak. This is episode 161 and on today's show We got Ilya Gorg joining us today. He's an internet plumber as he said before working at Google working on the HTTP to spec The precursor to this was known as speedy.

So if you've played with speed whatsoever, you know what what h2 is all about Or what it's what it's proposed to you about the spec is finalized You talk deeply about it. This whole conversation is the definitive conversation around h2 and what it's all about binary binary framing layer Pipelining multiplexing header compression also known as hpack server push TLS time to glass Upgrading support adoption. You name it. We covered it.

This is the definitive conversation around the h2 spec We had three awesome sponsors co-chip code school and also dream host. Our first sponsor is co-chip They're a hosted continuous delivery service focusing on speed security and customizability You can set up co-chip in your app in a matter of seconds and automatically to pull your code when your tests have passed Co-chip supports your get up in your book or project So no worries there and you can get started today with our free plan or you can go with a premium plan I say it's 20% off using our code It'll save you 20% off any plan you choose three months use the code the changeable podcast head to co-chip.com Slash the change all get started and now on to the show. All right, we're back We got Ilya the internet plumber himself on here third time on the show. You think you're I think three times put some into an elite group of people That's right smoking jacket.

No, yes Previously on episode 55 episode 1 44 back in February we talked about you know I've archived Ilya we talked about nightly, which was his turn if it changed all nightly pretty pretty fun stuff Everybody's enjoying that nightly email. We've been shipping out, but welcome back to the show. Yeah, thanks Thanks We're inviting me back to always fun chatting with you guys and today's show is all about HTTP to which will be a tongue twister for me I don't know about you a little bit you get tired of saying HTTP to one whatever I'm tired every other sentence I say can say just be so I don't notice anymore Have you found a way to say it faster or more? I don't know that people can actually understand it oven geeks Well, if you're talking to geeks, you can actually say just H2 which is a valid name for the a opn upgrade token Which we'll talk about at some point H2 I like that that is a lot easier to say H2 is a lot easier But better than HTTP to even want to try to say it just well I still stumble over it, but anyways, so you know welcome back Do you want to give another introduction for those who may not know you are no you're on 55 and 144 We talked about you are and you describe yourself as an internet plumber at Google So anything else you want to add to that?

No, I think that's that's pretty much it. So I work on a developer relations team at Google in particular focus on web performance So work very closely with the Chrome team and as of late I've been doing lots of work in the web performance working group So trying to define and improve existing APIs in the browser to follow developers and build some better applications and H2 is not related to the web group It's part of the IETF effort But it's definitely something that was a big effort within Chrome we'll talk about speedy of course and something that I've been Very passionate about and I guess since this shows all about H2 since you've cracked that and we can say that which saves me some But we're gonna talk about H2 then we've got to go back and talk about H1 and talk about the history of this spec that we've been living in and since you've you've been hanging out in that area What's the best way to talk about the basics of what the original spec was and how it turned into speeding how it turned eventually into H2? Sure, so I'll try to get a brief But as you said it's good to kind of rewind the time clock and understand how we got to where we are today Right, and of course everything starts with HV 0.9 which is Tim Berners Lee basically creates this very simple protocols literally one line that says yet You give it the name of the resource you hit enter and you get back a resource right? So this is about 93 94 and the idea was I just want to retrieve a text document because that's what it's supposed to be H2P, right?

I pre-text transfer protocol, right? Then after that this web thing became kind of popular and people figured that hey We would like to actually fetch other things as well so Somebody had this crazy idea of putting images into a document and other people started inventing other mechanisms to styles things like stylesheets and then of course later we got JavaScript and We started kind of building up these use cases We also realized that it's nice to be able to say like cash a resource instead of having to fetch it all the time So all of these use cases were emerging from the community There wasn't an official effort around it like a working group or an ITF effort now This was basically just people picking up H2P and just building servers and just saying like hey I dreamt up with this cool feature caching so here it is and then other browser and server implementers which is kind of pick and choose and say like okay I like this feature and whatnot and the way we arrived at HV 1.0 which was in 1997 it wasn't actually an official standard in the sense that somebody sat down and wrote everything from beginning to net rather It was a document that tried to capture the existing best practices So in the period of like four years starting from when HV first came out when somebody first introduced it to like four years later There was just a lot of emerging behavior and in HV 1.0 We tried to just like document it and that's all it was there wasn't even like a large attempt to rationalize at all It was just like here the best practices and you'll find on a common web server today So that's that's point nine is that going into one oh everything right so that that's what became one oh If I know it was an attempt to capture like this best practices and a common usage patterns on the web And then once we did that in with one oh there was a second effort Which was one point one which took another about two years or so to actually go back and start like cleaning up the spec So to introduce common language common terms into the specs that it becomes kind of rational and more easier to implement for new servers and user agents and Effectively when HV 1.1 came out which is in 1999 That's you know, that's the web though. That's the HP protocol rather that we've built the web on So there was that initial burst of creativity We captured it we kind of cleaned it up and then we just kind of left it there And as you were well aware if you think about if you've been online in 1999 and then you've visited the web recently the web is very different Right, so we have not just pages. They're full-out applications.

We have video We have all kinds of interesting things happening on the web and during this time the web was evolving But the issue protocol basically stayed where it was and in around 2007 and 2008 The Chrome team when we're working on Chrome realized that as we're building the browser That the protocol had a number of deficiencies which didn't allow us to build and present the pages as quickly as we wanted to the user So there was this new effort kind of a bunch of studies that we started under the name of speedy Which was around Experimenting with the pro call issue protocol and trying to figure out what kind of changes We would have to make through the protocol to address some of the limitations and specifically one of the challenges with HP has been the fact that It's a serialized request response pro call in the sense that if I send if I have a connection And I send the request and say hey, I would like to get the index.html file You have to wait until you get the full response back before you can reuse that connection to ask for a second thing Which may not seem like a big problem and it wasn't really a problem back in the 90s when all you were fetching is a document And maybe a couple of resources but now on average age is fetching over 100 resources Which are things like images JavaScript CSS and all the other stuff and now that you have a hundred resources And you have to do some serial it becomes a bottleneck So, you know, there's work arounds without of course you just open more connections But turns out that's also not great because if you open too many connections it can actually hurt user experience because now you run into troubles with congestion control and Opening unbounded number of connections may hurt the server that you're trying to talk to so realistically we have to cap that and Kind of through experimental Deployments we and other browsers determine that kind of arbitrarily six Connections per origin is the optimal number. So what that means is with hd1 you can fetch up to six resources in parallel Which kind of worked, you know in between 2000 and 2007 but in 2007 realized that that's not sufficient like this does not scale clearly Developers are putting more and more resources. They're building more ambitious applications than to do and to fix this issue We need to kind of rethink the protocol and that was effectively the inception of speedy and Speedy tried to change some of the basics of how messages are exchanged within hsp To address this thing Yeah, totally. So when you talk about the connections and limiting that that's why we get things like You know sprites for example with images.

That's why you have, you know all of your Websites images in one single file so you can sort of sprite around and you see assess to move that as a background image That's where you get practices like in cat needing a job You know seven or eight different JavaScript files that support your your site to operate down into one and that's where you get all these sort of I guess workarounds to get into this six, you know This glorified six number where you guys have figured out that that's the best number to focus on Yeah, exactly. That's a really good point. So we call these things like concatenation and as optimizations today right like bundle all your JavaScript files into app.js or you use an asset pipeline to create this thing on the fly for you And really it's exactly as you pointed out is a workaround It's a workaround for the fact that hsp provides limited parallelism or lack of parallelism if you want to call it that So we've been forced down this path of doing things like sprite images and concatenating files Which actually has a lot of negative side effects So for example say you take see how you've developed a beautiful application which is modular and has Everything all the logic is split into different files and you like it's just a well-engineer software project, right? And now you want to ship it to the client so they can execute it Well today in order to do that the best practice in air quotes there Is that you put all those files into one giant application.js and you ship it to the user Now moments later you realize that hey I made a mistake or maybe to update something so you change a single character or single byte in that one file And now the user has to re-download the entire file Right so you changed one byte and you have to download the whole thing all over again Which of course is expensive data wise it also slows down your application because now you have to download this giant thing when you only Change one little part of it and this is just not a good a user experience So these these best practices or these optimizations like concatenation actually prevent us from deploying effective caching strategies And the reason we were willing to put up with that in the past is precisely because the The latency trade-off of like this this constraint of lack of parallelism was so bad that we just kind of brushed it The brush the caching concerns under the rug and said that's okay We'll just re-download it all over again And so all this led into What was kicked off by Google trademark by Google even speedy spd y it's not an acronym But it looks like it might be and were you a part of the team when this kicked off?

Really like you were the founder of this project or what part did you originally play when sweetie? No, I actually came in after the project got started So I think it's actually interesting to talk about why it started in the first place and it all came down to This one experiment, which was done by a couple of engineers on the chrome team And what they tried to do was they took I think the top 100 websites They and they tried to simulate what would happen in terms of loading performances of the websites If we varied bandwidth and latency independently So say you have a one megabit connection and then you load all the pages And you just measure the how long it's up to load each page And then you double that to two megabits And you measure that again and see if there was a difference right like intuitively you would expect or you would hope rather That going from one to two would make things significantly faster because you can just download a lot more stuff more quickly So they kept increasing that bandwidth and then separately they ran the same experiment for latency So if you just keep increasing or decreasing latency from let's say 100 milliseconds to 200 milliseconds What's the impact and the thing that they realized was that after about five megabits per second, which is more than The average broadband connection speed in the United States So basically if you're on broadband in the United States or in most of the countries You already exceed that five megabits upgrading from five to say 10 megabits Will only give you like a single percentage point improvement in the page loading speed And that's why we're spending too much on their internet too potentially to not get a faster web Not because they're watching Netflix Right, well, that's that's a good point, right? So there are there is some type of traffic on the web that is bandwidth constraint So these large streams like video are definitely bandwidth constraints So if you want to watch HD video, yeah, please go ahead and get yourself a you know 100 megabit connection But upgrading from let's say a five megabit to a hundred megabit connection Does not or will not significantly improve your browsing experience, which is kind of sad, right? If you think about that makes sense Yeah, because you don't see you know ISPs marketing their you know their round trip times or their latency All right, so you can't just go buy their latency, right?

Well, yeah, and they're in there is the actual rub here, right? So repeating the same experiments with Changing latency they saw a linear performance improvement as in the lower the latency There's a very direct relation between lower latency and improving the performance So if you actually rather performance of loading websites or browsing the web So if you really want to have a faster experience of browsing the web, you should find a connection that has the lowest latency But as you said, I'm not aware of single ISP out there that is actually advertising the sort of thing in their marketing, right? Which is a whole other discussion that we should have at some point Yeah, I mean some of that of course out of their control right there's a lot of factors that play in latency And they're just one player in a game of you know, what is it actually like a mesh network is that prepared to say? Yeah, that's where there are many hops between you and the server and the ISP is perhaps the first couple of hops In practice, it turns out that ISPs or the last mile as we call it can contribute significant to mental latency So that's typically because the area is under-provisioned So everybody starts watching Netflix and all of a sudden there's not no capacity and latency suffers and all these things happen So there's definitely a lot that carriers can do to improve this sort of thing But you know, that's that's probably a whole separate discussion So the outcome of this whole experiment was basically the realization that the web is not going to get faster Unless we either decrease latency, which you know, we can't as an outsider Or we re-examine our protocols and figure out what is it that prevents us from utilizing the bandwidth in a better way And one way to improve latency is to like pipeline requests Right?

So instead of serializing every request one after the other, what if we were able to just say, well, I need these 50 resources So let me just send you all 50 requests at the same time HTTP does not allow us to do this but what if it could and that was effectively the premise for speed Like what do we need to do to change the protocol to allow that sort of thing and further if we're going down this path We also know that as the web has evolved and we've put more and more different kinds of resources these resources have different priorities So for example, if I send you 50 resources, the HTML file is very very important to me And the image file is important but not as important as HTML So it'd be kind of nice if I could communicate that to the server and say Here's 50 resources, but I could really use your help when getting HTML file first because that allows me to display something to the user So hd also the part that hd1 rather does not allow you to do that hd2 does And then there's other use cases like dependencies. Let's say you are trying to stream a video file It'd be kind of silly for the server to send you a frame A subsequent frame before it was able to send you the first frame ahead of it Right, it's like I don't want the second frame before I can get to the first one So there's some notion in there of saying like here's two requests But please deliver the other request or the response after you sent me the first one And kind of between all of this that was the foundation of CD and that started in around 2008 We ran some experiments. We saw some significant improvements in terms of the actual performance So at first we were actually able to deploy it or implement the first version in chrome And we also implemented the server portion on some google servers And we ran an experiment where we opted in some users into using CD And there were significant performance improvements for the user for the user that we saw that we're using CD And this is mainly like document-based sites not so much like video-based sites or these non-based sites like non-document sites No, no, it wasn't it wasn't that specific We saw improvements across the board just because we're able to remove these bottlenecks But it is true that the biggest benefits were typically for sites that had a lot of requests because now we were able to pipeline and eliminate These unnecessary Like queuing latencies and head-of-line blocking We're definitely getting into HTTP to landscape here. Um, is there anything else on speed we should cover in terms of like You know as the pre-cars it was the originating experiment to get to what is now the h2 spec So is there anything else we need to cover on speed to sort of migrate into talking deeply about h2?

I think we covered the main points. I guess I'll just say that speed was meant as an experiment And with time because we did see good improvements of performance It was actually becoming well adopted outside of google So we had firefox that enabled speedy as well in their browser Safari announced that they will support it. I think it was from safari 9 I also added support for speedy. So in effect it was becoming a de facto protocol and similarly server support was emerging So there was apache and genx and other servers that were all supporting it and kind of on a on the basis of that h2 working group Said, well, look that clearly there's something here So let's start an effort to modernize HTTP and they did a call for proposals Google and others submitted their proposals google submitted the speedy protocol and after a couple of rounds of discussions and feedback The speed is back was adopted as a basis or starting points with h2 protocol And then from that point forward as h2 development proceeded now speedy was also being developed But it was effectively kind of like an experimental branch where we were able to prototype new ideas Contestums see if they pan out and then that feedback would get merged into h2 So that's kind of history and so h2 development started in around.

I think it was 2011 Before we get into that. I do have kind of a big picture question around the The idea of replacing you know the current application layer protocol h2p was something better The goal being to make the web faster reduce latency, you know network computers, you know Have lots of different technologies at play The network stack for for those who aren't quite familiar the networking side has many layers Depending on how you look at it there's seven or five and Regardless of the way you look at it h2p is that you know that top layer That's the application layer where developers kind of play and integrates with your apps and whatnot Um below that, you know, there's the session layer where you have encrypted connections There's transport layer. This is where you know TCP and UDP are often used and then you have your IP layer Which we're mostly familiar with IP addresses and whatnot was there efforts to Swap out lower down to say well, maybe TCP is not the best way to deliver the web and Was a research that went into that were there efforts? They're trying to do that or have tried to do that No, so funny that you asked yes, even at the very beginning back in 2007 We knew that we would probably need to solve problems at multiple layers But at the same time it didn't make sense to try to tackle everything at once Right, I think that'll have been just too much to bite off Yeah, so initially the focus was on h2p.

So what can we do to solve that? And then once we unblock those issues We will immediately run into the performance issues at the layer below and that in particular here This would be the question about TCP like we solved issues at the application layer of h2p We removed kind of like blocking but turns out that TCP also has its own failure modes with head of wing blocking And now there's actually a different effort within within chrome called quick, which is effectively Think of it as h2p2 over udb. There's some other things But it's an experiment that we're working on now which is using udp to mitigate some of these issues that we're That we're running into within tcd But that's probably a whole other episode So long story short there are some, you know Ancillary efforts that aren't just seeing it the application layer they're going further down like the the tcp layer the transport layers There's a mention that's kind of hanging out there or the udp layer That's right. Yeah, and tcp continues to improve right so there's there's a lot of work happening in all parts of the stock I also think it seems like the further down you go the more tightly ungraded of the operating systems that you are And so perhaps more difficult even as a wide As far as adoption goes with a rollout, you know, we see how ipv6, you know Still out there in droves over years and years of it being pretty much done, I guess or available So perhaps starting at the top and working your way down is even the most effective way to to do it So that makes some sense We're getting into the transition between speeding h2 Um, you cover a little bit of how speedy was being adopted Can you maybe reiterate or at least tell us why why the need to you know move away from speedy as a thing and and uh, And do the h2 thing inside?

Sure, so the intent to the intent here was to standardize on an upper call, right? So to bring in we came up with some ideas and within speedy for how to address the particular issues that we thought were important To move the web forward in terms of performance Um, then you take that to itf and there's just a lot more people around the table with different kinds of experiences different perspectives on what is important And that was intent, uh, behind taking it to to itf and that's the intent of itf and that's where h2p2 was developed So over the period of about two years, there was 14 drafts 14 kind of big milestones along the way, uh, that we went through and now actually back in May So about a month ago now, uh, these specs have been officially published So there's actually two specs one is the h2p2 specification and another one is the hback or the header compression Specification which we can we actually haven't talked about this yet, but we can you know in a little bit And I guess one thing I'll mention here is you kind of hinted at this a little bit earlier Modifying something as big as h2p is a big task, right? And I think this is why to some degree It took us a long to get to this point because there's so much built on h2p that any thought of trying to modify it Uh, would be like a just a huge undertaking. So one interesting Point to keep in mind with h2p2 is that h2p2 does not actually modify any of the semantics of the h2p protocol So the headers and the header names the methods kind of your all your restful stuff All of that is exactly the same and that is intentional So in fact some people criticize h2p2 for not tackling some of the higher level issues that they think should be addressing h2p2 But that was explicitly out of scope when the when we chartered the whole h2 effort because we knew that there's so much that we could do But we wanted to focus on kind of this low-level performance and framing So any application that you have today that runs where h2p you can put that over h2p2 And everything will work as us because nothing has changed domestically and I think this was very very important Whether we should modify some other things kind of less semantic h2p is now beyond that right and that that's like now that we finished h2p2 We can now start that discussion So it's backwards compatible in any way that's what you're trying to say That's right, right so you can deploy like you'll have to swap out the server because it'll need to understand h2p2 But you don't have to modify your application like there's nothing about your application that will be like be h2p2 incompatible Now that said there are things that you can do within your applications to make it perform better or use p2 but that's a very different discussion right like this is the Come back to earlier point like you were concatenated files or perhaps you shouldn't do that now because it actually hurts performance Yeah, we were just talking about actually in the last show we were talking about that And it was sort of the discussion of things that have become in quotes We said earlier which was have become best practices around you know the h2p1 spec that you're doing not to concatenate and You know all that stuff and now that might not be a good thing since it's you have different things for like pipelining or multiplexing and support that better now Right exactly the problem with that I guess and we're just complicated of course I'm application developers I'm always thinking about like how does it complicate my workflows and stuff and and like that sounds great as far as h2p2 actually simplifies my workflow Because I don't need to do all that stuff anymore But it's not like h1's going anywhere anytime soon, so we're still gonna have to support both for you know for years to come Um, let's do this Let's get a high-level overview because we haven't actually talked about what it brings to the table We know what's trying to do which is improve performance reduce latency You know kind of be a more modern protocol for more modern web Ilya, why don't you give us kind of the big tentful tentful features of h2 and then we'll take a break and come back and we'll dig deep in each one All right, sure.

Um, so let's see where to start So we talked about the fact that everything needs to be one is kind of has a serial request response model And that was one of the main things that we wanted to address with h2p2 We know that in h2p1 world we also had these six connections and we want we don't want that either So the first premise that h2p2 started with in speed as well is that we want to optimize and transfer everything over a single tcp connection Like there should be there shouldn't be a reason why we need multiple connections Right opening multiple connections the same server doesn't actually give you anything in terms of like throughput, right? You can transfer everything over the same connection or six Except that you will actually get better performance of the transport layer if you're reuse the same connection because there's just a lot of mechanisms within like say tcp and other protocols that are optimized for Making the best use of available band So one tcp connection is the start and then if you have one speed connection What do we need to do to actually be able to send and receive multiple requests and responses at once? So to do that, we added this notion of framing where a message So an in h2p world a message is a collection of headers Which you just keep all your pairs like get this and you know this header that content length with a number And the actual payload so a message can be split into many different frames So for example headers can be transferred independently of the body and body itself can be split into many different chunks Which we kind of had before right with chunk encoding But you couldn't interleave multiple messages like you couldn't say here's a little bit of the body of the request or the response for this Request that you said and here is the other bit with another request like it you couldn't interleave those And that's what vine refraining provides introduces this notion of a stream ID So in h2p world the requests are we refer to them as streams So you open multiple streams to the server and each one of the streams carries say h2p headers and get headers for all of your requests The server receives all of those streams each stream has an id and it just starts generating responses And in order to send data back it just packages each chunk of data and it depends that stream ID plus some other metadata and sends it back to server And now all of a sudden because we can split these messages into smaller chunks we can actually interleave them So say you get two requests and one is for uh, I don't know a css file another one for an image You start sending the css file because it's the most it has a higher priority Which is something the client communicated to you, but then the server blocks maybe your application is kind of slow So then the server can say okay, well i'll pause that and i'll start sending you the image data And then once the css data once is available once again, it resumes that so now that the data can flow over the single connection It can be prioritized. We no longer need multiple connections, right?

Everything is transferred within the same stream So that's multiple accenting and prioritization and prioritization Another thing that was added Now that we have this notion of streams is a flow control. So if you're familiar with things like tcp4 control This is a similar thing, but it allows you to Express things like I want to receive so here's a request here's a stream and i'm willing to receive up to x many bytes of the stream And then I will tell you when to resume it And this is kind of cool because it actually opens up new opportunities for the client and server to interact in ways which it couldn't be for For example, say images, right Many or not many some image formats allow progressive rendering where you can fetch a little bit of the file and you can render a preview of the image Well before we have to fetch basically the entire image And then display it now with something like flow control. We can say well I have a lot of very important things that need to fetch But I also want to render a preview of the image because that would help me get the page displayed to the user more quickly So i'm willing to accept like 10 kilobytes of the image because that's sufficient for me to render a preview But then after that, I want all the other more important stuff and then once I receive that I can resume that stream and receive the rest of the image So this is fundamental. So this is something that was fundamental in you and previously not possible an h2 provides that and then The other interesting feature that was added is server push So the idea here is Today in h2 one you send a request and get a response and there's a one-to-one correlation But what if the server could actually send you back multiple responses?

And concretely the use case the use case here is you send a request or let's say a page About dot html and then we send you back the html and you immediately come back to us and say well Yes, I also want the style sheet that you declared and In that file, but the server already knew that like the server could know already know about that So why can't they just say here's the index HTML file and by the way, I know that you will also need the style file Or the style sheet file. So here it is as well. So like don't waste the round trip There's absolutely no reason to block on a round trip. I know you will need this so please have it and the sounds kind of crazy But we've actually been already using this.

This is what inlining does right because when you're in line Right the contents of a file you're actually saying don't don't come back and ask me for this likely I know you will need this so here just have it So it effectively formalizes and enables this sort of interaction at the protocol layer Which is nice because one of the side effects of inlining is that you can't cache it independently Right and so now you're inflating the size of the other file It has problems with prioritization or you can't prioritize it You have to invalidate it more frequently so push enables that which is really cool And once again, it's kind of a new capability that you just didn't have access to before So some of these features are kind of direct They're directly addressed to limitations of the previous limitations of the protocol And some of these features just enable fundamentally new patterns of interactions between the client and server that I think We're yet to explore really effectively And we'll talk about I guess the adoption and the current state of servers a little bit later But I think there's just a lot of room for innovation for how we deliver web applications with hd2 Good deal. That's definitely a good overview of hd2. We'll break here. There's one more isn't there?

Yes, there's one more thing. There's one more thing I almost forgot header compression So I mentioned that this is a separate specification and The problem that this was trying to address is that hsp1 allows you to transfer data in compressed form For example, you can jesip your content your text content, right? Which is very nice because it just happens that jesip is very effective like compressing text content typically reducing it It's not file size by like 30 to 80 percent Which is huge savings But the problem is that the actual metadata about the request things like your headers your cookies and all the rest was always transferred uncompressed And over time because we realized so much on headers and cookies and other things this stuff is kind of a crude And we're now sending sometimes megabytes of parameter that metadata I was recently looking at one website which had a lot of analytics and other vegans that During a single page load it was generating one megabyte of traffic of just uncompressed hsp headers What would you try to say? I mean the cookies are maxed out at a certain size, right?

Uh, yeah, different browsers actually have different ways to enforce it But you know if you send enough requests it all adds up pretty quickly It turns out that on average the request response even without cookies adds about 800 bytes of metadata Which doesn't seem like much but then you have to multiply it by a couple hundred requests per page And then if you if you do add cookies you're very quickly approaching you know So pretty significant territory like hundreds of kilobytes So header compression was our way to address that to say like we should be able to compress this And that's what hpack is now hpack actually provides two different mechanisms One is it uses Huffman coding to with a static dictionary just to compress value So you just give it a string and there's a predefined dictionary Which is used to compress these transfer data And then the other mechanism it has is that the client and server keep state about what data has been exchanged So think of something like say the user agent header, right? Which is kind of long string which describes the vaguely describes I should say the user agent in some properties about the device That data does not change between requests, right? But in hv1 we keep sending it on every single request which adds hundreds of bytes of data So with hv2 the way it works is you just send it once That goes into what we call a dynamic table Which basically just remembers that okay this this thing has been sent and it's let's say it's id is 55 So next next time on the next request I can just say 55 And you immediately know that oh, okay He wants to communicate that you you're also sending this header So that significantly reduces the amount of metadata that's being transferred We're talking by a couple of orders of magnitude Where the the lowest overhead of a hv2 stream now is about nine bytes As compared to say nine hundred yeah, nine hundred bytes with hv1 Which makes it very appealing for many other use cases hv2 or hv1 general is very popular outside of browsers as well API traffic in all the rest And but one of the issues has been kind of this high overhead of hv headers And with hv2 that's no longer concerned So you can actually use it for much and many other use cases Good deal. So you got one TCP connection you got request and stream With multiplex and prioritized streams binary framing layer Header compression also known as hback that sort of comprises hv2 in sort of one whack there Yeah, let's take a break for quick we'll come back And talk more deeply about each of these sections here, but we'll hear from a sponsor and we'll be back Dreamhost now has managed VPS hosting built for speed and scalability including solid state drives And that's awesome these VPS's are built for open source developers And now include one click installs of nojs custom Ruby and rvm support Speed speed and more speed is what it's all about Their VPS servers use SSD hard drives and are 20% faster than traditional SATA drives All virtual private servers from Dreamhost include SSD storage Ubuntu 1204 LTS web-based control panel scalable ram which is super awesome You can go from 1 giga ram and easily scale to 8 gigs if you need it Nojs once we can install Ruby version manager unlimited bandwidth unlimited hosted domains unlimited 24 seven support Go check them out and learn more at dreamhost.com slash the change log All right, we're back and so deep dive here on all the details of hdb2 I think the next question really is where from here We got service support bronzer support security concerns Where's the best place to start with the how hdb2 actually becomes a thing to develop each other Let's start with how you how it's implemented and how a conversation between a client and server Goes from h1 to h2 and if they're just how does that all work really?

Right so the upgrade cycle I think this will take a short detour into the security discussion as well or not security Uh-huh So in order to upgrade to hb2 we have to somehow figure out if the client and server support it Right, previously we just assumed that hb1 Now we need to somehow figure that out and ideally we'd like to do that with without any additional latency Because that would kind of defeat the whole purpose of doing the performance optimization So it turns out hp actually provides some mechanisms to do upgrade and negotiation So there's this upgrade flow But for some interesting reasons that is actually not practically useful for hb2 in particular One of the things we learned when we first started experimenting with speedy Was that there's a lot of existing middleware on the web things like proxies even antivirus software running on users computers and other things that are Looking at the flows over port 80 as they as they happen And oftentimes they when they detect something that doesn't smell like ht1 or something that they don't understand They assume that it's either bad malformed or malicious and just shut it down And this is actually not new we actually had the same experience with web sockets because they also flow over port 80 And it turns out that if you try to deploy web sockets over just on encrypted connections Oftentimes things just fail and you have no idea why they fail It's just for some reason the connection is aborted or it hangs or something else But then you switch that same connection into 443 or you run it over TLS and everything's fine So at practice you'll find that today the best practice with deploying web sockets is over TLS And we ran to the same issue with speedy where we you know, we would try to make connections over the unencrypted Channel and sometimes these things would just fail and like 20 to 30 percent of the time connections will just fail Which is obviously unacceptable right if you try to browse the web like imagine trying to open google and 20 to 30 percent of the time It just fails like that. That's not gonna happen Right, that's more than back UX. That's just like we failed the user, right? This is a horrible round.

That's no UX. Right. Exactly. No experience Like for that reason alone, we had to deploy speedy over HTTPS Because it provides an end-to-end encrypted tunnel which means that these intermediaries and other things just see encrypted data flow and encrypted data flow They develop the same so they can't really choke on it or do bad things to it And it turns out that That same reasoning applies to HTTP to so there's the spec itself does not mandate that you use HTTPS It actually provides mechanisms for you to Use HTTP to over unencrypted connections and you can certainly do that But the browsers that have implemented HTTP to which are Chrome and Firefox have a rich chip support for HTTP to Have said that they will only do HTTP to over HTTPS on the public web why discrepancy between the spec and the implementations Well, so the spec just says like there's no reason say you're in a controlled environment Say you have two servers that you control and control the path between them There's no reason for the spec to mandate HTTPS from that And we know that HTTP is being used in variety of different environments now, you know, as a With my security head on I will tell you that even though you think you have a clear path You should still use HTTPS between your server to server communication or some sort of encrypted tunnel Because we know that you know malicious people do malicious things and they sniff on traffic and they will they can do things Bad things with that traffic never so you should yeah, yeah, so you heard your first box Using crypto connections.

It's a good thing. But at the same time, it's like there's no reason There's no fundamental reason why we need to require that Right, so the spec says yes, we can use it but in practice on the web if you want to deploy it You need HTTPS and further the browsers will only negotiate HTTP to over TLS another question is okay So now that we have TLS tunnel, how do we actually know that the client and server support HTTP to in particular and there? There's actually a mechanism called a LPN negotiation, which is the Which is used to negotiate which protocols are supported during the TLS handshake. So the LPN that's right Okay, so what happens there is when the client sends a start to the establishment of the secure channel The client can server negotiate the parameters for the secure connection things like what kind of crypto you're going to use and other stuff And as part of that handshake, there's now an extension that negotiates or rather the client advertising switch protocols that supports And then and one of those protocols is HTTP to if you support it and then the server sees that and it can Confirm that and say okay.

Well, I also support just be too So I will be talking to you in HTTP to so this negotiation happens as part of the TLS handshake and it's important because it does not add actual latency So if you're using HTTPS already that would happen over TLS the negotiation bit And then your server would just automatically know to encode frames with HTTP to format as opposed to HTTP And this is great because it actually makes the whole thing transparent Like the server can take care of all of those details for you and there's no extra latency So just as a quick aside Certain people would complain about the practical requirement of HTTPS to you know Because it raises the barrier to entry on the web it costs money at least until the FF gets their thing rolling It complicates things. I'm you know, it's a hobby project. It's my blog. I don't need security.

What do you say to that? Well, that's that's a very long discussion. So Yeah, there's no there's no such thing as a sensitive traffic on web You can A malicious observer can infer a lot about your behaviors based on Navigation patterns of things that you may not consider to be sensitive But which in fact leak information about you those around you and all the rest so the argument of oh, but I'm just navigating like I'm just serving a personal blog right does not stand in my box and you know, many different people have made very good arguments in both directions I happen to be on the side that like I believe that we shouldn't grip all these things because the The incidentary kind of damage of revealing all of these bits adds up and you can infer a lot about the user and their intent just by observing these traffic patterns Yeah, so then there is the questions the practical concerns over how does the fact my performance? Obviously doing crypto requires more work than not doing crypto.

So how expensive is that it turns out that a decade ago that was actually very expensive Modern CPUs are optimized to execute crypto very very quickly. You don't need dedicated hardware for that sort of thing You did before for example Google Facebook Twitter all the big companies run TLS drilling software like we're not buying additional hardware And further actually as an interesting side effect of deploying things like speed and h2 because we require far fewer connections It actually can decrease your operational cost because you have to maintain fewer sockets You have to do fewer handshakes and handshakes actually the most expensive part of TLS So we've seen studies where if you run a load test against an HTTP once we're in enable h2 The resource usage is actually lower because of all the things we just mentioned So yes, there are costs to it. The whole certificate question It depends on what kind of certificate you need whether you need to support older browsers or not In all the rest. So yes, it adds a bit more complexity But practically speaking it has a requirement because we're just not willing to accept 30% failure rate Right on so while we're talking complexity and knee-jerk reactions I admit that I had a knee-jerk reaction around the binary framing layer when I first heard about it Um being a fan of HTTP and the plaintext aspect of it, of course a binary communication protocol Doesn't doesn't let you just see what's going on down the wire I'm sure you've had that uh come at you What do you say about people that complain about that particular aspect of h2?

Sure, so I think there's a couple of different perspectives that we should untangle there One is implementation second one is comparability right as a user So it turns out that implementation wise binary protocols are actually simpler and easier to implement correctly I happen to have some experience with implementing both. I've implemented both h2 and h2 in Ruby And from my own experience I can tell you that parsing text protocols Which have kind of very ambiguous semantics about things where they terminate how they terminate in all the rest Is much harder than a length prefix binary protocol, which is very particular, right? It just says how many bytes I'm gonna send you And then the type of frame and you ask specific rules for how to parse the data. So in practice, it's actually Easier to implement binary protocols.

It is more provably correct In the sense that there's far fewer edge cases that you have to consider and performance wise Because you're twiddling bits is actually better as well for the server because now you're not parsing strings you're parsing just bits and bytes Right, so that's that's implementation bits There are other implementation concerns with things like hpack and all the rest and there's more complexity on the server in terms of dealing with priorities But that's kind of a separate discussion. Then from the observability part I am sympathetic to The use case of well, I just want to open telnet and type and get and make a request or like a server But realistically, I don't think that's a it's a very compelling use case or very common at that And if you need observability then it is a question of tooling like maybe we just need better tools We already have great plugins for wire shark That will parse all the binary data and show you all of the all the things that you used to Similarly, if you open say chrome dev tools or any dev tools in a browser It doesn't matter whether you're running where you want to be to like all of it is already parsed and you can see all the data there Right and further like if you use TLS, then that's already binary training Right like so there's nothing new here It's just a question of tooling and it like I do see some kind of short-term pain where we go from something that was easily inspectable because you can just run Like a grep on a stream right to something that needs to take the stream parse it and then grab it But that's I think that's very easily solved So it's essentially it sounds like the binary framing layer has allowed you to Provide a more testable provable method to do it rather than with text based where before it had more edge cases that took to work around and more Potential to fail. Yep, exactly. Yep We talked about security.

We talked about So things there doesn't make sense to dive into the hpack whatsoever I know there was kind of two components of that we talked a bit about that But not quite that deeply on how that changes the headers to the fact that it's got the frame layer in the data The header frame and the data frame. I think that particular aspect is most interesting to implementers Um as a user, I'm more thinking about how do I use this, you know, who's using this? What's the upgrade process? How does it impact browser clients and server clients?

Um, maybe we can go there and talk about adoption and who's adopting it how it's rolling out and then how do we adopt it? Sure. Yeah, so guess let's start the beginning with the spec itself So the spec has been finalized. It's been published as rc.

So I believe it's rc 7540 and 7541 Think that's correct. So that's that's out in the wild as of a month ago back in february Firefox was the first browser to enable hp2 in their stable browser So if you're using firefox and you're talking to server that is capable talking hp2 then you're already talking hp2 Which is great chrome has already also enabled support that was shortly after firefox So both browsers support hp2 in stable today i.e has indicated that they will ship hp2 support and actually actually let me make that stronger So i.e their new browser edge is shipping with windows 10 which is coming out I believe at the end of july. I want to say 29th of july and edge supports hp2 So as of early august you can get your hands on our users real users will start getting their hands on on stable version of windows windows 10 Which will have edge which will talk hp2 Safari also supports speedy Safari or apple in general does not generally comment on their plans But there's no absolutely no reason to believe that they will not support hp2 sooner rather than later and in fact because we do not want Maintain to maintain both protocols speedy and hp2 and we want to move people towards hp2 chrome has actually announced that we will deprecate speedy in chrome in early 2016 Which is kind of a nod to the community to say that like hey you've deployed speedy and but there's a better thing and official thing called hp2 So let's move over there. So there's kind of this year race period where we've more or less formalized hp2 to deprecating speedy So that's client support There's also a growing and a good list of implementations for hp2 at for various languages So kind of libraries and all the rest If you go to the hst if you just search for hp2 wiki You will come to a github site that has a list of implementations and maybe we can share that in the notes afterwards So you'll find I think most every popular language you'll find support for it Is that hdb2.gov.io is that the one you're talking about?

Yeah, that's one and at the top there you'll see a link to implementations Yes, and that contains a table. It's just a wiki page on github that contains a list. So if you're working on on an implementation, please do add it there So that's so let's see that's client in terms of servers. There are some really good implementations out there already.

So There is there are a number of big sites that have enabled it on their servers. So twitter, google, facebook, a bunch of others. There are also open source implementations So Coming out of this whole h2f where there's actually a couple of new servers that have emerged for example in ng http and h2o Both built by the hp2 community in in Japan They're very good the second one earlier h2o h2o. Yeah, some high quality h2o nice Yep, and they're actually they're very good.

You can try and deploy them today. You can play with them rather And there is support coming to kind of more popular Service as well. So engine x and has announced that they will support h2 or they will add support h2 sometime and I think q3 or q4 of this year So they support speedy today, but that will be replaced with h2 Yep varnish Said that they're working and that's likely coming kind of in early 2016 That's my understanding. Then there is Apache traffic server already supports hb2 There is a couple of modules for Apache that implement hb2 so What else node there's there's good implementations for hb2.

I mentioned that I built a Ruby version It's a library not a server, but you can build it or a client or server with it So all that to say like the support is coming is growing the client support is there the servers are coming online There is lots of usage of it on the web today already Which I think is really important to note hb2 is not some kind of like a new fangled thing That has not been tested recall the fact that speedy has been been in development in parallel with hb2 and effectively We've been testing all of these ideas for a very long time for like five years plus So a significant portion of the traffic on the web today is already using hb2 and this is a very well tested protocol So it's safe to deploy and production ready Awesome. Well, I think we will take a break here and we come back We'll talk about straddling the line between h1 and h2 and also if you're interested in the nitty-gritty details how you can learn more Let's take a break and we'll be right back All right, put them away put them back put the books back on their shelf You don't need them and learn to code by doing with code school code school offers a variety of courses JavaScript h2 ml css ruby ios get and many many more to help you expand your skills and learn new technologies Code school knows that learning the code can be a daunting task And they've combined experienced instructors with proven learning techniques to make coding educational and memorable It gives you the confidence you need to continue pass those rough tough hurdles that you will definitely face learning the code Code school also knows that languages are a moving target They're always updating their content to give you the latest and the greatest learning resources You can even try before you buy roughly one out of every five courses on code school is absolutely and totally free This includes instructor classes on get ruby jquery and much more which allow free members to play full courses with coding challenges all included You can also pay as you go one monthly fee gives you access to every code school course And if you ever need a breather take a break you can spend your account at any time Don't worry your account history your points your badges they'll all be there and ready to pick things up again Get started on surfing your skills today at code school dot com. Once again, that is code school dot com All right, we are back. We are talking with elia gorgic about htp2 aka h2 aka the new hotness in Performance It's here.

It's arrived. The spec is finalized support is coming h1's gonna be around for a long time, isn't it? It's not going to wear. No, we're not gonna fall on our way.

It's with us for a while But the nice news is is that as People that deploy websites Doesn't matter too much because we can just wait for the client to say they support The new hotness and then just kind of upgrade them and not worry about anything else Is there anything at an application level? I mean server push I thought maybe would play in but really I just like pushing assets It's not like server push as opposed to long bowling Is there anything at the application level somebody who's building a web app that they have that they could like leverage in h2 feature wise? Yeah, definitely there's it comes back to the question of Weather and what you can do to optimize for h2, right? So we said earlier that any application will continue to work over h2 So if you just deploy a new server that happens to talk h2 everything will just work Then the question becomes can I make it better?

And the answer is there's probably most definitely yes And this is where we have to get into a whole separate discussion about like let's reexamine some of our existing In air quotes best practices to revisit them and see if we can undo some of the damage So I don't think we have enough time to get into all of these but things like the main charting is clearly an anti-pattern on On h2 so you want to avoid that and there's actually some good and nifty tricks that will allow you to do that without changing anything in your application So I'll just I have a Resource that talks about that in particular that we'll mention later then there's concatenation So you can actually start on doing some of that and then the question becomes like well If I want to optimize for both protocols because that client's in both camps How do I do that? And I think the practical answer there is you can make it arbitrarily complex in the sense that you can actually say Well, I know which protocol the client negotiated so sort of one version of the website here and another version of said there Perhaps it's a little too involved. I think more interesting question that becomes like maybe there's a happy middle As more and more users migrate to where they speak to I think you will see a very quick rise in adoption of h2 in terms of capabilities I think the server basically by the end of this year will have very good service support I expect very good client support as well and at that point the majority of the user users are on h2 I think you I think that should be sufficient to know most websites to say well Now we're willing to sacrifice some of the performance in h2 one without adding too much complexity in our in our actual development process In order to optimize for h2 so perhaps we'll undo some of the concatenation Maybe we won't ship a hundred files, but we can now consider sending 10 files, right? Which may have some slightly negative repercussions for h2 one, but it makes things much better because we can do caching and all these other things Similar things with server push.

There's lots of cool opportunities there for total automation. So one cool example is jedi jedi java hc web server has had support for h2 for a long time And they actually have this really cool mode where the server observes the request that come in So say the client requests the index file and then and then the client also comes back and asks asks the css file and java's your file There's a refer header that it can use to infer Kind of that map of well. Yes, there's an income back for these other things It observes that traffic pattern and then automatically starts server push for future clients So you as a developer don't have to do anything the server just takes care of all of all these things So I think we're going to see a lot of innovation in that space. There's new capabilities in terms of like well, can you push Cash and validation.

So if I told you to cash something for a year, can I push a record that says well, please delete that after you cash So I think those are the new and interesting things that we're still yet to explore But we're in pretty good shape Awesome. So I think by now our listeners are probably in two camps camp 1 is thinking h2 tp2 sounds awesome I can't wait till nginx supports it. So I just get better performance kind of out of the box From my web app and then camp 2 is probably like I want to dig in. I want to understand hpack I want to understand server push and the binary framing and stuff I think for the second group we have good news with regards to your book high performance browser networking.

You want to tell us about that? Yeah, so that's a book I've been working on or worked on but the right it's available today And the book actually came out. Let's see a year and a half ago And in the original version I talked about the earlier draft of h2 and speedy and I recently updated that So the print version does not have the latest content But the good news is that you can go online and read the up-to-date content for free in your browser And you can find it at hpbn.co And if you just add slash h2 You can just go to the chapter directly or you can just cancel the entire book and it covers all the things we talked about here But probably much more coherently and in more detail So you can learn about kind of all the nitty gritty of the protocol and there's also a follow-up section on Well, now that we have this, how can we what should we revisit and how can we optimize our applications? Also, so we'll link that up in the show notes and you did hear him write the print versions not up-to-date So whatever you do don't buy the book So I should say we've been updating so the beauty about writing technical books as they go out of date the moment you hit publish So I've been updating the content kind of in small and incremental bits ever since it was published And we've actually released updates print updates since then and the hp2 one is just much newer So it hasn't yet made it into the print version So if you were to buy a copy today when we're recording this it wouldn't have the latest hp2 but the plan is to definitely have it there And you mentioned potentially like a smaller volume just for these new protocol coming in place and sort of diving deep into that That's right.

Actually, if you go to the O'Reilly website and search for hp2 now They published the the new chapter the new hp2 chapter as a separate ebook So if you just want to read that you can get that as a kindle PDF or one of whatever version you prefer Gotcha Well, we'll definitely go up the chapter 12 is the chapter it is So if you're on the high performance browser networking Side and you're already there then just navigate the chapter 12 and you'll see the new chapter there for hp2 Can you restate the the URL that you said was the blessed one for you because I got the long version Which I think was a redirect. Are you tracking that or something like that? Yeah, it's just it's just more convenient to sort in person So it's hpbn which is the high performance browser networking is just the first letters of the book Dot co slash hp2. Yep, and that should take you to I guess chapter 12.

Gotcha. Perfect Well, Jerry, what else we got to cover. I know we talked about pretty much everything the only thing we didn't talk about really was was Something was sort of off topic but sort of in this can't to a degree which was Ilya, your focus on time to glass and Jared and I before they called we talked about time to glass I was like, well, what is what is time to last year? Do you know what this is a turn you've heard before and Jerry?

What did you say? Uh, google glass time to google glass How to get our website on the google glass fast like you thought it was So I want to take someone to buy google glass. That's your time to glass. Yeah, I think today.

That's pretty long quite the weightless So we talked about h2 and what that's bringing we talked quite a bit about straight line and and What you're suspected best practices to support h2h1 at the same time It's just a you know sort of monitor your your usage and as the number teeters more towards h2 Support more performance enhancements for that are prescribed by h2 versus h1 But what is this? Thousand milliseconds time to glass challenge that was in one of your slides as you talked about h2 Sure, so the the general premise is that we want to make pages visible or respond to user input within less than a second And that means like from the moment that you type in that you're relative to you hitting enter regardless of what type of network you're on Whether you're on a mobile phone with a crappy connection or on a fast gigabit connection You should have something visible to you within a second and if you accept that as a challenge then it becomes a question of Well, okay So what are the mechanics behind that like if you factor in all the latencies for setting up a connection and making the request and getting a response You very quickly arrive at the conclusion that you actually don't have a lot of time like a second seems like a long A lot of time, but it's actually not very much like it puts very hard constraints on how quickly your server has to respond how you structure your pages And a lot of other details and actually if you're curious about this particular topic, I'll do another plug I actually have a Udacity course, which It's probably take you a couple of hours to go through but we kind of talked through in great detail About how the browser actually constructs the page like from receiving the first HTML bytes to parsing CSS and executing JavaScript What does that pipeline look and what do you need to think about to allow the browser to paint something very quickly? Because you can take the same page with the same assets and you can make it block not rendering a thing for like arbitrarily long say five or ten seconds And you can make the same page render something very quickly But then continue progressively adding contents later and then the latter is the behavior that we want to enable So this is both on the browser like there's a lot of things that the browser has to do to make this happen But also on the developer because there are certain things that you do that the browser can just can't work around And the reason why it's accompanied in your HTTP to us is because a lot of this is really depending upon This entire conversation we just have which was h2 being more supported and that's gonna like getting to one second on h1 It's probably a lot harder when you're impossible, but actually it's more possible Yeah, exactly that's a question about like I need to fetch this many assets. I don't have this much time But hb2 or h1 places strict Restrictions on how many things I can fetch in parallel so that prohibits me from saying hitting that well Whereas with hb2 we can actually work around a lot of those things right not work around h2 fixes all those things And one of the reasons why I wanted to bring that into I know it's sort of slick off topic in addition to the conversation We just have but if you've been following along and change a lot weekly which we ship out every Saturday the last two weeks We've we've talked about this time to glass Term and one of them leads to something you had done I can't recall a link but To recommend right now, but I'll put it in the show notes for those listening and even to the issues of weekly But I want to bring that up since you were here on the show we talked about time to glass recently in change a lot of those listeners That read that email once every week sort of close to something gap on on the term itself And what that means for h2 and getting closer to one second or one thousand milliseconds or one second time to glass Yeah, it's a really interesting topic if you're interested in kind of web development in general But also just performance because it turns out that a lot of A lot of these cases right now are driven by mobile which places a lot of restrictions on latency and also emerging markets So there's explosive growth in terms of number of users coming online in places like India, Brazil and Russia and everywhere else and networks are kind of slow there and also expensive and We need to think really really carefully about how we build pages and all the rest.

So that's that's probably enough Of material there to talk about on another three episodes Yes, uh actually I went back and looked episode 54 sorry issue 54 of She's what we actually linked to this chapter 12 that we just been talking about here So that was the one link and the one was talking about rails and turbo links and how Turbo links aren't as bad as they are they're gonna spin seeing this all along and bonus We actually linked out to your talk on breaking the 1000 millisecond time to glass mobile barrier Which is what we've been talking about here. So there you go Circle completed Well, it's definitely fun having you back for it for three times. We'll get your jacket out to you soon enough Uh, so you can weigh that with pride It's got the changelike emblem on there and it's got I've been on the changelog three times on the back where the office or local meetups Whatever, you know, sweet flavor. Whatever particular fancy on that part there.

Uh, do you any thoughts on the closing here? No, just I really appreciate you coming on the show. We really enjoy it I've been interested in the inner to what's coming up on pipeline with h2 for a while So I'm glad we got the chat Good deal. All right.

Well, that's uh, that is the show and for now. Let's say goodbye buddy. See ya. See ya.

Thanks You