r/programminghorror • u/dex4er • 22d ago
GitHub's HTTP header
This is the crazy thing I've found after analysis why the CI test fails with "Maximum header size exceeded" today.
482
u/Belialson 22d ago
Company I work at was proxing some endpoints to us, it was a big shop. They used to set a cookie with json containing entire user’s session including every page visited and it’s details.
Same effect but unfortunately on production
92
u/buckypimpin 22d ago
isnt a cookie max 4KiB?
125
u/RegisteredJustToSay 22d ago
That's per cookie (name + value). You can store many times that if you break it up over multiple keys, tho max 180 per host on chrome. You could fit ten text copies of Alice in wonderland in that space.
You're right though, of course. Just don't want to give anyone here the idea that you're limited to this little space per domain.
37
u/RustOnTheEdge 22d ago
Alice in Wonderland is roughly 170kb, so it would fit 4.3 times in all the available header space.
18
15
u/RegisteredJustToSay 21d ago
Weird, specifically checked gutenberg and saw it was under 80 kB but now on desktop it says 170 kB. Was on mobile though so may have been a presentation issue. Thanks for the correction!
11
24
u/caboosetp 22d ago
You can fit quite a bit in 4k before it causes issues that a browser restart can fix and push off the issue.
5
u/Chesterlespaul 22d ago
Interesting idea. Obviously don’t do this. But I wonder sort of data this could be useful for.
15
u/groumly 22d ago
Asp (can’t remember if it was the og or the .net one, I think .net) did a lot of crap like that back in day.
It was used to turn stateless web framework into a stateful mode iirc.
12
u/lenswipe 22d ago
Ah yes __VIEWSTATE
I remember seeing that hidden field in a form, decoding it and was like ".....no fucking way"
6
u/groumly 22d ago
Oh, that rings a bell, yes. Wait, so was it actually stored in cookies then? Or did I get confused and it was just passed in a hidden form on every damn link of every damn page?
9
u/lenswipe 22d ago
no it was a hidden form field that got posted between pages
5
1
u/Chesterlespaul 22d ago
Oh that would be so useful back in the day! Reactive state set from web requests.
133
u/n9iels 22d ago
Proof that managing CSP headers is an absolute nightmare
-23
u/AyrA_ch 22d ago
They could have gone with
*.blob.core.windows.netto massively shorten the header.69
u/Alikont 22d ago
That's a generic domain for all Azure Blob storage accounts. So I could register my own Storage Account and get the matching url.
29
u/AyrA_ch 22d ago
They also allow
raw.githubusercontent.comwhich serves user defined files, so serving unspecified 3rd party content is not the primary concern here to not use an asterisk.4
u/Joniator 22d ago
Cant be sure, maybe they only query well known paths, then raw.githubusercontent.com would be fine, some random guys blob who controls that storagr wouldnt.
But thats grasping at straws, i dont think they had concern for anything but that was the fix for some issue they didn't understand enough to come up with a reasonable solution.
5
u/reerden 22d ago
Which makes it even dumber as it’s trivial to add your own CNAME to storage accounts.
3
u/RustOnTheEdge 22d ago
Which came with significant downsides back in the day. Only http or you’d need to use azure front door or something, iirc
2
u/reerden 22d ago
This is still the case, but I don’t think GitHub has any issue affording front door or another content delivery service.
If you run into limitations of a single storage account that you need multiple, especially this many, then surely you can afford something like cloudflare or front door.
1
u/Online_Matter 22d ago
Should they have used a reverse proxy then?
3
u/n9iels 21d ago
In terms of managing CSP headers that would be easier in theory. In praktische however, most analytics libraries have statics urls and marketing collegues don't care and will still create tickets to "whitelist absolutelyrandom.whatever.example.com".
1
u/Online_Matter 21d ago
That's what I was thinking and you're painfully right. Lot of urls in the post seems to be for the same project however.
41
u/steeltownsquirrel 22d ago
I'm off to write the first hit song in the blob.core genre
1
16
12
u/hursofid 21d ago
Yeah. I've been troubleshooting why Teams voice/video freezes on Linux and have discovered that it downloads a fucking 200-something megabytes of cookies IIRC.
Fuck microslop.
-10
u/des_the_furry 20d ago
Weak ahh OS can’t handle 200 mb 😹
9
4
10
u/TheSpiffySpaceman 22d ago
standard Microsoft fare, unfortunately.
I dare you to write an Azure DO extension and retain your sanity. I promise you that it's impossible.
8
4
u/andyfsu99 21d ago
Can someone ELI5 why this is this way?
4
u/GuyWithTwoThumbs 21d ago
Apparently too hard to write a proxy service to forward requests to relevant services, where you'd have to re-authenticate the user to each service as authentication cookies for each domain would not be passed by the browser to this proxy service. This is just how CSP works, and its an important security measure to have. Easier to have the client authenticate with the relevant cookies, etc. directly to the relevant services.
1
u/Hurtfulbirch 21d ago
A CSP tells the browser what 3rd party domains are allowed through. For things like fonts, scripts, images, etc. It’s to prevent xss attacks mostly, I think.
4
u/ryanmgarber 19d ago
GitHub becoming one of the buggiest, most annoying services to exist was not something I ever expected to happen and yet Microslop found a fucking way
6
u/GoddammitDontShootMe [ $[ $RANDOM % 6 ] == 0 ] && rm -rf / || echo “You live” 22d ago
Christ. And that isn't even the full list in the screenshot.
2
u/topgamer7 19d ago
I had to try to set up CSP for my work. It's a body nightmare if any of your js is third party. You have to call out every domain that code is loaded from at all
2
u/tootac 18d ago
There are bigger headers when looking just at popular sites (there are probably crazier examples):
https://hereket.com/posts/http-header-body-size/
64
u/Julius_Babies 22d ago edited 20d ago
I saw this post a while ago and just ran into this. The HTTP Engine my app uses cannot handle header values longer than 8192 characters by default, which causes my in-app update mechanism (which relies on GitHub releases) to fail when trying to download the update. I cannot provide a fix for this as this fix would require - you guessed it - an update.
This has been reported as a bug on the GitHub Community discussions already: https://github.com/orgs/community/discussions/211142
Edit: It's now fixed