r/programming • u/theBumbleSec • Feb 25 '21
An Exploration of JSON Interoperability Vulnerabilities
https://labs.bishopfox.com/tech-blog/an-exploration-of-json-interoperability-vulnerabilities14
u/yawkat Feb 25 '21
Parsing differential vulns (be they http request smuggling or json parsing or iOS plists) are so unnecessary. It's in their nature that they'll keep popping up, and trying to make tens of parsers work exactly the same way all the time is an exercise in futility.
The only solution is fully parsing the data and then re-emitting it. That way you can be sure only the degrees of freedom your model defines will appear in the output.
But it seems like we still need a few hundred WAF bypass exploits until this fact filters through...
6
u/Eluvatar_the_second Feb 25 '21
This, it seems really weird to have such thin layers that you can just pass the direct request on to the next layer.
1
u/sievebrain Feb 26 '21
Did you read the article? It points out that re-encoding doesn't solve the problem with some JSON engines.
9
5
u/Y_Less Feb 26 '21
From a developer's perspective, why would you waste computation by re-serializing a JSON object that you just parsed and validated when the string input is readily available? This assumption should be sound.
That assumption is absolutely NOT sound. Why are you using request data like that at all? Why is the original text version of data being passed around your system? Read it in, pass around the parsed internal format form between your components (as you will need to anyway, since not every one of those should be re-parsing the data), and them emit new when you make the request. This also allows for adding and removing and data in the middle-end, since the output is an entirely new form. JSON comes in, in-memory data gets processed, JSON goes out. This is suggesting also passing JSON around the edge.
2
u/cameron314 Feb 25 '21
I like how that one ruby parser returns 3e14159... (pi!) as the result of parsing 1e4096 :D
1
Feb 25 '21
How is it compared to XML? Better/worse/similar?
3
u/mpyne Feb 26 '21
XML is probably worse overall (XML-specific flaws are the #4 threat on the OWASP Top 10 Application Security Vulnerability list even though the list was compiled after JSON was much more predominantly in use).
The float/int problem isn't present in XML, even if that is because XML doesn't natively care about typing numeric values, but the other four major categories are all represented in some way.
4
u/evaned Feb 26 '21
but the other four major categories are all represented in some way.
I think this claim will require some support.
The first vulnerability listed is JSON parsers silently omitting part of the tree because of duplicate values. I would be shocked if an XML parser would do this. At last AFAIK, the closest analogue I know of would be an error in program logic rather than the parser -- your program logic expects that only one instance of a tag is present, but there are multiple, and then the program logic does the first/last precedence inconsistently. The main argument I'd make here is that's the place for a schema -- make a schema that requires that said element is present only once. If you use a schema, which IMO you should anyway (and should in JSON), then anything that validates won't have this problem. By contrast, a typcial JSON Schema validator will do nothing to help you prevent this vulnerability in JSON parsers, because they typically do validation after parsing, after the parts of the tree have been omitted. For example, I did a search for an online JSON Schema validator and picked the first link -- it has no problem validating an "invalid" input with duplicate keys. (You could argue that such input is not invalid JSON, but then I would argue that virtually every JSON parser is horrendously buggy.)
The stuff about string truncation because of weird characters I can totally see affecting XML parsers, so I'll give you that one. However, the "using comment truncation" is something I do not see affecting XML parsers, because parsers don't usually support grammar-level extensions like that AFAIK, and even if they do they're far less frequently used.
1
u/mpyne Feb 26 '21
The first vulnerability listed is JSON parsers silently omitting part of the tree because of duplicate values. I would be shocked if an XML parser would do this.
The error is differences in handling of duplicate values by different parsers, not mere omission. If every JSON parser always omitted the same duplicates there would be no issue.
XML suffers from the same issue, where different XML parsers can lead to different XML document trees based on how they handle complex XML document trees. XML directly supports duplication of tags but the general issue of different parsers coming to different parse trees remains. This is a common source of bugs with security-minded XML applications like SAML.
You are right that these need application involvement to get to the confusion, but since an application ultimately produces and consumes XML, and since XML invites so many ways to get to "confusion" here (including duplicate tags confusing applications in the same ways duplicate keys in JSON confuse applications) I can't fully give it a pass.
By contrast, a typcial JSON Schema validator will do nothing to help you prevent this vulnerability in JSON parsers, because they typically do validation after parsing, after the parts of the tree have been omitted.
Doing this with a JSON schema is admittedly a hard luck case even if you look beyond the first Google hit :). But a JSON linter is a possibility and does the right thing and those come up on Google as well.
However, the "using comment truncation" is something I do not see affecting XML parsers, because parsers don't usually support grammar-level extensions like that AFAIK, and even if they do they're far less frequently used.
So my reference to that was to real bugs affecting SAML libraries, which can be confused into minting tokens for the wrong user by the crafty placement of XML comments in otherwise legitimate SAML XML documents.
And of course, many applications have been caught out trying to parse XML comments incorrectly (since it's difficult to do correctly) and while, yes, "they should have just used a parsing library", parsing libraries are complicated, XML seems simple enough if you use it for simple tasks, and soon you're writing your own parser that will let through hyphens that mangles a real parser.
2
u/argv_minus_one Feb 26 '21 edited Feb 26 '21
real bugs affecting SAML libraries
The bug in the linked article appears to be the result of some extremely sloppy programming in the vulnerable XML parsers. In those parsers, there is a “get this element's text content” accessor which only returns the content of the first text node! When you're getting the text content of an element, you basically never want only the first text node because there might be a comment/tag/PI in the middle of the text! The DOM
textContentproperty concatenates all text nodes (including ones in descendant elements), which is the right thing to do and is not vulnerable to this attack.XML seems simple enough
I invite anyone who thinks this to actually read the productions in the XML spec. Simple, it ain't.
1
u/mpyne Feb 26 '21
I invite anyone who thinks this to actually read the productions in the XML spec. Simple, it ain't.
Exactly! All the stuff you point out about how it's theoretically possible to use XML safely for security applications misses the bigger question of why you would do this in XML in the first place as it is far too complicated to make this a good idea.
Even namespaces can gum things up, so to be security conscious you even have to ensure that all your XPath expressions fully write out the namespace for the tags they will use each time!
For example, an XPath expression like
/soape:Envelope/soape:Bodywould need to be converted to instead read as:
/*[localname()="Envelope" and namespaceuri()="http://schemas.xmlsoap.org/soap/envelope/"][1]/*[localname()="Body" and namespaceuri()="http://schemas.xmlsoap.org/soap/envelope/"][1]each and every single time.
1
u/masklinn Feb 26 '21
Definitely better, no contest.
To starts with, the xml spec has built-in security issues, things like DTD, and entity definitions, and encoding concerns, …
Second, XML is a way bigger and more complex format, even if you strip out the stuff from (1), so more opportunities to fuck up.
Third, because XML is so much more complex it’s more likely implementors will need to optimise it, which provides further opportunities for vulnerabilities.
And that’s before you account for the ancillary-but-really-required stuff like namespaces.
0
33
u/headhunglow Feb 25 '21
Finally some real programming content on /r/programming! Great writeup! However, I'm wondering why any JSON parser would support comments?