r/PythonLearning • u/Owlbuddy121 • Feb 22 '26
JSON vs TOON
Anyone have thoughts on this?
What’s your opinion on using a Toon-style JSON approach? Curious to hear different perspectives and real-world experiences.
2
u/Patient-Definition96 Feb 22 '26
Toon is silly. Why is it even a thing if we already have JSON. Doesnt make sense at all.
1
u/Slackeee_ Feb 22 '26
Because te people feeding all their data into LLMs realized that they can save some money by using TOON instead of JSON. That is it's main purpose, reducing token counts.
1
-2
u/Owlbuddy121 Feb 22 '26
TOON is really usefull with LLMs as due to small size of data set we can save tokens with LLMs
56
u/Outrageous_Let5743 Feb 22 '26
toon is just csv with extra steps
11
2
1
1
1
1
1
1
0
u/flixflexflux Feb 23 '26
JSON is just CSV with extra steps to loose the implicit structure CSV had.
4
u/Key_Mango8016 Feb 22 '26
This was designed for LLMs, since it uses much less tokens
2
u/I1lII1l Feb 22 '26
Yeah but do LLMs know it well, is it present in large amounts in their training data?
2
u/Key_Mango8016 Feb 22 '26
No idea. Not arguing for it, just stating facts
3
u/I1lII1l Feb 22 '26
I was not asking you per se, anyone who might know and participates in the open discussion.
1
u/Key_Mango8016 Feb 22 '26
I just gave ChatGPT the sample in OP’s post and found that it immediately understood what it means (I’m not surprised). I suspect that in practice, using this format would be possible as a drop-in replacement for JSON.
Personally, I don’t know if it’s worth it for production systems I own, because the vast majority of token usage in those systems comes from images & audio.
1
u/ComprehensiveJury509 Feb 22 '26
What's the point of saving tokens, if you then have constantly explain the format to the LLM? It was a completely ridiculous idea from the start, made by people who apparently don't understand how LLMs work. It's also laughable to come up with an entirely new format to fix issues that are probably completely irrelevant in a year or two, given how fast things move.
-2
21
Feb 22 '26
Thoughts:
JSON is more readable where data is non-uniform (ie not like this example). In this situation TOON is good, but CSV would be better. TOON appears to work best when a mixture of uniform and non-uniform data is needed, a scenario which is unusual but perhaps not that unusual.
The standard for TOON is still evolving and not finalised. That may be an issue for long term support.
Compatibility is important. A lot of software has support for JSON and CSV. TOON is currently supported in the most popular languages, but not in some of the less popular ones.
Overall, what I'm seeing is not terrible. It's something I might consider using in future, for the right use case. It's not something I'm going to rush off and start using right away tho.
3
u/Ok_Space2463 Feb 22 '26
I feel like embedded data would be a problem with toon because its doesnt have the indent or syntax wrapping?
1
u/bradfordmaster Feb 25 '26
I'm really bugged by the [2]. Why would you need the length encoded like that, such that every write has to touch the header? And so merging is complex, residually in parallel
1
Feb 25 '26
Yeah that is a concern. Having opening and closing brackets would work better for its use case.
1
u/natur_e_nthusiast Feb 25 '26
It might be a handy checksum
1
u/doctormyeyebrows Feb 26 '26
A checksum isn't handy if it has to be updated in place every time data is added
1
1
u/its_a_gibibyte Feb 26 '26
What about just JSON in a better layout:
{ "Columns": ["id","name","role"], "data": [ [1, "Alice", "admin"] [2, "Bob", "user"] ] }1
u/kozeljko Feb 26 '26
Why is "data" not aligned with "Columns" 🤢
1
u/its_a_gibibyte Feb 26 '26
Sure, you could also do:
{ "data": [ ["id","name","role"], [1, "Alice", "admin"], [2, "Bob", "user"] ] }1
u/robhanz Feb 26 '26
Eh I prefer the explicit columns element. That way there’s no “magic” row in the data collection you have to remember.
1
u/robhanz Feb 26 '26
Right. The issue is the redundant data, primarily. TOON is still more compact, but it loses a lot of the edge if you do it this way.
28
u/remic_0726 Feb 22 '26
encore un format en plus...
2
1
-3
u/PaulMorel Feb 22 '26
Yeah, but toon is more concise. I'm old enough to remember when we all preferred json over xml because it was more concise.
2
u/Aaron_Tia Feb 22 '26
Is it really a gain in real world application ?
How to distinguish 1 and "1" or true and "true"Types look like magically deduced, which will for sure leads to impossibility to migrate in many places.
1
u/Momostein Feb 25 '26
Yeah, if you're looking for maximum performance, you'll want to switch over to binary formats like protobuffers etc.
TOON would only be a bandaid fix.
21
u/Cybasura Feb 22 '26
JSON fundamentally is a clean dictionary-like data structure that is actually really nice, just fell short of the comment support by it's foundational design
TOON basically took JSON and somehow made it harder to dynamically manage
3
u/Frytura_ Feb 22 '26
Dynamically manage?
Isnt that the goal of a mapped out object that you then ask to spit the TOON / JSON data as a string?
2
u/Cybasura Feb 22 '26
Dynamically manage as in like programatically get/set/assign values into the dataset during runtime of the application
1
u/thee_gummbini Feb 24 '26
Its a serialization format though?
1
u/Cybasura Feb 25 '26
What? Yes I know that, I'm talking about runtime usage, modification functionalities
You know, CRUD? Create, Read, Update, Delete?
I didnt say it wasnt a Data Serialization File Format/Type, did I?
I was referring to importing the dataset file, manipulating it and the moment-to-moment use case operational workflow of working with this
1
u/thee_gummbini Feb 26 '26
But... Once you deserialize it... It should be the same? TOON doesn't introduce any runtime types, it deserializes to the same types as JSON would. The only differences are in the serialization, it being a serialization format.
The CRUD operations are the same, since neither JSON or TOON are databases, you load, modify, and write.
1
u/UnicodeConfusion Feb 23 '26
> just fell short of the comment support by it's foundational design
This is the part that frustrates me more than anything. Aside from naming files by the content (i.e. pom.xml instead of pom.mvn - which would let me know the intent of the file instead that's is an 'xml' file) Ugh I'm old.
1
u/Cybasura Feb 23 '26
Yeah whenever I use JSON, I have little to no gripes alot of the time but the second I want to write comments by muscle memory - I weep at the thought of the (loss of) potential
1
8
u/followthevenoms Feb 22 '26
Someone reinvented csv?
1
u/flying-sheep Feb 25 '26
CSV is not standardized, so people using slightly different dialects to encode and decode leads to countless subtle yet devastating data corruption bugs.
If TOON doesn't have that massive problem, I'm all for it.
1
u/exhuma Feb 25 '26
It kinda is: https://www.rfc-editor.org/rfc/rfc4180.html
But the standard came too late (2005) and even today many people don't know it exists
1
u/flying-sheep Feb 25 '26
Yeah, should have said “effectively not standardized”: most languages / popular libraries today are older than that and therefore don’t use the standard by default.
-1
u/AmazedStardust Feb 22 '26
It's basically CSV with cleaner nesting. It's meant for saving tokens when feeding data to AI
2
u/Own-Improvement-2643 Feb 22 '26
What is the cleaner nesting here? How is it any cleaner than csv?
1
u/Deykun Feb 23 '26
Isn't CSV so stupid that delimiters can be different? I’ve always disliked that.
1
Feb 25 '26
It’s flexible. The field and record delimiters can be any character. Very useful if you want to use values with commas or new-lines.
0
u/_ryuujin_ Feb 22 '26
its csv but easier to marshall back into an obj, since the obj def is defined in the header.
it looks cleaner and more compact for lots of records with well defined obj definition. it has its place.
1
u/ChomsGP Feb 23 '26
the first row on a CSV is also a header and can define the same field names, plus you don't need to tell it how many rows it has upfront
like I have no idea if the [2] in there is needed, first time I hear about TOON, just saying the example in OP is pretty bad/pointless
1
u/_ryuujin_ Feb 23 '26
yes you can do everything in a csv, but having a standardize format allows for easier marshall and unmarshalling the data. vs a custom format each time.
array count is nice as it could tell you much to read for this one obj def. maybe another obj def will start at the end of the 'array', its like a header for binary data, where you have msg len. before another set begins.
5
5
u/riansar Feb 22 '26
I think json looks easier to use if you have large objects you can ctrl f the property you are looking for which doesn't seem to be the case with toon
4
4
1
u/nullshipped Feb 26 '26
Why they call it toon?
1
3
3
u/Nidrax1309 Feb 24 '26 edited Feb 24 '26
If you format it like a complete bitch instead of
{
"users": [
{"id": 1, "name": "Alice", "role": "admin"},
{"id": 2, "name": "Bob", "role": "user"}
]
}then yeah, json looks less readable
1
u/ReasonablePresent644 Feb 24 '26
I think the goal was to compare how readable both formats are with the same number of lines. So yea JSON is more verbose but easier to read for complex structures.
1
u/Awes12 Feb 26 '26
Yeah, if you do something completely ridiculous, it's more readable. That's like saying C++ is more readable than python if you make the python code the same amount of lines as the c++ code.
7
2
2
u/captdirtstarr Feb 22 '26
Is there a URL for official TOON documentation?
1
u/ping314 Mar 10 '26
Yeah, https://github.com/toon-format/spec is the address the English edition of Wikipedia links to.
2
2
u/CraigAT Feb 22 '26
I am not sure what TOON adds over a CSV file! (Obviously it does add an object name and a line count, but those are easily specified or calculated in code)
Also JSON allows you to have another level (or multiple levels) within objects i.e. Each user here could have multiple roles. I am not sure how that would be implemented in TOON.
2
2
u/WhiteHeadbanger Feb 22 '26
TOON is readable.
JSON is serializable to and from dict.
So TOON is preferable for visual representation, and JSON to code stuff.
I would just stay in JSON.
1
u/UndeadBane Feb 24 '26
I someone tries to force me to read this thing with object of >6 fields, some of which may be absent, I will hurt them.
2
u/katlimruiz Feb 22 '26
Toon seems good but It is not about that. Json is ubiquitous, it parses and serializes everywhere. Distribution will always win (obviously the product has to be good enough)
2
u/questionsalways2233 Feb 22 '26
CSV is a nightmare- you have to hold your breath and hope it comes in right. As a data scientist, the cost of JSON in size is 1000% worth it. You just know it is going to work. I don't know why anyone would want to use a CSV-like structure as a modern replacement for anything
2
u/yes-im-hiring-2025 Feb 22 '26
LLMs are heavily trained on XML/markdown/JSON formatted data specifically. The TOON format is just CSV with extra steps - and it's worse for the LLM to work with than standard JSON or XML or md.
Don't pinch pennies for the performance. Input token costs should be actively managed, yes, but this isn't the optimisation you think it is. You need to either redesign your system formats where JSON is appropriate or just use pd.to_csv() instead of this TOON formatting.
TOON doesn't solve anything imo.
1
u/Laicbeias Feb 23 '26
Im not sure. I think json is great. But ai is dropping lots of data into context and toon just makes that shorter while staying descriptive. Its for those scenarios where AIs load a lot of context. And the simpler something is the easier AI handles it. They are smart enough for that.
Ive seen some guys using japanese mixed with english to drop more information into these llms. Basically encode your project into japanese as precontext. Since japanese is shorter and more expressive per token.
2
2
u/SlinkyAvenger Feb 22 '26
What's it look like when it's nested? Does it provide a way to do faster parsing ala SAX?
2
u/hyrumwhite Feb 22 '26
Toon is a format designed to be consumed by LLMs. If that’s not your use case, json is the better pick.
2
2
2
1
u/sugarfairymeliora Feb 22 '26
JSON is more widely used in the industry. With JSON you have a stable file format that will work with everything, whereas TOON is not so well-known and flexible. Plus, from my experience it is easier to convert JSON into CSV or any other file format, then TOON. LLM's also understand JSON formatting better, so if you are going to start developing an LLM or start working in that industry JSON's are more preferable.
1
1
1
1
u/the_reven Feb 22 '26
Eg, just use yaml instead! Seems like a waste of time tbh, json, yaml, csv, xml. They seem to cover everything quite well. Yank is very easy to read. Json is easy to read
1
1
u/Domingues_tech Feb 22 '26
Imagine the trees we could’ve spared if someone had said, “Guys… it’s a spreadsheet. Calm down.”
1
u/AbsorberHarvester Feb 22 '26
Some more time and Zoomers (with AI help, of course) will be presenting "new bicycle" - transfering BINARY DATA! (with formatting, as in toon) it can cut data size to half/third of "string".
1
u/CamilorozoCADC Feb 22 '26
Toon doesn't even make sense since LLMs are trained with JSON (because TOON usage is just not widespread yet) and using JSON means that you (and the LLM) already can use JSON Schemas for structured input/output and JSON query languages to handle your data like JSONpath or JSONata
If the objective is to analyze data and reduce token usage a better approach is to let the LLM use a toolset to analyze your data, for example using MCP tools to manipulate excel or CSV files instead of chucking a full blown dataset into the LLM
1
u/Concurrency_Bugs Feb 23 '26
When you have a large JSON with more than a single level, TOON would be unreadable.
Not to mention many existing frameworks work well with JSON
1
1
1
u/Anpu_Imiut Feb 23 '26
People even forget the real reason why Json is great. It is not about vusibility. Rather think about saving gbs of data in json vs others. It is efficient while you can open it with a proficient jaon reader.
1
1
1
u/SecondThomas Feb 23 '26
There is not the question of if there is a better alternative for Json, of course there is. It's all about adaptation and compatibility.
1
u/Last8Exile Feb 23 '26
I think format where schema is decoupled from data will save even more tokens.
Going full circle back to binary? Nah, too good to be true.
1
1
u/oculus42 Feb 24 '26
TOON tells you how many rows and you can parse the data as it arrives. For small objects it's pretty close to irrelevant. For large objects, it's probably more worthwhile to use some of the async transfer alternatives where the server can specify top level keys essentially as promises, which allows different sections to respond at different times even though the client receives one result...
There have been libraries that do real-time parsing of JSON, but they were more popular when dial-up and low-end DSL were a consideration.
The file size is almost nearly irrelevant because any quantity of regular data will compress close to perfectly with LZ-based compression like gzip or brotli, and they are parsed into the same dictionary structure.
It's interesting, but I don't readily see the value proposition for most data sets.
1
1
1
1
u/pardoman Feb 24 '26
Toon exists to reduce the tokes needed when feeding JSON data to LLMs, where the data contains arrays of identical objects (like the image OP posted). It’s not ideal for most use cases tho.
1
u/Agreeable-Nerve-65 Feb 24 '26
Toon is useful for humans, JSON for machines.
I’d keep JSON as the source of truth and use Toon only as a presentation layer.
1
u/Devel93 Feb 24 '26
Redundant! Why do I have to specify the length of the array? The number of items should determine the length, this is a format I am not allocating memory
Why not just use a CSV? It's the same thing!
How do you represent nested data?
Why would you introduce yet another markup language? Pun intended 😝
1
1
1
1
1
u/joe_chester Feb 25 '26
If you need a very small, efficient data exchange format, go with protobuf. If you need a simple, human readable/debuggable format, just use JSON.
1
u/Hour-Inner Feb 25 '26
My understanding is that some people are using TOON for ai workloads. Even if it’s less human readable, less text means less tokens
1
u/helpprogram2 Feb 25 '26
If I’m gonna use something besides json it’s gonna be buffers.
0 benefit in isn’t this
1
u/indeem1 Feb 25 '26
Because I have not Seen that answer here yet, I Heard there are many that use it for ai to reduce amount of tokens. I dont see an advantage for readability for people although or real advantages to use it in Production elsewhere
1
1
1
1
u/moshujsg Feb 26 '26
Yes now hit an api that returns 50 fields in toon mode, then start looking at the data. How will you know what fiekd a specific line it is? Also like, what are you really improving on? You usualky dont type out jsons like that enough for that to be a problem, you usually just read jaon like that, at which poimt ita just more readable vs toon.
1
1
u/Fabulous-Possible758 Feb 27 '26
Interchange formats are actually where LLM coding will actually do really well and make lot of this moot.
1
1
1
Feb 22 '26
[removed] — view removed comment
4
4
u/Slackeee_ Feb 22 '26
It is a data format with the purpose of reducing the token count when feeding your data to LLMs. It is also said to increase accuracy of LLMs.
So for 99% of use cases it has no real advantages.1
u/quts3 Feb 22 '26
Why isn't this the top comment.
There is actually a real need for a human readable structured data format that is optimal for LLM prompting. For most things it's markdown with embedded Json, but i can see people finding weaknesses in that approach when they poke. Never tried yaml prompt engineering in and out.
Unfortunately that 1% use case is strong enough to justify a unique data language.
1
u/Slackeee_ Feb 23 '26
Unfortunately that 1% use case is strong enough to justify a unique data language.
There are many strong 1% use cases that justify (and have) their own unique data format. That's usually a good thing and nothing to worry about, but it can be a bad thing when the people in that 1% now come out acting as if there data format is an actual revolution and that the other 99% now have to adapt and use their format without bringing them any actual advantages.
Which of course will be happen when changes happen in an overhyped field that is ground down to a carcass by people trying to get your attention for the money they make with that attention.
In the end it is pretty simple: you want to feed your data to an LLM? Great, TOON might be what you want, just check if it is a good fit for you. If it is you most likely will have nothing to change in your infrastructure other than writing a converter for your data that you want to feed to the LLM (and in the future maybe a TOON to JSON converter for extracting data, if LLMs start to answer in TOON). You don't need to change anything else in your infrastructure. Luckily enough current AI tools are pretty good in writing those converters, so you likely can just vibe-code it, if you want.
For most of us in the 99% TOON is nothing more than a "good to know that it exists if I ever will need it" that doesn't affect us at all.
27
u/AccomplishedPut467 Feb 22 '26
TOON looks cleaner to read for me. Is TOON offers faster lookups for data analyzing? I'am new.
Also, i think TOON looks very similar to CSV files.