r/LocalLLaMA
•
u/ConfidentDinner6648
•
1d ago
•
2
News CCORE-WEB-CODER drops this weekend
[UPDATE]
Preview Model: ccore/CCORE-Web-Coder-4B
Everything is open:
ThreeJS-3D-Scenes: 4,146 Three.js scenes
ccore/ThreeJS-3D-Scenes-v2: 3,445 scenes
ccore/ThreeJS-3D-GLB-Assets: 3,918 assets
All trained into 4B models you can run locally.
The CCORE-WEB-CODER drops this weekend.
4
8
u/AryanEmbered 1d ago
interesting. would have liked to see benches.
Also the lfm 3 model seems to be much nicer to code with than the older qwens, wonder how these would stack up.
So you've essentially trained an older quenon the dataset of threejs scenes and are seeing the 4b and that 2b checkpoint replicate them well enough.
honestly good use of the small models.
maybe if 5 years from now these capabilities get distilled into a 500M model it might be good enough to do realtime ingame scene/asset generation locally.
10
u/ConfidentDinner6648 1d ago •
![]()
I'm also training a 0.8B Web Coder, I just haven't posted about it yet because I'm not sure how good it'll turn out. The 4B is already doing pretty well.
I've trained a few LFM2.5 models too, but had problems with its small tokenizer. It tends to lose track of variable and function names, especially with longer outputs. LFM3 looks interesting, but I'm not a fan of its license. I definitely want to train one though.
The thing is, I have a limited training budget, so between spending money on a 4B that I know can handle the task and a 1.7B that I'm still unsure about, I decided to go with the 4B.
But I definitely want to experiment with the 1.7B models soon. There's a lot of potential in these smaller models :)
2
u/Danmoreng llama.cpp 1d ago
0.8B runs extremely well on CPU: https://github.com/Danmoreng/qwen35-cpu
3
u/Kahvana 1d ago
Very nice! Really like to see these type of specialized models!
What song did you use for the trailer?
[edit] found it! "Eternal Eclipse - Shape of Lies"
https://www.youtube.com/watch?v=9Ta-8SilgVs
1
u/ConfidentDinner6648 1d ago
I didn't know it was that one; it was a song from a library of background music.
2
u/Andreea_tucker04 1d ago
A prompt-to-page demo with one follow-up change in plain English would show how it handles iteration
1
u/ConfidentDinner6648 1d ago
That's actually what I'm working on right now. The current models are focused on one-shot generation, but I'm training a new 4B model for web development using a harness, with multi-turn interactions and follow-up changes in plain English.
The idea is to let it build something, receive feedback, modify the existing project and keep iterating instead of generating everything from scratch.
I'm still testing it, but the initial results look promising. I'll share some examples once it's ready.
1
u/stylome-ai-detection 1d ago
1
u/ConfidentDinner6648 1d ago
logico que é feito por ia eu tive que escrever falando pq eu to trabalhando
1
u/Andreea_tucker04 1d ago
I want to see how it handles a change that affects something it already built.
2
18
u/HAL_9_TRILLION 1d ago
This actually seems a bit like it could be the future? Hyperspecialized 4B models for everything. Maybe they could be loaded on the fly as needed behind the scenes by your very own hyperspecialized 4B agent? A first step to a world where six 5090s aren't needed to run something wildly useful? A man can dream.
4
u/Chirimorin 1d ago
One orchestrator model that delegates (sub)tasks to a set of separate expert models? That does sound like it could be the future, in a way it's similar to what Strata does (dynamically loading experts) but applied through subagents (who get a full task instead of 1 token) with separate expert models.
1
u/finevelyn 1d ago
Yeah but it's hyperspecialized to make a visual prototype / demo, and that's why a small model suffices. How many demos does one need?
2
u/bonobomaster 1d ago
Truely awesome and impressive.
I got a question: Did you try 9B parameters and if so, is it a big jump in quality?
I'm asking because my experiences with 4B models are, that they are just too stoopid, but at 9B, there are actually some smart cookies.
Furthermore, it seems, that 9B can be run by a lot of people as well, GPU wise that is.
2
u/stephen_holograf 1d ago
He’s mentioned in other threads that he’s using 4b because he can do full param fine tuning on a single rtx pro 6000 and so it’s relatively cost effective to just churn these out.
5
u/ConfidentDinner6648 1d ago
yes, that's exactly why I'm focusing on 4B models. It's relatively affordable to full fine-tune them on a single GPU, and I also want to see how far they can go compared to much larger models.
Right now I'm testing their capabilities on specific tasks. First with one-shot 3D generation, and now with web design, presentations and infographics. The goal is to reach high-quality outputs comparable to something like MiniMax 3.1 Flash, but within these specialized domains.
The next step is agentic training. I want to see if I can turn the 4B into a capable agent that can work with a harness, rather than just generate everything in one shot. After that, I'll decide which direction to take.
For everyday tasks, I actually believe we can eventually get a small model approaching Luna-level capabilities. Of course, it depends on the quality of the training data and the compute available, but I suspect the amount of compute required could be much lower than people generally assume if we put significantly more effort into data quality than is typically done.
3
u/enginetown 1d ago
How wouldn't a 9b be just as cost effective? Also wouldn't it prove your pipeline even further? If you're using Qwen 3.5 4b I feel like testing out the pipeline on the 9b variant could either be a big improvement or relative to the same results but I doubt it'd be worse and I know these aren't try one path and get the model but why not try your best proven path on it at least once It would be super cool to see.
2
u/Ylsid 1d ago
This is really cool but "game ready" might be saying a bit much if the examples are anything to go by lol
2
u/ConfidentDinner6648 1d ago
Fair point haha. By "game-ready" I mean assets that can be exported as GLB and used in game engines, not that every generation is production-ready without adjustments. Some are already usable, especially for low-poly games, while others still need work. These are raw model outputs, without manual cleanup. There's definitely room for improvement, but I think the results are pretty interesting for a 4B model.
1
3
3
3
2
2
2
2
1
u/Swimming_Beginning24 1d ago
Have you tried Nvidia Dory 4B as the base model? Maybe it could give you a little more oomph per parameter
-3
u/changed_later__ 1d ago edited 1d ago
Anyone else sick of the "Make up a claim that they said so you can prove them wrong" intro to everything these days?
8
u/ConfidentDinner6648 1d ago
ccore/Qwen3.5-4B-3D-Coder was the first to consistently generate complex 3D scenes.
2
u/ps18pmcom 1d ago
From your experience do you think LFM 1.7B would be capable as well? I've heard that it has beaten a 9B parameter model in certain tasks.
5
u/banana_slurp_jug 1d ago
It’s not wrong though: nobody expected a 4B model to be able to do something like that
2
u/MuzafferMahi 1d ago
Wtf I was literally going to drop my qwen 3.5 4b frontend coder THIS saturday. I don't know what to do, cause I'm pretty sure this guy is gonna beat the fuck out of me. Maybe I should release w the same name before him to kinda fuck with him lmao
2
u/MuzafferMahi 1d ago
Just realized the guy who posted this is the ccore. Aaagh shit. You beat me before man! I don't got much compute lmao.
10
u/LogFabulous9386 1d ago
A 4B model trained specifically for Three.js sounds promising.