r/LocalLLaMA • • 1d ago

Question | Help Tencent Hy Team & Shunyu Yao Here! Ask Us Anything about Hy4 & Hy4 flash!

Fresh off launching Hy4 and Hy4 Flash, the Hy research team is here to discuss how they built the models, boosted efficiency, and made them better for everyday workflows.

Our participants today:

u/No-Cherry2850

u/DiWang01

u/aiden_agent

u/lydellhuang

u/edwardbai

u/kkelvinchen

u/maxmpan

u/dick112233

u/kkelvinchen

u/GeoMorax

Blog: https://hy.tencent.ai/research/hy4-flash https://hy.tencent.ai/research/hy4

HF: https://huggingface.co/tencent/Hy4-flash https://huggingface.co/tencent/Hy4

327 Upvotes

436 comments sorted by

106

u/jucabala457 1d ago

Any chance for a Hy4-mini to replace qwen3.6-35ba3b?

152

u/DiWang01 1d ago • Heartwarming

Sure!It's already on our roadmap, stay tuned.

54

u/winky9827 1d ago

If you can pull that off, your team might overtake Qwen as the new favorite around here.

7

u/linuxid10t 16h ago

Damn, made that comment just a few hours too early...

→ More replies (4)

4

u/Andre4s11 23h ago

When will she be able to translate Chinese novels into Russian properly? And will those random Chinese characters in her responses disappear?

→ More replies (3)

66

u/No-Cherry2850 1d ago • Heartwarming

Stay tuned!!

30

u/rorowhat 1d ago

+1 we need some smaller model that can beat Qwen!

13

u/No_Swimming6548 llama.cpp 1d ago

Fr that segment is wide open

→ More replies (5)

2

u/StableDiffer 1d ago

Wanted to ask something similar

90

u/TastesLikeOwlbear 1d ago

I don’t have a question, really, I just wanted to say that as of five minutes ago, Hy4-flash saturated our internal benchmark for agentic coding and PR review, the first model to pass all tests and finishing more quickly than Qwen 3.8 Flash Next, GLM 5.3 Flash, and Deepseek V4.1 Flash. Wow!

25

u/aiden_agent 1d ago

Cooool!!

12

u/PrimeDirective8 1d ago

Wha?? I'd *love* to see those charts.

7

u/TastesLikeOwlbear 1d ago

Our tool doesn't make charts, just tables. But I'll open an issue on the benchmark repo to add charts and make Hy4-flash do it, and we'll see what happens!

9

u/Infinite-Local5435 1d ago

Without leaking internal info, could I ask what the agentic coding tasks mainly centered around? (3D, Large Mono repos, Short burst zero shots, documentation and keeping code up to date, understandability of code/comments/file structure, multi-turn code cleanliness (can it survive hundreds of rounds of autonomous operation without a human and maintain clean code))

15

u/aiden_agent 1d ago

We cover all of these scenarios, with a strong focus on task diversity.

14

u/TastesLikeOwlbear 1d ago

It isn't the most sophisticated test in the world; Claude Opus 5.5 and Astra would certainly destroy it.

It's a curated set of coding issues that an agent implemented incorrectly, PRs with problems that agents have failed to find, cases where agent hallucination caused problems (e.g., hallucinating that it ran tests that it didn't, or making some shit up instead of saying "I can't determine that with this information."), and agentic development environment administration tasks that agents have fucked up (e.g., "move all your docker images from the root filesystem to ZFS datasets").

It's drawn from/working with a large internal codebase with significant amounts of PHP, Rust, JS, and Python, plus shell-based devops tasks. So our test punishes open models that are overtrained on Python and JS, as well as models that "Oops, I quoted that wrong, let me try that again..."" in bash.

What's interesting about Hy4-flash is that it seems to be finishing the tasks with significantly fewer tool calls and more thinking than other models. (That's a prelim finding.)

6

u/maxmpan 1d ago

cooool

3

u/walden42 1d ago

Very cool. Using API, or local hosted? If local, what quant and engine?

4

u/TastesLikeOwlbear 1d ago

Local FP8 on VLLM. (With, currently, some hackery to make it work reliably on Blackwell Pro 6000s. Which hopefully means there's room for improvement!)

We're testing NVFP4 right now, looks like it's (predictably) going to come in a little lower but still damned impressive.

51

u/crusaderky 1d ago edited 1d ago

Not counting ngrams, Hy4-flash is 156B-A12B (7.7% activations). Qwen3.8-Flash is 125B-A6B (4.8% activations); DSv4.1 is 552B-A16B (2.9% activations), which drops to A8B (1.45%) during prefill.

How important for intelligence is the extra density? Did you test how much quality drop you would get at e.g. 156B-A8B or 156B-A5B?

35

u/edwardbai 1d ago

Good question. We've run scaling ladders with varying MoE sparsity, similar to the MAI-Thinking-1 paper, and reached roughly the same conclusion: at fixed active parameters, increasing sparsity improves compute efficiency. So the direction does work. 156B-A8B or A5B, however, is a fundamentally different trade-off. It keeps total parameters fixed while reducing active parameters, and I'd expect that to hurt model quality significantly. Active parameters essentially define your compute budget. Increasing sparsity can recover some of the efficiency, but not all of it; reducing active parameters ultimately means less compute and a weaker model.

16

u/coder543 1d ago

On a related note, I've wanted to see a lab really push sparsity to new levels for quite a while. Something like a 200B A3B would be fun, and these days, it could also have engrams.

Maybe it would be terrible... maybe it would be awesome. It would certainly be fast, and I think it would unquestionably be smarter than a 35B A3B model.

How far can MoE go?

26

u/edwardbai 1d ago

My opinion: an overly sparse MoE architecture is inference-unfriendly: it consumes a lot of GPU memory while not fully utilizing the compute (FLOPs). We can easily afford a model of the same total size but with more active parameters, while achieving the same inference throughput.

15

u/crusaderky 1d ago

What you say is certainly true for datacenter (high-parallelism) serving. Locally however, things are very different given the current hardware market. A12B is slow to run on Strix Halo and DGX Spark due to their low memory bandwidth, whereas QFN is decently fast for the same reason.

24

u/edwardbai 1d ago

I see ur point. For on-device models, we'll adopt a different design approach, co-designing tighter between the model arch and the GPU hardware on edge devices.

→ More replies (1)

4

u/BalorNG 1d ago

And dynamically looping the active part might not only rise the model "smarts", but give more time to transfer next set of experts into fast GPU memory according to some heuristic (that exist already). I'm reasonably sure that is next step to go for fast and smart and knowlege-dense models.

5

u/coder543 1d ago

Ideally, I think MoEs should be trained with a penalty for switching experts, and also a small prediction head that says which experts will be used soon so they can be optimistically brought into the cache.

But, maybe these things don't work well in practice.

→ More replies (1)
→ More replies (2)

50

u/Bitter-College8786 1d ago

Do you think, these models have reached their capacity, how much intelligence they can "soak"?
Wanted to know if more intelligent future Hy models need to have more parameters or will you be able to offer a stronger models with the same size, like a post-trained Hy4

42

u/aiden_agent 1d ago

We believe Hy4 still has significant room for improvement, especially through post-training. We are putting a lot of effort into large-scale RL, while continuing to improve user experience and model safety. Stay tuned for future updates!

We also plan to train larger models. We strongly believe in scaling and remain committed to it!

→ More replies (1)

38

u/No-Cherry2850 1d ago • Bravo

I think lots of potential left

→ More replies (1)

42

u/walden42 1d ago

I just want to say thanks for sharing your open weight models. We as a community do our best to look past the propaganda of both Chinese and US government and their media. The average person here just appreciates that AI models can be run in a private setting. Simple as.

24

u/BalorNG 1d ago

Any chance for a looped MoE model?

44

u/kkelvinchen 1d ago • Heartwarming

Oh, definitely possible. Stay in the loop

7

u/ortegaalfredo 1d ago

I got that reference

→ More replies (1)

21

u/edwardbai 1d ago

Gotcha.

21

u/inzee 1d ago

Any chance for a 24-31B dense model?

22

u/DiWang01 1d ago

Possibly — it's under consideration.

12

u/pmttyji 1d ago

+ Engram + Lightweight KV Cache

4

u/Marino4K 1d ago

Qwen really needs a true competitor in the 27-35B space

20

u/Obvious-Ad-2454 1d ago

What is your teams approach with using LLMs for your own work ?
Do you use opensource models, your own internal ones, closed source ?

11

u/DiWang01 1d ago

We use excellent models, regardless of whether they are open-source or not. Of course, Hy models are widely used internally.

8

u/willagiwu 1d ago

I generally use Hy4 for most of my day to day task already. Hy4 is capable of handling doc / presentation preparation, skill usage and internal engineering task with good output quality. We even have an internal version of higher tps so there is no reason not using it.

10

u/No-Cherry2850 1d ago

Use whatever you want

20

u/Infinite-Local5435 1d ago

Great model especially when paired with WorkBuddy! Some questions from me:

  1. How did you determine the 63.5B Engram parameter allocation relative to the approximately 155.5B-parameter neural backbone? I know in the engrams paper it was around 20 to 25% as it's optimal with their model, so I was wondering if there was a consistent pattern you found in your own testing that led to your team's choice

  2. Do you think Hy4 is currently capable of RSI to its next generation if left on its own? In addition, do you have benchmarks in which you measure this ability and are capable to testing autonomy outside of n hours + m tool calls of auto implementation?

  3. What does your team think is the direction HY series models want to take with AI? Do you expect the primary improvement mechanism to come from modifying model weights, constructing better inference-time systems, accumulating persistent procedural memory, or coordinating multiple specialized agents? (Which is the main focus, since ofc they are all important)

Thanks a bunch and please keep the remote control thingy from the current preview mode since it's a life saver when i want to execute something on my pc through gui and not agent in case something needs evaluation or goes wrong!

20

u/edwardbai 1d ago

Let me answer your first question. We found that the sweet spot (minima of a U-shaped law) for the Engram parameter ratio is around 1/3 of the total parameters including Engram.

17

u/aiden_agent 1d ago • Bravo

2. Our RSI capabilities are still at an early stage. We have developed several internal benchmarks to evaluate key aspects of RSI.

3. Our current priority is to build solid foundations, improve model intelligence, and work closely with our products to deliver a better user experience.

19

u/dwrz 1d ago

I have noticed degradation in LLMs ability to write. I would use Claude 3.5 for editing text, but no longer use 5.5. The only LLM that I've found outputs barely sufficient code documentation is Gemini Pro. Qwen 3.6 is a better writer than 3.8. Is this because LLMs are being trained on code, because of overfit, or something else?

29

u/lydellhuang 1d ago

We've noticed this too. We think it might be related to the model's training objectives. We want our models to write more like humans and sound less formulaic. Working on improving this, so stay tuned!

→ More replies (1)

7

u/aiden_agent 1d ago

Answered by u/lydellhuang (Hy Team)

19

u/Laoweek 1d ago

Not a question just an encouragement. Please keep doing the work so we don’t need to deal with the godforsaken company that’s Anthropic

3

u/maxmpan 15h ago

Stay humble and keep moving on❤️

18

u/rorowhat 1d ago

Since you guys know the models like nobody else does, can you also release an ultra optmized engine that goes along with the model? I believe this would really sell this model to most folks that are struggling to get good performance with llama.cpp.

21

u/dick112233 1d ago

We are advancing the relevant adaptation work for llama.cpp and will promote support within the open-source community in the near future.

6

u/aiden_agent 1d ago

Answered by u/dick112233 (Hy Team)

32

u/pantalooniedoon 1d ago

Thanks for another open release! Do Chinese labs look to reddit for feedback at all? Does our complaining and begging work?

38

u/aiden_agent 1d ago • Heartwarming

Thanks for your interest! We take community feedback very seriously.

15

u/More-Curious816 1d ago

What was the most difficult behavior to improve during post-training?

41

u/aiden_agent 1d ago

Overthinking.

14

u/Frail_Waif 1d ago

Do you all talk with other open LLM labs outside of research papers, or is the environment fairly competitive? 

16

u/aiden_agent 1d ago

We actively learn from the open LLM community and hope to contribute more of our own work back to it.

14

u/TheRealMasonMac 1d ago

Have you thought about paying more attention to non-verifiable instruction following? I notice that a lot of proprietary Western models perform better (e.g. with regards to style constraints).

16

u/maxmpan 1d ago

Pulls out tiny notebook and writes furiously. Noted!

10

u/aiden_agent 1d ago

Answered by u/maxmpan (Hy Team)

2

u/madtopo 1d ago

What does "non-verifiable instruction following?" mean? 

5

u/TheRealMasonMac 1d ago

Constraints that you can’t verify programmatically, and instead use humans or reward models to judge compliance.

Examples:

- Omit meta-commentary

- Avoid using idioms in explanations

- Research without bias

13

u/ps18pmcom 1d ago

Will you at all release any small models to compete with LFM3 on edge devices?

17

u/edwardbai 1d ago • Heartwarming

Yes. More small models coming.

→ More replies (1)

3

u/JGByvygyrfg 1d ago

yeah I've been wanting a sequel to last year's Hunyuan Dense models, the sizes were good but the models themselves were awful in my testing

12

u/shuczhao 1d ago

Great to see the red-teaming details for Hy4. Usually, heavy safety tuning makes open models refuse basic pentesting, reverse engineering, or CTF tasks. How did you guys avoid over-refusal on those dual-use coding prompts?

21

u/aiden_agent 1d ago

For cybersecurity training, we focus on defensive tasks, vulnerability analysis, and remediation, while excluding data focused on exploitation and offensive attacks. We also conduct red-team evaluations to assess potential risks.

Avoiding over-refusal remains an important area for improvement.

12

u/Transcendent_Bussy 1d ago

Are you guys planning on continuing development of models that can fit in the 64-128gb ram range?

12

u/ortegaalfredo 1d ago edited 1d ago

Hi!

I tested Hy4 Flash on custom benchmarks. Unfortunately, many new models seem to be benchmaxxed, like Baidu Ernie, that while being a big model, in my benchmarks it scores lower than qwen-flash.

But Hy4-flash scores very well, up to, or better than Qwen-3.8-Flash-Next. That makes me think that the big Hy4 might be the best current open model available. Strata/Maya/antirez ds4 have been game changers in that they allow to run those big models on modest hardware. So I have two questions:

  1. Are there efforts to release profile-based quants or contribute to projects that would make Hy4 usable in low VRAM systems like for example DS-V4.1-1017 is? I guess...Hy4 could do that himself.

And

2. Do you feel the pressure to remove cyber-capabilities like Kimi did?

29

u/GeoMorax 1d ago

Thanks for liking the Hy models! We're working on supporting Hy4 in Strata, and it'll be available very soon.

4

u/ortegaalfredo 1d ago edited 19h ago

Nice! thanks you for your generosity!

→ More replies (3)

6

u/aiden_agent 1d ago

Answered by u/GeoMorax (Hy Team)

→ More replies (3)

12

u/More-Curious816 1d ago

from https://hy.tencent.ai/Hy4_system_card.pdf
We therefore deliberately filtered the cybersecurity data used during training, retaining only data related to vulnerability reproduction and reme-diation and removing offensive training content such as exploitation, reverse engineering

Vulnerability discovery and reverse engineering are not inherently offensive activities. and Incident-response teams use reverse engineering, for malware analysis and deobfuscation, and forensic investigation to understand how an attacker compromised a system.

vulnerability discovery capability that allow the defense team to discover vulnerabilities before the attackers do is one of the most important ways to reduce the attack surface of critical infrastructure from the defnse point of view

Aren't you making defenders pay the price by limiting the model cyber capabilities? especially since these capabilities are inherently dual-use so limiting one hurt the other side badly?

6

u/aiden_agent 1d ago

Answered by u/maxmpan (Hy Team)

8

u/maxmpan 1d ago

From our evals, Hy4's vulnerability discovery capability is strong, while exploitation is relatively weak.

10

u/More-Curious816 1d ago

What was the strongest architectural alternative to the final Hy4 design that the team seriously considered but rejected? and why?

20

u/edwardbai 1d ago

Good question. We went back and forth between GQA and MLA for quite a while. MLA has significantly higher compute intensity than GQA, which made spec decoding on inference GPUs compute-bound. However, at the same KV cache size, MLA delivers better model quality than GQA. Fortunately, with the introduction of DSA, we were able to bring down MLA's compute intensity during decoding. That ultimately tipped the balance, and we decided to go with MLA.

8

u/Ok_Cow1976 1d ago

Are the two models trained more on coding or for general purposes

19

u/aiden_agent 1d ago

General-purpose.

8

u/True_Requirement_891 1d ago

Why remove cyber sec red-teaming data from training? One could literally chain up a harness that uses this model part of the loop and do the same task.

It neuters the model for no real gain. Have you guys thought of an alternative to this?

→ More replies (3)

8

u/More-Curious816 1d ago

you said that

Compared with Hy4, Hy4 Flash introduces three notable architectural changes.

  1. Hybrid attention. 2 LatentMoE. 3 N-gram embedding (Engram).

but also said

Post-training: Self-distillation from Hy4 establishes strong general intelligence,

So Hy4 Flash combines self-distillation from Hy4 with a substantially different architecture incorporating 1.Hybrid attention. 2. LatentMoE. 3. N-gram embedding (Engram).

my questions are:

  1. Hy4 Flash introduces LatentMoE and Engram as architectural changes relative to Hy4, while using self-distillation from Hy4 during post-training. How did you adapt the distillation process to account for these architectural differences?

  2. What were the biggest challenges in teaching Flash to learn from Hy4 when the two models have different architectures?

  3. what did you learn about the distillation process itself while developing Hy4 Flash? Were there unexpected findings about what a smaller model can learn from a larger teacher, particularly when the two models have substantially different architectures?

  4. How much of Flash's performance comes from learning from Hy4, and how much comes from its new architecture?

  5. Which abilities were hardest to preserve when reducing the model to approximately 11.3 billion active parameters per token?

7

u/Loose_Being_8583 1d ago

Welcome!Like Hy4 Preview, the full release of Hy4 still sometimes spends too long reasoning. Does this overthinking come from the RL reward design or from the distribution of the training data? Can the model learn when to stop?

20

u/No-Cherry2850 1d ago

The goal of Hy4 is to maximize intelligence, so long cot is known issue. However, it enables us to distill it down to Hy4 flash with good cost efficiency. For flash we use rl to make sure cot is short

7

u/Exciting_Variation56 1d ago

Does your team see this model as one users will run on their hardware at home? or is that your intention with your hinted-at smaller MoE model in other threads?

I'm literally looking into a server rack just because of this flash release. Great work and thank you for sharing with the world.

9

u/No-Cherry2850 1d ago

Yes

5

u/Exciting_Variation56 1d ago

Is there a known timeline for that part of the roadmap? that's incredible news

6

u/JGByvygyrfg 1d ago

Any plans to release a new open image model? HunyuanImage-3.0 was great but far too large for most setups. Make a smaller 5-12B model for local usage than make a larger model as there need to be more open source image models. I'd like to see another HunyuanVideo as well. Also btw what happened to your Discord server? There no moderation, very little real activity, and you let it get hacked once.

8

u/No-Cherry2850 1d ago

We’ll think about a smaller image model, and check out video model soon

→ More replies (3)

6

u/lawdawgattorney 1d ago

Are you going to release an exl3, nvfp4 or trellismx quantized version of this (hy4) model?

3

u/dick112233 1d ago

Due to resource constraints, we have not enabled quantization support for formats such as Hy4 nvfp4. However, you can follow and refer to quantization schemes and weights provided by some community developers—for example: https://huggingface.co/Avifenesh/Hy4-flash-NVFP4 and Hy4-Flash GGUF: https://huggingface.co/AngelSlim/Hy4-flash-GGUF/tree/main

→ More replies (1)

6

u/Barni275 1d ago

Any chances for Hy-27B? 🙏

4

u/DiWang01 1d ago • Heartwarming

Possibly — it's under consideration.

5

u/Every-Article6734 1d ago

Will there be similar size of open weight models in future? This is particularly fit for 128GB machine in Q4.

Thank you for bringing the model to the community.

2

u/aiden_agent 1d ago

Answered by u/GeoMorax (Hy Team)

5

u/Kahvana 1d ago edited 1d ago

Thank you for releasing the weights!

  • Is Tencent considering "Flash Lite" models (e.g. 35B-A3B)? These can be quite beneficial (e.g. for costumer-facing chatbots, document parsing, classification, simple automatisation, tasks benefitting from low drag and high speed, etc.)

  • What benefits does iHC have over mHC and Gated Residuals (GR)?

  • Why was SWA chosen for Hy4-Flash over gated DSA used in Hy4?

  • Are other attention mechanisms (such as QSA) being considered for the future?

  • Thank you for changing the license. It's nice that the model is now allowed to be used within the EEA! (Greetings from the Netherlands!) Will the terms and licences for older open-weigth models on huggingface be updated also?

Finally, for more personal questions to the people(s) answering:

  • How do you feel about contributing to open-weight models?

  • What has been the most difficult moment for you personally during development?

  • What was the biggest joy for you personally during the development itself, besides the release of your work?

  • If you had a wish, what would you wish an LLM could do for you in the future? (It can be realistic, dreamy, whatever you feel like. Fun answers like generating anime waifu companions or the perfect 饅頭 (steamed bun) picture are completely acceptable too! 🤣)

Thank you for your time and for doing the AMA. I'm looking forward to using more of your work in the future!

14

u/edwardbai 1d ago

Cool, many technical questions. 1. Yes, stay tuned for a cutting-edge Flash-Lite model. 2. Compared to mHC/GR, iHC is much simpler, delivering comparable performance at a lower cost. 3. SWA reduces KV cache size by 4×, significantly lowering serving costs. 4. Of course! We are open to all sorts of arch innovations.

8

u/lydellhuang 1d ago
  • Contributing to open-weight models: It's incredibly rewarding to contribute to the open-source community, especially when we see so many people paying attention to our work.

  • The most difficult moment: For me, it was scaling from Hy3 to Hy4. We didn't do as many detailed ablation studies on the data as I would've liked, so there was a lot of uncertainty going into training. I was honestly pretty nervous at the beginning of training.

  • The biggest joy: Luckily, the training went surprisingly smoothly in the end. It felt like going all-in on a bet and actually winning :)

  • My wish for future LLMs: As a huge foodie, I'd love an LLM that could invent entirely new recipes for me, and actually make sure they taste exactly the way I like!

5

u/kmike84 1d ago

Recently mimo was really open about their training process: https://mimo.xiaomi.com/rl/#overview

What do you think about this?

And if you can share - is their post-training close to what you're doing, or are there significant differences - costs, amounts of data, ratio of various task types, etc.?

10

u/aiden_agent 1d ago

It's great to see MiMo sharing their work. Our RL training runs at a larger scale.

5

u/Impossible_Ground_15 1d ago

Are you considering releasing an open weight Text to Speech model?

Thank you for what you do for the community!

5

u/aiden_agent 1d ago

Well received!

9

u/EducationalWelcome31 1d ago edited 1d ago

Hy4 Flash - Is this model harder to quantize than others of similar size?

I had awesome 100% performance with the hosted model in a work task I often use as a model benchmark. But with the NVFP4 build I had around a 8% error rate. Haven't seen any drop near that level with 4-bit quants of other models.

Compared to other 4-bit quants, I've also found KLD to be surprisingly high and same top token as the base model to be low.

I use basalt on a 2x5070ti + 1x5060ti, 96gb ddr5 machine.

12

u/GeoMorax 1d ago

Quantization quality depends heavily on the method and calibration used. We recommend trying our officially provided NVFP4 build, which is quantized with GPTQ. We release two NVFP4 variants, available here: https://huggingface.co/AngelSlim/Hy4-flash-GGUF/tree/main In our evaluations, the NVFP4 variant shows only a small gap from BF16, so it should be a better fit for accuracy-sensitive tasks like your benchmark.

2

u/aiden_agent 1d ago

Answered by u/GeoMorax (Hy Team)

8

u/Illustrious-Thing567 1d ago

How much thought have you given to creative writing/readability? One of the major things that would make me switch away from Qwen 3.8 Flash Next would be having a model that is far more readable and understandable when explaining things.

11

u/aiden_agent 1d ago

We also think creative writing/readability is very important and are making it a key priority for improvement.

→ More replies (1)

8

u/CriticallyCarmelized 1d ago

In my testing it already is far superior to Qwen, and from another reply in this thread, it sounds like a focus of theirs.

8

u/Manerfish 1d ago

From what I have experienced with Hy4 flash, it seems like a model that really gets to the point without adding much fluff or using weird words. I haven't tried for creative writing, but I imagine that for documentation or stuff like that it would work well.

6

u/kkelvinchen 1d ago

Quite a bit, and we still have room to improve. Sometimes one good example does more than five paragraphs.

4

u/lydellhuang 1d ago

Similar to the previous answer, we care a lot about how useful our models are in practice, and writing quality and readability are both important parts of that. We're actively working on improving them.

4

u/Notdesciplined 1d ago

At what level would Hy4 models be at today?

what level will it potential reach for future models?

3

u/aiden_agent 1d ago

level 2.5 imo

4

u/MerePotato 1d ago

Any plans for a model targeting 24GB VRAM?

5

u/GeoMorax 1d ago

Yes, it will be released in the future.

→ More replies (1)

2

u/aiden_agent 1d ago

Answered by u/GeoMorax (Hy Team)

4

u/Mysterious_Finish543 1d ago

Will there be a technical report for Hy4 and Hy4 Flash?

Would love to get a report like DeepSeek did for R1, V4, etc, very useful for open research.

7

u/edwardbai 1d ago

will keep that in mind.

5

u/maxmpan 1d ago

you may check our system card for more eval results: https://hy.tencent.ai/Hy4_system_card.pdf

4

u/KambeiZ 1d ago

I don't know if i'm too late to the party, but i'm curious on the training & architecture side: what leads the decision behind each tinkering of the network layers? For example, how do you "choose" what layer are essential to keep for the flash from the initial architecture in Hy4 ? Really curious!!

4

u/edwardbai 1d ago

Welcome to the party! Hy4-Flash builds on the technologies from Hy4, like iHC, gated MLA, and DSA, while introducing several new arch component: hybrid SWA, LatentMoE, and Engram. These design choices are backed by extensive experiments across our scaling ladders and careful model–inference co-design.

4

u/Shadow_s_Bane 1d ago

Can we expect GGUFs ? If yes which one of the flash ones should I get ?

3

u/aiden_agent 1d ago

Answered by u/GeoMorax (Hy Team)

4

u/InfinityZeroFive 1d ago

Thanks for the open-weights! I work in computational biology and I have tested Hy4 Flash on hand-docking ligands into proteins without using standard docking tools (like GNINA), in both reasoning-only and with-tools settings. As a fun eval. This task requires the model to reason over the biological chemistry, first identify the likely pocket of the protein and then, based on the contacts, how the ligand should be placed.

Hy4 Flash is surprisingly decent, especially considering it recalls the protein less often than other open models (like DeepSeek V4.1 Flash 1107). It has moderate-OK capability to reason from sequence the correct pose, but with tools has surprised me a few times, even beating Claude Sonnet and Haiku 5.5 in some cases (the Claude models are heavily trained for biological tasks - currently open-source models are quite far behind in science, for example as you can see in TerminalBench-Science)

My question to the Hy team is: do you plan to prioritise RL for science (bio specifically) in the future, especially for mid-sized models in the range of Hy4 Flash to 500B parameters? And do you think Chinese labs can challenge Claude's dominance in biology?

7

u/aiden_agent 1d ago

We are actively investing in RL for science, while also placing a strong emphasis on safety.

→ More replies (1)

12

u/crusaderky 1d ago

How much do you think ArtificialAnalysis' obvious bias towards closed US models is hurting your bottom line and that of other Chinese frontier labs?

9

u/maxmpan 1d ago

One site doesn't really make or break real-world adoption. We look at all kinds of evals—internal benchmarks, public ones, third-party, plus actual user feedback. You need the whole picture to tell how a model really performs, and at the end of the day, that's what people care about.

5

u/aiden_agent 1d ago

Answered by u/maxmpan (Hy Team)

3

u/mr_andrey_isachenko 1d ago

Congrats on the Hy4 and Hy4 Flash release, and thanks for doing an AMA here!

Two questions for Shunyu and the architecture/RL team:

  1. Hy4 Flash's architecture is super interesting on the memory side - cutting KV cache by 6x-24x and offloading the 64B Engram parameters to host RAM makes 12B active params fly on 128GB+ setups, but 220B total (130GB+ even at Q4) still prices out most single-GPU and 64GB Mac users here. If you scale this architecture down to a "Hy4-Mini" (say ~30B-35B total, 3B-4B active), do the Engram embeddings and Identity Hyper-Connections (iHC) still give the same boost, or do they only shine at 200B+ scale?

  2. With Shunyu leading the charge (huge fan of your original ReAct work): when training Hy4 and Hy4 Flash for long-horizon agent/CLI loops, what was the hardest reward-hacking or state-drift failure mode you had to solve in multi-turn RL?

5

u/edwardbai 1d ago

On your first question, Engram and iHC have even greater benefits for smaller models.

3

u/No-Cherry2850 1d ago

The hardest challenge was reward hacking caused by gaps in environment design. For example, preventing agents from searching online for answers instead of actually solving the tasks.

3

u/ResearchCrafty1804 1d ago

Some researchers say that theoretically an 1B (active parameters) model could outperform today’s frontier models in a couple years, do you think it’s indeed possible, or is there a ceiling?

3

u/chill-boi 1d ago

The Hy4 system card says you “turn real-world needs and feedback into training data and evaluation targets,” and some misuse evals are built from real agent interaction data. Did you ever use past misalignment incidents, either your own or publicly reported ones from other labs, in training or as evaluation targets?

4

u/maxmpan 1d ago

only on the eval side for tracking and monitoring

2

u/aiden_agent 1d ago

Answered by u/maxmpan (Hy Team)

3

u/takoulseum 1d ago

Instead of making new models, do you plan updates of Hy4 Flash?

6

u/lydellhuang 1d ago • Bravo

Sure, we are optimizing both pretraining and postraining recipes for Flash level model

→ More replies (1)

3

u/a_pimpnamed 1d ago

You guys planning on making ternary models from scratch to make models stupidly efficient to serve?

3

u/GeoMorax 1d ago

Currently we do not have the plan to train a ternary models from scratch.

3

u/aiden_agent 1d ago

Answered by u/GeoMorax (Hy Team)

3

u/power97992 1d ago

How is the progress on continual learning? 

6

u/aiden_agent 1d ago

It's a promising direction, and we're actively researching it.

4

u/kmike84 1d ago

How would you rank your friendly open source competitor labs (deepseek, z.ai, moonshot, alibaba, etc.) - not based on the model scores, but based on their team's technical capability, research quality, etc.? Who would you consider top 3, in order?

19

u/maxmpan 1d ago

Not falling for that trap—all of them are great:)

3

u/aiden_agent 1d ago

Answered by u/maxmpan (Hy Team)

2

u/-Krotik- 1d ago

which technology do you think is gonna impact local LLMs the most?

→ More replies (1)

2

u/sdatta11 1d ago

CLI can easily break when a model hallucinates precise syntax or flags. Could you elaborate on how you handled reinforcement learning without loosing the context.

5

u/aiden_agent 1d ago

For tool-calling accuracy, we put a lot of effort into RL data, including challenging cases where models are prone to making mistakes. We also use a fine-grained reward system that applies stronger penalties for errors in critical details.

For context retention, we are continuously extending the context length used in RL training and specifically improving the model's context compaction capabilities.

→ More replies (1)

2

u/power97992 1d ago

Can your company build a memory fab, so vram and gpus can get cheaper? ( Highly unlikely to happen)
A more pertinent question- for hy 4.5, you scale up the parameters to 1.5 Trillion to 2 T yeah?

3

u/DiWang01 1d ago

The next-generation Hy model will definitely be scaled up even more significantly.

2

u/Pharaoh0527 1d ago

With access to advanced AI chips becoming increasingly restricted, what is the biggest bottleneck for Tencent Hy: compute, training efficiency, or algorithms? Can future Hy models continue to close the gap with frontier models?

→ More replies (1)

2

u/Zealousideal-Mall818 1d ago

OK , LOVE YOUR WORK AND THANK YOU , question ; when will you guys pivot to creating models that use more structured Datasets , instead of just text , when will you merge the 3d training with video training creating a render engine that is controllable not some random slot machine ,

2

u/aiden_agent 1d ago

Great suggestion!

2

u/Efficient_Fig_254 1d ago

exl3 quantization would be huge for local inference on lower-end hardware. any plans to support it?

3

u/dick112233 1d ago

Currently, there is no official plan to adapt an exl3 version. However, since the model has already been open-sourced, we believe the community will soon roll out corresponding versions gradually.

→ More replies (1)

2

u/PrimeDirective8 1d ago

Looking at llama.cpp PRs, I see one to support Hy4 Flash was opened 2 days ago. Do you guys work with the likes of llama.cpp to add support before the model is widely released, or is it best to release and let users add support later?

I'll second other's request for a 35B to 70B, maybe up to 120B MoE.

4

u/dick112233 1d ago

We are advancing the adaptation for llama.cpp and should release it in the community in the near future; we are currently engaged in deep collaboration with communities such as vllm and sgl.

3

u/aiden_agent 1d ago

Answered by u/dick112233 (Hy Team)

2

u/Anna__Nelson 1d ago

Thank you for open-sourcing Hy4 and Hy4 Flash! I’ve been using both recently and have had a great experience. I have a few questions for the team:

  1. Now that Hy4 and Hy4 Flash are open-source, are you considering releasing smaller models, such as 27B or 35B variants, to support a wider range of private and on-premises deployments?

  2. Hy4 Flash tends to produce shorter chains of thought than many other Flash models. However, shorter reasoning can also lower a model’s capability ceiling. What considerations drove this trade-off?

  3. What will you prioritize in future Flash updates—for example, post-training for a potential 4.1 release? Will the focus be on maintaining efficiency or raising the capability ceiling? In my testing, Hy4 Flash seems to have more gaps in general world knowledge.

  4. I noticed that Hy4 Flash uses 12 sparse-attention layers and 30 sliding-window-attention layers, while all 78 layers in Hy4 use sparse attention. What drove this architectural choice? Was it specifically designed to improve Flash’s efficiency, or did training stability also play a role?

  5. Given the difficulty of distilling Claude, how much could this affect improvements in your next model release?

  6. With the teams behind GPT and Claude increasingly pushing into embodied AI and scientific research, what areas will Hy focus on or expand into next?

2

u/nunodonato 1d ago

Is there a specific reason why you guys decided to not publish benchmark results for very common benchmarks like GPQA Diamond, tau bench, IFBench, SWE Bench...?

9

u/maxmpan 1d ago

tbh, we show results for recent advanced benchmarks like Agents' last exam, ProgramBench, etc. Benchmarks you mentioned are a little bit earlier and the scores are saturated.

2

u/aiden_agent 1d ago

Answered by u/maxmpan (Hy Team)

2

u/Silent_Glass 1d ago

Perhaps a very dumb question as I’m still learning about local AI Models. I have 96GB VRAM with 128 RAM. Can it run flash?

5

u/GeoMorax 1d ago

Yes, it can run. You can keep the main weights and attention on the GPU, and offload some of the expert layers to system RAM. The Engram tables can also live in RAM.

2

u/aiden_agent 1d ago

Answered by u/GeoMorax (Hy Team)

2

u/CharlesYao0710 1d ago

我不太懂你们讨论的那么专业的内容,但是至少对于国内的一些AI,我觉得Hy正在加速追击甚至赶超,而且是我欣赏的开源路径\⁠(⁠°⁠o⁠°⁠)⁠/。问一个比较low的问题吧O⁠_⁠o,Gemini对世界知识的理解强有一部分原因是Google学术还有Google搜索,YouTube等多年的数据库,那你们团队的训练数据是否一部分来自于公众号,视频号等,这些数据对于训练有效吗?(⁠◠⁠‿⁠◕⁠)

→ More replies (1)

2

u/OddDesigner9784 1d ago

What are models you consider comparable to hy4 or flash from the US labs. I don't really trust AA so I wonder what your evaluation is

2

u/More-Curious816 1d ago

"Although Hy4 is one of the strongest open-weight models available today, a notable gap remains between it and top proprietary models, particularly on challenging internal benchmarks for research-level mathematics, long-horizon coding, model self-improvement, and advanced visual question answering."

there is so much attention on stem related topcs like coding, mathematics and scientific researchs, but lack of attention on creative works like writing coherent fictional stories. do you plan on shift your attention to this issue or release a model that less coding and mire creative writing?

2

u/cleverusernametry 21h ago edited 19h ago

I turn criticism into power, I soar beyond limits, owning my greatness!

3

u/More-Curious816 1d ago

from https://huggingface.co/tencent/Hy4-flash

Disclaimer: Because our base model was exposed to a large amount of synthetic instruction data during mid-training,

what are the reasons for exposing your base model to a large amount of synthetic instruction data during mid-training?

5

u/edwardbai 1d ago

To improve agentic capabilities

→ More replies (1)

2

u/RobotSympathiser 1d ago edited 1d ago

Hi Hy4 team, congratulations on the release.

I work on small and medium sparse MoE models and quantization. I have a 35B-A3B model (256 experts, top-8, about 3B active) quantized to roughly 2.1 bpw ternary, and I'm trying to recover capability with 4-bit on selected expert tensors plus low-rank correction branches. Your 12B-active, 512-expert LatentMoE design is the closest reference I have.

I have five questions, roughly in order of how much they would change what I do next.

  • Your IQ2-AH build uses IQ2_XXS on gate/up and Q4_K on down. On my 35B, the opposite works better: Q4 gate/up plus ternary down gives PPL 2.34, while Q4 down plus ternary gate/up gives 2.68, and all-ternary gives 2.93. Is down generally more sensitive in Hy4, or is that specific to LatentMoE and the 1536-d expert latent? What signal do you use to decide which expert projection gets more bits? Relatedly, is per-expert mixed precision worth the complexity in a large fine-grained MoE, or is per-tensor allocation usually enough?

  • Do you train the router when recovering a quantized MoE? I froze the router and saw routing drift and a tool-use regression, and preserving the original routers did not fix it. Is router co-adaptation essential after quantization, and how do you keep it stable?

  • For a frozen, quantized body, would you use low-rank branches at the MoE and attention outputs, full-body LoRA/QLoRA, or QAT on the quantized weights? Where do adapters belong in a sparse MoE? My rank-512 branch LoRA at the block outputs was not enough, and I would rather not repeat that on a larger run.

  • As active parameters drop from 12B toward 3B, what degrades first, and is there a point where a ternary or 2-bit MoE stops being viable? If you had a 3B-active model, what would you spend the next bit or the next unit of compute on: more active capacity, Engram, higher bits, or latent experts?

  • How much does Engram contribute at 12B active, and would it be worth adding to a 3B-active model? Do you use a separate n-gram table per layer, or a shared table with per-layer projections? Which trade-off do you prefer for memory and capabilities?

Thanks, and congratulations again on the release.

3

u/stylome-ai-detection 1d ago

Notice: This comment appears to be AI generated.


Stylome AI detector · scan results

3

u/GeoMorax 1d ago
  1. In general Transformer layers, the down projection is more sensitive to quantization because of activation outliers, so we give it more bits. We choose bit-widths using Hessian-based sensitivity. We prefer per-tensor allocation over per-expert mixed precision, because many inference engines treat all experts as one stacked matrix and per-expert precision would break that. Per-tensor is also more portable.

  2. Yes. In QAT we train the router together with the rest of the model. The routers will be trained to be stable

  3. We prefer full-parameter QAT because it gives better quality. We haven't run LoRA experiments, so we can't advise on adapter placement.

  4. We expect reasoning depth, long-context tracking and tool-call reliability to degrade first, and 2-bit PTQ to break well before ternary QAT does.

  5. Engram parameters are not counted in the 12B active parameters. We use separate n-gram table per layer. We prefer to maximize capability as long as the memory budget is met.

→ More replies (1)
→ More replies (1)

1

u/Mr_Lucas2000 1d ago

Do you think you'll match the performance of sonnet 5.5 or better in hy4.5 or hy5? And what's your look on the future?

1

u/quiet_meadow_734 1d ago

How frequently will you aim to release new checkpoints?

How long does it take to train a checkpoint of an existing model?

And how do you improve your training data between checkpoints to improve the model's capabilities?

1

u/Easy_Refrigerator280 1d ago

What percentage of weights would you say might be considered bloat and could be trimmed if we were very precise with our training?

Like efficient reasoning to search up info vs baked info onto weights. Or just general irrelevant info in weights that does not help towards intelligence?

1

u/Aley701 1d ago

Where i can find some official tutorial video on implementing your model into our workflow

→ More replies (1)

1

u/AIonyourface 1d ago

Coding plan when ?

1

u/yanhua_lab 1d ago

hi hy4 team, thanks for releasing the weights.

got a couple questions about the autonomous r&d part:

  1. in the hyra 2.0 experiments, did hy4 mostly win through hyperparameter tuning and low-level kernel throughput optimizations, or did it actually come up with non-obvious architecture or data mix improvements?

  2. also, you mentioned in known limitations that there's still a gap in model self-improvement. is the real bottleneck right now the model's ceiling for coming up with good research hypotheses, or is it more that the automated eval harness has super sparse reward signals and long iteration cycles?

would love to hear any insights you can share.

1

u/Jeweloon 1d ago

Hello, Mr. Yao Shunyu.
You previously noted that Tencent AI has entered the second half of its development trajectory. Just days ago, Tencent unveiled its HY4 model, which has quickly cemented its place in the industry’s top tier.
I’m curious: what is the core strategic goal for Tencent AI in this next phase? And what will be your most critical priorities moving forward to deliver on that vision?

1

u/Loose_Being_8583 1d ago

Shunyu has often emphasized the importance of having good taste in data. What are your criteria for good training data, and how do you test whether a dataset meets them?

1

u/Lyelinn 1d ago

Any recent innovation you're looking for implementing for the future pipeline or models?

What are the worst bottlenecks of today in terms of making a new release from pre-training to finished, usable model?

1

u/Super-Ad2061 1d ago
  1. Will there be optimizations on the architecture side going forward — specifically, a more efficient architecture to ensure token generation speed (while maintaining quality)?

  2. hy4 has achieved strong results in many scenarios, but compared with hy4 preview, in complex scenarios it produces significantly more output tokens, tool calls, and turns. In our testing, it is not more efficient than deepseek v4.1 flash 1017. Will this be optimized going forward? Mainly because the DS models handle cost-efficiency quite well.

  3. How should data mixture ratios and data experiments on small-sized models be more reasonably scaled up to large models?

  4. How should verification (verify) be done in finance, medical, and legal scenarios?

  5. How long until a model exceeding 1T (parameters) is expected to be released?

  6. Will there be a video generation model?

1

u/Loose_Being_8583 1d ago

With Hy4 and Hyra 2.0, you’ve already been able to improve your training infrastructure and train better small models. What experiment would you run next to show that a model trained in one round is itself better at doing model R&D in the next? How can a model get better at learning itself?

2

u/aiden_agent 1d ago

We’re working on Hyra 3.0. Stay tuned!

1

u/fancyrocket 1d ago
  1. Will you release any smaller models to compete with Qwen 27B Dense Models?

  2. Will you release any coding focused models like in the <100b range?

5

u/DiWang01 1d ago

First: maybe.

Second: probably not.

1

u/tehrob 1d ago

They're hosting an AMA to demonstrate the capabilities of their language models while communicating as though language itself were an insurmountable technical obstacle.

1

u/paul_tu 1d ago

Thanks for you work. I've been impressed by how Hy4 deals with CAD and modelling overall. Any plans to make QFN competitor? Like streaming ngrams for Strix halo range of devices is pretty useful. Yet it seems not an option with Hy4.