r/Qwen_AI • • 25m ago

Funny I asked Qwen a question.

0 Upvotes

r/Qwen_AI • • 2h ago

Other qwen should've kept things free.

Post image
0 Upvotes

for the same price I can get more of 6.1 sol max and gemini 4 hydrogen xhigh? Who are they trying to win over, non tech people (majority of users) will use the more familiar chatgpt and tech people or people with expertise will straight up use lesser capable models for under 1$. only thing keeping them relevant is their local models, they should either make it free or far less $ for the monthly subscription + include a full uncensored model right there in their webapp. or focus to excel in a specific domain, maybe tutoring for example. I only used 3.8 max till now because it was free, nothing else honestly.


r/Qwen_AI • • 2h ago

Help 🙋‍♂️ Is it possible i can squeeze in a decent Qwen model in a P52 thinkpad?

2 Upvotes

With all the talk about Qwen4, Strata, and new inference engine, is it possible to run anything stronger than 35-A3B. I don't mind slow token speeds aslong as It's in the double digits.. Is there any inference engines well suited for this?

Specs:

4GB VRAM

32GB DDR4 RAM

Intel-Xeon E-2176M (12) @ 4.400

Is there hope guys?..


r/Qwen_AI • • 3h ago

Discussion The effects of Anthropic making distillation harder is clear

Thumbnail
youtube.com
0 Upvotes

Not sure what I expected, but here are some Qwen 4 Max outputs.


r/Qwen_AI • • 4h ago

Discussion Seriously impressed with Basalt on 24GB laptop RTX 5090. Qwen 3.8 27B at 162.2tps decode and prefill at 5431tps. Qwen 3.8FN at 248tps decode and 3335tps prefill.

Thumbnail
gallery
21 Upvotes

As the title says, I'm seriously impressed. Especially so with 27B. I was using llama.cpp and could get around 40 - 50tps, but damn, this just blew it straight out of the water. Insane speeds for such a dense model.

Obviously, 27B hits my graphics card hard, but I found it interesting that on Qwen FN, I literally didn't even notice any temp increases. It's like it has no effect on my hardware, but it's operating.

Definitely worth a try if you're on Blackwell architecture.

It makes me even more excited for Qwen 4's release.


r/Qwen_AI • • 4h ago

CLI Saetta: a single-model engine for Qwen3.8-Flash-Next on Apple Silicon (Zig + hand-written Metal), with Claude Code / pi / opencode in one command

Thumbnail github.com
1 Upvotes

I've been building saetta, an inference engine that does one thing: run Qwen3.8-Flash-Next as fast as possible on an M5 Max for coding agents. No generic graph, no other models. It's a Zig host with hand-written Metal kernels, including prompt matmuls on the GPU's neural accelerators. It ships as a single binary and needs no Python.

What it does

  • OpenAI-compatible API (/v1/chat/completions) and Anthropic Messages API (/v1/messages), with streaming, thinking and XML tool calls

  • saetta start claude (or pi, opencode) starts the server and your agent together, without editing your agent config

  • saetta download coder fetches the model itself: pinned revision, resumable, every file hash-checked

  • Two packs: the original 4-bit MLX pack (105 GB, 512 experts) and the expert-pruned GSQ-RCO Coder variant (76 GB, 256 experts)

  • saetta monitor: a live view of GPU load, memory, prompt-cache reuse and recent requests

Numbers (measured on my M5 Max)

  • Prompt reading: about 1,650 to 1,950 tokens/s from 4K to 70K tokens

  • Decode: 63 to 82 tokens/s, with speculative decoding from the model's MTP head on greedy requests

  • Agent follow-up turns re-read only the new tokens: a 16K conversation answers the next turn in about 0.2 s, and real Claude Code sessions reuse about 20K of their roughly 20K-token prompt per turn

  • Cold 64K prompt: about 37 to 40 s to first token

Demo: Claude Code running on the local model writes a snake game in Python, in real time (GIF in the README).

Install:

curl -fsSL https://github.com/antonellof/saetta/releases/latest/download/install.sh | sh
saetta download coder
saetta start claude

Repo: https://github.com/antonellof/saetta (MIT)

Feedback welcome, especially from anyone running agents on long prompts locally.


r/Qwen_AI • • 4h ago

Funny Please, Qwen 4 35B A3B or such :D

Thumbnail
suno.com
43 Upvotes

r/Qwen_AI • • 5h ago

LLM 大家觉得qwen4max适合做项目规划吗

2 Upvotes

r/Qwen_AI • • 5h ago

Model So, Qwen 3.5 plus and 3.6 plus is not longer in the app?

2 Upvotes

Just saw qwen 3.5 plus and 3.6 plus gets removed from the model options in Qwen Studio and Qwen App

Guess they're gonna retire older models with newer ones as well in the future

Soooo, any substitution suggestion for qwen 3.7plus? I'm using it for creative writing, and for my personal consumption and entertainment. That's can generate story that's not stiff?

Preferably the models that I can access in the Qwen Studio and Qwen app.


r/Qwen_AI • • 7h ago

News Smaller Qwen4 models 🔥

Post image
553 Upvotes

r/Qwen_AI • • 7h ago

LLM They put official benchmark without their biggest opponent(aka Deepseek)

Post image
24 Upvotes

r/Qwen_AI • • 7h ago

News Qwen announces Qwen4 smaller models & Qwen4-Max!

Post image
27 Upvotes

r/Qwen_AI • • 7h ago

Discussion Is token plan worth it?

1 Upvotes

I am considering subscribing to standard or pro plan, had ollama cloud before but it is really bad, usage is too small and it is so slow with constant disconnects.

Anyone recommends pro plan or standard plan? Is usage generous?

I mainly use it for coding, doing research, and some blender with comfyui work


r/Qwen_AI • • 8h ago

LLM I Added LLama.cpp Support to Zikporx AI Chat Client

Thumbnail gallery
1 Upvotes

r/Qwen_AI • • 9h ago

News Qwen4 Smaller Models Open Weights Confirmed via Official Qwen X account.

Thumbnail
gallery
86 Upvotes

*all benchmark images can be reached from their Official X account.

“More to come! The open weights of smaller Qwen4 models will be released. Stay tuned. 👀”

https://x.com/alibaba_qwen/status/2122148127520289151?a

As we all heard that Qwen4 Max just dropped which I been excited for as a newly subscriber, I think most of us didn’t realize that they officially confirmed again that we will have Open Weights for Qwen4. So no need to sweat Local Community. Next up we have the summit on the 17th and conference on the 19th to look forward to for more answers.

I had to bring this up due to various commenters that might’ve been afraid they weren’t releasing any open weights at all. Just to clear your mind this is word for word from their official X account. Enjoy!

Also the other context before it.

“Meet Qwen4-Max, your new AI teammate for work, creativity, and discovery!

Qwen4-Max is built on a new MoE architecture with native vision and a 1M-token context window.

Highlights:
- Outperforms Qwen3.8-Max with up to ~50% fewer tokens, while average cost per task drops over 60% in our internal evaluation.
- Advances real-world productivity through greater proficiency with professional tools, stronger long-horizon execution, and higher-quality deliverables.
- Handles real tasks, including 3D modeling, game development, manufacturing, finance, law, healthcare, and frontier scientific discovery.
- $1.5/M tokens for input, $4.5/M tokens for output, and $0.075/M tokens for implicit caching — all lower than Qwen3.8-Max, with implicit caching down by 70%.

The API is live on QwenCloud. You can also try Qwen4-Max on Qwen Studio, QwenWork and Qoder. “


r/Qwen_AI • • 9h ago

Discussion Using local Qwen3.5 for NPC dialogue in Nine Thrones

Post image
2 Upvotes

Development-build screenshot: Baroness Isolde responding through the optional local Qwen3.5 4B dialogue system.

Hi everyone! We're Red Tower Studios, developing Nine Thrones, a party-based fantasy RPG with tactical turn-based combat, now in Early Access on Steam.

Our optional offline NPC dialogue uses Qwen3.5 4B and 9B. The screenshot shows an actual 4B-generated reply from Baroness Isolde in a development build.

We're working on keeping these conversations connected to the game world. Character prompts include personality, the current situation, relationships, and relevant recorded knowledge, with firsthand knowledge distinguished from unconfirmed reports.

Game systems remain responsible for changes to the world. An NPC saying something happened doesn't make it a completed game event. The dialogue pipeline also checks for unsupported factual claims, though keeping conversation consistent is still an ongoing development problem.

Traditional authored conversations remain alongside the optional AI dialogue. We also use generative AI in our art workflow, followed by Procreate edits and work in our in-game tools.

Disclosure: this is our paid Early Access game, with more work ahead on the world and simulation:

https://store.steampowered.com/app/5093250/

For anyone using Qwen for character dialogue, what has helped most with keeping replies grounded in the context you provide?


r/Qwen_AI • • 9h ago

Help 🙋‍♂️ Awareness of Qwens 4 Max False Positive Censoredship

Thumbnail
gallery
0 Upvotes

Anyone using the Qwens 4 Studio web interface for chat should know that I tested my narrative fictional fanfiction story and creative writing, and it triggered another false-positive censorship attack—unlike Qwens 3.8 Max. I’m sharing photos to raise awareness. You can see the difference between 3.8 Max and 4 Max for fanfiction writing: 3.8 is moderate, while 4 Max triggers another false censorship attack. I hope the development team will pay attention to this post.


r/Qwen_AI • • 10h ago

Funny Someone buy my a r9700 ai sapphire pls. Ty 😘🤣

Post image
44 Upvotes

r/Qwen_AI • • 11h ago

News Where Qwen models land on my AI-slop benchmark: the Qwen3.8 and 3.7 Max/Flash models sit in the sloppier half, Qwen3.7 Flash is among the cleaner ones

0 Upvotes

Direction first: #1 = sloppiest, #58 = cleanest. I've been building a benchmark called The Slop Index that scores short-form AI writing (email, Slack messages, social posts, essays) on padding, templating, rhythm and known AI tells. There is no LLM judge in the mechanical score. Every rank below is the mechanical rank only, out of 58 models, so it doesn't move day to day.

Here is where the Qwen models in the roster landed (overall mechanical rank):

ModelOverallEmailSocialEssay
Qwen3.8 Flash#18#9#6#29
Qwen3.7 Max#22#33#13#39
Qwen3.8 Max#23#31#36#6
Qwen3.8 Max Prime#27#24#38#15
Qwen3.7 Flash#45#41#46#52

A few things I noticed:

  • Four of the five sit in the sloppier half overall. Qwen3.7 Flash is the exception at #45, and it is also the cheapest model in that group at $0.03 in / $0.13 out per million tokens.

  • Qwen3.8 Flash is much sloppier than Qwen3.7 Flash on this score, but these are different models, so I wouldn't read it as a trend.

  • Qwen3.8 Max is #6 for essays but #36 for social posts. The same model can read very differently depending on the format.

Limitations: it only covers short-form writing, there is no fiction yet, and the mechanical axes miss repeated sentence constructions. The roster also has the Qwen3.7 and 3.8 models, not Qwen 4.

Links:

For context, I'm one of the people behind https://slashy.com, an email client that writes drafts that sound like you. This benchmark started as how we pick models.


r/Qwen_AI • • 12h ago

News Artificialanalysis lists Qwen4 Max at an Intelligence Index of 43

Post image
193 Upvotes

r/Qwen_AI • • 12h ago

Discussion I Fine-Tuned Qwen3-8B Twice on an RTX 5090. Unsloth used 33% Less PyTorch GPU Memory, but Barely Saved Any Time.

1 Upvotes

Imagine a small AI team that regularly fine-tunes a language model for customer support or internal applications. Every update consumes GPU time, and with a limited budget, choosing the right training framework matters.

An optimized framework might offer faster training and lower memory usage. But does that actually make the complete job faster and cheaper?

I wanted to explore that question by fine-tuning Qwen3-8B on a single NVIDIA RTX 5090 using two implementations: Unsloth NF4 and standard QLoRA with Hugging Face Transformers, PEFT, and bitsandbytes.

The results were surprising. Unsloth achieved approximately 2.1× higher measured non-padding-token throughput and used 33% less peak PyTorch-allocated GPU memory. Yet the complete 500-step training job finished only 21 seconds earlier.

That difference between training performance and total runtime became the most interesting part of the experiment.

The experiment didn't go exactly as planned

I originally intended to compare CQ 4-bit training against Unsloth NF4. However, the CQ implementation encountered kernel compatibility problems during the initial RTX 5090 tests, preventing a reliable training comparison.

Rather than publish incomplete results, I changed direction and compared Unsloth with a conventional QLoRA implementation. Both used the same locally stored, prequantized NF4 version of Qwen3-8B. This was a comparison of training implementations, not different quantization formats.

Testing on a single RTX 5090

Everything ran on a single RTX 5090 with 32 GB of VRAM through a GPU cloud platform.

For the dataset, I used Alpaca with 49,402 training examples and 2,600 validation examples.

Both implementations used LoRA rank 32, alpha 32, dropout 0, and adapters targeting attention and MLP projections. I configured a maximum sequence length of 512, microbatch size 2, gradient accumulation 4, and learning rate 2e-5.

Each run completed 500 optimizer steps, with validation every 50 steps and checkpoints every 100 steps.

The aim was to compare training speed, peak PyTorch memory allocation, complete runtime, and validation loss to understand which implementation might offer a practical advantage for a team with limited GPU resources.

I instrumented optimizer-step timings with CUDA synchronization, excluding the first 10 warmup steps. Evaluation and checkpoint saving occurred outside those measured windows.

Unsloth was faster and used less memory

The training-step measurements initially looked like a clear win for Unsloth.

Its median optimizer-step duration was 0.766 seconds, compared with 1.568 seconds for standard QLoRA. Measured non-padding-token throughput was approximately 861 versus 413 tokens per second. The memory results showed a similar advantage.

Unsloth reached 8.71 GiB of peak PyTorch allocated memory, while standard QLoRA reached 12.93 GiB, a reduction of approximately 32.6%.

These are PyTorch allocator measurements rather than total device-wide VRAM consumption. Still, the difference suggests Unsloth could provide more memory headroom for certain workloads. I didn't test larger batch sizes or sequence lengths, so that potential benefit remains unverified.

At this point, I expected Unsloth to finish the complete training job substantially earlier. It didn't.

The unexpected result: almost identical total runtimes

Unsloth completed its 500-step run in 30.39 minutes. Standard QLoRA took 30.74 minutes.

The difference was just 21.26 seconds, or approximately 1.15% of the standard run's total runtime.

I went back through the logs because the measured optimizer-step timings suggested a much larger advantage. Across the steady measurement windows, Unsloth spent approximately 371 seconds in optimizer steps, compared with 783 seconds for standard QLoRA. Yet that difference didn't appear in the complete job durations.

The surprising part was that the measured training-step advantage didn't translate into a comparable reduction in total runtime. Evaluation, checkpoint saving, and other overhead weren't separately measured, so I couldn't determine exactly where the difference disappeared. The final evaluation alone took 136.25 seconds for Unsloth versus 99.70 seconds for standard QLoRA, highlighting why a complete workflow needs more detailed timing.

That doesn't establish why the total runtimes were so close, but it demonstrates why measuring training steps alone is insufficient.

For a team paying for GPU time, the practical implication is important: a faster measured training step doesn't automatically mean a proportionally cheaper training job.

Under the same GPU hourly rate and straightforward time-based billing, these particular runs would have nearly identical compute charges.

What about the final model?

Both runs completed successfully and produced final LoRA adapters.

Unsloth finished with a validation loss of 1.3629, compared with 1.3731 for standard QLoRA.

The results were close, but I wouldn't claim one implementation produced a better model based on these numbers alone.

I completed only one full run per method, and the recorded token counts weren't identical. Unsloth processed 319,476 non-padding tokens in the steady measurement windows, compared with 323,394 for standard QLoRA. Supervised token counts and padding behavior also differed.

Those differences suggest the training workloads weren't perfectly equivalent, even though the main configurations matched.

This makes the comparison useful as an observation of two working training stacks, but not a definitive framework benchmark.

What I'd change next time

I'd start by making both implementations use identical tokenized examples, attention masks, and supervised labels.

I'd also measure training, evaluation, checkpoint saving, and remaining overhead separately. That would help explain the gap between optimizer-step speed and complete runtime.

Finally, I'd repeat the tests across multiple runs and longer workloads to see whether the findings remain consistent.

So which framework would I choose?

For a small AI team working with limited GPU resources, I'd lean toward Unsloth when memory headroom is the main constraint. In this experiment, it used approximately 33% less peak PyTorch-allocated GPU memory, which could provide more flexibility for future workloads.

However, if the goal is simply to reduce the cost of this particular 500-step training job, neither implementation had a meaningful advantage in total runtime. Both finished in approximately 30 minutes, so the GPU rental cost would be nearly identical under the same hourly rate and billing conditions.

For me, the main lesson was that framework selection should be based on the complete workload and the resources that matter most, not just the largest advertised or measured speedup.

The experiment didn't establish a universal winner, but it clarified what a team should measure before choosing one.

For those who regularly fine-tune LLMs, what matters most in your framework selection: GPU memory headroom, training throughput, or the total cost of completing a job?


r/Qwen_AI • • 12h ago

Benchmark 103.7 GiB Qwen3.8-Flash-Next on Strix Halo with 128GB unified memory, baseline numbers

0 Upvotes

Been messing with my strix halo box and finly got the full UD-Q4_K_XL Qwen3.8-Flash-Next loaded and running.
Setup:
Nimo minipc with ryzen AI max+ 395
Radeon 8060S + XDNA2
128 GB unified memory
BIOS VRAM reservation 512 MiB, about 117 GiB GTT
Gufo, GPU only, no MTP
FCLK locked at 2000 MHz
Baseline results:
16K prefill 1888 t/s
16K decode 34.1 t/s
Model is the ~125B MoE trunk with ~6B active plus the extra n-gram embedding and MTP modules. Total om-disk size 103.69 GiB across GGUF shards. It fits and runs without drama under the current memory settings. This is just the clean GPU-only baseline. Already have the GPU+NPU and MTP numbers, including one high decode result that needs propr context. Wanted to post the basid does-a-100-plus-GiB-quant-actually-fit and what-does-the-speed-look-like data first.


r/Qwen_AI • • 13h ago Heartwarming

News Alibaba launches Qwen4-Max with 1M context and $1.50 input pricing

Thumbnail
runtimewire.com
197 Upvotes

r/Qwen_AI • • 13h ago Heartwarming

News Qwen4-Max is here. How competitive is it?

Post image
191 Upvotes

Alibaba has officially released Qwen4-Max.

- New MoE architecture

- Native vision + 1M context window

- Over 60% lower task costs (according to Alibaba)

- API pricing starts at $1.5/M input tokens

It also targets complex tasks like 3D modeling, coding, finance, and healthcare.

Smaller Qwen4 models are coming soon.

Can Qwen4-Max challenge the top-tier AI models?