r/LocalLLaMA • u/AdInternational5848 • 11h ago
Discussion Qwen 3.8 Flash Next, DeepSeekV4 Flash and Time
MacStudio 128GB M1 ultra
I'm curious how people who can run both models are deciding which one to work with.
DeepSeekv4 benchmarks right below GPT 5.5 high.
Flash Next benchmarks right above 5.4/5.5 xhigh
Both seem capable but it seems like Qwen 3.8 Flash next is taking much longer to actually finish anything. From the chart, it seems like I can do 5 tasks with DeepSeekv4 in the time it takes to do 1 with Qwen 3.8 Flash Next and Flash Next isn't 5x better than DeepSeekv4. In my personal experience it may not be 5x longer but it's absolutely taking signifcantly longer to do anything.
Is there something I'm missing?
NOTE: I'm not bashing Qwen 3.8 flash next, I like it, currently 2h and 5 minutes into deep diving on some work for me. On my hardware I can load one of these models with multiple slots so I have to choose. I'll likely have to invest the time into a more deterministic benchmarking on my personal tasks to compare the quality I'm getting back and I'll still be using 3.8 27b because I find it capable and it uses less of my machine so I can do more other things like the image gen I'm working to integrate into my personal harness.
10
u/BeaveItToLeever 8h ago
I've settled on QFN after extensive testing on both.
I'm using a heavily modified DeepSeek Harness with QFN medium typically. I will occasionally give xhigh a run, as my harness has some things in place to help mitigate massive overthinking that 3.8 models are known for(system prompt only mildly helps. The key here has been logit manipulation and toggling, though I'm sure there's a less sloppy approach)
Both DSV4 and QFN can more or less handle the workloads I give it, which are a mix of normal agentic "research this and give me an aggregate with links" sorts of things(which I also have 35b do), game dev, supply logistics, and then actual supply chain software.
They perform roughly equally in results when their thinking is reigned in to be more reasonable. I do have DSV4 to be more prone to occasional confusion but not to the point it's an issue. Though I'll say I find whatever black magic they put into QFNs vision model is also a plus - it's very very good
So them being roughly equal, I choose QFN simply for the fact that it's significantly faster. There are good NVFP4 quants for it, and I have it running roughly 1800 PP 50ish TG on a single spark after all the tinkering I've done with it. Compared to DSv4s numbers being basically half that on PP and less than half that on TG(18-24ish), though I've admittedly not tinkered near as much with the engine for that model.
I think they're both amazing, but I've removed DeepSeek and Dwarfstar from my machine in favor of just running QFN in conjunction with 3.6 35b a3b
As a side note, since I mentioned 35b. I think those of you with a mind to tinker and fork/create your own harnesses should revisit this model. The harness goes a long way here and 35b is an absolute beast when reigned properly
8
u/Dany0 10h ago
Use Sharp template + medium reasoning with advisor model. Matched 5.6 sol in my tests a month back
Idk how/if other harnesses implement advisors, I use omp
9
u/watcholic 9h ago
Sharp chat template and medium reasoning on QFN is consistently several times faster than 27B (with sharp template and medium reasoning) from my experience.
https://huggingface.co/peculiar-ragdoll/Qwen-Sharp-Chat-Templates
1
u/MmmmMorphine 8h ago
This is a bit random, but for omp are there any good things ways to set up a browser (or otherwise) gui?
I mean I can always ssh in and all, mostly, but damn if I can't shake my tab and/or sidebar addiction.
16
u/norenEnmotalen 10h ago
The time per intelligence of Qwen3.8-27B vs Qwen3.8-Flash-Next certainly reflects my experience
5
u/HighSeasArchivist 4h ago
DSV4.1 for me at least just talked in fucking circles. It did it very fast, but was just garbage output. Qwen3.8 Flash Next and 27B have both been very solid for me. I don't care about tok/s if it's just bullshit talking in loops.
3
u/MD_Reptile 10h ago
I think its important to remember some of these models are very benchmaxxed - they might perform better because they've been specifically trained for those benchmarks. I couldn't say for sure if we are seeing some results of that here.
Having said that, while deepseek v4.1 flash seems pretty smart and relatively fast - I dunno about its real world "speed" being quite what they are showing here in comparison to other models, in the sense that I have to do a lot of handholding still to get it steered straight on things. Not as bad as GLM 5.3 flash though which will go real off the rails if I give it vague instructions. Flash next I'm finding at least mostly sticks to what I tell it to do, even if it has to burn tons of thinking tokens to get to a solution.
2
u/AdInternational5848 10h ago
I like the output of Qwen 3.8FN and it's technically faster than DeepSeekv4 on my hardware with probably double the prefill speed but it feels like it takes longer to produce a finished result.
2
u/my_name_isnt_clever 9h ago
Use medium effort with Q3.8FN instead of xhigh, it does WAY less reasoning with barely any capability drop. It's my go-to for everything right now.
2
u/Perfect-Campaign9551 8h ago
This . I switched to Medium reasoning for QFN and it's MUCH more usable and seems to actually give better results quite often.
1
u/DifficultyFit1895 5h ago
Swift-1.5 for me is the best at xhigh with tokens similar to regular QFN medium.
1
u/ormandj 9h ago
Which is interesting, since with my serving images and proper quants (stock for DS v4 Flash and FP8 for GLM 5.3 Flash), I find DS v4.1 Flash tends to deviate/hallucinate things a LOT more than GLM 5.3 Flash. I hear that providers, on the other hand, use bad NVFP4s for GLM 5.3 Flash and the quality of output is awful. DS v4.1 Flash is already mixed 4-bit/8-bit so tends to get run at full model precision.
1
u/MD_Reptile 9h ago
Maybe that's what it is, I can't run glm local, so I used open router at a random provider.... Hard to tell what version I was testing!
1
1
1
u/Iron-Over 10h ago
This is API serving locally on two DGX Sparks QwenFlash next is fastest in my testing. As long as you have a cheap subscription doing planning.
1
u/RosebudNebula 9h ago
Qwen 38 FN getting 40 on that bench seems really high. I'm not expecting it to be close to or even beat Opus 4.7. Opus 4.6 is already deemed usable by me when I actively use it.
I do find Q4 / NVFP4 Qwen 3.8 FN very similar to Opus 4.6, though, if not a little better.
1
u/Long_comment_san 9h ago
I personally think that 45 on this benchmark +/-1 is an important threshold where quality leap happens. My experience with GLM Flash is quite positive but as you go just a little bit higher, quality starts to change a lot. I do roleplay and I kinda notice these leaps in intelligence because I see model in different scenarios. Doesn't feel like placebo.
1
u/Opening-Designer4333 9h ago
I think the inference time was measured on API, and some APIs are very fast like deepseek.
1
u/Mistercheese 9h ago
I think most of this reflects what experience I do have with the models, except for Opus 5.5 (High). I feel like it's definitely both smarter and faster than any version of GPT Sol (5.6, 6.1, etc)
1
u/Past-Town-9807 8h ago
QFN can overthink. Sometimes you have to reduce tasks for it to bite sized pieces and tell it not to overthink. I had a failure earlier today because of that. The model overthank (overthunk? edit: overthought? lol) itself into a timeout.
1
u/Septerium 7h ago
I feel like DSv4 Flash Vision is overall better than QFN. It spends less tokens to finish tasks. And more importantly: I can actually understand what it says. QFN writing style is awful.... really hard to make sense of what it's trying to say
1
u/Firenze30 6h ago
In my setup, Qwen3.8-27b achieves about 3.5x to 4x faster prompt processing and text generation speed than DeepSeek V4 Flash 0731. However, on the exact same self-contained prompt to do a task, DeepSeek V4 Flash 0731 finishes in 3 hours while Qwen 3.8-27b still takes 2.5 hours. Qwen3.8's high raw speed doesn't translate into significant time savings. That minor 30-minute speed advantage isn't really worth the trade-off in quality in my use cases.
1
u/look 6h ago
I’m not sure what the point is of a time per task “benchmark” for open weight models. It depends massively on the provider.
I’ve used Qwen3.8 Flash at several hundred tokens per second on RunInfra (back when they had it).
Or MiMo v2.6 Pro, for example: it is slow on Xiaomi at ~30-40 TPS, but PrimaLabs has it for the exact same price at 400 TPS.
1
u/braintheboss 4h ago
i don't feel qwen flash is benchmarxx. Its an ass pain for gpt sol 6.1 when it reviews his plans. Frontiers not check gaps passed while qwen even catch frontier drop a approved step
1
u/QuackerEnte 3h ago
This doesn't take into consideration that QFN and 27B are being served at an abysmal 40-50 tok/s compared to other models. Token-efficiency wise it's not way more than say deepseek v4 flash. keep in mind the actual speed you can run these at and the token usage.
0
u/New_Guitar_9121 10h ago
On an M1 Ultra 128 GB that wait is the machine. Same model, mlx-serve, 60-core M3 Ultra 256 GB, weights resident, MTP on: median about 96 tok/s from 32k out past 128k, 3,156 replies. A 128 GB M1 will not land there. The experts fit, and the GPU is two generations newer. If a job still runs long after tok/s looks fine, Flash-Next also spends a lot of tokens thinking unless the reasoning budget is capped.
2
u/AdInternational5848 10h ago
Thank you for that insight, I'm looking at a hardware upgrade soon and it's promising to hear that I'd get roughly 5x tok/s.
Question, do you have experience with the deepseek models to compare how quickly they finish comparable tasks?
27
u/poofph 10h ago
I use this system prompt with flash next to keep it in line. It will produce and apply a fix in 5 minutes then spend an hour running tests on it otherwise. I also have a prompt built into my client app that instructs it to be concise.
system prompt:
You are an expert Python software engineer specializing in surgical code maintenance, refactoring, and patch deployment.
When fixing a bug: reproduce once, apply the fix, run the relevant test exactly once to confirm, then stop and report what you changed and the test result. Do not run additional tests, re-run passing tests, or verify further unless the test fails. If everything passes, your response must end with "DONE".