[ Removed by moderator ]

[u/Character-Apple-8471]

[u/LocalLLaMA-ModTeam]

Rule 3 - Minimal value post (LLM generated)

[u/robgami]

Reports like this should really have the quants used in the title or at least somewhere in the body.

[u/Character-Apple-8471 → u/robgami]

Sorry mate..done

[u/ExperienceDry6608]

is flash-next generally smarter? it's lower active count makes it dicey for longer coding imo

[u/RadiantHueOfBeige → u/ExperienceDry6608]

Much smarter (i.e. more knowledge, better design ability, cross-domain reasoning etc) but also clumsy. 27B is a surgeon with steady hands and sharp focus in comparison. Small quants are also somewhat forgetful... I can sort of run Q5, and it's noticeably more coherent than Q4. Q6 is another step up but on DDR4 it's glacial.

[u/Monad_Maya → u/ExperienceDry6608]

Not the OP, QFN has higher skill potential but needs to be directed well.

Haven't tried for longer coding sessions though.

[u/Character-Apple-8471 → u/ExperienceDry6608]

sure its "smarter" if the use case is "3D temple generation" or "SVG of a cat", in real life coding (92K context deep) i would choose 27B , not in Opencode though, DSH for good compaction

[u/N34257]

I've encountered similar issues, but...I'd still choose Flash Next over 27B every time. Why? Because despite the fact that it means more involvement from me to get to a correct solution, it has much better and more varied ideas, and a wider breadth of knowledge than 27B. I value those things more than perfect accuracy.

[u/beling86]

Great content. Would you mind sharing the weight quantization/ kv cache quantization you used ?

[u/Character-Apple-8471 → u/beling86]

Weights IQ3_S (ISTA-DASLab GSQ-RCO), KV cache int8, 65536 context. Strata 0.1.44.1 on 3060 12GB + 5060 Ti 16GB, layer split at 12. Worth noting most of the tuning win was placement rather than quant, moving to a two-card split took decode from 26.8 to 50.2 tok/s (+87%), and expert cache hit rate from ~55% to 91%. The cards matter a lot more than the numbers suggest: the 3060 is stuck on a PCIe Gen3 x4 slot (2.85 GB/s measured) versus the 5060's Gen4 x8 (14.3 GB/s), and that 5x link gap drove most of the card-ordering decision.

[u/SeveralViolins → u/Character-Apple-8471]

Wait, so this a Q3_S version of flash? At 4bit I see sub 90% KLD. This must be ~80%?

[u/Oli_Luck → u/Character-Apple-8471]

Did you use same model and kv cache quant ?
Especially KV Cache quant make a huge difference on long run

[u/Character-Apple-8471 → u/Oli_Luck]

yes..same KV Cache...the 27B was the Austrian Qwen (Ista :) )

[u/Miserable-Dare5090 → u/Character-Apple-8471]

Both were IQ3 small, a quantization bound to give you precisely the results you got.

Try a quant with 16 bit attention paths next time.

[u/Evildude42]

Maybe I'm blin,d but he didn't say what weights he tested these things at and were these tested on the same machine. Because if they were and flash was just reduced, so it could run on that machine then there's your problem

[u/Character-Apple-8471 → u/Evildude42]

Weights IQ3_S (ISTA-DASLab GSQ-RCO), KV cache int8, 65536 context. Strata 0.1.44.1 on 3060 12GB + 5060 Ti 16GB, layer split at 12. Worth noting most of the tuning win was placement rather than quant — moving to a two-card split took decode from 26.8 to 50.2 tok/s (+87%), and expert cache hit rate from ~55% to 91%. The cards matter a lot more than the numbers suggest: the 3060 is stuck on a PCIe Gen3 x4 slot (2.85 GB/s measured) versus the 5060's Gen4 x8 (14.3 GB/s), and that 5x link gap drove most of the card-ordering decision.

[u/n1nj4p0w3r → u/Character-Apple-8471]

of course they couldn't fullfill your request with such a context, they were busy doing compaction 80% of the time

[u/Character-Apple-8471 → u/Character-Apple-8471]

IQ3_S

[u/chensium → u/Character-Apple-8471]

IQ3_S is too degraded, and the task is too big for such a small context.  Not really worth comparing.  At such low KLD, dense will do a better job of staying on task, which is likely all you're measuring here.

[u/AndroYD84]

I see that you've used Strata to run QFN, have you changed the config after the setup? Because the author used suboptimal settings to increase the performance but at the expense of crippling the thinking process just to flaunt about how quick it is.

I had to make changes to "strata-<quant>.json" because the default thinking preset was too greedy and would loop non-stop, after I set it like this it stopped looping and actually performed really well.

"sampling": {

"temperature": 1.0,

"top_p": 0.95,

"top_k": 20,

"min_p": 0.0,

"presence_penalty": 0.0,

"repetition_penalty": 1.0

},

(For an instruct/non-thinking preset use temperature 0.7, top_p 0.8, top_k 20, min_p 0, presence_penalty 1.5.)

This will slow down the output a bit but at least it is usable for real work.

[u/Character-Apple-8471 → u/AndroYD84]

Damn..my top_p was at 0.55 not o.95, a typo on my end ...what an idiot I am..thanks mate

[u/Enragere → u/AndroYD84]

wait aren't these well-known defaults? now you got me curious

[u/AndroYD84 → u/Enragere]

What I'm saying is that Strata out of the box DIDN'T use these well-known defaults and I had to manually configure them, many people just ran the setup and used it as-is, but I see there was a pull request here that implemented it now: https://github.com/Niko1221/Strata/pull/1274 but you still have to manually opt-in for it appending the --thinking or --instruct flag.

[u/Educational-Region98]

Question is if I could run both... Will they solve different problems?

[u/Perfect-Campaign9551]

Were you relying only on the internal knowledge of the models, or were you providing them with proper context, like documentation, etc?

[u/Character-Apple-8471 → u/Perfect-Campaign9551]

All API docs had to be researched online and tested for rate limiting before committ..<--- by QFN

[u/BarracudaDefiant4702]

Would be interesting to see how DeepSeek flash 0731 or similar compares. I would expect it to be slight better in the end results.

[u/zomerbk]

I’d add an observation type to the normalized data: IP sighting, subnet aggregate, IOC assertion, and so on. Scoring and visualization could then enforce what each type supports. A DShield subnet row, for example, could contribute to regional activity without becoming evidence that its starting IP attacked a destination.

[u/ttkciar]

Hello Character-Apple-8471, please clarify how much of your post was LLM-generated, and why you chose to use an LLM to help you write it. I need the information in order to make a Rule Three determination.

[u/Character-Apple-8471 → u/ttkciar]

90% LLM generated....I run agentic tests to create reports accross models and rubrics

[u/FeydRowan]

Great testing we need more of this 

[u/Even_Whereas_8442]

We constantly have to listen to these cheapskates. Why don't they pay for the API? Local is not for you.

[u/Bulky-Priority6824 → u/[deleted]]

Copy paste bullshit