r/LocalLLaMA • u/Thrumpwart vLLM • 8h ago
Discussion Very impressed with LFM2.5-24-A2B in Pi Agent
Just downloaded the new LFm2.5-24B-A2B LiquidAI model for testing. They claimed some pretty good performance, and I've had good experiences with their prior releases.
Some context - I use Ornith 1.5-35B pretty regularly in multiple workflows - sometimes on it's own, and sometimes as part of an AI-Council in Pi Agent. It's been very reliable, provides surprising insight "smarter" models miss, and is a great workhorse in Pi.
I swapped LFM 24B in to replace Ornith in a few of my workflows - it is holding up REALLY well. It's very fast (~95 tps sustained in llama.cpp using ROCM on an AMD W7900 - latest llama.cpp build). The Q8 takes up 30GB vram with full 128k context. It hasn't failed me yet in my mechanical tasks - it just keeps going and going and going and seems to tick all the boxes for a "little" model I run alongside a larger or smarter model on my other GPU.
For "insight" I'll have to test it longer in my council-mode workflow but I will report back.
Just wanted to give my $0.02 (CAD) on this model - it is very competent with Pi and worth your time to check out. The speed alone is a nice upgrade, and for my more basic tasks, even long running recursive tasks, it is holding up very nicely.
Kudos to the LiquidAI team, this one is going to get alot of use on my chonky boi.
8
u/Charming_Barber_3317 7h ago
I heard its good for mechanical tasks but not good for coding? Is that correct?
8
u/Thrumpwart vLLM 7h ago
I've used it for coding in prolog and some limited python - it does those very well. I have not yet thrown it at a very difficult coding challenge but will over the next day or so.
2
u/sleight42 7h ago
What else do you use it for?
2
u/Thrumpwart vLLM 6h ago
The model? or my coding workflows?
1
u/sleight42 6h ago
Yes?
4
u/Thrumpwart vLLM 6h ago
Right now I'm using it on difficult logic puzzles to test a lateral-thinking pi extension I've been developing. Next I'll be using it in council-model to perform comprehensive analysis and evaluations of my other projects along with access to my knowledge base and pi-web-access. This is where Ornith shined - it performed different, distinct lines of analysis and evaluation than some of the larger models in the council, and tended to find discrepancies and offer proposed improvements that the bigger models missed.
The ultimate goal is to see if it can suitably ingest a large project repo, understand the various moving parts, understand the goal, and iteratively work through the project workflow to find places to optimize or try new approaches.
I generally use Deepseek Flash as the orchestrator, with council members being Qwen3.8 Flash Next, Qwen3.8 27B, and Ornith1.5 35B as council members. Sometimes I'll mix it up and replace 27B with Gemma 4 31B, and sometimes I'll try Glm5.3 to review the councils findings.
1
u/Not-reallyanonymous 3h ago
Haven’t used it but based on benchmarks and what Liquid AI is all about:
It can code. It might even be competent at coding. It probably isn’t good at coding, where good is defined as “relative to other models of a similar age and size.” I wouldn’t say Liquid AI is about “mechanical tasks,” but they do lean “utility models.” This might be a good model alongside Qwen or Glimmer as a code implementer, for example, or embedded in some workstation software to write adapters on the fly. Probably not the first choice for “I am a software developer writing software with AI.”
5
u/txgsync 7h ago
Thanks for the review! I'll see how it compares to Gemma4-26B-A4B. If its tool-calling is more reliable it might be a nice daily driver for smaller basic agentic workflows: daily briefing, automated podcasts, summarizing open pull requests, categorization, etc. I was terribly disappointed with Nemotron Lightning for this purpose; gpt-oss-20b was, believe it or not, quite a beast in that role too...
4
u/Thrumpwart vLLM 6h ago
I haven't had a tool call fail yet in Pi. YMMV of course but I would be interested in a head to head analysis between it and Gemma 4 26B as I've never really used that one.
7
u/truthzealot 7h ago
It’s one of the models I tested yesterday against qwen3.8 27b 2bit GSQ RCO and it didn’t do as well. It does seem fast though as an MoE. The LFM2.5 is blazing and seemingly as capable. Unfortunately fast doesn’t beat correct so I’m sticking with qwen. They did a killer job and the GSQ RCO is magical.
5
u/mesolimbic-bliss 7h ago
I had better luck with MiniCPM-V-4.7-35B-A3B, its so freakin fast and also scores as good as ornith and LFm in my work cases
1
u/Thrumpwart vLLM 6h ago
Thanks alot! Another one to test.
Seriously though, thanks for heads up. Llama.cpp support I assume?
2
u/mesolimbic-bliss 6h ago
Yes gguf and I run it on 2x3060 with 24GB VRAM total and its so dangum fast i cant stop using it
1
5
u/Midaychi 7h ago
Hows its corpus and writing style though? Gemma models have always been the king there- just end up stinking agentically
6
u/TheRealMasonMac 6h ago
I wouldn't expect it to compete with Gemma. I asked it on their playground to define MILF, as a true LLM researcher would, and it got the answer half-wrong. It got the acronym right but added, "with an attractive daughter." I don't think Liquid models are that great at world knowledge unless it's encyclopedic to be honest.
1
u/Thrumpwart vLLM 6h ago
Yeah Gemma 4 31B (and the new Argon is AMAZING!) was always very good at legible, clear writing. I haven't tested this yet on any writing tasks (I don't really use any) but I'll give it a try when my other tests complete.
1
u/sabine_world 6h ago
Writing style is okay, it's more.. stiff and direct. Though I tested it with pretty stock settings.
You can probably shape it a little with a system prompt.
I tested it out by poking around my obsidian vault/journal had it analyze entries from last year around the same time compared to now.
Gemma is a little more, natural feeling in those kinds of analysis, this model is more like.. focused on bullet points and such. Like a presentation style. Though you can just ask it to present things how you like.
I think it's kinda cool though I wanna try it for some coding stuff at some point.
2
u/moderately-extremist 5h ago
It's definitely fast. I'm getting about double the pp and tg speeds compared to qwen3.6 35b-13b. Tool calls are working fine (using hermes agent). I'm going to daily drive this for a bit and see how it goes.
4
u/Bulky-Priority6824 3h ago
Anything can seem like an upgrade after daily driving Ornith
3
u/Dabalam 2h ago
I don't find it better than Ornith (only faster) and it doesn't seem that OP makes that claim either. For a lot of tasks It is "smart enough" though and in those cases the speed is pretty great.
It would be somewhat surprising and impressive if this smaller model was straight up better across the board. Within the "science coding" domain I am generally in, Ornith is hard to beat until you get to like IQ3 Qwen 27b (and even that won't always beat Ornith 1.5). Even the tiny quants of Qwen Next Flash tend to outperform both though.
1
u/CiobanuXashi 5h ago
Sustained 95 tps at q8 with full 128k context, and it holds up on the long runs. A lot of model for $0.02 (CAD).
1
u/Rahulb1732 3h ago
I had high hopes from this models and from minicpm V4.7. Tried both threw a few coding implementations tasks and lfm failed tool calls a lot which was extremely disappointing. I’ll try it again with an updated template and cross my fingers cause I really want to replace qwen3.6. All the 35B models have really not beaten qwen3.6 well enough for me to replace it.
1
u/UltramanQuar 2h ago edited 2h ago
It is quick but not that accurate, gave it some tasks related to finding cyber stuff and doing reports on that, didn't perform well. I was using custom harness with deepagents
1
u/MooseEfficient2151 2h ago
liquid's been quietly cooking lately, nice to see it holding up so well as a drop-in replacement
1
u/bugra_sa 1h ago
The broken template is a good reminder that model quality and integration quality get mixed together way too easily. I'd retest tool use with a fixed set of prompts repeated 20 times, then count malformed calls, retries, and wrong arguments separately. One clean demo won't show whether the template fix actually made the agent reliable.
-6
u/THECANADIANBEEF2017 5h ago
Great little model, soon as I get enough karma , I've got a good drop to make.
31
u/fourinthoughts 7h ago
There are problems with the template from the initial release that made tool calls frequently fail, so it got dismissed and deleted.
There's a fix posted on HF by peculiar ragdoll today that fixed the tool call problems I was having with the Unsloth quants (q8 and q4 xl}.
On Strix Halo llama.cpp ROCm, I'm getting 65+ tps at q4 quant with q8 kv and the default sampling is great for ngram bursts. I haven't done any stress testing, but it's definitely a very competent model with only 2B active that can fit a 16GB GPU