Am I right in thinking this is a tiny model which has been trained well to reason, and that's it? Makes me think of a smart person who doesn't know anything about a given topic, but with the right tools will go and research the heck out of it. I really like the sound of this... why have models train on learning anything when you can just train them how to learn and let them get on with it from something as small as a Pi Zero and an internet connection.
This has been my dream ever since. Instead of encoding "all the knowledge" into those parameters, how about just making a model that has the same size, but all (or rather most) it does is reasoning? Just give it the ability to browse the net (e.g. language specifications, documentation and best practices) and just have it do its thing. Why does my coding agent need to know the population of New York, know a cheese cake recipe or the general lifespan of an ostrich? Just give it the bare minimum knowledge to think and reason about, and let it figure out the rest.
Sadly that's not how LLMs work, since all they do is "token prediction". At least the models we have to today ...
I think this is a well known concept, which we can't deliver yet. LLM/transformer give us reasoning engine as a byproduct of its design, but it is quite ineffective. If we can distill reasoning, if reasoning can be achieved without general knowledge, it will be a very effective machine.
Some amount of knowledge is required for reasoning. Maybe such model can dynamically knowledge domains to have taxonomy. For example, model can't effective reason about development task, if it has no knowledge about development best practices. But population of New York or recipies can definitely be loaded run time with tools.
>Some amount of knowledge is required for reasoning.
This is the root of problem. If you think about STEM universities, they don't really teach you things you need in the real world. They teach you what you need to know in order to go out there and accumulate the necessary information which can then be used to solve problems. Giving a person access to the internet or a super powerful calculator (like Mathematica) won't mean that they can do anything useful. They need tons of experience to use these tools in an effective way. That experience is basically all that implicit adjacent knowledge that we pick up along the way getting our degrees. And LLMs pick that up during pre-training. Drop this part and the outcome will be worthless.
Take mathematics as an example. Humanity has found math notation, which allowed to express math rules — distill them to the core. Before math was expressed in prose — a very inefficient way, very similar to current LLMs.
In my school, math teacher was giving me prose, which I was converting to math notation. I could argue, that this prose→reasoning conversion is not required at training, and can be obtained at inference time with search tools.
Yup, you still need knowledge. Even if you have access to all the data and tools, you still need to know what to search for, what tools to use and to understand what the user is asking.
Our computers can already do everything, have access to all the tools and information, yet they still need a human/intelligence to use it and apply to specific problems.
Even defining the problem requires knowledge.
As for the tools, if the model has access to 1000 tools, how would it know which one to use if it doesn't have any knowledge itself?
What if I ask for "table tennis spin" it had a "magnus effect calculator", how would it know to make the connection between the two?
Model can use tools to get that knowledge. In your example, read Wikipedia page about table tennis. Imagine a reasoning engine with a big enough context, that knows nothing. A path built from first principles to understand "table tennis spin" — does not look very long for me.
How would it know about Wikipedia and when to use it? From the tool description? If we had 100k such tools, then that wouldn't even fit in the context.
This is only one example, plus if the topic is more complex, maybe it had to search/learn everything (what is table tennis, what is spin, what is a human, what is a ball), etc. So it would be like spawning a baby human, have it spend an (instant) life learning about the world before providing an answer. Maybe this could work in 10 yesrs, if models get stronger with huge context lengths and almost instant data retrieval. Is it the best way to go about things though? Most animals have most of their core abilities embedded in their DNA and "instincts". A cat doesn't have to learn what a bird is in order to hunt it, it's already "embedded" in its neural pathways, or even deeper, at a full-body level. Those type of systems are a lot more efficient than the learned ones. Maybe the best future AI, will have everything already embedded, instead of just being a strong reasoning machine. All AI responses should be instant and like "reflexes" instead of reasoned steps.
Imagine you only know how to cook (use fry pan skill) and know how to cook omelette (recipe). You get the task to cook doner kebab. How many Wikipedia pages do you need to read to get a good understanding? I guess its max 5.
I think grounding your abstract problem to an example makes it more trivial, than it sounds in general.
> How would it know about Wikipedia and when to use it?
2 general concepts "You have to get good understanding of subject area before you do actions" + "Wikipedia is a good source of knowledge of subject areas" will get a model there.
> spawning a baby human, have it spend an (instant) life learning
Humans spend 99% of their life on boring repeating tasks, not learning anything, just navigating on heuristics.
(what is turkish)->(parse lots of potentially relevant/irrelevant context because I have no way of knowing which if any of this informs the doner kebab before I've looked at it)
>dish made of meat
(what is meat) -> (parse lots of potentially ir/relevant context because I don't know if the specific origin/chemistry/mechanics or whether maillard reactions are important before I learn about them)
>cooked on a vertical rotisserie.
(what is a rotisserie) -> etc etc etc
Seems significantly less efficient than just having the various (how to cook > meat, tools > rotisserie, how to cook > seasoning > tomato; lettuce; cabbage; onion with sumac; fresh or pickled cucumber or chili; various sauces, etc) just already built in to the weights.
Yes, but still "how to cook" is not atomic. It involves knowing how to move stuff, how to measure, what "cooked" looks like in different environment (i.e. different lighting) or variations in ingredients, how to recover from specific failures (i.e. a good cook can fix accidentally adding too much salt, by counter-balancing with an ingredient that absorbs the extra salt). And this is only one skill.
It's a bit how deep image neural nets work, where simply detecting shape primitives is not enough, the net is also the connection and relation between those primitives.
Even saying, the AI should just have the "cooking" or "coding" skill, trivializes the problem.
> Humans spend 99% of their life on boring repeating tasks
But we are also non-stop unconciously learning about the world non-stop, from the analgous stream of inputs and seeing the immediate result/feedback. Even looking at static picture is like over-training a specific dataset.
Just boiling water would be difficult. Do I just add heat until I see bubbles? Or should I have a world model in which I understand that boiling water will be of varying temperatures at varying altitudes and given different liquids.
Because if the recipe just says "boil for 10 minutes" but the thing being cooked really needs a temperature of 212F for 10 minutes, the thing isn't going to be cooked if you're not actually at 212 for 10.
It probably can't use all the truth in its content window, not yet anyway.
E.g. you put a graph in its content window, and you ask it to find a Hamiltonian cycle, can it do it?
Probably this could be a next step in the future for more powerful AIs, a layer that abstracts the facts in its content window away, and a layer that solves this types of abstractions.
This is me vibe-splaining something I don't know a lot about, but I doubt there is such a thing.
If "all the knowledge" is what our models now do, what exactly would be the most extreme "none of the knowledge +search" ?
> language specifications.
It would load in all the knowledge to figure it what "language" means, then it would continue trying to decode what "specifications" means.
That might sound absurd, but to figure out the population of New York It's either: Just going to google it, or derive from primary sources.
But how is it ever going to interpret the primary sources? It needs to understand the question, how complex a question is, and how complete an answer is and how things relate. Thats just _too_ much language.
There might be a way to compact this down into a LLM-native language such that the request of `the population of New York` or `use best practices` is encoded without our messy human language for a reasoning model to work with, but the encoding itself has to be done by the "all the knowledge" llm. Now it seems we just rebuild something related to MoE with extra step afaict.
Basically: you can't teach people to think without giving them some facts and ideas to think with. It's like trying to teach woodworking without giving the students any wood.
No, but you could teach them rapid skill acquisition first, then the basics of woodworking, and they'd be off. They could then apply the same approach to machining. Instead, people absorb the meta-skill through osmosis. The first thing we should teach, starting with reading, is how to learn.
It would also reduce training costs to nothing. Current methodology requires continual retraining to scoop up new facts. If you can do a one time "this is how to think" - that could conceptually work forever, just plug in a new database layer that can be queried as required.
But isn’t that what “training” is anyway? They train LLM today like that and the database becomes the parameters. You can post train on smaller corpus for purpose-built stuff.
Couldn't you build some internal knowledge that would stay and you could teach a model this way. A very fast local memory of some sort. You could also specialize model this way so it is very skilled in your domain. The more you use it, the smarter it gets. I guess the problem is for the model to decide whether the information stored in memory is sufficient or not.
You could, but it's driving in the wrong direction to try to build that knowledge into the model weights because you'll always run into a capacity limit sooner with a small model than with a larger one. The thing the model is specialised for is linguistic understanding and the reasoning process itself, and you max that out at the expense of domain-specific knowledge. If you take "as few weights as possible" as a given, I think the interesting question is how small you can make the model with externalised memory. The openclaw and hermes people are all over this sort of memory problem: using the local filesystem or a local database of some sort is exactly a "very fast local memory" where the more you use it, the more knowledge it gathers. Whether that translates to it being "smarter" is a deeper question than it looks.
Because reasoning is an emergent byproduct of training it on all knowledge. It still doesn't "know" things in this form and just generates tokens, no matter how weird we spin it.
So if you don't train it on a large dataset of a lot of words with a lot of sensible connections, it won't be able to reason, as it won't be able to make proper connections between words and sentences.
You can try training a really small model and seeing the gibberish outputs when you train it on only a small dataset.
Minmaxing the dataset to extract maximum generation with minimal data does sound like fun, but if you want to build SoTA models as a company, the economic tradeoff of doing that vs slapping a few more GPU's together is terrible.
I think small expert models could be pretty powerful from open weight providers.
Imagine, for example, a model that's primarily train on typescript and general programming. It would be faster to train and it could be a lot smaller than a generalist model. It might be the best model to pick when you are doing typescript programming. And if you could squeeze that into 3B parameters a lot of consumer hardware could run it locally.
You could even expand it to just "webdev tech" or the like.
I think you could probably train a model to consider boolean logic, modal logic, and mathematics reasonably well, but there is still a pretty big leap between that and thinking about things.
Even the most basic questions such as put a ball in a cup and place it on a table upside down then pick up the cup and put it in a box.
Requires knowledge of things not mentioned in the question (notably gravity).
Strict definition of all terms quickly gets you into a quagmire of complexity. Some base level of knowledge about things is required for you to give it instructions. If it only knows how to reason, it lacks any idea of what to aim to achieve.
There is quite a pronounced disconnect between the vast stores of written data that models are trained on and robust consideration of a topic. I do wonder if the path can be directed by the order of training.
For example if you train a model to basic literacy using tinystories, then math and philosopy texts, then psychology, and sociology texts, and then finally the mass data of everything from conversations and rants, to code and fiction.
Does that end up with a significantly different model to one that is trained on books on acting, creative writing, and fantasy novels, before introducing the same final mass data set.
How much does it's current ability allow it to contextualise new training data?
>Even the most basic questions such as put a ball in a cup and place it on a table upside down then pick up the cup and put it in a box.
That reminds me - this used to be my go-to question for smaller models and on which they would always fail miserably on:
A small strawberry is placed in a large cup. The cup is placed upside down on the kitchen table. Someone then lifts the cup as-is and puts it in the microwave. Where is the strawberry when the cup is in the microwave?
Here's what the 1.9GB VibeThinker-3B-GGUF:Q4_K_M answered:
Answer: The strawberry is still on the kitchen table – it fell out when the cup was turned upside‑down, and the subsequent lift‑and‑microwave move doesn’t change that.
So it seems there is definite progress here. Both specialized and yet improved common sense on things outside its domain of specialization.
Is that learned common sense or has it learned the structure of that particular problem?
What happens if you ask
A small strawberry is placed in a large cup. The cup is placed upside down on a saucer on the kitchen table. Someone then lifts the cup and saucer as-is and puts them in the microwave. Where is the strawberry when the cup is in the microwave?
> Even the most basic questions such as put a ball in a cup and place it on a table upside down then pick up the cup and put it in a box.
I do not think this is a great example. First, it is not a question. Second, it seems very related to robotics. A model itself cannot put a ball anywhere, it can just call tools and answer in text, image, etc.
An LLM seeing "put a x in a y and place it on a z upside down then pick up the y and put it in a z2." and then a question about what happens could check a rag for properties of those x,y,z,z2 and still answer. Alternatively, this could be useful for coding, for example. And that is a very extreme example. Some basic language plus tool use could go quite far. I think it is a very interesting direction vs here is a gpu the price of a car.
I wasn't explicitly stating the question, It was paraphrasing a common test question for world knowledge.
That you don't need to have a ball, cup, table, or even the ability to perform physical actions in order to consider where the ball ends up is in-itself required knowledge.
The thing is we tried that for decades, using more formal logic to build reasoning engines. And we never got it to be even a fraction as good and generic as learning-based LLMs are today.
Choosing between a model that can only "reason" and a model that has extensive knowledge and "reasoning", the latter will be undeniably better. Advanced reasoning requires cross-domain knowledge, superb pattern recognition, which can only be gained through the same mechanisms which give you a knowledgeable model.
Except for the most basic of tasks, such as "turn on my lights" or "cross-reference these two lists", I wouldn't trust a small model to be as conscientious and reliable as one with deep knowledge.
other way around. it's trained to generate long CoT to reason through problems (and does it well!) but has ~no tool calling capability, and ~no ability to manage more than 1-2 messages.
Yeah but don't you think like that's an oversimplication with the metaphor if we assume this model can do a smart human-level analysis and distillation of knowledge, no? I mean if that were true (i.e. its just like that) then yeah there is no need for massive models but I really would doubt that.
Even recent massive models do not work anything like a smart human does at the moment so why are we assuming this can?
"The right tools" in this case might presumably include, eg, a set of repos + docs and specs on the various technologies being used. Or a library of text/images and background docs on style and techniques use to create them.
That plus this model should give you a very powerful and focussed assistant.
I have been obsessed with the idea of this for a while, theres a Qwen with Opus reasoning distilled that works nicely as well. I think the next frontier is optimizing the models to be more capable on less hardware especially if it can learn on the fly.
Even Karpathy acknowledged that this would require some baseline of human knowledge. The idea wasn't pure logic/reasoning, but some subset to bootstrap from.
Looks like we are seeing small but mighty model breakthroughs, outpacing the pure capital firepower of SOTA providers. I love rooting for the little guy, but is it too soon to call it? To play devils advocate, could it just be the benchmarks are not efficient enough to capture success of real developer workflows?
I think people are going to continue to be surprised by the capability of small models.
Now, if you ask this model to have a conversation with you, it's gonna fail and be incoherent. But boy, does it sure reason through math problems well.
I've just started using qwen3.6:35b a couple days ago running on my framework desktop and rather impressed. It runs really well and reminds me of probably the first Claude model I used. It's the first local model that's actually working for me in a coding agent I've tried. Very exciting!
Interesting. I wonder if there is opportunity to train a set of small model variants to excel at a certain stacks. Eg Qwen3.6-27B for Node + React or Qwen3.6-27B for Rust + TUI
This is always how I've imagined small/consumer-hardware models going in time. If I only ever code in Python, give me a model that does just that (plus some general CS, algorithms, structure, etc.) and does it super-fast and well. Make it small enough that if I need a Python back end and an HTML front end, another specific model can load alongside and collaborate on the front end.
Or give me a pure shopping model that has a general understanding of products and product categories, and then will playwright/scrape/API into shopping sites to compare options and find me what I want. Etc.
Qwen 3.6 27B is an anomalously strong all-around model for its size, but when we run our evaluations, we generate 10 coding submissions/language/model (110 total). So full discosure, the per-language per-model performances can be noisy (I do not think Qwen3.6 27B is better than Fable 5 in agentic workflows when writing Kotlin, given enough samples, although we do find some interesting anomalies that hold up under large sample sizes).
It is, but it's way too slow on a Strix Halo due to its limited bandwidth.
(I'm still sad that they didn't make a 122B-A10B version of it, as it's the kind of model that fits best on a Strix Halo, and for 3.5 it was comparable in performance to the dense 27B version).
Yeah the speed is vastly different but it's getting ~10 tps. And the MoE model is like 50 or something.
Might use it if it proves to be much smarter I dont really monitor my agents while they're running.
Off the top of my head since it seems to be the quick info you're looking for: IIRC, with these two, the 27B is a dense model, meaning it's all active at inference. Meanwhile, the 35B is a Mixture of Experts (MoE), so only part of its network (3B?) is active at any time.
Thanks! Dense models have been slow on my compute, but I'll give it a try. If its not toooooo slow then it's fine I mostly fire and forget agents anyway.
Edit: seems fast! I'll try it out some more, thanks again.
I'm running qwen36.:35b:iq4 IQ4_XS quant. Takes 18 GB of RAM with 131k context window. Seems to be really good. Have it running local stuff via Hermes, using a cloud model via Ollama (Deepseek V4-Pro) for heavy lifting.
I settled on opencode after trying goose and aider as well. I'll probably try some more but opencode worked similar to Claude code which is my main agent.
I serve the model with ollama and am thinking about replacing ollama but haven't looked into it.
I have openwebui for chat if I want that too, but don't really use it.
I’m beginning to suspect the closed SOTA labs were doing all these optimisations, keeping quiet about it, and just charging us out the yinyang for inference.
There is some base level of intelligence any model needs to be useful, even in narrow tasks.
Could you teach a 5 year old to drive a car? A 10 year old? A 12 year old? To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball. By the time a human in in their mid teens they have acquired the base knowledge...
Small models need to have enough base knowledge to be able to be good enough -- even in a seemingly narrow regime. Where is that? Obviously they don't need all the obscure knowledge of a frontier model but there is some base level which is probably more than it would first seem.
I don't see what reading has to do with knowing not to reverse on a highway. It's not like they put up big glowing signs that say "wrong way" like in a video game.
Most of the world follows the Vienna Convention on Road Signs and Signals, where all the important road signs are understandable without reading. This is how no entry signs look around the world [1]
Especially important in places like Europe, where it's common for the driver to be able to read, but unable to speak the language of the country they are currently driving through. I can't speak any Polish, but can travel on Polish roads just fine
This. It’s bizarre to claim that it’s impossible to drive safely in a country where you don’t speak the language. I’ve driven plenty in remote parts of the Middle East despite not reading Arabic, and never once went into oncoming traffic.
I've had some close calls with roundabouts with one-way on/off roads, especially figuring out the bike lanes. None of it required reading but would have been safer for sure.
I agree that drivers should know not to reverse on a highway regardless of local signage.
But in situations that could be ambiguous, I think this is a regional difference - the US, Australia, part of the rest of the Americas use lots of text on road signs (including literal "wrong way" signs); Europe and much of the rest of the world use far less text (including purely pictographic "wrong way" signs). Especially important in Europe where drivers just can't learn 20+ languages.
There literally are "no u turn" signage where you are supposed not to do that. They literally put up signs for it. It is not glowing in the sky, and it doesnt need to be, and doesnt help making a point strawmanning it.
And it's trivial to learn "no u turn" and a few other basic signs as a graphic, even when you can't read them (or can't read in general).
It's also trivial to do a u turn even when you can read, know what the sign says, and you feel like doing one because no car is coming anyway, and millions of people do that everyday too.
> To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball. By the time a human in in their mid teens they have acquired the base knowledge...
It is really strange to see comments like this here, where people seem to reduce some basic human action into how it would work in a text-only computer game. Driving itself requires mainly muscular memory how to operate the car, which why people who drive a lot can just go on autopilot and think something completely different when driving long distances. That is of course a form of kno, but you only get it through repetition. Of course driving in traffic requires far more, basic understanding of traffic law etc, but most of driving is muscle memory, understanding the vehicle and anticipating future occurrences. Why we apes are so good at this is because we have some million years of evolution of just using our bodies and seeing what happens. And of course we all seen the gif of an orangutang driving a golf cart (how real it is I’m uncertain), so there’s that.
I think might help to think models not as some future replicants, but models with certain capabilities in certain domains. It probably doesn’t make much sense to ask Opus 4.8 to drive you around as it doesn’t make sense to except a small image model made for edge devices to be able to write a novel. Perhaps we should just think of them as tools with certain applications they are made for.
> Could you teach a 5 year old to drive a car? A 10 year old? A 12 year old? To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball. By the time a human in in their mid teens they have acquired the base knowledge...
I would be interested to see a formal study of this. I say this not out of anything other than a observation that I think the only real blockers are a) judgement, and b) physical reflexes/strength. As a kid I was certainly aware of ice,snow, and rain, because I road my bike year round and had low confidence in my own ability to control my bike on snowy or wet terrain, especially during season changes. That translated into learning to drive in northern Canada in the winter and applying those lessons to driving.
In an environment devoid of consequences, I have seen kids operate driving simulations (both real simulations, and video games) with a degree of precision that is shocking, including seeing several 9-11 year olds play the simulations and games with a much higher degree of confidence than adult drivers. Children have an awareness that the simulations are consequence free, unless given other motivation. Adults that are consistent drivers have muscle memory and preconceived expectations that govern the decisions they make when playing the game. I am curious about the level of training and exposure required for children to overcome their lack of awareness of the hard limits and consequences of driving and driver error, versus the amount of training and exposure required for expert drivers that are novice gamers to stop applying their learned experience to consequence free simulations.
A 10 year old definitely,and 5year old is close, but not unrealistic, To drive a car you don't need to be able to read... To drive a car on the road with other people is a whole other story :-)
I suspect plenty of five year olds can do a respectable job in Mario Kart, Gran Turismo, etc driving games. Gaming has too low of stakes to judge them on perfectly adhering to the rules of the road, but the ability is there.
Being able to drive a car properly also depends on having the right exploration-exploitation balance. A three-year-old is likely to explore too much in a situation where mistakes can be dangerous.
This requires not only knowledge, but also the control systems that develop with the prefrontal cortex. LLMs don't do much control yet.
I was driving a tractor since 12, including on the road with small farm equipment, and indeed, mostly out of the necessity, but I also received a lot of tuition (from licenced drivers) to know how to behave.
This is more of a question of the definition of "drive a car" than any specific issue about intelligence. Drive a car without errors? Impossible, and now we're into a subjective discussion about what feels intelligent. Pass the DMV test? Probably. How complicated are the conditions? There are plenty of drivers with bad judgement. It's a quicksand sort of discussion.
While I agree with your assessment, probably could've chosen a better example, as in many countries young kids even as young as 8 will learn how to drive.
Several US states will give you a permit to drive a farm vehicle on public roads at 14. Illinois recently passed an amendment to allow farm kids to drive a semi-truck at 16. And there is absolutely no minimum age for driving so long as you are on private land - I have seen 8 year olds driving a pickup truck round a farm...
The mere fact that children as young as 16 can drive on public roads in the US is kind of crazy. On private land no one should give an ass about it, but children in traffic? Crazytalk.
> To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball.
Conflation. That's to drive a car safely. To just drive a car one only need know to press gas to move, press brake to stop, turn steering wheel to change direction and maybe use a gear stick to shift into drive/park (car can be modified to abstract that away). Not much more complex than riding a bicycle; maybe even less since no need to learn to balance.
Millions of people do drive who can't read. It's very common in parts of Asia, Africa, Latin America, etc, especially rural, but even in cities.
There are places where oral exams and audio-assisted testing is allowed. And there are places where people just drive (and drive fine) not bothering with a license.
If it writes functional Python instead of cosplaying as a Java programmer and cramming code with classes and accessors, it's already better than Opus...
Lots of confusion about what this model is actually focused on.
It is a cheap specialist for closed-world, verifiable reasoning tasks like math, self-contained coding problems, and similar.
"Closed-world" means the needed information is already in the context. It is not a tool-using agent that can discover missing context. "Verifiable" means answers are hard to generate but easy to check.
So no open ended research, repo wide agent work, factual Q&A, or SVG generation. More of a compact reasoning module for bounded problems.
I just tried the quantized Q4_K_M from [1] in my RTX 2070 Super, it ran at 110 tok/s with 1800 tok/s prefill, and found the same solution to your prompt. It generated valid LaTeX for the answer but its reasoning trace uses mostly compact ASCII math notation. Took 3min 22s to answer, spending 22k tokens almost all on thinking.
The interesting thing about models this small is they should be able to be put on a single Taalas chip (the HC1 already runs a Llama 3.1 8B model). We're already at the point where half-decent reasoning could be run on an ASIC (and at mind-boggling speeds).
Having some success while testing this model out as a replacement for GPT-5 nano in source code security review. Running on RTX 3090 (24 GB VRAM) via vLLM. It's not great on structured output (as noted in the model card) but I'm working around that in my harness.
Thanks for that, and credit to you for the idea, I quickly vibe coded (heh) a working minimal tool calling harness thats allows it to make multiple tool calls each turn. Currently its working pretty damn well all things considered: https://github.com/NickalasLight/VibeHarness.git
I think this is predicted? Part of the story is how they were able to preserve core reasoning ability while cutting knowledge like "pelicans have wings."
> these findings motivate the Parametric Compression-Coverage Hypothesis, which views verifiable reasoning as compressible into compact reasoning cores, while open-domain knowledge and general-purpose competence require broad parameter coverage over facts, concepts, and long-tail scenarios.
So I think the takeaway here is, this is a super fast companion model to larger models, that reasons quickly. Perhaps this technique can be used to train a highly optimized reasoning "expert" in MoEs.
This model doesn't support tool calling, was not part of its training. It's focused on Python (and I think C++) competitive programming and mathematics tasks, i.e. tasks with verifiable rewards. So if you have a task that fits that description, the size-to-capability ratio is good.
These kinds of models might be more useful as tools to be used by larger orchestrator models, than being the orchestrators themselves.
I'm not seeing any mention of tools in the paper, much less a bias towards "curiosity" to use those tools when it encounters gaps in its knowledge. So perhaps this is a good proof-of-concept that single-pass code generation is viable with this small a model - but we're still a long way from a viable solution.
try it again but give a careful explanation of what a bicycle and a pelican is and how the pelican would sit atop the bicycle. Then give it a reference to the SVG tags you want it to use with documentation.
Imagine you want to make a smaller model that is really good at one thing, say, driving a car. You could remove the parameters that lead it to correctly answer, "What is the powerhouse of the cell?" or, "Who was the first president of the United States?"
It would look really dumb if someone asked it that, but that's fine. You're trying to make a model that is optimized for efficiency for a specific task. As much as possible, you should prune uncorrelated things.
In this case, I’d expect it should make a web search tool call to find the Python library best suited for SVG generation and manipulation, and then use what it learns there to execute the task you’ve asked it to do (either asking if you’d like to incorporate the library as a dependency or to roll its own implementation of a subset of the features if that was your preference),
Assuming tool calling hasn’t been entirely stripped out of this model.
Prompt: If A goes to B who then goes to C, can A send something to C?
Response:
We need to interpret best. The phrase "If A goes to B who then goes to C, can A send something to C?" could be a puzzle about the concept of sending something (like passing a ball) and the relationships.
Scenario: A gives something to B, and B passes it on to C.
Question: Can A also give the same thing to C?
Answer: Only if A can obtain a second copy (e.g., the thing was duplicated). Otherwise, after handing it to B, A no longer holds it and cannot “send” it unless a copy exists.
[Lots of other unnecessary commentary and "scenarios" that make even lesser sense]
Ran the same query and there is a ton of stuff, but it looks like it's reasoning through the ambiguity of the sentence. It still gets the right answer. Moreover, if we consider the FLOPs expended to get to the answer, and compare that to opus, I think it's still a net win.
My hunch is that Opus scale models probably have shortcuts encoded into the model that handle these ambiguities cases, wheres this model has learned a program to reason through the edge case (crystalized vs fluid intelligence). Remembering that probablity (frontier) vs calculating it on the fly (vibethink)
> LLM-based Query Quality Filtering. We utilize capable LLMs to assess query quality, filtering out samples with incomplete descriptions, unreasonable conditions, invalid logic, or an inability to effectively assess target knowledge points.
Really enjoying seeing these really capable SMLs.
Note that on HF they state:
"This model was not trained on tool-calling or agent-based programming data. We therefore do not recommend using it for tasks that involve function calling, API orchestration, or autonomous coding agents." - https://huggingface.co/WeiboAI/VibeThinker-3B
So we can't just hook it up to a coding harness like pi.dev or something.
How would you best utilize a model like this for coding? I take it it's not meant for vibe coding a full app, and the reasoning probably makes it unsuitable for autocomplete. Would you use it to implement specific functions? I looked at one of the coding benchmarks used, Live Code Bench, and it seems to be problem descriptions with sample input and output, and then a solution with a single function or class.
Seems like a really good model to use in an IDE when you still want control over the code structure then.
Not sure if it's suited for that. If you read the article it's stated that it is basically a research project to see how far they can push it with small models.
It's terrible at hunting security bugs (I expected it to be, but I wanted to be sure). I added it to a benchmark I made with a corpus of some Mythos-discovered bugs, and it found zero. The smallest pretty successful models remain Qwen 3.6 and Gemma 4 (but I haven't tested the very small variants of those yet).
The lack of tool use will hinder it a lot I think, since bug hunting requires collecting context across a code base and stitching it together. It might be good in a more narrow sense, i.e "is there a bug in this block of code" and not considering how it interacts with the rest of the code base.
That's also more aligned to its leetcode style training data, the code under test is fully in the context window. It might be interesting to have a bigger tool use model go through the effort of collecting the context, and feeding it into this kind of model for analysis only. It becomes more of a thinking tool, instead of the orchestrator.
I really hope that in a couple of years I can have a laptop that runs a reasonably good coding agent locally, that I can run fast and do most of my programming with, without running my laptop hot. I could keep open code and use other models when needed, but really for most of my work, I'm already breaking it down so that I can review code changes eventually, and I just need something reasonably decent and fast and unlimited. I think its coming.
I'm very optimistic here as well. And it's also worth noting that tooling is improving along with the models. I really think we have to treat models and tools as a package. The models is your engine, but you need a chassis to run it.
I find what makes frontier models actually work well isn't just the capability of the model, but how well the harness is tuned to its expectations. I wrote a about this in a bit more detail here. https://yogthos.net/posts/2026-06-08-dirge-code.html
I hope so too. But I fear that it will feel inadequate if we know there is always a $20 online model that is an order of magnitude better. I don't think there will be a "good enough" local model so long as frontier models look so much better.
Sounds like something that could be pretty useful as a 'validation' subagent. Provide it the details/context related to a larger LLM's run or turn in a harness and have it act as a gatekeeper. At this size and speed it looks like it could be economical to have it run every turn or even every tool call and inform the main agent about the result and success/failure.
I have been thinking about how to use this. Since it doesn’t support tool calling I have been considering a dual model deployment, where a small tool calling llm drives the majority of the user experience, and vibe thinker is tapped for reasoning by the other llm.
So who has suggestions on small models with excellent tool calling capabilities?
Gemma 4 E4B and Qwen 3 4B are pretty good, but fine-tuning makes them really good. There are tradeoffs at this size, so you'll have to find (or make) a finetune that does what you need.
I recently came across this model and I would love to try it with my coding agent soon.
I really like the idea of small models that can reason but do not have too much knowledge. Also, no emphasis on tool calls. I think the agent should do the heavy lifting and reach half way.
I use really small models, like Qwen 3.5 0.8B to 9B - no tool calling, no MCP, no skills, nothing. No multi-turn chat even. Models are given very specific tasks using a vast number of system prompts and all the response handling is done in the agent(s).
That is a good point. I do think these models would be good in the decision making. The large models are trained to use tool calling. Perhaps the small models can generate the text that would express their decision but not generate good JSON to reply with correct syntax. I do not know but this is my hunch.
Two model, one is optimised for system, reasoning etc, second is optimised for specific language ( rust or go ? ) , both small enough to run on local computer, will it work ?
I gave this a run on llama.cpp locally. My GPU is Ge1080, so I needed quantized version for even such a small model and…
This. Is. Amazing. I am flabbergasted.
I am not into the whole GenAI thing and I have very little need for anything agentic, but Python, C++ and Maths is exactly what I mostly used these for, so this might actually become my main work horse. This is so cool.
I even used it for stuff it is not built for, asking complex qustion on history (Battle of Tours 732) and literature (Joyce’s Portrait of Artist) and it was surprisingly good, even though it started to hallucinate names and details (such as claiming Joyce’s father was a priest). For 3B I expected it to mainly spout complete nonsense.
Yeah, I thought so too. It's supposedly good at coding, too. But, if its English responses are insane, why should I trust it to understand my programming-related instructions?
Well, only one way to find out! I mean, maybe it only says insane things and goes in circles if you ask non-coding questions, but it doesn't exactly inspire confidence.
GRPO skips the value network that makes PPO expensive — it scores candidates relative to each other within a group. that's what makes verifiable-reward training practical at 3B scale
So I went ahead and quickly vibecoded a working harness with a barebones tool interface and some constraints on output (credit to noperator for the idea).
github: https://github.com/NickalasLight/VibeHarness.git
Its meant for a Windows machine using ollama but I'm sure anyone who wants to mess with it can point claude code at it to convert it for your own operating system and requirements. After install you can ask it to do something with "vibe 'create me a poem about cheese in cheese.txt'" its workspace is by default the directory the cli was located in when you called it.
Ask an existing LLM harness like Claude Code. "Install this and run the demo program" or "Set this up with llama-server" or "Give me an Open WebUI page for this model" will work.
What are the implications of local SOTA inference, given the insane datacenter "investing" ?
It surely cannot be justified only for training at this scale, and since models nowadays are improved more and more by fine tuning than re-training from scratch.
Will a viable local model crash the US economy ?
More importantly, are the LLM companies aware, and are they deliberately buying out all the RAM and GPUs in order to prolong the inevitable ? Probably not, but I wouldn't be surprised if that is the case.
Does python coding depend on political facts of the world?
It might appear not, but actually, the process of reasoning is not an isolated act. The right and wrong way of doing things is codified in social evolution that absorbed all facets of life. Why should you optimize a piece of code for performance? Why performance is needed? What is a bug? What features and UI themes would be more intuitive for humans?
There is a butterfly effect. Everything affects everything to some extent.
True, but this model provides something of a lower bound on just how much world knowledge is really needed for unrelated reasoning tasks. That lower bound appears to be quite low indeed. Lower than I thought it would turn out to be.
The paper makes a clear claim: "it provides an important
and concrete proof: on well-constrained, verifiable reasoning tasks, first-tier performance is no longer the exclusive
domain of ultra-large models" And that's exciting.
Am I right in thinking this is a tiny model which has been trained well to reason, and that's it? Makes me think of a smart person who doesn't know anything about a given topic, but with the right tools will go and research the heck out of it. I really like the sound of this... why have models train on learning anything when you can just train them how to learn and let them get on with it from something as small as a Pi Zero and an internet connection.
This has been my dream ever since. Instead of encoding "all the knowledge" into those parameters, how about just making a model that has the same size, but all (or rather most) it does is reasoning? Just give it the ability to browse the net (e.g. language specifications, documentation and best practices) and just have it do its thing. Why does my coding agent need to know the population of New York, know a cheese cake recipe or the general lifespan of an ostrich? Just give it the bare minimum knowledge to think and reason about, and let it figure out the rest.
Sadly that's not how LLMs work, since all they do is "token prediction". At least the models we have to today ...
I think this is a well known concept, which we can't deliver yet. LLM/transformer give us reasoning engine as a byproduct of its design, but it is quite ineffective. If we can distill reasoning, if reasoning can be achieved without general knowledge, it will be a very effective machine.
Some amount of knowledge is required for reasoning. Maybe such model can dynamically knowledge domains to have taxonomy. For example, model can't effective reason about development task, if it has no knowledge about development best practices. But population of New York or recipies can definitely be loaded run time with tools.
>Some amount of knowledge is required for reasoning.
This is the root of problem. If you think about STEM universities, they don't really teach you things you need in the real world. They teach you what you need to know in order to go out there and accumulate the necessary information which can then be used to solve problems. Giving a person access to the internet or a super powerful calculator (like Mathematica) won't mean that they can do anything useful. They need tons of experience to use these tools in an effective way. That experience is basically all that implicit adjacent knowledge that we pick up along the way getting our degrees. And LLMs pick that up during pre-training. Drop this part and the outcome will be worthless.
Take mathematics as an example. Humanity has found math notation, which allowed to express math rules — distill them to the core. Before math was expressed in prose — a very inefficient way, very similar to current LLMs.
In my school, math teacher was giving me prose, which I was converting to math notation. I could argue, that this prose→reasoning conversion is not required at training, and can be obtained at inference time with search tools.
Yup, you still need knowledge. Even if you have access to all the data and tools, you still need to know what to search for, what tools to use and to understand what the user is asking.
Our computers can already do everything, have access to all the tools and information, yet they still need a human/intelligence to use it and apply to specific problems.
Even defining the problem requires knowledge.
As for the tools, if the model has access to 1000 tools, how would it know which one to use if it doesn't have any knowledge itself?
What if I ask for "table tennis spin" it had a "magnus effect calculator", how would it know to make the connection between the two?
Model can use tools to get that knowledge. In your example, read Wikipedia page about table tennis. Imagine a reasoning engine with a big enough context, that knows nothing. A path built from first principles to understand "table tennis spin" — does not look very long for me.
How would it know about Wikipedia and when to use it? From the tool description? If we had 100k such tools, then that wouldn't even fit in the context.
This is only one example, plus if the topic is more complex, maybe it had to search/learn everything (what is table tennis, what is spin, what is a human, what is a ball), etc. So it would be like spawning a baby human, have it spend an (instant) life learning about the world before providing an answer. Maybe this could work in 10 yesrs, if models get stronger with huge context lengths and almost instant data retrieval. Is it the best way to go about things though? Most animals have most of their core abilities embedded in their DNA and "instincts". A cat doesn't have to learn what a bird is in order to hunt it, it's already "embedded" in its neural pathways, or even deeper, at a full-body level. Those type of systems are a lot more efficient than the learned ones. Maybe the best future AI, will have everything already embedded, instead of just being a strong reasoning machine. All AI responses should be instant and like "reflexes" instead of reasoned steps.
Imagine you only know how to cook (use fry pan skill) and know how to cook omelette (recipe). You get the task to cook doner kebab. How many Wikipedia pages do you need to read to get a good understanding? I guess its max 5.
I think grounding your abstract problem to an example makes it more trivial, than it sounds in general.
> How would it know about Wikipedia and when to use it?
2 general concepts "You have to get good understanding of subject area before you do actions" + "Wikipedia is a good source of knowledge of subject areas" will get a model there.
> spawning a baby human, have it spend an (instant) life learning
Humans spend 99% of their life on boring repeating tasks, not learning anything, just navigating on heuristics.
>Doner kebab or döner kebab[a] is a Turkish
(what is turkish)->(parse lots of potentially relevant/irrelevant context because I have no way of knowing which if any of this informs the doner kebab before I've looked at it)
>dish made of meat
(what is meat) -> (parse lots of potentially ir/relevant context because I don't know if the specific origin/chemistry/mechanics or whether maillard reactions are important before I learn about them)
>cooked on a vertical rotisserie.
(what is a rotisserie) -> etc etc etc
Seems significantly less efficient than just having the various (how to cook > meat, tools > rotisserie, how to cook > seasoning > tomato; lettuce; cabbage; onion with sumac; fresh or pickled cucumber or chili; various sauces, etc) just already built in to the weights.
I'm just playing devil's advocate here.
Yes, but still "how to cook" is not atomic. It involves knowing how to move stuff, how to measure, what "cooked" looks like in different environment (i.e. different lighting) or variations in ingredients, how to recover from specific failures (i.e. a good cook can fix accidentally adding too much salt, by counter-balancing with an ingredient that absorbs the extra salt). And this is only one skill.
It's a bit how deep image neural nets work, where simply detecting shape primitives is not enough, the net is also the connection and relation between those primitives.
Even saying, the AI should just have the "cooking" or "coding" skill, trivializes the problem.
> Humans spend 99% of their life on boring repeating tasks
But we are also non-stop unconciously learning about the world non-stop, from the analgous stream of inputs and seeing the immediate result/feedback. Even looking at static picture is like over-training a specific dataset.
Just boiling water would be difficult. Do I just add heat until I see bubbles? Or should I have a world model in which I understand that boiling water will be of varying temperatures at varying altitudes and given different liquids.
Because if the recipe just says "boil for 10 minutes" but the thing being cooked really needs a temperature of 212F for 10 minutes, the thing isn't going to be cooked if you're not actually at 212 for 10.
It probably can't use all the truth in its content window, not yet anyway.
E.g. you put a graph in its content window, and you ask it to find a Hamiltonian cycle, can it do it?
Probably this could be a next step in the future for more powerful AIs, a layer that abstracts the facts in its content window away, and a layer that solves this types of abstractions.
This is me vibe-splaining something I don't know a lot about, but I doubt there is such a thing.
If "all the knowledge" is what our models now do, what exactly would be the most extreme "none of the knowledge +search" ?
> language specifications.
It would load in all the knowledge to figure it what "language" means, then it would continue trying to decode what "specifications" means.
That might sound absurd, but to figure out the population of New York It's either: Just going to google it, or derive from primary sources.
But how is it ever going to interpret the primary sources? It needs to understand the question, how complex a question is, and how complete an answer is and how things relate. Thats just _too_ much language.
There might be a way to compact this down into a LLM-native language such that the request of `the population of New York` or `use best practices` is encoded without our messy human language for a reasoning model to work with, but the encoding itself has to be done by the "all the knowledge" llm. Now it seems we just rebuild something related to MoE with extra step afaict.
Education had this sad 15 year period where it thought “competences” are all you need.
Turns out that without the world knowledge to have a base of facts, it is not.
Basically: you can't teach people to think without giving them some facts and ideas to think with. It's like trying to teach woodworking without giving the students any wood.
No, but you could teach them rapid skill acquisition first, then the basics of woodworking, and they'd be off. They could then apply the same approach to machining. Instead, people absorb the meta-skill through osmosis. The first thing we should teach, starting with reading, is how to learn.
“Theoretical tennis”
Competences were always supposed to be supported by demonstrable knowledge and skills and behavior.
So I don't think it's true that relevant knowledge was deprioritized. At least it wasn't supposed to be.
It would also reduce training costs to nothing. Current methodology requires continual retraining to scoop up new facts. If you can do a one time "this is how to think" - that could conceptually work forever, just plug in a new database layer that can be queried as required.
But isn’t that what “training” is anyway? They train LLM today like that and the database becomes the parameters. You can post train on smaller corpus for purpose-built stuff.
Any sufficiently general superintelligence will deduce the existence of rice pudding and income tax from Cartesian first principles.
Reasoning alone can’t always predict all the bits of knowledge you’d need to sufficiently solve a problem, that you would research when planning.
I mean, this really doesn't sound useful even if LLMs worked that way.
First, if you know nothing you don't even know what you're missing or what to search for.
Then, without unlimited context, you have to do research for every task all over again every time.
> First, if you know nothing you don't even know what you're missing or what to search for.
RAG on the initial prompt would be the first thing to try.
> Then, without unlimited context, you have to do research for every task all over again every time.
Thing is, we're really really good at building very fast search engines. Doing research all over again every time shouldn't be a problem.
Couldn't you build some internal knowledge that would stay and you could teach a model this way. A very fast local memory of some sort. You could also specialize model this way so it is very skilled in your domain. The more you use it, the smarter it gets. I guess the problem is for the model to decide whether the information stored in memory is sufficient or not.
You could, but it's driving in the wrong direction to try to build that knowledge into the model weights because you'll always run into a capacity limit sooner with a small model than with a larger one. The thing the model is specialised for is linguistic understanding and the reasoning process itself, and you max that out at the expense of domain-specific knowledge. If you take "as few weights as possible" as a given, I think the interesting question is how small you can make the model with externalised memory. The openclaw and hermes people are all over this sort of memory problem: using the local filesystem or a local database of some sort is exactly a "very fast local memory" where the more you use it, the more knowledge it gathers. Whether that translates to it being "smarter" is a deeper question than it looks.
The model they built knows a fair bit apparently. You can't get 94.3 on AIME26 knowing nothing.
Because reasoning is an emergent byproduct of training it on all knowledge. It still doesn't "know" things in this form and just generates tokens, no matter how weird we spin it.
So if you don't train it on a large dataset of a lot of words with a lot of sensible connections, it won't be able to reason, as it won't be able to make proper connections between words and sentences.
You can try training a really small model and seeing the gibberish outputs when you train it on only a small dataset.
Minmaxing the dataset to extract maximum generation with minimal data does sound like fun, but if you want to build SoTA models as a company, the economic tradeoff of doing that vs slapping a few more GPU's together is terrible.
I think small expert models could be pretty powerful from open weight providers.
Imagine, for example, a model that's primarily train on typescript and general programming. It would be faster to train and it could be a lot smaller than a generalist model. It might be the best model to pick when you are doing typescript programming. And if you could squeeze that into 3B parameters a lot of consumer hardware could run it locally.
You could even expand it to just "webdev tech" or the like.
I think you could probably train a model to consider boolean logic, modal logic, and mathematics reasonably well, but there is still a pretty big leap between that and thinking about things.
Even the most basic questions such as put a ball in a cup and place it on a table upside down then pick up the cup and put it in a box.
Requires knowledge of things not mentioned in the question (notably gravity).
Strict definition of all terms quickly gets you into a quagmire of complexity. Some base level of knowledge about things is required for you to give it instructions. If it only knows how to reason, it lacks any idea of what to aim to achieve.
There is quite a pronounced disconnect between the vast stores of written data that models are trained on and robust consideration of a topic. I do wonder if the path can be directed by the order of training.
For example if you train a model to basic literacy using tinystories, then math and philosopy texts, then psychology, and sociology texts, and then finally the mass data of everything from conversations and rants, to code and fiction.
Does that end up with a significantly different model to one that is trained on books on acting, creative writing, and fantasy novels, before introducing the same final mass data set.
How much does it's current ability allow it to contextualise new training data?
>Even the most basic questions such as put a ball in a cup and place it on a table upside down then pick up the cup and put it in a box.
That reminds me - this used to be my go-to question for smaller models and on which they would always fail miserably on:
A small strawberry is placed in a large cup. The cup is placed upside down on the kitchen table. Someone then lifts the cup as-is and puts it in the microwave. Where is the strawberry when the cup is in the microwave?
Here's what the 1.9GB VibeThinker-3B-GGUF:Q4_K_M answered:
Answer: The strawberry is still on the kitchen table – it fell out when the cup was turned upside‑down, and the subsequent lift‑and‑microwave move doesn’t change that.
So it seems there is definite progress here. Both specialized and yet improved common sense on things outside its domain of specialization.
Is that learned common sense or has it learned the structure of that particular problem?
What happens if you ask
A small strawberry is placed in a large cup. The cup is placed upside down on a saucer on the kitchen table. Someone then lifts the cup and saucer as-is and puts them in the microwave. Where is the strawberry when the cup is in the microwave?
The hard part was always the number of 'r's
> Even the most basic questions such as put a ball in a cup and place it on a table upside down then pick up the cup and put it in a box.
I do not think this is a great example. First, it is not a question. Second, it seems very related to robotics. A model itself cannot put a ball anywhere, it can just call tools and answer in text, image, etc.
An LLM seeing "put a x in a y and place it on a z upside down then pick up the y and put it in a z2." and then a question about what happens could check a rag for properties of those x,y,z,z2 and still answer. Alternatively, this could be useful for coding, for example. And that is a very extreme example. Some basic language plus tool use could go quite far. I think it is a very interesting direction vs here is a gpu the price of a car.
I wasn't explicitly stating the question, It was paraphrasing a common test question for world knowledge.
That you don't need to have a ball, cup, table, or even the ability to perform physical actions in order to consider where the ball ends up is in-itself required knowledge.
The thing is we tried that for decades, using more formal logic to build reasoning engines. And we never got it to be even a fraction as good and generic as learning-based LLMs are today.
I dont think think my point is getting across. This is in the context of how much world knowledge a model needs to be trained on, not llm vs not llm.
Choosing between a model that can only "reason" and a model that has extensive knowledge and "reasoning", the latter will be undeniably better. Advanced reasoning requires cross-domain knowledge, superb pattern recognition, which can only be gained through the same mechanisms which give you a knowledgeable model.
Except for the most basic of tasks, such as "turn on my lights" or "cross-reference these two lists", I wouldn't trust a small model to be as conscientious and reliable as one with deep knowledge.
other way around. it's trained to generate long CoT to reason through problems (and does it well!) but has ~no tool calling capability, and ~no ability to manage more than 1-2 messages.
see the warning at the top of https://huggingface.co/WeiboAI/VibeThinker-3B
Yeah but don't you think like that's an oversimplication with the metaphor if we assume this model can do a smart human-level analysis and distillation of knowledge, no? I mean if that were true (i.e. its just like that) then yeah there is no need for massive models but I really would doubt that.
Even recent massive models do not work anything like a smart human does at the moment so why are we assuming this can?
"The right tools" in this case might presumably include, eg, a set of repos + docs and specs on the various technologies being used. Or a library of text/images and background docs on style and techniques use to create them.
That plus this model should give you a very powerful and focussed assistant.
I have been obsessed with the idea of this for a while, theres a Qwen with Opus reasoning distilled that works nicely as well. I think the next frontier is optimizing the models to be more capable on less hardware especially if it can learn on the fly.
> Am I right in thinking this is a tiny model which has been trained well to reason, and that's it?
i remember karpathy mentioning in dwarkesh podcast. But is reasoning really possible without all the knowledge.
Even Karpathy acknowledged that this would require some baseline of human knowledge. The idea wasn't pure logic/reasoning, but some subset to bootstrap from.
Then smaller the models are, the longer they have to reason when dealing with complex problems. The trade-off is real.
Sure it is small, 3B. But on Pi Zero, I can tell you from my experience, you'll be disappointed.
Looks like we are seeing small but mighty model breakthroughs, outpacing the pure capital firepower of SOTA providers. I love rooting for the little guy, but is it too soon to call it? To play devils advocate, could it just be the benchmarks are not efficient enough to capture success of real developer workflows?
I think people are going to continue to be surprised by the capability of small models.
Now, if you ask this model to have a conversation with you, it's gonna fail and be incoherent. But boy, does it sure reason through math problems well.
I've just started using qwen3.6:35b a couple days ago running on my framework desktop and rather impressed. It runs really well and reminds me of probably the first Claude model I used. It's the first local model that's actually working for me in a coding agent I've tried. Very exciting!
Try 27b, it's significantly smarter than 35b-a3b (although it is slower, it's not so bad with MTP).
At least according to gertlabs, Qwen3.6 27B outperforms every SoTA (closed) model at Kotlin: https://archive.vn/RYBCL / https://gertlabs.com/rankings?mode=agentic_coding&language=k...
Interesting. I wonder if there is opportunity to train a set of small model variants to excel at a certain stacks. Eg Qwen3.6-27B for Node + React or Qwen3.6-27B for Rust + TUI
This is always how I've imagined small/consumer-hardware models going in time. If I only ever code in Python, give me a model that does just that (plus some general CS, algorithms, structure, etc.) and does it super-fast and well. Make it small enough that if I need a Python back end and an HTML front end, another specific model can load alongside and collaborate on the front end.
Or give me a pure shopping model that has a general understanding of products and product categories, and then will playwright/scrape/API into shopping sites to compare options and find me what I want. Etc.
Qwen 3.6 27B is an anomalously strong all-around model for its size, but when we run our evaluations, we generate 10 coding submissions/language/model (110 total). So full discosure, the per-language per-model performances can be noisy (I do not think Qwen3.6 27B is better than Fable 5 in agentic workflows when writing Kotlin, given enough samples, although we do find some interesting anomalies that hold up under large sample sizes).
It is, but it's way too slow on a Strix Halo due to its limited bandwidth.
(I'm still sad that they didn't make a 122B-A10B version of it, as it's the kind of model that fits best on a Strix Halo, and for 3.5 it was comparable in performance to the dense 27B version).
Yeah the speed is vastly different but it's getting ~10 tps. And the MoE model is like 50 or something. Might use it if it proves to be much smarter I dont really monitor my agents while they're running.
3.5's 122-A10B is still great!
Hmm, I just assumed bigger was better. How's it different?
Off the top of my head since it seems to be the quick info you're looking for: IIRC, with these two, the 27B is a dense model, meaning it's all active at inference. Meanwhile, the 35B is a Mixture of Experts (MoE), so only part of its network (3B?) is active at any time.
Thanks! Dense models have been slow on my compute, but I'll give it a try. If its not toooooo slow then it's fine I mostly fire and forget agents anyway.
Edit: seems fast! I'll try it out some more, thanks again.
35b-a3b is only 3b active parameters, it's a MoE.
I'm running qwen36.:35b:iq4 IQ4_XS quant. Takes 18 GB of RAM with 131k context window. Seems to be really good. Have it running local stuff via Hermes, using a cloud model via Ollama (Deepseek V4-Pro) for heavy lifting.
If your framework desktop is the 128G Strix Halo, I recommend giving Qwen 3.5 122B-A10B a shot.
This Q5_K_M quant should be near lossless and fit with full 256K context in about 100GB of RAM: https://huggingface.co/AesSedai/Qwen3.5-122B-A10B-GGUF
3.6 scores better on coding across the board.
Edit: specifically Qwen 3.6 27B beats that on coding and agentic workflows.
Vibe thinker also beats Opus 4.5
I'll keep this in mind.
Could you please share which coding agent you are using with it?
Crush: https://github.com/charmbracelet/crush/
The Q8_K_XL MTP model from Unsloth: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF
I settled on opencode after trying goose and aider as well. I'll probably try some more but opencode worked similar to Claude code which is my main agent.
I serve the model with ollama and am thinking about replacing ollama but haven't looked into it.
I have openwebui for chat if I want that too, but don't really use it.
I am using Mistral Vibe.
npx @oh-my-pi/pi-coding-agent
Pi
It feels sometimes like optimizations are only starting.
I’m beginning to suspect the closed SOTA labs were doing all these optimisations, keeping quiet about it, and just charging us out the yinyang for inference.
Also as much landgrab as possible for data centres, infrastructure, that can kepe it running for the next 5-15 years.
Why does an M1 Max continue to remain capable, if not more capable with every passing year with LM Studio? :)
There is some base level of intelligence any model needs to be useful, even in narrow tasks.
Could you teach a 5 year old to drive a car? A 10 year old? A 12 year old? To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball. By the time a human in in their mid teens they have acquired the base knowledge...
Small models need to have enough base knowledge to be able to be good enough -- even in a seemingly narrow regime. Where is that? Obviously they don't need all the obscure knowledge of a frontier model but there is some base level which is probably more than it would first seem.
> To drive a car requires being able to read
Emphatically, it does not. Passing your drivers test may require being able to read, but plenty of illiterate people around the world drive just fine.
There is a reason we made all the common road signs recognisable purely by shape/colour, after all.
I don't think many drivers pay too much attention to signs apart from traffic lights.
Until they reverse on a highway and kill a family. Being able to drive isn't where parent poster put the bar
I don't see what reading has to do with knowing not to reverse on a highway. It's not like they put up big glowing signs that say "wrong way" like in a video game.
In Australia, you will see signs on freeway offramps pointing to any cars attempting to drive on to the freeway 'WRONG WAY GO BACK' [0]
Though it is true you don't need to be able to read to operate a vehicle, you /do/ need to be able to read to operate a vehicle safely.
And for those who can read: could you teach someone how to drive using an LLM? Sure. Safely? Probably not.
[0] https://www.transport.nsw.gov.au/operations/roads-and-waterw...
Most of the world follows the Vienna Convention on Road Signs and Signals, where all the important road signs are understandable without reading. This is how no entry signs look around the world [1]
Especially important in places like Europe, where it's common for the driver to be able to read, but unable to speak the language of the country they are currently driving through. I can't speak any Polish, but can travel on Polish roads just fine
1: https://en.wikipedia.org/wiki/Prohibitory_traffic_sign#No_en...
This. It’s bizarre to claim that it’s impossible to drive safely in a country where you don’t speak the language. I’ve driven plenty in remote parts of the Middle East despite not reading Arabic, and never once went into oncoming traffic.
I've had some close calls with roundabouts with one-way on/off roads, especially figuring out the bike lanes. None of it required reading but would have been safer for sure.
Then there is Hanoi.
>Though it is true you don't need to be able to read to operate a vehicle, you /do/ need to be able to read to operate a vehicle safely.
Not really. You just need to be able to decipher the sign, which is trivial, even if you can't read it or spell it.
I agree that drivers should know not to reverse on a highway regardless of local signage.
But in situations that could be ambiguous, I think this is a regional difference - the US, Australia, part of the rest of the Americas use lots of text on road signs (including literal "wrong way" signs); Europe and much of the rest of the world use far less text (including purely pictographic "wrong way" signs). Especially important in Europe where drivers just can't learn 20+ languages.
There literally are "no u turn" signage where you are supposed not to do that. They literally put up signs for it. It is not glowing in the sky, and it doesnt need to be, and doesnt help making a point strawmanning it.
> There literally are "no u turn" signage where you are supposed not to do that
These signs, you mean? https://en.wikipedia.org/wiki/Prohibitory_traffic_sign#No_U-...
https://en.wikipedia.org/wiki/Prohibitory_traffic_sign#/medi... yeah?
And it's trivial to learn "no u turn" and a few other basic signs as a graphic, even when you can't read them (or can't read in general).
It's also trivial to do a u turn even when you can read, know what the sign says, and you feel like doing one because no car is coming anyway, and millions of people do that everyday too.
Not reversing on a highway doesn't require reading, just driving sense.
And whole lot of people have done stupid shit like that while perfectly able to read, many even with masters and PhDs.
> To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball. By the time a human in in their mid teens they have acquired the base knowledge...
It is really strange to see comments like this here, where people seem to reduce some basic human action into how it would work in a text-only computer game. Driving itself requires mainly muscular memory how to operate the car, which why people who drive a lot can just go on autopilot and think something completely different when driving long distances. That is of course a form of kno, but you only get it through repetition. Of course driving in traffic requires far more, basic understanding of traffic law etc, but most of driving is muscle memory, understanding the vehicle and anticipating future occurrences. Why we apes are so good at this is because we have some million years of evolution of just using our bodies and seeing what happens. And of course we all seen the gif of an orangutang driving a golf cart (how real it is I’m uncertain), so there’s that.
I think might help to think models not as some future replicants, but models with certain capabilities in certain domains. It probably doesn’t make much sense to ask Opus 4.8 to drive you around as it doesn’t make sense to except a small image model made for edge devices to be able to write a novel. Perhaps we should just think of them as tools with certain applications they are made for.
> Could you teach a 5 year old to drive a car? A 10 year old? A 12 year old? To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball. By the time a human in in their mid teens they have acquired the base knowledge...
I would be interested to see a formal study of this. I say this not out of anything other than a observation that I think the only real blockers are a) judgement, and b) physical reflexes/strength. As a kid I was certainly aware of ice,snow, and rain, because I road my bike year round and had low confidence in my own ability to control my bike on snowy or wet terrain, especially during season changes. That translated into learning to drive in northern Canada in the winter and applying those lessons to driving.
In an environment devoid of consequences, I have seen kids operate driving simulations (both real simulations, and video games) with a degree of precision that is shocking, including seeing several 9-11 year olds play the simulations and games with a much higher degree of confidence than adult drivers. Children have an awareness that the simulations are consequence free, unless given other motivation. Adults that are consistent drivers have muscle memory and preconceived expectations that govern the decisions they make when playing the game. I am curious about the level of training and exposure required for children to overcome their lack of awareness of the hard limits and consequences of driving and driver error, versus the amount of training and exposure required for expert drivers that are novice gamers to stop applying their learned experience to consequence free simulations.
if you don't sit in the car you lack a lot of information. driving without them is almost a different skill.
(i'm above average in both)
A 10 year old definitely,and 5year old is close, but not unrealistic, To drive a car you don't need to be able to read... To drive a car on the road with other people is a whole other story :-)
I suspect plenty of five year olds can do a respectable job in Mario Kart, Gran Turismo, etc driving games. Gaming has too low of stakes to judge them on perfectly adhering to the rules of the road, but the ability is there.
Being able to drive a car properly also depends on having the right exploration-exploitation balance. A three-year-old is likely to explore too much in a situation where mistakes can be dangerous.
This requires not only knowledge, but also the control systems that develop with the prefrontal cortex. LLMs don't do much control yet.
Ask people who grew up on a farm in a rural area. Sometimes you have to even if you can't and you do.
I was driving a tractor since 12, including on the road with small farm equipment, and indeed, mostly out of the necessity, but I also received a lot of tuition (from licenced drivers) to know how to behave.
Different times though.
Not that different. Still happens all the time all over the world (the west included).
True story, they can already drive a tractor at 10 and I know people who learned to drive a proper truck at 13 too
You can teach a dog to drive a car.
https://www.youtube.com/watch?v=BWAK0J8Uhzk
And AI tried telling me that Uber for Dogs (dogs are the drivers) was a terrible idea…
This is more of a question of the definition of "drive a car" than any specific issue about intelligence. Drive a car without errors? Impossible, and now we're into a subjective discussion about what feels intelligent. Pass the DMV test? Probably. How complicated are the conditions? There are plenty of drivers with bad judgement. It's a quicksand sort of discussion.
While I agree with your assessment, probably could've chosen a better example, as in many countries young kids even as young as 8 will learn how to drive.
In some countries they even let kids as young as 16 drive, no wonder they have so many accidents.
Several US states will give you a permit to drive a farm vehicle on public roads at 14. Illinois recently passed an amendment to allow farm kids to drive a semi-truck at 16. And there is absolutely no minimum age for driving so long as you are on private land - I have seen 8 year olds driving a pickup truck round a farm...
The mere fact that children as young as 16 can drive on public roads in the US is kind of crazy. On private land no one should give an ass about it, but children in traffic? Crazytalk.
As far as just the general driving skill: https://www.youtube.com/watch?v=RZ_0ImDYrPY
> A 10 year old
https://www.youtube.com/watch?v=sLIAoW4QxIs
> To drive a car requires being able to read, to have judgement about ice or rainy conditions, to anticipate a child running after a ball.
Conflation. That's to drive a car safely. To just drive a car one only need know to press gas to move, press brake to stop, turn steering wheel to change direction and maybe use a gear stick to shift into drive/park (car can be modified to abstract that away). Not much more complex than riding a bicycle; maybe even less since no need to learn to balance.
>To drive a car requires being able to read
Millions of people do drive who can't read. It's very common in parts of Asia, Africa, Latin America, etc, especially rural, but even in cities.
There are places where oral exams and audio-assisted testing is allowed. And there are places where people just drive (and drive fine) not bothering with a license.
Note that these are Python-only results, the model will not do as well with other languages.
I'm glad to see more domain-focused SLMs, we need more of them! A programming focused MoE should work well across many languages.
If it writes functional Python instead of cosplaying as a Java programmer and cramming code with classes and accessors, it's already better than Opus...
Lots of confusion about what this model is actually focused on.
It is a cheap specialist for closed-world, verifiable reasoning tasks like math, self-contained coding problems, and similar.
"Closed-world" means the needed information is already in the context. It is not a tool-using agent that can discover missing context. "Verifiable" means answers are hard to generate but easy to check.
So no open ended research, repo wide agent work, factual Q&A, or SVG generation. More of a compact reasoning module for bounded problems.
To follow up on this, I had it solve a nasty ODE problem that I saw in the recent Mathematica 15 release post:
And surprisingly it found a valid solution! Extra impressive because it runs 25 tok/s on my measly RTX 2070 super. Apparently Mathematica 14.3 couldn't solve this ODE.Qwen 3.5 9B Q4_K_M solved this using 10K tokens in 5 mins on a RX 7600.
The answer is exactly what you have posted. I am impressed by Qwen!
How do you know it’s a valid solution? Are you able to verify it yourself?
This is a math problem with a math solution. You verify it with math
Interesting!
I just tried the quantized Q4_K_M from [1] in my RTX 2070 Super, it ran at 110 tok/s with 1800 tok/s prefill, and found the same solution to your prompt. It generated valid LaTeX for the answer but its reasoning trace uses mostly compact ASCII math notation. Took 3min 22s to answer, spending 22k tokens almost all on thinking.
[1] https://huggingface.co/prithivMLmods/VibeThinker-3B-GGUF
How do we know the solution isn't in the weights though?
If it can code well then once you put it in a loop with an interpreter it can do anything.
The interesting thing about models this small is they should be able to be put on a single Taalas chip (the HC1 already runs a Llama 3.1 8B model). We're already at the point where half-decent reasoning could be run on an ASIC (and at mind-boggling speeds).
Yeah, if they can fit an 8B model that's really good at improving the output by thinking, running at 16K tok/s on Taalas would be mind-blowing.
Given this and the quality of open models, it makes no sense to me that there’s a future for Anthropic et all?
Packaging a capability into a consumable form will still be business.
It's like web hosting; all the open source tools are there and free, and yet website tools, hosts, etc flourish.
It’s true, but hosting prices are still within spitting distance of rolling it yourself.
SOTA providers are expecting some level of margin. Companies everywhere have a tight eye on their AI bills right now.
The motivation is there if the models get good enough, even if it’s more painful.
Exactly, it's like web hosting.
And I don't see a lot of web hosts with twelve-digit valuations.
it's not like anthropic will stop improving their models
Having some success while testing this model out as a replacement for GPT-5 nano in source code security review. Running on RTX 3090 (24 GB VRAM) via vLLM. It's not great on structured output (as noted in the model card) but I'm working around that in my harness.
Can't you just force it to do structured output via constrained generation?
Yes, I did end up figuring out a clean way to allow normal reasoning inside <think> and then force JSON _after_ the closing </think>. Example here: https://gist.github.com/noperator/6c711ab19027ea8056442df839...
Thanks for that, and credit to you for the idea, I quickly vibe coded (heh) a working minimal tool calling harness thats allows it to make multiple tool calls each turn. Currently its working pretty damn well all things considered: https://github.com/NickalasLight/VibeHarness.git
> but I'm working around that in my harness.
How?
Maybe limiting logits to what is syntactically correct? E.g. {"hello" has to be followed by whitespace or colon. Any other logits get dropped.
I tried generating the classic pelican svg, but it failed horribly just showing me a rectangle and a black circle...
I think this is predicted? Part of the story is how they were able to preserve core reasoning ability while cutting knowledge like "pelicans have wings."
> these findings motivate the Parametric Compression-Coverage Hypothesis, which views verifiable reasoning as compressible into compact reasoning cores, while open-domain knowledge and general-purpose competence require broad parameter coverage over facts, concepts, and long-tail scenarios.
So I think the takeaway here is, this is a super fast companion model to larger models, that reasons quickly. Perhaps this technique can be used to train a highly optimized reasoning "expert" in MoEs.
The only real essential item here is tool calling capability is it not? So I assume they tested a strong read/write/edit tool consistency?
This model doesn't support tool calling, was not part of its training. It's focused on Python (and I think C++) competitive programming and mathematics tasks, i.e. tasks with verifiable rewards. So if you have a task that fits that description, the size-to-capability ratio is good.
These kinds of models might be more useful as tools to be used by larger orchestrator models, than being the orchestrators themselves.
I'm not seeing any mention of tools in the paper, much less a bias towards "curiosity" to use those tools when it encounters gaps in its knowledge. So perhaps this is a good proof-of-concept that single-pass code generation is viable with this small a model - but we're still a long way from a viable solution.
try it again but give a careful explanation of what a bicycle and a pelican is and how the pelican would sit atop the bicycle. Then give it a reference to the SVG tags you want it to use with documentation.
Here's what I got
https://9ol.es/tmp/pelican.png
with https://9ol.es/tmp/prompt_pelican.txt
using prithivMLmods/VibeThinker-3B-GGUF:Q4_K_M
That’s all I needed to hear
As in, you learnt that a useless test that no one should be using was tested here, that's what you meant right?
right?
Its for reasoning not generating art?
Can you explain this a bit more
Imagine you want to make a smaller model that is really good at one thing, say, driving a car. You could remove the parameters that lead it to correctly answer, "What is the powerhouse of the cell?" or, "Who was the first president of the United States?"
It would look really dumb if someone asked it that, but that's fine. You're trying to make a model that is optimized for efficiency for a specific task. As much as possible, you should prune uncorrelated things.
SVG generation is a useless test, what's there more to know?
What if you're reasoning about how to generate SVG correctly?
In this case, I’d expect it should make a web search tool call to find the Python library best suited for SVG generation and manipulation, and then use what it learns there to execute the task you’ve asked it to do (either asking if you’d like to incorporate the library as a dependency or to roll its own implementation of a subset of the features if that was your preference),
Assuming tool calling hasn’t been entirely stripped out of this model.
(Edit) No tool calling, per this comment: https://news.ycombinator.com/item?id=48640189
Beats Opus 4.5 on reasoning you say?
Prompt: If A goes to B who then goes to C, can A send something to C?
Response:
We need to interpret best. The phrase "If A goes to B who then goes to C, can A send something to C?" could be a puzzle about the concept of sending something (like passing a ball) and the relationships.
Scenario: A gives something to B, and B passes it on to C. Question: Can A also give the same thing to C? Answer: Only if A can obtain a second copy (e.g., the thing was duplicated). Otherwise, after handing it to B, A no longer holds it and cannot “send” it unless a copy exists.
[Lots of other unnecessary commentary and "scenarios" that make even lesser sense]
Ran the same query and there is a ton of stuff, but it looks like it's reasoning through the ambiguity of the sentence. It still gets the right answer. Moreover, if we consider the FLOPs expended to get to the answer, and compare that to opus, I think it's still a net win.
My hunch is that Opus scale models probably have shortcuts encoded into the model that handle these ambiguities cases, wheres this model has learned a program to reason through the edge case (crystalized vs fluid intelligence). Remembering that probablity (frontier) vs calculating it on the fly (vibethink)
> Multi-level Quality Control.
> [...]
> LLM-based Query Quality Filtering. We utilize capable LLMs to assess query quality, filtering out samples with incomplete descriptions, unreasonable conditions, invalid logic, or an inability to effectively assess target knowledge points.
I am a human and I don't know how to interpret this prompt.
> If A goes to B who then goes to C, can A send something to C?
"If John travels to the location of Mary, and later Mary reaches Paul, is John enabled to have an item delivered to Paul"
Or what did you mean.
If A goes to B who then goes to C does C know A?
Really enjoying seeing these really capable SMLs. Note that on HF they state: "This model was not trained on tool-calling or agent-based programming data. We therefore do not recommend using it for tasks that involve function calling, API orchestration, or autonomous coding agents." - https://huggingface.co/WeiboAI/VibeThinker-3B So we can't just hook it up to a coding harness like pi.dev or something.
How would you best utilize a model like this for coding? I take it it's not meant for vibe coding a full app, and the reasoning probably makes it unsuitable for autocomplete. Would you use it to implement specific functions? I looked at one of the coding benchmarks used, Live Code Bench, and it seems to be problem descriptions with sample input and output, and then a solution with a single function or class.
Seems like a really good model to use in an IDE when you still want control over the code structure then.
Not sure if it's suited for that. If you read the article it's stated that it is basically a research project to see how far they can push it with small models.
It's terrible at hunting security bugs (I expected it to be, but I wanted to be sure). I added it to a benchmark I made with a corpus of some Mythos-discovered bugs, and it found zero. The smallest pretty successful models remain Qwen 3.6 and Gemma 4 (but I haven't tested the very small variants of those yet).
https://swelljoe.com/post/will-it-mythos/
The lack of tool use will hinder it a lot I think, since bug hunting requires collecting context across a code base and stitching it together. It might be good in a more narrow sense, i.e "is there a bug in this block of code" and not considering how it interacts with the rest of the code base.
That's also more aligned to its leetcode style training data, the code under test is fully in the context window. It might be interesting to have a bigger tool use model go through the effort of collecting the context, and feeding it into this kind of model for analysis only. It becomes more of a thinking tool, instead of the orchestrator.
I really hope that in a couple of years I can have a laptop that runs a reasonably good coding agent locally, that I can run fast and do most of my programming with, without running my laptop hot. I could keep open code and use other models when needed, but really for most of my work, I'm already breaking it down so that I can review code changes eventually, and I just need something reasonably decent and fast and unlimited. I think its coming.
I'm very optimistic here as well. And it's also worth noting that tooling is improving along with the models. I really think we have to treat models and tools as a package. The models is your engine, but you need a chassis to run it.
I find what makes frontier models actually work well isn't just the capability of the model, but how well the harness is tuned to its expectations. I wrote a about this in a bit more detail here. https://yogthos.net/posts/2026-06-08-dirge-code.html
I hope so too. But I fear that it will feel inadequate if we know there is always a $20 online model that is an order of magnitude better. I don't think there will be a "good enough" local model so long as frontier models look so much better.
Seems like most people have settled on Opus 4.6 as the breaking point (me as well).
Once I can spend 10k to run Opus 4.6 at home, I'm done.
Sounds like something that could be pretty useful as a 'validation' subagent. Provide it the details/context related to a larger LLM's run or turn in a harness and have it act as a gatekeeper. At this size and speed it looks like it could be economical to have it run every turn or even every tool call and inform the main agent about the result and success/failure.
I have been thinking about how to use this. Since it doesn’t support tool calling I have been considering a dual model deployment, where a small tool calling llm drives the majority of the user experience, and vibe thinker is tapped for reasoning by the other llm.
So who has suggestions on small models with excellent tool calling capabilities?
Gemma 4 E4B and Qwen 3 4B are pretty good, but fine-tuning makes them really good. There are tradeoffs at this size, so you'll have to find (or make) a finetune that does what you need.
Qwen3.6-35B-A3B is pretty amazing. I'm using it with 96k context on 24GB VRAM through ollama.
Maybe bonsai 8b would make the duo, if you do try it, pls post here as I'm a bit curious too.
granite 4
I still cannot trust evaluations and benchmarks. How can you prove that the test datasets are truly unseen examples?
I think the only way to prove that these models are truly as good as they claim is to wait and see if they are getting adopted in practice.
> the only way to prove that these models are truly as good as they claim
It would be actually to progress towards the solution of the "black box" problem, the goal of "transparency".
You have to implement a reasoner (etc.), you conceive the best architecture for it - then implement and test it.
SLM when trained for single use case often beats the LLM. That's both the advantage and limitation.
I recently came across this model and I would love to try it with my coding agent soon.
I really like the idea of small models that can reason but do not have too much knowledge. Also, no emphasis on tool calls. I think the agent should do the heavy lifting and reach half way.
I use really small models, like Qwen 3.5 0.8B to 9B - no tool calling, no MCP, no skills, nothing. No multi-turn chat even. Models are given very specific tasks using a vast number of system prompts and all the response handling is done in the agent(s).
https://github.com/brainless/nocodo
Maybe no tool calling, but seems it could be really good at deciding which tool to use and when?
That is a good point. I do think these models would be good in the decision making. The large models are trained to use tool calling. Perhaps the small models can generate the text that would express their decision but not generate good JSON to reply with correct syntax. I do not know but this is my hunch.
Notable:
Qwen2.5 is ancient by LLM standards.Two model, one is optimised for system, reasoning etc, second is optimised for specific language ( rust or go ? ) , both small enough to run on local computer, will it work ?
I gave this a run on llama.cpp locally. My GPU is Ge1080, so I needed quantized version for even such a small model and…
This. Is. Amazing. I am flabbergasted.
I am not into the whole GenAI thing and I have very little need for anything agentic, but Python, C++ and Maths is exactly what I mostly used these for, so this might actually become my main work horse. This is so cool.
I even used it for stuff it is not built for, asking complex qustion on history (Battle of Tours 732) and literature (Joyce’s Portrait of Artist) and it was surprisingly good, even though it started to hallucinate names and details (such as claiming Joyce’s father was a priest). For 3B I expected it to mainly spout complete nonsense.
If you could pair it somehow with a model that can code and describe code this could be a very powerful combo.
I tried actually talking to it. It reminded me of GPT-2.
It's not supposed to be a chat model. It is crazy good at math.
Yeah, I thought so too. It's supposedly good at coding, too. But, if its English responses are insane, why should I trust it to understand my programming-related instructions?
Well, only one way to find out! I mean, maybe it only says insane things and goes in circles if you ask non-coding questions, but it doesn't exactly inspire confidence.
Man just need something like this with tool calling.
The absolute worst name for a model I've seen
GRPO skips the value network that makes PPO expensive — it scores candidates relative to each other within a group. that's what makes verifiable-reward training practical at 3B scale
Why was this downvoted.
So I went ahead and quickly vibecoded a working harness with a barebones tool interface and some constraints on output (credit to noperator for the idea). github: https://github.com/NickalasLight/VibeHarness.git
Its meant for a Windows machine using ollama but I'm sure anyone who wants to mess with it can point claude code at it to convert it for your own operating system and requirements. After install you can ask it to do something with "vibe 'create me a poem about cheese in cheese.txt'" its workspace is by default the directory the cli was located in when you called it.
What is the idiots guude to run this one local now?
Ask an existing LLM harness like Claude Code. "Install this and run the demo program" or "Set this up with llama-server" or "Give me an Open WebUI page for this model" will work.
[dead]
Use LM Studio.
how do I get these weights in particular?
omlx makes it quite easy
BF16 with no QAT quants == half backed bread
this is a good model. I benchmark reasoned answers to qwen 3.6 27b (no think)+ bash and it held up.
I don't get this beating opus, It just hardcoded the tasks for bench , It does even respond normally
A alot randomness in it
Please don't hype
What are the implications of local SOTA inference, given the insane datacenter "investing" ?
It surely cannot be justified only for training at this scale, and since models nowadays are improved more and more by fine tuning than re-training from scratch.
Will a viable local model crash the US economy ?
More importantly, are the LLM companies aware, and are they deliberately buying out all the RAM and GPUs in order to prolong the inevitable ? Probably not, but I wouldn't be surprised if that is the case.
Wake me up when it does OCaml fine.
[flagged]
[flagged]
[dead]
[flagged]
[flagged]
[dead]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
3B param on par with opus 4.5 sounds interesting. Will read the full article before making my mind
Does python coding depend on political facts of the world?
It might appear not, but actually, the process of reasoning is not an isolated act. The right and wrong way of doing things is codified in social evolution that absorbed all facets of life. Why should you optimize a piece of code for performance? Why performance is needed? What is a bug? What features and UI themes would be more intuitive for humans?
There is a butterfly effect. Everything affects everything to some extent.
True, but this model provides something of a lower bound on just how much world knowledge is really needed for unrelated reasoning tasks. That lower bound appears to be quite low indeed. Lower than I thought it would turn out to be.
This thing is just bonkers.
[dead]
The paper makes a clear claim: "it provides an important and concrete proof: on well-constrained, verifiable reasoning tasks, first-tier performance is no longer the exclusive domain of ultra-large models" And that's exciting.