Needle 2 is a useful reminder that many edge-AI products do not need a general assistant.
Cactus describes Needle 2 as a 45 million parameter, 14 MB agentic language model built for tool calling, structured extraction, and device control on constrained hardware. The stated runtime target is tiny: 28 MB deterministic session RAM, no GPU or NPU requirement, and more than 500 tokens per second decode on a Raspberry Pi 5. The model is Apache 2.0 licensed and shipped with a dependency-free C++ runtime.
The important design choice is schema-first control. Needle 2 maps messy user language into typed function calls, enum fields, array arguments, and an empty-call refusal path. The page also describes a learned confidence score, with low-confidence cases re-asked or escalated to the cloud.
Grey Haven’s read: local AI gets interesting when the product narrows the job until small models are enough. Lights, thermostats, small robots, appliances, inspection tools, and field devices do not need a model that can debate philosophy. They need a local dispatcher that can select the right action, fill the right arguments, refuse off-topic requests, and preserve privacy when the network is unavailable.
Operators should watch the failure boundary. A tiny model that acts locally must make uncertainty legible. The useful pattern is not “cloud versus edge.” It is a tiered control loop: act locally for narrow high-confidence commands, ask a clarifying question when inputs are ambiguous, and escalate only when the decision requires broader context. That pattern is likely to matter more than raw benchmark comparisons for industrial devices and low-cost consumer hardware.
Source: Cactus, “Needle 2 - The 14 MB Agentic LLM for Tiny Devices.”
