On-Device AI Models Just Got a Major Upgrade — Apple Is Eyeing PrismML to Change Everything
A Caltech-born startup has done something that seemed implausible just months ago: compressed a 27-billion-parameter large language model to run entirely on an iPhone 17 Pro — and Apple has taken notice. PrismML claims to have successfully compressed Alibaba’s Qwen 3.6 model to run locally on that device, achieving a compression ratio exceeding 90% with no claimed performance loss. The implications for on-device AI models — and for the future of private, powerful mobile intelligence — are enormous.
Apple has held meetings with PrismML about ways it could use the startup’s technology to run much larger AI models directly on iPhones. That’s a quietly seismic sentence. It signals that Apple’s long-standing push for local AI execution may be about to hit a genuine inflection point — one that could redefine what your iPhone is capable of doing without ever touching a cloud server.
What Makes On-Device AI Models So Hard to Scale?
The fundamental problem with running AI models on iPhone isn’t ambition — it’s physics. Standard LLMs store each weight — the numerical values that encode the model’s learned knowledge — in 16-bit or 32-bit floating point format. That precision is expensive: more bits per weight means more memory, more bandwidth, and more power at inference time.
The Qwen 3.6 model is so large because of its 27 billion parameters, a figure that allows it to perform more advanced tasks than smaller models. The models that typically run on phones normally have just a few billion parameters, limiting their capabilities. Apple’s own current on-device flagship tells this story well. The new on-device model Apple unveiled at its Worldwide Developers Conference this June features 20 billion parameters but employs a sparse architecture, with only 1 billion to 4 billion parameters active at any given time. In contrast, PrismML’s model keeps all 27 billion parameters simultaneously active during runtime — a difference the company views as its core competitive advantage.
That distinction matters for on-device AI capabilities. A 27-billion-parameter model kept fully active can handle nuanced writing tasks, code generation, and contextual reasoning that sparse 1-4 billion active-parameter models cannot.
PrismML’s Breakthrough: The Science of Compressing Large Language Models
PrismML is a spin-off from the California Institute of Technology (Caltech). CEO Babak Hassibi is a professor of electrical engineering at the university and completed the mathematical research underpinning the technology with his co-founders during their time there. Caltech holds the relevant patents and has granted an exclusive license to PrismML.
The approach to compressing large language models that PrismML employs is radically different from conventional methods. Traditional large language models use 16-bit floating-point precision, leading to massive memory requirements. PrismML’s approach uses 1-bit weights (essentially on/off states) and ternary variants (weights of -1, 0, or +1). It’s a shift that sounds simple but took years of advanced mathematics to pull off without gutting model performance.
The company’s core technology involves a mathematical method that compresses the Qwen 3.6 model from approximately 54GB to under 4GB — a compression ratio exceeding 90% — with the company asserting no impact on performance. For context, it allows complex AI tasks to run directly on smartphones by shrinking a normally much larger language model to under 4GB, enabling all 27B parameters to operate on-device without claimed performance loss.
The ternary neural network architectures at the core of PrismML’s work have a deep scientific lineage. Ternary quantization is a process that maps real-valued parameters to a discrete set of three values {–α, 0, +α}, enabling efficient model compression and reduced energy consumption. Ternary neural networks offer a favorable trade-off between accuracy and computational saving compared to a binary quantized network, which often suffers from higher accuracy loss due to extreme quantization. PrismML’s proprietary twist on these ternary neural network architectures is what’s drawn Apple’s eye.
The Bonsai Family: Proof That Local AI Model Execution Works
Before tackling the 27-billion-parameter Qwen 3.6 model, PrismML had already shipped working proof through its open-source Bonsai series. PrismML, an AI lab co-founded by Caltech mathematician Babak Hassibi, has open-sourced its 1-bit Bonsai series of large language models. The flagship model, 1-bit Bonsai 8B, boasts 8.2 billion parameters and occupies just 1.15 GB of memory, making it 14 times more compressed than comparable 16-bit models. This model, along with smaller versions, demonstrates comparable performance to 16-bit models on standard benchmarks, despite its reduced memory footprint.
Local AI model execution at this scale also translates to real speed. The 1-bit Bonsai 8B model achieves impressive real-world speeds, with the iPhone 17 Pro Max processing approximately 44 tokens per second. PrismML’s innovation reduces energy consumption by 4 to 5 times compared to 16-bit models, with potential for further efficiency gains if future hardware supports 1-bit inference.
The company has also extended this local AI model execution approach to image generation. PrismML has already demonstrated production-ready on-device AI through Bonsai Studio, its image generation app, which generates 512×512 images on iPhone 17 Pro in roughly 12 seconds using the firm’s ternary Bonsai Image model. That’s running AI models on iPhone without any server roundtrip, for a task that most people assume requires cloud infrastructure.
Benchmark results back up the efficiency claims. On raw benchmark performance, Bonsai 8B scores an average of 70.5 across IFEval, GSM8K, HumanEval+, BFCL, MuSR, and MMLU-Redux — putting it above Llama 3.1 8B (67.1) and LFM2 8B (69.6), all of which are 14x larger in memory footprint.
Why Apple Needs This: The Cloud Dependency Problem
Reducing AI Memory Footprint to Break Free From Servers
Apple’s current AI strategy is a hybrid. For advanced features that need to reason over complex data with larger foundation models, Apple created Private Cloud Compute (PCC), a groundbreaking cloud intelligence system designed specifically for private AI processing. But that hybrid approach comes with a catch. Apple is now collaborating with Google and NVIDIA to run new Apple Intelligence workloads on Google Cloud, extending its PCC privacy commitments to third-party data centers for the first time. This year, Apple collaborated with Google to leverage technologies behind its Gemini family of models to build the next generation of Apple Foundation Models.
That reliance on external infrastructure creates real strategic tension. Apple has long positioned on-device AI as a core pillar of its privacy and security commitments, largely avoiding the multi-hundred-billion-dollar data center arms race among tech giants. However, Apple’s long-awaited major Siri upgrade still relies on Google’s Gemini model, with its most advanced features requiring Nvidia chips running in Google Cloud. This reality is noticeably at odds with Apple’s on-device AI vision, making PrismML’s technology potentially strategically valuable for Apple.
Reducing AI memory footprint aggressively — the way PrismML does — directly attacks this problem. Larger models running directly on iPhones would allow for more Apple Intelligence features to run on device instead of on Apple’s Private Cloud Compute servers, which could reduce Apple’s costs and further enhance user privacy.
On-Device AI Capabilities That Actually Change User Experience
The everyday impact of expanding on-device AI capabilities would be immediate and tangible. If Apple can run larger models on iPhone, the first impact would likely be in features that need context and speed. Siri could handle more requests locally, understand longer instructions, and respond with less delay. Dictation could improve with better language understanding. Writing tools could become more useful offline. Photos, screenshots, and visual search could process richer information without routing as often to cloud systems.
Local processing can reduce latency, work without a strong connection, protect more data on the device, and make AI features feel built into the phone rather than attached through a remote service. Those aren’t minor conveniences. For users in areas with poor connectivity, or anyone who simply values their data staying on their device, these benefits are the whole point.
The Technology Underpinning Running AI Models on iPhone
What Ternary Neural Network Architectures Actually Do
The science of making on-device AI models viable comes down to dramatically redefining how each “weight” in a neural network is stored. Ternary Bonsai models use a ternary weight representation, where each weight takes one of three values: {-1, 0, +1}, corresponding to 1.58 bits per weight. PrismML’s proprietary architecture applies this representation end-to-end across the entire network.
The payoff from applying ternary neural network architectures at scale is striking. At just 1.58 bits per weight, the models achieve dramatically smaller memory footprints compared to standard 16-bit models: Ternary Bonsai 8B sits at approximately 1.75 GB versus ~16.4 GB for a standard equivalent. That’s the kind of reduction that transforms what running AI models on iPhone actually means in practice.
Reduced memory footprint — storing weights in three states — means models often require substantially less storage than full-precision counterparts, enabling deployment on devices with limited flash or RAM. Lower inference energy from simpler arithmetic (adds, sign checks) typically consumes less power than floating-point multiplications, which is valuable for battery-powered devices.
What Independent Experts Are Saying
PrismML completed a $16.25 million seed funding round earlier this year, with participation from Khosla Ventures. Vinod Khosla, founder of Khosla Ventures, said in an interview that his interest in PrismML stemmed from the company offering a “fundamental breakthrough.”
Amir Salek, who founded Google’s TPU program, praised PrismML’s work, saying the “fundamental change in the power-to-compute equation” could unlock new AI hardware architectures. That’s not a throwaway endorsement — it comes from someone who built the very infrastructure that PrismML aims to make less necessary.
That said, PrismML’s claim that a fully 1-bit model can match the capabilities of higher-precision systems remains unproven outside the company’s own benchmark results. Extreme quantization techniques have historically struggled to preserve accuracy in complex reasoning tasks. Independent third-party benchmarks and real-world deployments will be critical in determining whether PrismML’s approach represents a true breakthrough or a more limited optimization.
What Comes Next: PrismML’s Ambitions and Apple’s Move
PrismML claims the model is capable of handling complex conversations, reasoning, fully autonomous agents, and software coding tasks. CEO Babak Hassibi predicts that within three years, the vast majority of AI computing users need will be performed locally.
PrismML says the company plans to continue compressing even larger models — including trillion-parameter models — to run on devices, eventually entering the arena alongside OpenAI GPT and Anthropic Claude. That’s an audacious roadmap. But given what the company has already demonstrated with running AI models on iPhone, it’s not an easy claim to dismiss.
Apple has not announced a partnership with PrismML, and the meetings do not confirm that the startup’s technology will appear in future products. Still, the fact that Apple is at the table matters. Apple is actively exploring third-party optimization technologies, which suggests the company recognizes a gap between what its own models deliver and what on-device hardware can theoretically support. This doesn’t necessarily mean acquisition (though Apple has bought AI startups before), but it reveals Apple views external expertise in model compression as worth investigating.
Why This Moment Is Bigger Than One Startup
The conversation around on-device AI models has shifted from “someday” to “right now.” What PrismML represents isn’t just a compression trick — it’s a structural challenge to the assumption that powerful AI requires server-scale compute.
PrismML’s sweeping goal is to enable a future where powerful AI can run locally, efficiently, securely, and faster, and where datacenter buildouts can do more with fewer resources and avoid ballooning energy costs. For Apple, that future aligns perfectly with its privacy narrative and its growing need to reduce dependence on cloud partners.
Apple’s strongest AI message remains privacy. Running more intelligence on the device supports that message because sensitive prompts, images, documents, messages, and personal context do not need to leave the phone as often. That fits the company’s long-standing position that personal data should be protected by design rather than handled through vague promises.
Whether Apple licenses PrismML’s technology, acquires the startup, or simply uses these conversations to sharpen its own internal roadmap, the direction is unmistakable: the era of on-device AI models powerful enough to rival cloud-based systems is closer than most people thought.
Frequently Asked Questions
What is PrismML and what does it do?
PrismML is a U.S.-based AI startup that spun out of the California Institute of Technology (Caltech). It specializes in extreme model compression, using 1-bit and ternary weight architectures to dramatically shrink large language models so they can run directly on consumer devices like iPhones, laptops, and embedded systems — without needing cloud infrastructure.
How did PrismML compress a 27-billion-parameter model to run on an iPhone?
PrismML used a mathematical compression method developed at Caltech to reduce Alibaba’s Qwen 3.6 model from approximately 54GB down to under 4GB — a compression ratio exceeding 90%. The approach uses 1-bit or ternary weight representations (values of -1, 0, or +1) instead of the standard 16-bit floating-point format, dramatically reducing the model’s memory footprint without a claimed loss in performance.
Has Apple officially partnered with PrismML?
No. As of the reporting available in July 2026, Apple has held exploratory meetings with PrismML, but no formal partnership or acquisition has been announced. The meetings indicate interest, not a confirmed integration into future Apple products.
What is Apple’s current on-device AI model?
Apple’s current largest on-device model is AFM 3 Core Advanced, which has 20 billion parameters and powers iOS 27 features like expressive Siri AI voices and improved systemwide dictation on iPhone 17 Pro and iPhone Air. However, it uses a sparse architecture where only 1 billion to 4 billion parameters are active at any given time, unlike PrismML’s fully active 27-billion-parameter model.
What is a ternary neural network and why does it matter for mobile AI?
A ternary neural network is one where each weight value is constrained to one of three states — typically -1, 0, or +1 — rather than a high-precision floating-point number. This approach, stored at approximately 1.58 bits per weight, drastically reduces the memory and energy requirements of a model while preserving most of its reasoning capability. For mobile devices with limited RAM and battery, ternary neural network architectures are one of the most promising paths to running sophisticated AI locally.
How much funding has PrismML raised?
PrismML completed a $16.25 million seed funding round backed by Khosla Ventures, Cerberus Capital, and Caltech. Khosla Ventures founder Vinod Khosla described the technology as a “fundamental breakthrough” and a “major technological breakthrough — a mathematical breakthrough — not just another small model.”
What are the risks or limitations of PrismML’s approach?
The primary risk is independent verification. PrismML’s performance claims — particularly that its extreme compression incurs no meaningful accuracy loss — are based largely on the company’s own benchmarks. Historically, aggressive quantization techniques have struggled to preserve accuracy on complex reasoning tasks. Until independent third-party benchmarks are published and real-world deployments are tested at scale, these claims remain unconfirmed outside PrismML’s own testing environment.
