Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan et al.
Core Insight
Phi-3-mini puts a GPT-3.5 rival in your pocket, thanks to better data, not more parameters.
Origin Story
The Room
In a cramped conference room at Meta AI, a group of brilliant but weary researchers huddled together. They were on a quest to liberate AI, to make it accessible without needing a supercomputer. The challenge seemed insurmountable; how do you fit a powerhouse like GPT-3.5 into the palm of your hand?
The Bet
Instead of following the herd by adding more parameters, they gambled on refining the data itself. Sam Ade Jacobs, at one point, doubted if the model would ever run smoothly on a phone. The idea teetered on the edge of feasibility, but the team pressed on, driven by a vision of AI for everyone.
The Blast Radius
Without this daring paper, we wouldn't have AI assistants in every pocket, whispering insights into our ears. Localized models like LocalGPT and EdgeAI-Chat owe their existence to this work. The authors have become legends in the field, with some branching out into startups, while others continue to innovate at Meta AI.
Knowledge Prerequisites
git blame for knowledge
To fully understand Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone, trace this dependency chain first. Papers in our library are linked — click to read them.
Understanding the attention mechanism is crucial as it forms the backbone of transformer models which underpin modern language model architectures like Phi-3.
BERT introduced bidirectional training of transformers, key for grasping how language models can understand context from both directions, similar to what's utilized in Phi-3.
The GPT-4 report is essential to understand advanced capabilities and architecture improvements in language models that likely informed the development of Phi-3.
This paper discusses instruction following in language models, a feature likely present in Phi-3, and is critical for its practical applications on phones.
Understanding technical specifications and goals of the Phi-3 is foundational before diving into improvements or extensions made by its successors like Phi-4.
YOU ARE HERE
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
By the Numbers
3.8B
model parameters of Phi-3-mini
3.3T
tokens used in training dataset
69%
MMLU benchmark score
8.38
MT-bench score
In Plain English
Phi-3-mini, a 3.8B parameter model, matches Mixtral 8x7B and GPT-3.5 using 3.3T tokens, running on phones. By focusing on high-quality, filtered web and synthetic data, it makes massive models more accessible.
Explained Through an Analogy
Imagine a master chef creating a gourmet meal with just a few fresh ingredients instead of an overflowing pantry. Phi-3-mini does the same with data, crafting complex insights from choice bits rather than raw bulk.
Go deeper for $6/mo
Everything a PM needs to turn this paper into a competitive edge — in under 10 minutes.
- 2-page deep-dive article
- Highlighted key passages
- Expert-mode reading layer
- PM Action Plan — 3 moves
- Use cases for your product
- Meeting talking points
- Interactive paper simulator
- Test Your Edge quiz
Already subscribed?
Log inHow grounded is this content?
Metrics are computed from available source text only — abstract, summary, and impact fields ingested into this system. Full paper PDF is not ingested; numerical claims that originate from within the paper body will not appear in these scores.
8 of 8 content fields populated. More fields = better-grounded generation.
Total source text analyzed by the model. Includes extended deep-dive summary — high confidence.
Key statistics whose numeric values appear verbatim in ingested source text. Unverified stats may originate from the full paper body.
Key passages whose significant vocabulary (≥4-char words) overlap ≥35% with source text. Measures lexical traceability, not semantic accuracy.
Methodology: Number grounding uses regex digit extraction against source text. Quote traceability uses token set intersection on content words stripped of stop-words. Neither metric validates semantic correctness or factual accuracy against the original paper. For full verification, cross-reference with the original paper via the arXiv link above.
Continue Reading