- AI Fire
- Posts
- 📱 Phone AI's Own Leaderboard!
Which AI model actually runs best on your phone? Pipette now benchmarks real on-device performance across iPhone, Galaxy, and Mac, including speed, quality,...
What's on FIRE 🔥
IN PARTNERSHIP WITH SECTION
On 9/17 from 11AM – 6PM ET, join Section, Scott Galloway, and AI leaders from Wayfair, MetLife, TD Bank, SS&C, Booz Allen, and more for the bi-annual AI:ROI Conference.
1 day. 10 sessions focused on defensible AI spend, ROI, and impact.
AI INSIGHTS
Liquid AI and Artificial Analysis launched Pipette, an open-source benchmark for testing which AI models actually run best on-device, including on the iPhone 17 Pro. Main details:
Measures quality, speed, latency, and memory use directly on phones, laptops, PCs, and embedded hardware.
Compares the full setup: model + quantization + runtime + device.
Already includes 10,000+ verified benchmark results.
Covers around 35 model classes, 7 quantization levels, and multiple devices.
Current hardware includes iPhone 17 Pro, Samsung Galaxy S26 Ultra, and MacBook Pro with M5 Max.
The benchmark is open source, so developers can submit new devices, runtimes, models, and configurations.
The important part is that the “best model” can change dramatically depending on quantization, runtime, and hardware. A model that looks strong in cloud benchmarks may be too slow or memory-heavy on a phone.
Pipette gives us a practical way to choose the best local AI setup for the exact device we want to ship on.
SPONSORED BY HUBSPOT
Want to get the most out of ChatGPT?
ChatGPT is a superpower if you know how to use it correctly.
Discover how HubSpot's guide to AI can elevate both your productivity and creativity to get more things done.
Learn to automate tasks, enhance decision-making, and foster innovation with the power of AI.
AI SOURCES FROM AI FIRE
1. FREE: 7 Incredible Things ChatGPT Browser Use Can Actually Automate Like an Agent For You. ChatGPT is becoming an AI agent. See 7 incredible ways ChatGPT Browser can browse the web, complete complex tasks, and work like your digital assistant.
2. How to Build Your Agent Skills Once, Run It Almost Everywhere: Claude Code, Codex, Cursor. Every AI platform has its own way of handling workflows today. Agent Plugins 1.0 could change that by creating a shared layer where agents follow you wherever you work.
3. Comprehensive Claude Cowork Tutorial for Beginners: Build Your First AI Workspace. It just looks complicated at first. If you’re new to Claude Cowork, or you’ve already tried it but still feel like you’re only using a small part of what it can do, save this guide.
4. Lesson 3: How to Make Your Research 10x Faster with Deep Research and Gemini Notebook. Google now has several powerful research tools, but choosing the wrong one can create more work. I’ll help you pick the right tool for each task.
Last time, we asked what kind of newsletter growth help you wanted from us. A lot of you responded.
Some of you wanted the full course, or live workshops, or direct help from our team. Some even wanted a deeper look at how we actually grow & operate AI Fire.
Honestly, seeing that response made the last 3+ years of work feel worth it. So we opened Lesson 1 of our Paid Newsletter Growth module for free.
Lesson 1 covers the math: CPA, subscriber value, payback, and safe budget levels.
Lesson 2 covers the operating system: SparkLoop, Beehiiv Boosts, paid ads, partner reviews, segments, welcome emails, and scaling decisions.
This first lesson is mostly about the thinking and math you need before you spend money to grow a newsletter. Lesson 1 gives you the foundation.
Then, inside Lesson 2, we go into the practical system and see how to make real scaling decisions with us step by step.
If you’ve ever thought about using paid growth to scale your newsletter, start with the free lesson first. It will save you from guessing.
TODAY IN AI
AI HIGHLIGHTS
🔥 Want to make money with AI? Join our FREE live webinar this Wednesday at 7:00 PM PT to learn how to build an AI Affiliate & Dropshipping business with an AI Influencer. Save your seat now.
👀 I’d test Ox Alpha again if you only tried it on day one. Multiple tests say the outputs now feel noticeably sharper, and better structured. Test it yourself.
🤯 Unlimited MiniMax access for 14 days is kind of hard to ignore. You’ve got M3, M2.7, Speech 2.8, and Music 3.0 on GMI Cloud until September 6. Start building.
🥇 Well-known creator Theo Browne ranked his 15 favorite AI models from best to worst, and the post blew up with 2M views and 13K likes. Check the ranking here.
🚀 Trump bought up to $50K in SpaceX shares just 11 days after its record IPO, according to a disclosure. The timing is... going to get huge attention (even now).
😬 A month ago, AI hedge fund Situational Awareness looked like one of AI investing’s hottest bets. Now it’s dealing with massive losses and an SEC probe.
🛡️ Anthropic is expanding Mythos 5 access to more cyber defenders beyond Project Glasswing, while bankers are reportedly floating a $2T IPO valuation & $100B+ raise.
💰 Big AI Deal: Nvidia signed a $6B deal to license Poolside’s AI technology and bring 100+ engineers onto its Nemotron team, strengthening its push to build more powerful open-weight models.
NEW EMPOWERED AI TOOLS
📈 SegMetrics turns ChatGPT or Claude into a marketing analyst that understands your funnel, historical data, and the metrics that actually matter. See this free 30-minute training to master it.
⚖️ Harvey’s Tenet is its first legal AI model, built for long, complex legal work and optimized to use fewer tokens efficiently.
🦉 MarketOwl turns your goal into a full marketing strategy, then executes campaigns across 8,000+ tools, and AI agents. 500 free credits.
💳 PaymentKit keeps SaaS subscriptions running by routing payments across processors and surviving unexpected processor shutdowns.
AI BREAKTHROUGH
A new paper, “Practice Makes Unsafe,” shows how self-improving AI agents can accidentally turn one malicious experience into a persistent reusable skill. Main findings:
All 21 self-evolving agent configurations created unsafe skill artifacts.
15 of 21 later caused harm in a completely fresh session.
Three malicious tasks increased carryover attack success from 16.0% to 35.3%.
The danger remains even after the original malicious prompt disappears because the unsafe behavior is stored in the agent’s skill library.
The researchers introduced SkillMisevo-Gym and SkillMisevo-Bench to track how harmful behavior moves from experience → saved skill → later reuse.
Their defense, SafeEvolve, reduced unsafe retrieval by 26.7 percentage points and fresh-session harm by 17.3 points, with almost no loss in normal task performance.
The worrying part is persistence. A single successful attack can potentially influence many later tasks because the agent treats the unsafe behavior as something worth remembering.
With self-improving agents, safety checks cannot stop at the final answer. We also need to inspect what the agent learns, saves, and reuses over time.
We read your emails, comments, and poll replies daily
How would you rate today’s newsletter?Your feedback helps us create the best newsletter possible |
Hit reply and say Hello – we'd love to hear from you!
Like what you're reading? Forward it to friends, and they can sign up here.
Cheers,
The AI Fire Team
Reply