Can AI Accurately Track Calories with a Picture?

Artificial intelligence is rapidly making its way into the fitness and nutrition space. From workout programming to habit tracking, AI tools are becoming more accessible and more powerful. One of the most appealing ideas? Simply taking a photo of your meal and having AI tell you exactly how many calories and macros you’ve eaten.

It sounds like a game-changer. But how accurate is it really?

A recent Pubmed Study put this idea to the test, and the results suggest we’re not quite there yet.

What the Study Tested

Researchers evaluated three of the most widely used large language models (LLMs): ChatGPT, Claude and Gemini. The goal was simple: determine how accurately these models could estimate:

• Food weight
• Calories (energy content)
• Macronutrients (protein, carbs, fats)

They tested this across 52 different foods, using images as the only input, just like a typical user would.

What They Found

The results were… messy.

1. Calorie & Portion Estimation Was Highly Inaccurate ChatGPT and Claude had just under 40% average error for estimating weight and calories Gemini performed worse, with errors around 65–70%. To put that into perspective, if your meal is actually 600 calories, the estimate could easily land anywhere between 360 and 840+ calories.

2. Macronutrient Estimates Were Even Worse Error ranged from 42% to 110%.

3. Bigger Meals = Bigger Errors Accuracy dropped further as portion sizes increased. Larger, mixed meals (think bowls, plates with multiple items) were significantly harder for AI to interpret correctly.

4. Some Foods Were Completely Misidentified In several cases, the AI didn’t just miscalculate but misidentified the food altogether. That’s not a small problem. If the AI thinks your beef is tofu, or your full-fat yoghurt is low-fat, your entire calorie and macro breakdown becomes meaningless.

Why Is This So Difficult?

At first glance, estimating calories from a photo seems straightforward. But in reality, it’s a complex problem involving several layers:

1. Portion Size Is Hard to Judge from Images

A photo lacks depth, scale, and context. Without reference points, even humans struggle to estimate portion sizes accurately.

2. Food Composition Varies Massively

Two identical-looking meals can have completely different calorie contents depending on:

Cooking methods Oils used Ingredients hidden beneath the surface

3. Mixed Meals Add Complexity

Meals aren’t always simple. A single dish might contain multiple ingredients layered together, making accurate breakdown nearly impossible from a single image.

The Bigger Problem: Nutrition Tracking Is Already Imperfect

Even traditional methods aren’t great.

• Food recall methods (e.g. “what did you eat yesterday?”) are notoriously unreliable
• Food frequency questionnaires are even worse for precision
• Even manual calorie tracking relies on estimation unless you’re weighing everything

So it’s not that AI is uniquely flawed, it’s just that this problem is inherently difficult.

So… Can You Use AI for Calorie Tracking?

Short answer: not accurately, yet.

But that doesn’t mean it’s useless.

Where AI Can Be Helpful

• Getting a rough ballpark estimate
• When logging meals quickly when precision isn’t critical
• Increasing awareness of food choices
• Supporting consistency when manual tracking feels too time-consuming

Where It Falls Short

• Precise calorie tracking
• Macro-specific goals (especially protein)
• Coaching or clinical-level nutrition planning

What This Means for You (Especially If You Train)

If your goal is:

Fat loss → a 300–500 calorie error matters Muscle gain → missing protein targets matters Body recomposition → precision matters

Then relying solely on AI image tracking is a mistake.

For now, the gold standard still looks like:

• Weighing food (when accuracy matters)
• Using established tracking apps/databases
• Building consistency over perfection

The Future: Promising, But Not Ready

AI is improving fast. As models become better at:

• Image recognition
• Contextual reasoning
• Integrating databases

We’ll likely see major improvements in this space.

But based on current evidence, including this study AI calorie tracking from photos is still too inaccurate to rely on for serious results.

Bottom Line

AI can make nutrition tracking more convenient—but not yet more accurate.

If you treat it as a rough guide, it can be useful. If you treat it as precise data, it will mislead you. And when it comes to your results, that distinction matters.

Resources:

📝Performance Evaluation of 3 Large Language Models for Nutritional Content Estimation from Food Images | Pubmed