Google Is Quietly Building AI Into the Pixel Camera App, and It’s Worrying Me
This week, Google’s Pixel 10 smartphones officially debuted, and with them, a bunch of generative AI features built right into the camera app. These days, phones typically use “computational photography,” a fancy term for all the lighting and post-processing effects they add to photos as you take them. But AI turns computational photography into a whole other beast, and I’m not sure we’re ready for it.
Techies like to ask themselves, “What is photography?” joking that the more post-processing you do to a photo, the less it resembles what happened in real life. The night sky is too bright, faces have fewer flaws than in the mirror, and so on. Generative AI in the Camera app is like the final boss of this moral conundrum. That’s not to say these features are useless, but ultimately, this is more of a philosophical debate than a technical one.
Should photos look like what the photographer sees with their own eyes, or should they be as attractive as possible, realism be damned? For now, it’s easy to leave these questions to the pickiest of people — who cares if the sky is too neon if it makes the photo pop? — but if AI is going to add entirely new objects or backgrounds to your photos, it’s time for everyone to ask themselves what they want from their phone cameras before they even open the Gemini app.
And given how Google uses AI in its latest phones, it’s entirely possible you’ll end up with an AI-powered photo and not even notice.
Pro Res Zoom
Perhaps the most impressive of Google’s new AI features is Pro Res Zoom. Google advertises it as “100x zoom,” and it works a bit like the completely fictional “zoom in and out” technology you’d see on old cop shows.
Basically, you can now zoom the lens 100x on a Pixel 10 Pro or Pro XL, and at first glance, it won’t look any different from regular software zoom (which relies on cropping, not AI). But inside your phone’s processor, you’ll still have the same problems that make “zoom and enhance” seem so ridiculous on shows like CSI: Crime Scene Investigation .
In short, the problem is that it is impossible to invent a resolution that the camera has not captured. If you zoom in so much that the camera lens sees only blurry pixels, the camera will never be able to accurately determine what is actually there.
That’s why this feature, while it may seem like a regular, non-AI zoom at first glance, is actually more of an AI-assisted edit than a full-on 100x zoom. With Pro Res Zoom, your phone will zoom in as much as possible, then use the blurry pixels it detects as a guide for its built-in diffusion model. The model will then figure out what the pixels should look like and stitch the result into your photo. It won’t match reality exactly, but if you’re lucky, it might be close enough.
For certain details, like rock formations or other mundane inanimate objects, this may be enough. But when it comes to faces or landmarks, you might feel like you just took a close-up of, say, a lead singer at a concert, without realizing that your “zoom” was just a clever Gemini request. Google says it’s trying to suppress hallucinations, but if you’re uncomfortable sharing or including a photo from Gemini in a creative project, the app will have the same problems, except that the branding may make you oblivious to the AI’s involvement.
Luckily, Pro Res Zoom doesn’t completely replace non-AI zoom — when zooming beyond the normal 5x hardware limit, you’ll now get two results to choose from: one with Pro Res Zoom applied, and one without. I ‘ve written about this in more detail if you’re interested, but even though there are non-AI options, the AI option isn’t explicitly shown when choosing.
It’s a much more informal approach to AI than Google has taken in the past. People may be used to AI editing their photos at their request, but having that feature applied automatically through the camera lens is something new.
Ask to edit
The AI integration doesn’t end after you take a photo, though. With the Pixel 10, you can now use natural language to ask AI to edit your photos directly from the Google Photos app. Just open the photo you want to edit, tap the edit icon, and you’ll see a chat window where you can use natural language to suggest edits to your photo. If you prefer, you can even speak instructions instead of typing them.
At first glance, I don’t mind this. Google Photos has dozens of different editing icons, and they can be difficult for the average person to use. If you just want to crop or apply a filter, you can do it without having to navigate an interface that might otherwise seem intimidating.
The problem is that in addition to using Google Photos’ traditional tools, the “Request an Edit” feature lets you suggest more outrageous edits, without making it clear when AI is being used to make those changes. You could ask the AI to replace the background of your photo with a completely new one, or, if you’re looking for a less drastic change, remove reflections from a photo taken through a window. The problem? Many of these edits will require generative AI, even seemingly less disruptive ones like glare removal, but you’ll have to rely on your intuition to determine when it’s being used.
For example, while Google Photos typically includes an “AI Enhancement” button among its editing options, it’s not the only way to add AI to your photo. The “Ask to Edit” feature will do its best to fulfill any request you make using all the tools at its disposal, and given my hands-on experience with it at a Google event, that includes generative AI. It might seem obvious that it would use AI to, say, “add a Mercedes behind me in this selfie,” but I can imagine a less tech-savvy user assuming they could ask the AI to “zoom out,” not knowing that changing the aspect ratio without cropping also requires the use of generative AI. Specifically, it requires asking the AI to imagine what might surround what’s in your photo in real life. Since it can’t know that, it runs a high risk of hallucination, no matter how unassuming “zoom out” sounds.
Since we’re talking about a tool designed to help less technical users, I’m concerned that there’s a good chance they might accidentally engage in fictional creation and think it’s a completely innocent, realistic shot.
Camera Trainer
Then there’s Camera Coach . This feature also brings AI to the Camera app, but it doesn’t add it directly to your photos. Instead, the AI suggests alternative framing and angles for whatever your camera sees, and tells you how to take those shots.
In other words, it’s a “what you see is what you get” approach. Camera Coach’s recommendations are just ideas, and while they’ll take more effort to implement, you can be sure that any shot you take will look exactly like it does in the viewfinder, without any artificial intelligence.
This almost immediately allays most of my concerns about unrealistic photos being passed off as the absolute truth. Camera Coach might suggest a shot that’s actually impossible to take, like if it wants you to enter a restricted area, but the worst you’ll get is disappointment, not a photo that passes off AI generation as the same as, say, zooming.
People need to know when they are using AI
I’m not going to solve the “what is a photograph?” question in a day. The point is that some photographs are meant to represent the real world, and some are meant to simply be aesthetically pleasing. I get it. If AI can make a photograph more visually appealing, even if it’s not entirely realistic, I can see the appeal. That doesn’t address potential ethical questions about where the training data is coming from, so I’d still ask you to be careful with these tools. But I do know that pointing to a photograph and saying “that never happened” is not a rhetorical panacea.
I’m concerned about how casually Google is rolling out new AI features, as if they’re identical to traditional computational photography, which still always uses your actual image as a basis, rather than a fictional one. As someone who’s still wary of AI , I’m immediately wary of AI-powered image generation disguised as “100x zoom.” Not everyone pays as much attention to these tools as I do, and it’s reasonable to expect these features to work as advertised, rather than risk hallucinations .
In other words, users need to know when their photos are using AI to be confident that their images are realistic and when they aren’t. Calling a telephoto zoom “5x zoom” or calling a zoom that overlays AI on many pixels “100x zoom” isn’t the same as calling the Photos app’s built-in natural language editor not clearly indicating when generative AI is being used and when it isn’t.
Google is aware of this problem. All photos taken on the Pixel 10 now have built-in C2PA credentials that can tell if AI was used in the photo’s metadata. But when was the last time you actually checked a photo’s metadata? Tools like Request an Edit are clearly designed with foolproof security in mind, and expecting users to manually review every photo to see which ones have been edited by AI and which haven’t is unrealistic, especially when we’re building tools that are specifically designed to help users reduce the number of steps before they get to the final photo.
It’s natural to expect AI to be used when the Gemini app launches, but its inclusion in previously AI-less tools like the Camera app requires more fanfare than quiet C2PA accounts and one vague sentence in a press release . Notifying the user of the intent to use AI should happen before they take a photo or edit it. Information shouldn’t be subtly tagged so that the user can find it later if they decide to look for it.
Other AI-powered photo tools, like Adobe’s, already do this, adding a simple watermark to any AI-generated project. While I won’t tell you how to feel about AI-generated photos in general, I will say that you shouldn’t find yourself creating them by accident. Of all Google’s AI camera innovations, I’d say Camera Coach is the only one that does this. For a big new product from the creator of Android, an ecosystem that Google proudly touted as “open” at this year ’s Made by Google conference, one of three transparency metrics isn’t what I expected.