Apple Has a New Open-Source AI Image Editor
Apple is a bit behind in the generative AI space, with the exception of some small features added in iOS 17. However, 2024 is shaping up to be a big AI year for Apple. All eyes are on iOS 18 , which should be equipped with artificial intelligence features, including an updated Siri.
In anticipation of this release, Apple researchers, in collaboration with the University of California, Santa Barbara, unveiled an open-source artificial intelligence model that understands natural language instructions. In short, you ask the AI to do something to change a photo, and it will do it.
What is Apple MGIE AI Image Editor?
Called “MGIE” (MLLM-Guided Image Editing), this new AI model takes standard user commands to achieve three different editing goals: “Photoshop-style modification, global photo optimization, and local editing.”
Photoshop-style modification includes actions such as cropping, rotating, and changing the background; global photo optimization includes adjusting effects for the entire image, including brightness, contrast or sharpness of the image; while local editing affects specific areas of the image such as its shape, size and color.
MGIE is mainly based on MLLM (Multimodal Large Language Model), which is a kind of LLM capable of interpreting visuals and audio in addition to text. In this case, MLLM is used to accept user commands and interpret them as the correct editing direction. The MGIE research paper explains that this is a traditionally difficult task because user commands can often be too vague for the system to understand correctly without additional context. (What does the program think “make pizza healthier” mean?) But the researchers say MLLMs like MGIE are effective here.
According to the research paper, MGIE is capable of performing many different types of visual editing. You can ask him to add lightning to a picture of a body of water and make the water reflect that lightning; remove an object in the background of the image, such as a person who has been unintentionally photobombed; turning some things into other things, such as a plate of donuts into pizza; increase focus on a blurry object; remove text from a beautiful photo, as well as many other features.
You can get an idea of how this technology will work by reading the full research paper, which includes examples of the editor’s work; it is available here .
Of course, this isn’t the first time AI has been used in photo editing. Photoshop has had a variety of AI editing tools for some time now , including those built based on user suggestions. But MGIE is perhaps the most realized command-based AI image editor concept yet.
How to try out Apple’s image editor MGIE for yourself
Since the model is open source, anyone can download and integrate it with their own tools. However, if you’re like me and don’t know where to start, you might want to try this demo hosted by one of the project’s researchers. You can upload the image you want to edit, enter the command, and then process it.
However, there is currently a fairly large request queue in the demo version. I’m currently one of 237, and I think that number may continue to grow as more people want to try this model.
It’s unclear if or how Apple will integrate MGIE into its own platforms. But if the company had a year to do this, it would definitely be 2024.