This AI Will Draw Whatever You Tell It To.
The technology is amazing, but let’s be honest: it gets a little boring sometimes. Computers and smartphones change gradually from year to year, software moves to a subscription, and innovation in general is not as common. However, from time to time new technical developments are actually surprising and exciting. That’s how I feel about this new AI that will draw almost anything you throw at it.
The software, called DALL·E 2 , developed by OpenAI, uses machine learning to generate new images based on a user-provided text description. You can see examples of how this works on the website: The developers allow you to mix and match different items, actions and settings to see how DALL·E 2 creates different images from these descriptions.
For example, “astronaut on horseback in photorealistic style” generates images of just that, resulting in different “photos” of an astronaut on horseback, similar to MTV promo pictures. Change the “photorealistic style” to “pencil drawing” and all of a sudden you’ll see a few sketches of this astronaut on horseback, only no one has drawn anything – it’s completely created by artificial intelligence.
However, it goes way beyond these impressive tech demos. From the tests I’ve seen, AI can take the most detailed requests and produce images that you might see hanging in a modern art museum. Director Karen X Chang demonstrated this by first asking DALL E 2 to draw a girl walking up an endless staircase, which she did with ease, and then asking DALL E 2 to draw a girl walking up an endless staircase in clouds made of cookies, which he did, with ease .
How DALL·E 2 works
The project really started in January 2021 with DALL·E . The goal was the same as that of DALL·E 2: to create a program that could generate new images from text. Since then, OpenAI has figured out how to greatly improve image quality by reintroducing the software called DALL·E 2.
DALL·E 2 developers train the system using labeled images. DALL E 2 probably sees thousands of images of cars tagged “car”, houses tagged “home”, coffee drinks tagged “coffee”, etc. It uses this knowledge base to make the best guesses about what what you are requesting. If it’s trained enough to know every part of your description, you’ll get amazingly accurate images of things you’ve never seen before, like “tax-paying monkey with a funny hat.”
Many machine learning programs work this way, but one of the differences with DALL·E 2 is that it is able to “fill in the gaps,” as OpenAI puts it. Essentially, DALL·E 2 looks at your textual description and thinks about the context you didn’t provide. If you ask “a dog playing basketball outside”, it is assumed that you are referring to the basketball court that can be found around the park, which means that you should be able to see the trees, grass, sky, etc.
However, the DALL·E 2 team openly acknowledges that there are gaps in knowledge about the system and can lead to some interesting and insightful discoveries. If the system has not been trained on “howler monkeys”, asking to draw “howler monkeys” may result in images of screaming monkeys rather than the particular type of monkey you are looking for.
The DALL·E 2 application goes beyond creating new images. You can use this tool as an AI-powered photo editor by removing image elements you don’t like or adding new components that weren’t there before. It can also generate new images based on existing images: if you need to draw the Mona Lisa but you can’t draw, DALL·E 2 will do it as if you can.
The developers of DALL·E 2 are committed to protecting DALL·E 2 from misuse, including the creation of images based on hatred or violence. As such, the team has opened up the software to a limited number of “trusted testers” but will eventually invite more people to test DALL·E 2 on their own.
However, there is no clear timeline on this front, nor is there much of an explanation for what makes a “trusted tester”. If you’re interested, it seems like it’s worthsigning up for the DALL·E 2 waitlist right now (I signed up when I wrote this article). If you meet the OpenAI criteria, you may be asked to test DALL·E 2 yourself.