reader comments
33 with 0 posters participating
On Wednesday, a San Francisco-based developer named Liu Liu released Draw Things: AI Generation, a free app available in the App Store that lets iPhone owners run the popular Stable Diffusion AI image generator. Type in a description, and the app generates an image within several minutes. It’s a notable step toward bringing image synthesis to a wider audience—with the added privacy of running it on your own hardware.
Introduced in August, Stable Diffusion (SD) is an AI image generator model that creates novel images from text descriptions (called “prompts”). Typically, people run SD through the commercial DreamStudio service, on a remote cloud machine with rented compute time, or locally on a PC using a custom open source implementation. When running locally, SD requires a fairly beefy GPU to generate images quickly, but some developers optimized the model to run on older GPUs with less VRAM (if you don’t mind waiting longer to see results).
In the same vein, Liu Liu has managed to optimize Stable Diffusion to run on the iPhone, a somewhat difficult process that the developer described in a blog post. “The main challenge is to run the app on the 6GiB RAM iPhone devices,” Liu Liu writes. “6GiB sounds a lot, but iOS will start to kill your app if you use more than 2.8GiB on a 6GiB device, and more than 2GiB on a 4GiB device.”
Upon first running Draw Things, the app downloads several necessary files—including the Stable Diffusion 1.4 model—to your iPhone. To use it, type in a prompt at the top of the screen, then tap “Generate.” Between generating images, tap the number in the top center of the screen to randomize the seed, which is a number that partially guides the generation of the image.
On our iPhone 11 Pro, generating a 384×384 image took a little over two minutes. It’s faster on an iPhone 14 Pro, according to Liu Liu, generating an image in about a minute. Either way, SD is computationally intensive. After successive generations, our iPhone got notably warm to the touch.
It’s worth noting that with Stable Diffusion, 384×384 images often generate relatively poor, low-detail results because SD’s creators trained the model using 512×512 images. When attempting to generate a 512×512 image on our iPhone 11 Pro, we received a warning and proceeded anyway, but the app crashed to a black screen.
In addition to regular image generation duties, Draw Things also supports inpainting, which lets you replace a portion of an image with AI-generated imagery, and loading extra image synthesis models such as the unauthorized “Modern Disney Diffusion” model (that generates Disney-looking characters) and the anime-powered “Waifu Diffusion” model (we tested, and it is possible to generate NSFW material using the app, so be warned). The inclusion of those capabilities means that Draw Things might not remain in the App Store for long if it becomes popular, since the content it generates may violate Apple’s terms of service.
Whatever its fate, Draw Things feels like an important proof-of-concept, showing that Stable Diffusion can run locally on the iPhone, even if slowly. If the app sticks around—perhaps with some filters or modifications—Liu Liu describes room for potential future optimizations that might speed things up: “I probably left 50% performance on the table still.”
This isn’t the first time Stable Diffusion has run on an iPhone. In September, developer Matt Waller ran Stable Diffusion locally on his iPhone XS, but he did not provide an App Store app that would allow others to replicate the feat. We’ll likely see more local AI image synthesis on smartphones as the devices continue to increase in computational power.
Go to discussion…
