Apple launches new AI for images: What makes it special

From UniGen to UniGen‑1.5

In May of last year, Apple’s team published a study titled UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation. It introduced the first unified multimodal large language model that combines image understanding and generation without separating these tasks into different systems.

Now, Apple has released a follow-up study detailing UniGen‑1.5.

What’s new in UniGen‑1.5

UniGen‑1.5 expands the original model’s capabilities by adding image editing features, while maintaining a single architecture for understanding, generating, and editing images.

Creating such a universal system is a complex challenge, as image understanding and generation require different approaches. However, researchers claim that a unified model can leverage its understanding abilities to improve image generation.

UniGen‑1.5 has human-level image recognition and editing skills

One of the key challenges in image editing is that models often struggle to correctly interpret complex instructions, especially when the changes are subtle or highly specific.

To address this, UniGen‑1.5 introduces a new stage called Edit Instruction Alignment. In this step, researchers train the model to generate a detailed textual description of how the edited image should look. This intermediate process helps the model better understand the task before producing the final output.

UniGen‑1.5 can better interpret complex instructions

Unified reward system

A key advancement in UniGen‑1.5 is the use of a single reward system for both image generation and editing. Previously, this was a challenge, as editing can involve anything from minor adjustments to full-scale transformations.

Limitations
However, researchers note that UniGen‑1.5 still faces difficulties with text generation and maintaining object identity:

• The model does not always render text accurately on images due to limitations of the lightweight detokenizer.

• There can be noticeable changes in object details, such as the texture of a cat’s fur or the color of a bird’s feathers.

Researchers emphasize that the model requires further refinement to address these limitations.

You may also be interested in:

• Apple is developing 8 new iPhones

• Apple set to launch anniversary iPhone