Pika combines multiple images and an audio file into editable videoBrief · digest for November 7
Brief≈ 1 min

Pika combines multiple images and an audio file into editable video

The team behind the content generation platform Pika Labs has rolled out an update that dramatically expands control over the output. Instead of the usual lottery with text prompts or assembling a clip from a single image, users now get a full-fledged multimodal builder.

Now you can simultaneously upload up to five different reference images and a separate audio track in uncompressed WAV format. A multimodal Diffusion Transformer (DiT) aligns visual elements with the audio spectrogram through cross-attention layers, matching virtual camera angle shifts to the rhythm and intonations of speech.

The main practical highlight of the update is the built-in targeted inpainting tool. If you don't like a character's hand movement or the color of a car in the generated clip, there's no need to regenerate the whole video: just highlight the problem area with your cursor and rewrite the action on the fly. Overall background lighting, perspective, and audio sync remain completely untouched.

Synthesizing a five-second high-resolution clip on Pika's cloud cluster takes 30 to 50 seconds. For ad creators, this opens up the ability to assemble scenes while accurately preserving a hero's appearance from different angles under pre-recorded voice acting.

Want a deep diveVotes help us choose what to break down in detail. This material has 0 votes.
Sources · 3