Animating with Stable Diffusion

While Stable Diffusion itself is a text-to-image model, that doesn't stop us from finding creative ways to use it to create animations! This is a quick page covering some of the top tools for Stable Diffusion animations as of April 2023 for the #DiffuseTogether competition.

Deforum - animate with prompts and camera movements

• Deforum Colab notebook - run in web browser with Google's GPU's
• Deforum automatic1111 extension - run locally in SD web ui
• Visions of Chaos - under Machine Learning > Image Generation > Text to Image 
• RunDiffusion.com - run SD web ui and Deforum in hosted environment
• Kaiber.ai - online tool to create animations based on Deforum
  
• Deforum Discord - ask for help, see what people are making

Stable WarpFusion - use videos as input, generated content sticks to video motion

• Subscribe to Sxela's Patreon to access the WarpFusion notebook 

Stability Animation API/SDK - use API key from DreamStudio to create animations using SDK, no GPU needed

• SDK documentation and install instructions
• Colab notebook using the API

Interpolations - smoothly blend between prompts and noise seeds

• Giffusion - nice webui inside Colab to create 
  
• Latent Blending - extra smooth transition between prompts
• Stable Diffusion Interpolation - interpolate seed, prompt pairs
• Stable Diffusion smooth interpolation - improvements on seed, prompt interpolation
• Stable Diffusion Videos - interpolation with audio reactivity

Other Animation Tools

• Digital Daydream - start with a prompt then let CLIP Interrogator day dream (example)
• Genmo - commercial AI animation web tool
• Infinite Zoom - auto1111 extension for zooms with outpainting
• Stable Diffusion KLMC2 Animation - quickly make animation from single prompt (example)
• Stable Diffusion KLMC2 Animation expanded - schedule prompts and lots more options
• Revel.xyz Animai - easily upload image and get back SD animation
• TemporalKit - auto1111 extension for video input with temporal coherence (example)
• Video Killed the Radio Star - create animations like this