Local Diffusion is a Stable Diffusion based image generator and interpolator running locally and native inside AE.
Since the ai is very performance hungry, please test on your system. You’ll preferably have a NVidia GPU of the 2000 series or newer (>=6GB VRAM, >=10GB for XL) or an M1/M2 Mac running Ventura with 32GB RAM or more.
Local Diffusion will run on other configurations too (including cpu only) but might be very slow (several minutes per image).
Features
New: Stable Diffusion XL
We now integrate the latest state-of-the-art image model! Use SDXL for unmatched fidelity, details and photorealism.
Versatile prompting
We have implemented a simple and effective prompting mechanism. Prompts are entered via mask names and can be weighted by the masks opacity.
Interpolate prompts
Keyframe the prompts weight to interpolate between objects and find interesting concepts.
Guide creation by source layer
The Source+Text mode allows you to specify a layer which guides the images creation. This way prompting and interpolation can be applied to your own footage.
<- “industrial robot” to “person made of clay”
Full control over generation exposed!
You get full control over the model used, the aspect ratio, guidance scale and the seed. This allows for fast iterations and experimentation.
Easy to use and performant
This is a native port into our ai framework and therefore easy to install and use. Local Diffusion makes use of some state-of-the-art performance optimizations and produces images fast!
Integrated Superresolution
We ship an integrated superresolution ai with the plugin. Up to 3072×3072 pixel output!
Negative prompts
Simply add a “-” in front of the text to use it as negative prompt.
More features to come…
While you play with Local Diffusion, we are already working hard on integrating more features 🙂
Tutorials & Documentation
Additional Infos
Compatibility
It runs on AE 2022 and above. Please test your configuration using the free trial before purchasing.
Limitations
GPU mode may be not as stable as cpu mode.
On Windows an Nvidia graphics card is needed for gpu version of the plugin.
Trial version
The trial is fully functional with a blue cross watermark which is removed once a valid license is entered.
Credits
Based on research and implementation of Stability Ai.
Version History
1.6.4 (Current version) – Oct 17, 2023
– fix another library loading bug
1.6.1 – Oct 16, 2023
– fix plugin initialization on non-nvidia systems
1.6.0 – Oct 9, 2023
– update underlying ai libraries for better performance and stability
1.5.6 – Oct 5, 2023
– fix installer bug which lead to missing dependency on some systems
1.5.5 – Oct 5, 2023
– make installer compatible with upcoming AE 2024
1.5.4 – Sep 21, 2023
– increase logging verbosity during plugin loading
1.5.3 – Sep 19, 2023
-better initialization logging
1.5.2 – Sep 12, 2023
– add stable diffusion xl model
1.4.5 – Aug 30, 2023
– fix plugin initialization on some systems
1.4.3 – Aug 22, 2023
-bug fixes
1.4.3 – Aug 22, 2023
-bug fixes
1.3.2 – Jul 7, 2023
-bug fixes
1.3.0 – May 30, 2023
– added new scheduler “Multistep DPM-Solver” which gives good results with much lower steps
– fixed bug in base model, This may lead to slightly different outputs than in the last version and therefore breaks backwards compatibility for the “base” model
1.2.0 – May 23, 2023
– support for negative prompts. Simply append a “-” before the prompt and it will be used for negative conditioning.
1.1.0 – May 5, 2023
– add “source + text” mode. this allows for image generation based on input layer and (interpolated) prompt
1.0.0 – May 5, 2023
Initial release
1.6.1 (Current version) – Oct 16, 2023
– performance improvements
1.6.0 – Oct 9, 2023
– update underlying ai libraries for better performance and stability
1.5.6 – Oct 5, 2023
– fix installer bug which lead to missing dependency on some systems
1.5.5 – Oct 5, 2023
– make installer compatible with upcoming AE 2024
1.5.4 – Sep 21, 2023
– increase logging verbosity during plugin loading
1.5.3 – Sep 19, 2023
-better initialization logging
1.5.2 – Sep 12, 2023
– add stable diffusion xl model
1.4.9 – Sep 1, 2023
– improve loading time on MacOS Big Sur
1.4.6 – Sep 1, 2023
– fix initialization bug on MacOS Big Sur
1.4.3 – Aug 21, 2023
– bug fixes
1.4.3 – Aug 21, 2023
– bug fixes
1.3.0 – May 30, 2023
– added new scheduler “Multistep DPM-Solver” which gives good results with much lower steps
– fixed bug in base model, This may lead to slightly different outputs than in the last version and therefore breaks backwards compatibility for the “base” model
1.2.0 – May 23, 2023
– support for negative prompts. Simply append a “-” before the prompt and it will be used for negative conditioning.
1.1.0 – May 5, 2023
– add “source + text” mode. this allows for image generation based on input layer and (interpolated) prompt
– improve installer
1.0.0 – May 5, 2023
Initial release