Discover step-by-step how to use ComfyUI for create amazing short videos with this video creation workflow.
Workflow : https://sneakyrobot.org/workflow/sneaky-robot-script-to-video-v1/
Buy me a coffee? https://buymeacoffee.com/sneakyrobot
facexlib -
python.exe -m pip install --use-pep517 facexlib
insightface-
python.exe "path to insightface" onnxruntime.
Comfyui: https://github.com/comfyanonymous/ComfyUI
Models: LTXV
NB=You can use either 1 of the models but I will advise Unet for 8GB Vram And the one with baked in VAE for Higher spec GPU's
LTXV Unet-
https://huggingface.co/Lightricks/LTX-Video/blob/main/unet/unet_diffusion_pytorch_model.safetensors
(Goes into the Models\Unet Folder)
LTXV VAE-
https://huggingface.co/Lightricks/LTX-Video/blob/main/vae/vae_diffusion_pytorch_model.safetensors
(Goes into the Models\VAE folder)
LTXV With baked VAE
https://huggingface.co/Lightricks/LTX-Video/blob/main/ltx-video-2b-v0.9.safetensors
(Goes into Models\Checkpoint Folder)
You can use the same clip model you use with flux.
Models: flux
flux 1 dev q4 -
https://huggingface.co/city96/FLUX.1-dev-gguf/blob/main/flux1-dev-Q4_0.gguf
t5xxl gguf-
https://huggingface.co/city96/t5-v1_1-xxl-encoder-gguf/blob/main/t5-v1_1-xxl-encoder-Q4_K_M.gguf
Clip L-
https://huggingface.co/black-forest-labs/FLUX.1-schnell/blob/main/ae.safetensors
VAE-
https://huggingface.co/black-forest-labs/FLUX.1-schnell/blob/main/ae.safetensors
ControlNet fp8-
https://huggingface.co/Kijai/flux-fp8/blob/main/flux_shakker_labs_union_pro-fp8_e4m3fn.safetensors
PuLID FLUX-
https://huggingface.co/guozinan/PuLID/blob/main/pulid_flux_v0.9.0.safetensors?download=true
Hyper lora-
https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-FLUX.1-dev-8steps-lora.safetensors
Models: SDXL
wild card-
https://civitai.com/models/293331/wildcardx-xl-turbo
controlnet-
SDXL CONTROLNET: https://huggingface.co/thibaud/controlnet-openpose-sdxl-1.0/tree/main
PulID-
https://huggingface.co/huchenlei/ipadapter_pulid/blob/main/ip-adapter_pulid_sdxl_fp16.safetensors
Extra Models---
Insightface:
https://github.com/Gourieff/Assets/tree/main/Insightface
Anthelope V2: https://huggingface.co/MonsterMMORPG/tools/blob/main/antelopev2.zip
Stable audio:
Checkpoint-
https://huggingface.co/stabilityai/stable-audio-open-1.0/blob/main/model.safetensors
F5 TTS: the voice samples should not be more than 15 seconds.
Whisper Huggin face space: For transcribing voice samples
https://huggingface.co/spaces/hf-audio/whisper-large-v3-turbo
How to Install Ollama:
https://www.youtube.com/watch?v=e3j1a2PKw1k&list=TLPQMDMxMjIwMjQt0gwpmA8D4Q&index=2
Prompts:
https://docs.google.com/document/d/1M4VnVn40YhnU_SU-i79ZKqjfhWlzwN02Wh0KwsTaBAU/edit?usp=sharing
------Contents of this video------------
0:00 intro
1:58 Create Story, character, and scene prompts
3:52 Create Character Dataset
8:01 Consistent Character Lora trainer
11:37 Consistant Scene Creation
13:38 Soryboard
13:51 Second method Consistant Scene Creation
16:00 Video Creation
19:07 Sound Effect Creation
20:56 Text To Speach Creation
23:41 Final Video Result
25:19 Install comfy Ui
26:05 Install PulId requirements
27:38 Install Models