Our Discord : Ultimate guide to the LoRA training. If I have been of assistance to you and you would like to show your support for my work, please consider becoming a patron on 🥰
Playlist of Stable Diffusion Tutorials, #Automatic1111 and Google Colab Guides, DreamBooth, Textual Inversion / Embedding, #LoRA, AI Upscaling, Pix2Pix, Img2Img:
Welcome to the ultimate beginner’s guide to training with #StableDiffusion models using Automatic1111 Web UI. In this video, we will walk you through the entire process of setting up and training a Stable Diffusion model, from installing the LoRA extension to preparing your training set and tuning your training parameters. We’ll also cover advanced training options and show you how to generate new images using your trained model. By the end of this video, you’ll have a solid understanding of how to use Stable Diffusion to train your own custom models and generate high-quality images.
You should watch these two videos prior to this one if you don’t have sufficient knowledge about Stable Diffusion or Automatic1111 Web UI:
1 – Easiest Way to Install & Run Stable Diffusion Web UI on PC by Using Open Source Automatic Installer –
2 – How to Use SD 2.1 & Custom Models on Google Colab for Training with Dreambooth & Image Generation –
0:00 Introduction speech
1:07 How to install the LoRA extension to the Stable Diffusion Web UI
2:36 Preparation of training set images by properly sized cropping
2:54 How to crop images using Paint .NET, an open-source image editing software
5:02 What is Low-Rank Adaptation (LoRA)
5:35 Starting preparation for training using the DreamBooth tab – LoRA
6:50 Explanation of all training parameters, settings, and options
8:27 How many training steps equal one epoch
9:09 Save checkpoints frequency
9:48 Save a preview of training images after certain steps or epochs
10:04 What is batch size in training settings
11:56 Where to set LoRA training in SD Web UI
13:45 Explanation of Concepts tab in training section of SD Web UI
14:00 How to set the path for training images
14:28 Classification Dataset Directory
15:22 Training prompt – how to set what to teach the model
15:55 What is Class and Sample Image Prompt in SD training
17:57 What is Image Generation settings and why we need classification image generation in SD training
19:40 Starting the training process
21:03 How and why to tune your Class Prompt (generating generic training images)
22:39 Why we generate regularization generic images by class prompt
23:27 Recap of the setting up process for training parameters, options, and settings
29:23 How much GPU, CPU, and RAM the class regularization image generation uses
29:57 Training process starts after class image generation completed
30:04 Displaying the generated class regularization images folder for SD 2.1
30:31 The speed of the training process – how many seconds per iteration on an RTX 3060 GPU
31:19 Where LoRA training checkpoints (weights) are saved
32:36 Where training preview images are saved and our first training preview image
33:10 When we will decide to stop training
34:09 How to resume training after training has crashed or you close it down
36:49 Lifetime vs. session training steps
37:54 After 30 epochs, resembling images start to appear in the preview folder
38:19 The command line printed messages are incorrect in some cases
39:05 Training step speed, a certain number of seconds per iteration (IT)
39:44 How I’m picking a checkpoint to generate a full model .ckpt file
40:23 How to generate a full model .ckpt file from a LoRA checkpoint .pt file
41:17 Generated/saved file name is incorrect, but it is generated from the correct selected .pt file
42:01 Doing inference (generating new images) using the text2img tab with our newly trained and generated model
42:47 The results of SD 2.1 Version 768 pixel model after training with the LoRA method and teaching a human face
44:38 Setting up the training parameters/options for SD version 1.5 this time
48:35 Re-generating class regularization images since SD 1.5 uses 512 pixel resolution
49:11 Displaying the generated class regularization images folder for SD 1.5
50:16 Training of Stable Diffusion 1.5 using the LoRA methodology and teaching a face has been completed and the results are displayed
51:09 The inference (text2img) results with SD 1.5 training
51:19 You have to do more inference with LoRA since it has less precision than DreamBooth
51:39 How to give more attention/emphasis to certain keywords in the SD Web UI
52:51 How to generate more than 100 images
54:46 How to check PNG info to see used prompts and settings
55:24 How to upscale using AI models
56:12 Fixing face image quality, especially eyes, with GFPGAN visibility
56:32 How to batch post-process
57:00 Where batch-generated images are saved
New how to install DreamBooth : https://youtu.be/pom3nQejaTs – Discord : https://bit.ly/SECoursesDiscord – Patreon : https://www.patreon.com/SECourses
Stable Diffusion Playlist (16+ Tutorial / Guide Videos) : https://www.youtube.com/playlist?list=PL_pbwdIyffsmclLl0O144nQRnezKlNdx3
Please join discord, mention me and ask me any questions. Thank you for like, subscribe, share and Patreon support. I am open to private consulting with Patreon subscription.
Prompts used to generate thumbnail photo and others on SD 1.5
portrait photo of prompt_instance_kw:1.2, photo of prompt_instance_kw,8k,hdr,smooth,sharp focus,cinematic lighting,intricate, highly detailed,fantasy, illustration, intricate, octane, digital painting, artstation, concept art,illustration, vibrant colors, cinematic
Negative
Negative prompt: bad anatomy, bad proportions, blurry, cloned face, deformed, disfigured, duplicate, extra arms, extra fingers, extra limbs, extra legs, fused fingers, gross proportions, long neck, malformed limbs, missing arms, missing legs, mutated hands, mutation, mutilated, morbid, out of frame, poorly drawn hands, poorly drawn face, too many fingers, ugly, ugly, duplicate, morbid, mutilated, out of frame, extra fingers, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, ugly, blurry, bad anatomy, bad proportions, extra limbs, cloned face, disfigured, out of frame, ugly, extra limbs, bad anatomy, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, mutated hands, fused fingers, too many fingers, long neck
Steps: 25, Sampler: Euler a, CFG scale: 8.5, Seed: 2430164950, Size: 512×512, Model hash: e02601f3
Prompts used to generate classification images on SD 1.5
face photo of a man, 8k, hd, smooth, sharp focus, Cinematography
Negative same as above
Prompts used to generate classification images on SD 2.1
face photo of a man, hdr, 8k, sharp
Negative same as above
Can i use the lora trained model for img2img afterwards?
how to train/insert/inject own photo of clothes,costume,dress
What is the difference from this system to the one where to use a LORA we need to type in the prompt <lora:…..:1.0> and a keyword? I believe that in the other way we can use a different model then just apply the LORA to interact with the model we are using. How to create that type of LORA? Thanks
Can you make a new video based on the latest version of DreamBooth? Time to time it is difficult to follow the steps since they changed several things
Hi , what model are you using for training, can i train LoRa on any model of my choice or is it supposed to be done on specific models. Thanks for amazing content
hello, please tell me, if I managed to run and use locally stable diffusion webui on an AMD processor, can I train LORA?
Who is this handsome devil?
Good outcomes at the end and BIRME was a great time saver – thank you! Video a bit lengthy in the middle with fiddling around with SD2.1
your ads is waaaaaaaay to aggressive, watched 2 adds, scrolled the video and got 2 more ads, scrollet a littlebit more and got ads right after… wtf. but I like your content!
Hello. stable diffusion started to get a bit big in file size and started to leave me with little space on my C drive and would like to copy it in another drive. Also once i copy it i will delete the original in C drive will that affect the program in any way?
everything is cool. but I have dissimilar people at the exit after training.I can't understand how to be..help me what am I doing wrong?
A lot of the important settings are now gone, like setting the amount of epochs to make,
nice turkish accent bro 🙂
I run Stable diffusion in Colab, is there any way to create LORA through Colab?
I don't get the dreambooth tab
When I install Dreamboots and applied it to SD after restart I don't have the Dreamboots button :/
amazing!
Wow, you saved me again. Thank you so much!!
Teşekkürler
Very helpful Videos!
I ask first and then TLDR you 🙂
Should I build with as many pictures as I can get my hands on, or should I select, let's say 100-200,
and then bite into the sour apple and go through all the .txt Files and explain everything in detail to the Ai?
The same question goes for the Classification Pictures. How important are they? Should I have very similar Pictures to the ones I want to train,
or should I throw a wide/wild collection at it? What is getting "classified" in that process?
And how intelligent is the Ai? When I look at the .txt Files, it is not that impressive.
Is the Ai baked into the "Web Gui", or is it in the Model Files?
I mean, will a prompt for Depth of Field or something universal like that bring the same effect no matter what Checkpoint I use?
I had big problems with creating a Model because of incompatibilities from versions of python, torch… and the worst was Xformers.
In the end, you helped me figure most of it out 🙂
But the real deal was deleting the "venv" Folder and letting it rebuild (what I already did a week ago when it refused to launch. I saw that folder pop up at first launch and it was suspicious from then on 😀 But just a few Days passed by and I had already forgotten 🙂
You should mention that from time to time I guess. Otherwise, you did a very good job explaining everything. I would say that I am over 90% satisfied.
That is very good! Stay on the track you are on.
Thank you. Thank you.
you never explained how to use this as a lora. The point of a lora is to append it to another base model. If you use the lora AS the checkpoint model you basically wasted your time.
Lora is ONLY useful if its a trigger weight in your prompt inside of a different checkpoint. Thats why they exist. (not for ram saving) and extracting a lora from what you did, wont give you any desired results I promise.
I get this error when I click train:
RuntimeError: Input type (struct c10::Half) and bias type (float) should be the same
Hello Dr., how would i make LORA out of my already trained models(ckpt) or diffuser files. Any tutorial on this ? Pl help.
@Software Engineering Courses – SE Courses Amazing video Congrats! quick ques- I'm now training, I'm getting the images at the classification folder, however, I'm not getting anything in the Sample folder of the module, I don't even have that folder inside my module. Clues?
Lora???? i only see dreambooth? did i miss something??
macos can’t use
4:00 The "About" section of Birme websites claims the whole thing is done in static javascript client side in your browser and nothing is sent server side thus no data is collected, so in theory you could just save the Birme webpage to your PC and run it all locally.
Thanks. I can't find the exact link to extension wiki. When paste the link , I get a JASON error. Can you post the link so I can see all available extensions? Thank you.
I think it would be great if you could do a video on the big differences between things like huggingface diffusers, dreambooth, textual inversion, LoRA, and how they might be related. LoRA always seems to get grouped with Dreambooth. I have a hard time understanding where the lines are drawn, and how these things are related (or not).
i got this error "Exception training model: 'expected Tensor as element 0 in argument 0, but got NoneType'." trying to training using LORA settings. using model: SD15NewVAEpruned.ckpt as a base model. anyone knows what is this error about?
Hello, me again. I saw that when creating Loras with the kohya-ss you get better results than when using dreambooth, could you make a video about this? Because I try to watch videos from other creators, but none are as organized and detailed as you. So I can't learn from them. 🙁
thank you. answered all of my questions. straightforward
Straight from the Automatic 1111 wiki as of Feb.06/23 "Currently, Lora networks for Stable Diffusion 2.0+ models are not supported by Web UI." Keep up the good work, I enjoy different views from different people. If we read enough we will learn.
34:55 if you hit enter while you have the terminal window open, it will resume the window. you have text selected and that freezes the command window.
So after all of that I'm left with a .pt file that is small (but doesn't work), or a ckpt that works but is a couple of GB. That was an hour wasted. I wanted to learn the kind of LoRa that they upload at Civitai. They are generally around 144 mb and look amazing. BUT HOW ARE THEY GENERATED!?
How do you make the LORAs smaller in size? Alot of the ones I see for download around the web are only a few hundred megsbytes at most, and anything I create is about the same size as the original cpkt
Abi Türk müsün? Video için teşekkürler.
The Dreambooth > Generate > Generate sample images looks ok and has some resemblance, but when I use the lora or the safetensors file as a model in txt2img and using <lora:NameOfPerson_2400:0.8>, the results are nothing like the samples generated in dreambooth… anything I'm missing?
Seems the dreambooth extension is broken beyond all repair for 2.x SD as we are all getting the same errors and the dev is "sick of the fucking shit". I really wanted to do LoRA but there are just no tools out there that work, and/or have any help vids like yours.
Its necessary that all the images (to training) are the equal size?
My friend, I just wanted to say: Your Videos are brilliant. It's hard to find tutorials that cover everything so detailed and easy as yours. Big thanks for your work! Question: Could you make a general introduction Video about how to use Automatik1111 WebUI? I think many of your viewers (including me) would love to see such a video, where you cover the basics. After that we can watch you videos on how to train SD or other advanced topics. Thanks a lot!
Maybe you can help me cause I'm using pretty much all of the same settings you are with my dataset and I keep getting out of memory errors with lora even though I have 8GB and it shows that you're only using 4.9GB of Vram
What kind of prompt do you use to create images for a style? Anyone have some good ideas for classification image prompts?
"probably possible to train on multiple gpu". I've one 3090 in my watercooled loop… Guess I will be adding the Titan X Pascal back in for faster /h lora training…
Güzel anlatım knk tşk🌹
With my 4090 tha maximum It/s is 1.80 there is any configuration to optimaze this?
Hi thank you for the video — this was great!
Is it possible to train a face and a different style at the same time? or train them separately and use them in the same prompt?
Thank you