Image generative AI - FOSS only (hackmas2024)

Try out open source image generation - and learn how to control AI magic. No prior knowlege needed. Bring ideas on what you want to generate.First theory / terminology you'll need for the toolsPropriety tools and why they suckThe 3 major base-models SD1.5, SDXL and FLUX.1/SD3 - diffusion vs flow modelsELI5: latent spaces, u-nets, [un]CLIP, VAE, cfg-scale, samplers and schedulersThen I'll show you how to use the 2 major web UIs:The beginner tool a1111 stable diffusion web uiThe pro tool comfyuiFast view over other tools that I haven't triedThen I'll explain what you can customize - with example workflows in comfyui Models: LoRAs - low rank adapters - diff/patches to base models for generation guidance, i.e. pose, canny edge, styles, concepts, etc. IPAdapter: Transfer styles, faces and do per-layer-prompting Hyper/turbo models: Generate 20 1kx1k images per second on a 4090 Conditioning: unCLIP use pictures as prompt. Latent space walks + Different CLIPs for different layersInitialization: Start from a (masked) image - inpainting, outpainting, overpainting, refining, transformingSpecial models for upscaling (SUPIR) and animation (AnimateDiff)Bring a laptop / tablet / phone if you want to try it. I'll rent a 3090/4090 and provide you a beginner setup for at least 20h during/after the talk.A matrix channel / mastodon hashtag for AI art would be cool. Maybe we can do a contest / exhibition?Slides: https://schickmas.at/2dd2f1cbd7c3 about this event: https://sessions.hack-mas.at/event/w4DZtGfeNK86AGbhr/image-generative-ai-foss-only