custom fork vladmandic

[motorist828]

Hi I found this interesting thread https://github.com/vladmandic/automatic
Many are upset about the slowdown in development of AUTOMATIC1111
A person wants to bring people together to work faster and fix problems
Will we ever see directml for this version?

[OldPixelReaper]

interesting

[Aptronymist]

Hi I found this interesting thread https://github.com/vladmandic/automatic Many are upset about the slowdown in development of AUTOMATIC1111 A person wants to bring people together to work faster and fix problems Will we ever see directml for this version?

I communicated with vlad last week asking the same thing, he's taking a look at this project and perhaps SHARK, to see what's going to work best for AMD users last I heard, but he said he wanted to not hack it together. I'm looking forward to it, he knows how to get things done quick and he's putting in the effort!

[Miraihi]

Tried SHARK just yesterday, and it's surprisingly slower than DirectML, has less features and crashes my drivers as a bonus.

[motorist828]

I tried the SHARK, as I understand it, it does not work on Vega video cards :(

[Omniphantasm]

Tried SHARK just yesterday, and it's surprisingly slower than DirectML, has less features and crashes my drivers as a bonus.

I've never even been able to get it to create a single image.

POS Crashes me back to prompt as soon as it makes the VAE or whatever. Can't find a single other person with the issue on a 7900XTX so I just gave up and have decided its a bad program as is logical.

[Lumpsack]

Also throw NCNN into the ring for AMD support, I don't truly know how it compares, but ChainNer uses it and its really fast compared with the CPU

[Lumpsack]

NM, I found the thread and put the question there - thanks for highlighting it.

[lshqqytiger]

I wanted to create a new branch for this, but it seemed difficult. (due to checkout master)
I created a new repository and you can try it.

I recommend not to use webui.bat or webui.sh.
Activate your virtual env using python venv or anaconda and start webui with python launch.py.

New installation guide:
(TBA)

Memory optimization for AMDGPUs is disabled by default. You can enable it on Settings > Stable diffusion > Enable memory optimization for AMDGPUs. Only works for DirectML backend.

If black images are generated, enable no-half on Settings > CUDA Settings > Use full precision for model (--no-half) and Use full precision for VAE (--no-half-vae).

You can apply sub quadratic attention (--opt-sub-quad-attention) on Settings > Stable diffusion > Cross-attention optimization method.

[motorist828]

Do share your exact steps

Delete the venv folder
run CMD in vladmandic-automatic-directml directory
python -m venv venv
venv\Scripts\activate
python launch.py

After that I needed to reinstall torch

pip install torch==1.13.1 torch-directml --force-reinstall --ignore-installed
launch, it works
python launch.py --medvram

[Omniphantasm]

Nice, that did it for me. Thanks

Now is there a particular way I need to start and/or update this folder in the future so I don't horribly break everything again? 😆

Holy hell, it's so much faster now. 512x512 set of 8 only took 30 seconds, it's actually faster than my old A1111 setup was going before I borked it the other night.

[motorist828]

Looks like everything is broken today.
used directml-fix.bat before launch

[lshqqytiger]

It worked well before I push, but it was broken for me, too. I'm finding way to make it work again.

[lshqqytiger]

Now, setup seems to be work well.
But there is a same issue with vladmandic/sdnext#448

[Yodrik]

I tried this new fork. *1

I think I goofed up the venv thing, but I ran into "RuntimeError: ... There is not enough GPU video memory available!" at first.
Subsequently I ran on --medvram which works to generate 512x512, but I was unable to generate 600x800 regardless of the other options enabled.

I took 7 samples while trying to adjust settings to get to 600x800,
(RX 6600) 512x512 image times @ Steps: 10, Sampler: DPM++ 2M Karras

	vladmandic-automatic-directml	stable-diffusion-webui-directml
Time taken	48.85s	-
-	45.56s
-	40.15s
-	45.28s
-	45.07s
-	46.20s
-	44.88s
Avg.	~45s	~16s

Unfortunately right now it almost feels like the --lowvram speed on this fork.

*1 stable-diffusion-webui-directml borked (steps run, but no image generated) after the testing and I had to reinstall python + reinstall stable-diffusion-webui-directml to get it to work again.
I mean to do more adjusting and take image time data in this fork before playing around with the vladmandic fork again.

[vladmandic]

i love the effort here! @lshqqytiger any chance you'd be interested in contributing to main repo directly?
i'm more than open to merge anything thats needed, just need to make it conditional - so it works for standard as well as directml use-case.

[lshqqytiger]

Can I access a branch for directml implementation? I will compare this repo with yours, and then I will push the necessary parts. I'll work only on that branch. You can merge it into master whenever you want.

[vladmandic]

as far as i know, github doesn't have per-branch user-rights, but i don't mind creating a branch and making you a collaborator in general - with understanding that you won't push to master blindly :)

[lshqqytiger]

okay thank you :)

[vladmandic]

done. and i've created a branch for you: https://github.com/vladmandic/automatic/tree/directml

[vladmandic]

Dicord: vlado#8083

[lshqqytiger]

This moved to https://github.com/vladmandic/automatic/tree/directml.
I didn't test this branch yet, and I just moved some parts with comments.
I'll test it tomorrow.

[motorist828]

@Miraihi torch 2.0 gives me a very good boost on ROCm linux
So this is more of a problem with DIrectML and other libraries.

[Miraihi]

I tried running SD on Linux ROCm, but looks like RX 580 doesn't support it. What a shame.

[Yodrik]

A day or two with the merged commit and I sort of settled on the last combo here. SDP was really off and on, either OOM or success. In the benchmark it didn't work out at all though. I understand from the code the benchmark test is 512x512 batches, so it serves as a simple test after install. High res is another story still.

(RX6600 8GB)

Precision	-nohalf/-upcast	MO	-arg	benchmark score
Full	-nohalf	SubQ		0.22 / error / error
Full	-nohalf	SubQ	medvram	0.27 / 0.27 / 0.24
Full	-nohalf	SDP	medvram	error / error / error
Full	-nohalf	doggettx	medvram	0.3 / 0.27 / 0.25
Full	-nohalf	invokeai	medvram	error / error / error
Full	-upcast	SubQ	medvram	0.46 / 0.4 / 0.37

[Cwieku666]

with normal benchmark I get 0.63 / 0.53 / 0.55 with rx 470 4gb (Full + upcast + sub-quadratic with medvram) for example somebody with rtx 3060 scored 6.67 / 7.68 / 8.48

quick question how I can change branch to main for new updates when I installed it with git clone -b directml https://github.com/vladmandic/automatic

[lshqqytiger]

$ git checkout main

[vladmandic]

you've enabled setting in cuda params that is not compatibile (rollback vae).
i just did the update to check for cuda available before even checking that setting.
update repo and after ui starts, go to settings and disable it.

[OldPixelReaper]

Hi, if I want to test it under AMD I need to run
git clone https://github.com/vladmandic/automatic or
git clone https://github.com/vladmandic/automatic/tree/directml
? Thanks :-)

[Yodrik]

As of the latest commits DirectML support is now part of the master branch, so https://github.com/vladmandic/automatic

*DirectML to use AMD in Windows. This is alongside the existing ROCm to use AMD in Linux already available.

[OldPixelReaper]

thx :-)

[Lumpsack]

Great piece of news, thanks @Yodrik - I've just installed main and its working well under AMD using directml natively.
haha, now just to get ROCm into Windows!

[OldPixelReaper]

For anyone with AMD on Windows, does the fork still work with the latest updates? For me it seems to run only on the CPU after the git pull. Even a fresh install doesn't do anything. From 1.2 it/s to 24 it/s :(

AMD RX 5700 XT, Win 11

[OldPixelReaper]

Maybe this will help you a little bit. Fresh installation, only a few extensions turned off and so, that's all I did. As I said, about 2 weeks ago it ran great. With the fork of lshqqytiger it also runs. I just wait and see now^^

config.txt
setup.log
console-startup-debug.txt
DxDiag.txt

[howDoILearnPython]

Wow... let me try that fork too.

[vladmandic]

He did all the work to bring DirectML support here.
I'll check the arg parsing, best guess is thar it somehow it skips it even if it's specified and dected by launcher.

[howDoILearnPython]

tried fresh install just 30 mins ago, with --use-directml --medvram
there are info said it's using directML backend.
However, i tried txt2img, according to the speed and task manager, it's apparently still using CPU rather than GPU.
let me try again and report to here later.

[howDoILearnPython]

tried again and confirm all setting is correct.
inference will cause CPU 100% full load with system memory being used, 20s/it.
FYI, for lshqqytiger fork, GPU will be 100% full load with VRAM being used, 5s/it for my vega 8, 3200G APU.

not sure if this help, following the my console output.

C:\vladmandic>webui --use-directml --medvram --listen
Using VENV: C:\vladmandic\venv
12:24:35-124855 INFO Starting SD.Next
12:24:35-124855 INFO Python 3.10.9 on Windows
12:24:35-218782 INFO Version: a2485cf Fri May 12 21:12:24 2023 -0400
12:24:35-624871 INFO Setting environment tuning
12:24:35-624871 INFO Using DirectML Backend
12:24:38-171820 INFO Torch 2.0.0+cpu
12:24:38-171820 WARNING Torch reports CUDA not available
12:24:38-187447 INFO Verifying requirements
12:24:38-265578 INFO No changes detected: Quick launch active
12:24:38-281203 INFO Running extension preloading
12:24:38-281203 INFO Server arguments: ['--use-directml', '--medvram', '--listen']
No module 'xformers'. Proceeding without it.
12:24:47-361189 INFO Libraries loaded
12:24:47-376815 INFO Available models: C:\vladmandic\models\Stable-diffusion 1
ControlNet v1.1.152
ControlNet v1.1.152
Image Browser: ImageReward is not installed, cannot be used.
12:24:53-049091 INFO Loading UI theme: black-orange
Running on local URL: http://0.0.0.0:7860

To create a public link, set share=True in launch().
12:25:00-893391 INFO Local URL: http://localhost:7860/
12:25:00-893391 INFO API Docs: http://localhost:7860/docs
12:25:00-893391 INFO Initializing middleware
12:25:01-830958 WARNING Torch FP16 test failed: Forcing FP32 operations
12:25:01-846584 INFO Setting Torch parameters: dtype=torch.float32 vae=torch.float32
unet=torch.float32
Loading weights: C:\vladmandic\models\Stable-diffusion\beautypromix_v1.safetensors ------ 0.0/3… -:--:--
GB
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
12:25:02-893533 INFO Model created from config: C:\vladmandic\configs\v1-inference.yaml
12:25:03-456076 WARNING Scaled dot product cross attention is not available on CPU
12:25:03-456076 INFO Embeddings loaded: 0 []
12:25:03-471698 INFO Model loaded in 2.2s (load=1.2s create=0.5s apply=0.5s)
12:25:03-706089 INFO Model load finished: {'ram': {'used': 6.71, 'total': 23.94}}
12:25:04-081117 INFO Startup time: 25.8s (torch=5.6s gradio=2.4s libraries=1.1s codeformer=0.2s
scripts=5.1s opts onchange=0.3s ui=3.5s start=4.3s scripts
app_started_callback=0.1s checkpoint=3.1s)
12:25:04-096742 INFO Memory used: 4.72 total: 23.94

[OldPixelReaper]

Seems to be running again, big thanks!
Starting SD.Next
16:11:40-458829 INFO Python 3.10.11 on Windows
16:11:40-521334 INFO Version: 554f262 Wed May 17 08:46:57 2023 -0400
16:11:40-927610 INFO Setting environment tuning
16:11:40-927610 INFO Using DirectML Backend
16:11:42-303086 INFO Torch 2.0.0+cpu
16:11:42-318712 INFO Torch backend: DirectML (torch-directml 0.2.0.dev230426)
16:11:42-396842 INFO Torch detected GPU: AMD Radeon RX 5700 XT