Optimizations for CPU

[latot]

Hi all! I'm not very specialist on this topic, but I has been some time generating images with comfy, then my gpu went off (to repair) and though try to run this on a epyc 7742 and 256gb of ram, I know it is not the most efficient things but is what I want to try now, the best optimization.

I read stable-diffusion.cpp is nice for cpu, but sadly I has not been able to optimize it properly.

All tested with 16 cores.

sd-cli: AVX2 + BLAS/BLIS(1thread) + O3 + CPU optimizations + force f32: 45s/it is the best one I could get (confirmed is linking to BLIS)
comfy: 30s/it
torch + diffusers + BLIS: 13s/it

In all my tests to get the best performance I use taskset -c "0-15", epyc 7742 have blocks of 4 cores per L3, and this ones are real cores, affects a lot, using comfy without this one I got 50s/it.

My current env vars for my tests are:

export TH=16
export OMP_NUM_THREADS=$TH
export MKL_NUM_THREADS=$TH
export OPENBLAS_NUM_THREADS=$TH
export NUMEXPR_NUM_THREADS=$TH
export FLEXIBLAS_DEFAULT=BLIS
export OMP_PLACES=cores
export OMP_PROC_BIND=close

export BLIS_NUM_THREADS=$TH
export LD_PRELOAD=/usr/lib64/libblis.so.4

You might think some of this are not very useful, but in my tests, this combination are the ones who get the best timing, I tested the threads, omp and ld_preload ones separately and they affect performance, when using sd-cli I set 1 thread, because the client already do things in parallel with --thread, I also tried with blis threads at the same time but there was no differene, tried without --threads and with blis 16 threads but sd-cli never got into the sampler step (took more time than the previous tries without even finish).

I don't have gpu right now, so all is running on CPU.

I still think is weird to get 45s/it with sd-cli... do you know other ways to optimize this and get better timing?

I use gentoo.