Have a look at the discussions here https://github.com/comfyanonymous/ComfyUI/discussions/4571, you can get a pretty good speed ups with 4090, from that thread "Using the Q8 GGUF I see 15 seconds at the fastest to 17 seconds at the slowest on my RTX 4090 with Euler 20 Steps for 1024x1024 images."