Just think about how recently we got that feature in the official ChatGPT image gen. And now we have that running locally — assuming that is, I can figure out how to get this running on my Mac — blows my mind.
There is difussion.cpp which is intended for those types of models. I set up krea-2-turbo with the help of ChatGPT 2 months ago, if you have a capable computer that's what I would suggest once it becomes supported.
Probably ComfyUI is one of the easiest way to get started with local image/video models. Or perhaps vLLM, if they have support for it already, would be something like `vllm serve <model> --omni --port 9080`
The capabilities of local LLM text-to-image is honestly pretty damn impressive. IMO, I think local image generation is currently ahead of local code generation. I can get an image in seconds locally with the quality being way higher than what I'd expect from a local model. However with coding it's much slower and much less impressive. I'm sure there's a reason for this and I'm not an AI expert so I'll let the smarter folks tell me why, but that's just been my observation thus far.
I've played with diffusion models on and off since the first release of Stable Diffusion - just for amusement, without a particular goal.
Recently, I've been helping a friend's wife with some basic vector images for her sewing hobby (she has what is essentially a CNC sewing machine) and have been super-impressed with FLUX.1-Kontext, which I've been running on my Macbook Pro with mflux. Its ability to (for example) take a photo of a human or an animal and return a line drawing which is recognisably them (rather than just a generic similarish image as I've experienced with other models) is excellent.
It's an older model now, but (AIUI) has the text-handling features baked in, and in my various testing is very reliable at giving me the outputs that I want, without the randomness I've experienced previously. It's big and relatively slow (~3 mins per 512x512 image edit on my M1 Max Mac) but excellent to work with. It's also very straightforward to set up, without the harness complexity of e.g. comfyui.
I am really grateful to the Chinese Labs for open sourcing their best models. If it was left to the Americans, we would be forced to pay obscene API fees to use them.
A lot of the previous Qwen models seem to have used Apache licenses, among others:
https://en.wikipedia.org/wiki/Qwen#List_of_models
Unfortunately, it looks like this model is using a much more restrictive license:
https://github.com/QwenLM/Qwen-Image-2.1/blob/main/LICENSE
Was going to post about this: the last image models with Apache 2.0 license seem to be from 2025, recent Qwen models are "non-commercial use".
A 7B diffusion model can now render CJK text better than Microsoft Windows.
Just think about how recently we got that feature in the official ChatGPT image gen. And now we have that running locally — assuming that is, I can figure out how to get this running on my Mac — blows my mind.
How do you use this model locally, similarly to using `llama-server -m <model>`?
(Of course I mean: outside direct use of Python, and in the most efficient way.)
There is difussion.cpp which is intended for those types of models. I set up krea-2-turbo with the help of ChatGPT 2 months ago, if you have a capable computer that's what I would suggest once it becomes supported.
why not just as you suggested i.e. https://qwen.readthedocs.io/en/latest/run_locally/llama.cpp.... then get the result either via a UI or wget/curl it back?
I am not sure that llama.cpp also supports image generation models.
it's multimodal, see https://github.com/ggml-org/llama.cpp/blob/master/docs/multi...
Multimodal doesn't guarantee input and output.
> Currently, we support image, audio and video input.
Probably ComfyUI is one of the easiest way to get started with local image/video models. Or perhaps vLLM, if they have support for it already, would be something like `vllm serve <model> --omni --port 9080`
on the linked GitHub page they list support Diffusers, ComfyUI, vLLM-Omni, SGLang, and LightX2V with links to each
They finally fixed their VAE. It really held back their models over the last 2 years.
The capabilities of local LLM text-to-image is honestly pretty damn impressive. IMO, I think local image generation is currently ahead of local code generation. I can get an image in seconds locally with the quality being way higher than what I'd expect from a local model. However with coding it's much slower and much less impressive. I'm sure there's a reason for this and I'm not an AI expert so I'll let the smarter folks tell me why, but that's just been my observation thus far.
I've played with diffusion models on and off since the first release of Stable Diffusion - just for amusement, without a particular goal.
Recently, I've been helping a friend's wife with some basic vector images for her sewing hobby (she has what is essentially a CNC sewing machine) and have been super-impressed with FLUX.1-Kontext, which I've been running on my Macbook Pro with mflux. Its ability to (for example) take a photo of a human or an animal and return a line drawing which is recognisably them (rather than just a generic similarish image as I've experienced with other models) is excellent.
It's an older model now, but (AIUI) has the text-handling features baked in, and in my various testing is very reliable at giving me the outputs that I want, without the randomness I've experienced previously. It's big and relatively slow (~3 mins per 512x512 image edit on my M1 Max Mac) but excellent to work with. It's also very straightforward to set up, without the harness complexity of e.g. comfyui.
I mean I’m sure it’s the reverse for an artist. They would be less impressed with the image and more impressed with the code quality
To generalize, LLMs are great at what you are not skilled at.
God I love the Qwen team. Easily the most diverse set of models from all the Chinese labs. Only Gemini/DeepMind comes close.
Boy do I love waking up to find a new awesome toy from the Qwen team waiting for me to play with! Pulling it now
I am really grateful to the Chinese Labs for open sourcing their best models. If it was left to the Americans, we would be forced to pay obscene API fees to use them.
Very impressive, and kind of worrying a 7B model can have such capabilities. The implications are huge. And Qwen does no watermarking (yet) yeah?
They always had a fourier space mark in their models even without the VAEs are usually pretty easy to detect.
Ah I wasn't aware, thank you
Image gen you eyeball one frame and stop, code needs hundreds of tokens all correct in sequence, one bad line and the whole thing fails.
Interesting in the example of assembling the Cheers team how the otherwise great result genericizes Shelley Long.
The result seems a pretty good representation given the source image wasn't that great. I think that Woody Harrelson comes across much worse.