I am using the StableDiffusionPipeline from the Hugging Face Diffusers library in Python 3.10.2, on an M2 Mac (I tagged it because this might be the issue). When I try to generate 1 image from 1 prompt, the output looks fine, but when I try to generate multiple images using the same prompt, the images are all either black squares or a random image (see example below). What could be the issue?
My code is as follows (where I change n_imgs from 1 to more than 1 to break it):
from diffusers import StableDiffusionPipeline pipe = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5") pipe = pipe.to("mps") # for M1/M2 chips pipe.enable_attention_slicing() prompt = "a photo of an astronaut driving a car on mars" # First-time "warmup" pass (because of weird M1 behaviour) _ = pipe(prompt, num_inference_steps=1) # generate images n_imgs = 1 imgs = pipe([prompt] * n_imgs).images I also tried setting num_images_per_prompt instead of creating a list of repeated prompts in the pipeline call, but this gave the same bad results.
Example output (for multiple images):
[edit/update]: When I generate the images in a loop surrounding the pipe call instead of passing an iterable to the pipe call, it does work:
# generate images n_imgs = 3 for i in range(n_imgs): img = pipe(prompt).images[0] # do something with img But it is still a mystery to me as to why.
3 Answers
Apparently it is indeed an Apple Silicon (M1/M2) issue, of which Hugging Face is not yet sure why this is happening, see this GitHub issue for more details.
I think it might be a PyTorch issue given that a pure MPS version of the code (in Swift) worked fine last time I tested:
import MetalPerformanceShadersGraph let graph = MPSGraph() let x = graph.constant(1, shape: [32, 4096, 4096], dataType: .float32) let y = graph.constant(1, shape: [32, 4096, 1], dataType: .float32) let z = graph.matrixMultiplication(primary: x, secondary: y, name: nil) let device = MTLCreateSystemDefaultDevice()! let buf = device.makeBuffer(length: 16384)! let td = MPSGraphTensorData(buf, shape: [64, 64], dataType: .int32) let cmdBuf = MPSCommandBuffer(from: device.makeCommandQueue()!) graph.encode(to: cmdBuf, feeds: [:], targetOperations: nil, resultsDictionary: [z:td], executionDescriptor: nil) cmdBuf.commit() You will encounter this error now if you ran that or anything related to MPS using M1/M2 apple devices.
NotImplementedError: The operator 'aten::index.Tensor' is not current implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on . As a temporary fix, you can set the environment variable PYTORCH_ENABLE_MPS_FALLBACK=1 to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.
I'm actually looking for a workaround here too in case you guys know.
