I am quite familiar with Dask distributed for CPUs. I'd like to explore a transition to running my code on GPU cores. When I submit a task to the LocalCUDACluster I get this error:
ValueError: tuple is not allowed for map key This is my test case:
import cupy as cp import numpy as np from dask_cuda import LocalCUDACluster from dask.distributed import Client cluster = LocalCUDACluster() c = Client(cluster) def test_function(x): return(x+1) sample_np = np.array([0,1]) sample_cp = cp.asarray(sample_np) test_1 = test_function(sample_cp) test_2 = c.submit(test_function,sample_cp) test_2 = test_2.result() test_1 output:
array([1, 2]) test_2 output:
distributed.protocol.core - CRITICAL - Failed to deserialize ..... ValueError: tuple is not allowed for map key How do I correctly distribute tasks on CUDA cores?
UPDATE:
I managed to get it working by first installing the Dask Distributed and Dask CUDA release.
However, I noticed that only 1 worker is available, but I have 600 CUDA cores. How do I distributed individual tasks on these 600 CUDA cores? I'd like to parallelize tasks on these 600 cores.
Versions:
dask 2.17.2 dask-cuda 0.13.0 cupy 7.5.0 cudf 0.13.0 msgpack-python 1.0.0 1 Answer
It looks like this question has an answer in the comments. I'm going to copy a response from Nick Becker
Dask's distributed scheduler is single threaded (CPU and GPU), and Dask-CUDA uses a one worker per GPU model. This means that each task assigned to a given GPU will run serially, but that the task itself will use the GPU for parallelized computation. You may want to look at the Dask documentation and explore Dask.Array (which also supports GPU arrays).
