×

Welcome to TagMyCode

Please login or create account to add a snippet.
 

Ray to avoid mini tasks

1
0
 
1
Language: Python
Posted by: dan darm
Added: Feb 28, 2021 8:10 PM
Views: 4783
Tags: ray
  1. import time
  2. import ray
  3.  
  4. ray.init(num_cpus = 4)
  5.  
  6. def tiny_work(x):
  7.     time.sleep(0.0001) # replace this is with work you need to do
  8.     return x
  9.  
  10. @ray.remote
  11. def mega_work(start, end):
  12.     return [tiny_work(x) for x in range(start, end)]
  13.  
  14. start = time.time()
  15. result_ids = []
  16. [result_ids.append(mega_work.remote(x*1000, (x+1)*1000)) for x in range(100)]
  17. results = ray.get(result_ids)
  18. print("duration =", time.time() - start)

Write a comment

To post comments, please log in or register.