Use Scala Future to do Multiple Concurrent Tasks and Wait for Results

06 Jan 2016

Suppose we need to do multiple tasks concurrently and wait until all of them been finished. Scala Future is the way to go. In the following code, we execute multiple tasks using 8 threads and return the results until all the tasks done.

import scala.concurrent.{ExecutionContext, Await, Future}
import java.util.concurrent.Executors
import akka.util.Timeout

def compute[T:ClassTag](tasks: Seq[Task[T]]):Seq[Task[T]] = {

  // customize the execution context to use the specified number of threads
  val num_threads = 8
  val pool = Executors.newFixedThreadPool(num_threads)
  implicit val ec = ExecutionContext.fromExecutor(pool)
  implicit val timeout = Timeout(5 seconds)

  def r(t:Task[T]): Future[Task[T]] = Future {
    t.compute
    t
  }

  val ret = Await.result( Future.sequence( tasks.map( r(_) ) ), timeout.duration )
  pool.shutdown
  ret
}