Java ExecutorService and Thread Pools: An Introduction

Introduction Creating a new Thread for every task works for a demo, but not for a real application: threads are expensive to create, too many of them exhaust memory, and you have to manage starting, joining and results by hand. The ExecutorService in java.util.concurrent separates what to run - tasks - from how they are run - a pool of reusable threads. This tutorial is an introduction to ExecutorService and thread pools in Java. We will start with what a thread pool is and how to create one with Executors. We will then submit tasks, get their results through Future, and run many tasks at once with invokeAll(). Next we will handle exceptions thrown by tasks, and shut a pool down properly. Lastly, we will look at the different kinds of pools, and at virtual threads.

(Note - This tutorial follows synchronized and volatileClick to read tutorial: Java synchronized and volatile Explained with Examples. The tasks are written as lambda expressionsClick to read tutorial on Lambda Expressions.) What is a thread pool? A thread pool is a fixed set of worker threads plus a queue of tasks. When you submit a task, it goes into the queue; each worker repeatedly takes the next task and runs it. The threads are created once and reused, the number of threads stays under control however many tasks arrive, and your code never touches a Thread directly. A task is either -

a Runnable - void run(), which returns no result; or
a Callable<V> - V call() throws Exception, which returns a result and may throw a checked exception.

Running tasks on a fixed thread pool

//FirstPool.java
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

public class FirstPool {
  public static void main(String[] args) throws InterruptedException {
    ExecutorService pool = Executors.newFixedThreadPool(3);
    for (int i = 1; i <= 6; i++) {
      int taskId = i;
      pool.submit(() -> {
        String worker = Thread.currentThread().getName();
        System.out.println("Task " + taskId + " running on " + worker);
        sleep(100);
      });
    }
    pool.shutdown();                                     // accept no new tasks
    boolean finished = pool.awaitTermination(5, TimeUnit.SECONDS);
    System.out.println("All tasks finished: " + finished);
  }

  static void sleep(long ms) {
    try { Thread.sleep(ms); } catch (InterruptedException e) { Thread.currentThread().interrupt(); }
  }
}

OUTPUT of the above code (the order varies from run to run)

Task 1 running on pool-1-thread-1
Task 3 running on pool-1-thread-3
Task 2 running on pool-1-thread-2
Task 4 running on pool-1-thread-1
Task 5 running on pool-1-thread-2
Task 6 running on pool-1-thread-3
All tasks finished: true

Explanation of the code

Executors.newFixedThreadPool(3) created a pool of three threads, named pool-1-thread-1 to -3. Six tasks were submitted, so each thread ran two of them - the threads were reused.
The lambda can use taskId because it is effectively final; the loop variable i itself is not.
shutdown() stops the pool from accepting new tasks, but lets queued tasks finish. awaitTermination() waits for them, up to a time limit. Without a shutdown, the pool's threads would keep the program running.

Getting results with Future submit() returns a Future - a handle to a result which will be available later. future.get() waits until the task has finished and returns its result. Submitting several Callables first and calling get() afterwards lets the tasks run in parallel.

Running Callables in parallel and collecting their results

//FutureResults.java
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

public class FutureResults {
  static int slowSquare(int n) {
    try { Thread.sleep(500); } catch (InterruptedException e) { Thread.currentThread().interrupt(); }
    return n * n;
  }

  public static void main(String[] args) throws Exception {
    ExecutorService pool = Executors.newFixedThreadPool(4);
    long start = System.currentTimeMillis();

    List<Future<Integer>> futures = new ArrayList<>();
    for (int n = 1; n <= 4; n++) {
      int value = n;
      Callable<Integer> task = () -> slowSquare(value);
      futures.add(pool.submit(task));                   // returns immediately
    }
    int sum = 0;
    for (Future<Integer> f : futures) {
      sum += f.get();                                    // waits for each result
    }
    long elapsed = System.currentTimeMillis() - start;
    System.out.println("Sum of squares 1..4 = " + sum);
    System.out.println("Took about " + (elapsed / 100) * 100 + " ms for four 500 ms tasks");
    pool.shutdown();
  }
}

OUTPUT of the above code

Sum of squares 1..4 = 30
Took about 500 ms for four 500 ms tasks

Four tasks of 500 ms each finished in about 500 ms in total, because the four pool threads ran them at the same time. Run one after the other, they would have taken two seconds. Future also has isDone(), cancel(), and a get(timeout, unit) which gives up after a time limit. invokeAll() and invokeAny()

invokeAll(tasks) - submits a whole collection of Callables, waits for all of them, and returns their Futures in the same order.
invokeAny(tasks) - returns the result of whichever task finishes successfully first, and cancels the rest. Useful for asking several equivalent sources and using the fastest answer.

invokeAll() and invokeAny()

//InvokeAllAny.java
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

public class InvokeAllAny {
  static Callable<String> fetch(String server, long delayMs) {
    return () -> {
      Thread.sleep(delayMs);
      return server + " answered after " + delayMs + " ms";
    };
  }

  public static void main(String[] args) throws Exception {
    ExecutorService pool = Executors.newFixedThreadPool(3);
    List<Callable<String>> tasks = List.of(fetch("london", 300), fetch("mumbai", 100), fetch("tokyo", 200));

    for (Future<String> f : pool.invokeAll(tasks)) {
      System.out.println("invokeAll: " + f.get());
    }
    System.out.println("invokeAny: " + pool.invokeAny(tasks));
    pool.shutdown();
  }
}

OUTPUT of the above code

invokeAll: london answered after 300 ms
invokeAll: mumbai answered after 100 ms
invokeAll: tokyo answered after 200 ms
invokeAny: mumbai answered after 100 ms

invokeAll() kept the order of the task list, while invokeAny() returned the fastest server's answer.

Exceptions in tasks An exception thrown inside a submitted task does not crash the pool or the program - it is captured in the Future. Calling get() then throws an ExecutionException, whose cause is the original exception. If you never call get(), the exception is silently lost - so always check the results of submitted tasks.

Handling an exception thrown by a task

//TaskExceptions.java
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

public class TaskExceptions {
  public static void main(String[] args) throws InterruptedException {
    ExecutorService pool = Executors.newSingleThreadExecutor();
    Future<Integer> ok = pool.submit(() -> 10 / 2);
    Future<Integer> failing = pool.submit(() -> Integer.parseInt("ten"));
    for (Future<Integer> f : new Future[] {ok, failing}) {
      try {
        System.out.println("Result: " + f.get());
      } catch (ExecutionException e) {
        System.out.println("Task failed: " + e.getCause());
      }
    }
    pool.shutdown();
  }
}

OUTPUT of the above code

Result: 5
Task failed: java.lang.NumberFormatException: For input string: "ten"

Shutting down - and try-with-resources (Java 19+) A pool must always be shut down, or its threads keep the JVM alive. shutdown() lets queued tasks finish; shutdownNow() interrupts running tasks and returns the ones that never started. Since Java 19, ExecutorService is AutoCloseable: its close() shuts down and waits for all tasks, so a pool can be used in try-with-resources.

An ExecutorService in try-with-resources

//CloseablePool.java
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;

public class CloseablePool {
  public static void main(String[] args) {
    AtomicInteger processed = new AtomicInteger();
    try (ExecutorService pool = Executors.newFixedThreadPool(4)) {
      for (int i = 0; i < 20; i++) {
        pool.submit(() -> {
          try { Thread.sleep(20); } catch (InterruptedException e) { Thread.currentThread().interrupt(); }
          processed.incrementAndGet();
        });
      }
    }                                                    // close() waits for all 20 tasks
    System.out.println("Processed " + processed.get() + " orders after the pool closed");
  }
}

OUTPUT of the above code

Processed 20 orders after the pool closed

The counter is an AtomicInteger because it is shared by the pool's threads, as explained in synchronized and volatile. Kinds of executors

newFixedThreadPool(n) - exactly n threads; the usual choice for CPU-bound work, with n about the number of cores (Runtime.getRuntime().availableProcessors()).
newSingleThreadExecutor() - one thread, so tasks run one at a time, in order.
newCachedThreadPool() - creates threads as needed and reuses idle ones; good for many short tasks, but unbounded.
newScheduledThreadPool(n) - runs tasks after a delay, or repeatedly at a fixed rate.
newVirtualThreadPerTaskExecutor() (Java 21+) - one cheap virtual thread per task, ideal for large numbers of tasks which mostly wait on I/O. See Java 21 Virtual Threads TutorialClick to read the Java 21 Virtual Threads tutorial.

For production systems, ThreadPoolExecutor can be created directly to set a bounded queue and a policy for when the queue is full, and CompletableFuture lets you chain asynchronous steps together instead of blocking on get(). Summary In this tutorial we first saw that a thread pool reuses a fixed set of worker threads to run tasks from a queue, and ran six tasks on three threads with Executors.newFixedThreadPool(). We then collected results from Callable tasks through Future, running four tasks in parallel, and used invokeAll() and invokeAny(). Next we saw that exceptions in tasks are wrapped in an ExecutionException. We then shut pools down with shutdown(), and with try-with-resources since Java 19. Lastly, we went through the kinds of executors, including virtual threads. This completes the concurrency introduction. Next, we move on to tooling and good practices with Java naming conventions and code style.

Core Java Tutorials
Java synchronized and volatile Explained with Examples Java Naming Conventions and Code Style Guide Stack vs Heap Memory in Java and Object References

Java ExecutorService and Thread Pools: An Introduction

Click on a category to view all articles