Java 8 Custom Collector with Collector.of() Tutorial

Introduction

This tutorial explains how to write your own custom Collector in Java 8, both with the factory method Collector.of() and by implementing the Collector interface in a class. We will start with the four functions every collector is made of - supplier, accumulator, combiner and finisher - and the characteristics that tell the stream how it may use them. We will then build a payroll-summary collector with Collector.of(), a collector without a finisher, and a reusable "top N" collector class, and check that they give the same result in parallel. Lastly, we will list the rules a correct collector must follow and when a custom collector is worth writing.

(This tutorial builds on the Collector basics tutorialClick to read: Collector basics tutorial, which introduces the Collector interface and the predefined collectors.)

What a collector is made of

A collector describes a mutable reduction: start with an empty container, add every element of the stream to it, and turn the container into the result. The interface Collector<T, A, R> has three type parameters - T, the type of the stream's elements; A, the type of the mutable container (often hidden as ?); and R, the type of the final result - and four functions -
  1. supplier() - a Supplier<A> that creates a new, empty container.
  2. accumulator() - a BiConsumer<A, T> that adds one element to a container.
  3. combiner() - a BinaryOperator<A> that merges two partial containers, used when a parallel streamClick to read tutorial: Java 8 Parallel Streams Tutorial with Examples has filled several of them.
  4. finisher() - a Function<A, R> that turns the container into the result.
Anatomy of a Java 8 Collector - supplier, accumulator, combiner and finisher, and the IDENTITY_FINISH, UNORDERED and CONCURRENT characteristics
A fifth method, characteristics(), returns a set of hints: IDENTITY_FINISH (the container is already the result), UNORDERED (the result does not depend on element order) and CONCURRENT (one container may be filled by several threads at once).

Collector.of() - a collector from four lambdas

Definition of Collector.of() in java.util.stream.Collector -
static <T,A,R> Collector<T,A,R> of(Supplier<A> supplier, BiConsumer<A,T> accumulator, BinaryOperator<A> combiner, Function<A,R> finisher, Characteristics... characteristics) AND static <T,R> Collector<T,R,R> of(Supplier<R> supplier, BiConsumer<R,T> accumulator, BinaryOperator<R> combiner, Characteristics... characteristics)
Where,
  • The first form takes all four functions. It is the quickest way to write a one-off collector.
  • The second form has no finisher: the container is the result, and the collector automatically gets the IDENTITY_FINISH characteristic.
  • Further characteristics can be passed at the end, such as Collector.Characteristics.UNORDERED.

Java 8 code example of custom collectors

The examples summarise a list of employees. PayrollStats is the mutable container for the first collector: it counts employees, adds up their salaries and remembers the top earner. TopN is a reusable collector class that keeps the n greatest elements of any type.
Employee.java
//Employee.java
public class Employee {
  private final String id;
  private final String name;
  private final String department;
  private final double salary;

  public Employee(String id, String name, String department, double salary) {
    this.id = id;
    this.name = name;
    this.department = department;
    this.salary = salary;
  }

  public String getId() { return id; }
  public String getName() { return name; }
  public String getDepartment() { return department; }
  public double getSalary() { return salary; }

  @Override
  public String toString() { return name; }
}
PayrollStats.java - the mutable container
//PayrollStats.java
// A mutable container that a collector fills, one employee at a time
public class PayrollStats {
  private int count;
  private double total;
  private Employee topEarner;

  public void add(Employee e) {
    count++;
    total += e.getSalary();
    if (topEarner == null || e.getSalary() > topEarner.getSalary()) {
      topEarner = e;
    }
  }

  public PayrollStats combine(PayrollStats other) {
    count += other.count;
    total += other.total;
    if (other.topEarner != null
        && (topEarner == null || other.topEarner.getSalary() > topEarner.getSalary())) {
      topEarner = other.topEarner;
    }
    return this;
  }

  public String report() {
    return String.format("%d employees, total %.2f, average %.2f, top earner %s",
        count, total, count == 0 ? 0.0 : total / count, topEarner);
  }
}
TopN.java - a collector class implementing the Collector interface
//TopN.java
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.EnumSet;
import java.util.List;
import java.util.PriorityQueue;
import java.util.Set;
import java.util.function.BiConsumer;
import java.util.function.BinaryOperator;
import java.util.function.Function;
import java.util.function.Supplier;
import java.util.stream.Collector;

// A reusable collector that keeps the n greatest elements, in descending order
public class TopN<T> implements Collector<T, PriorityQueue<T>, List<T>> {
  private final int n;
  private final Comparator<? super T> order;

  private TopN(int n, Comparator<? super T> order) {
    this.n = n;
    this.order = order;
  }

  public static <T> TopN<T> of(int n, Comparator<? super T> order) {
    return new TopN<>(n, order);
  }

  @Override
  public Supplier<PriorityQueue<T>> supplier() {
    return () -> new PriorityQueue<>(order);          // smallest of the kept ones at the head
  }

  @Override
  public BiConsumer<PriorityQueue<T>, T> accumulator() {
    return (queue, element) -> {
      queue.add(element);
      if (queue.size() > n) queue.poll();             // drop the smallest
    };
  }

  @Override
  public BinaryOperator<PriorityQueue<T>> combiner() {
    return (left, right) -> {
      for (T element : right) {
        left.add(element);
        if (left.size() > n) left.poll();
      }
      return left;
    };
  }

  @Override
  public Function<PriorityQueue<T>, List<T>> finisher() {
    return queue -> {
      List<T> result = new ArrayList<>(queue);
      result.sort(Collections.reverseOrder(order));
      return result;
    };
  }

  @Override
  public Set<Characteristics> characteristics() {
    return EnumSet.of(Characteristics.UNORDERED);
  }
}
Java 8 code using Collector.of() and the TopN collector
//CustomCollectors.java
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Comparator;
import java.util.List;
import java.util.stream.Collector;

public class CustomCollectors {

  public static void main(String[] args) {
    List<Employee> staff = Arrays.asList(
        new Employee("E-1001", "Tom Jones", "Finance", 5200),
        new Employee("E-1002", "Harry Major", "Finance", 4100),
        new Employee("E-1003", "Ethan Hardy", "Sales", 6300),
        new Employee("E-1004", "Nancy Smith", "Sales", 5800),
        new Employee("E-1005", "Deborah Sprightly", "IT", 7000));

    // 1. Collector.of() with a supplier, accumulator, combiner and finisher
    Collector<Employee, PayrollStats, String> payrollReport = Collector.of(
        PayrollStats::new,          // supplier: a new, empty container
        PayrollStats::add,          // accumulator: add one element to a container
        PayrollStats::combine,      // combiner: merge two partial containers
        PayrollStats::report);      // finisher: turn the container into the result
    System.out.println("1. sequential: " + staff.stream().collect(payrollReport));
    System.out.println("   parallel:   " + staff.parallelStream().collect(payrollReport));

    // 2. Without a finisher the container itself is the result (IDENTITY_FINISH)
    Collector<String, StringBuilder, StringBuilder> initials = Collector.of(
        StringBuilder::new,
        (sb, name) -> sb.append(name.charAt(0)),
        StringBuilder::append);
    StringBuilder result = Arrays.asList("Tom", "Harry", "Ethan", "Nancy").stream()
        .collect(initials);
    System.out.println("2. initials: " + result + ", characteristics: "
        + initials.characteristics());

    // 3. A reusable collector class implementing the Collector interface
    List<Employee> topThree = staff.stream()
        .collect(TopN.of(3, Comparator.comparingDouble(Employee::getSalary)));
    System.out.println("3. top 3 by salary: " + topThree);

    // 4. The same collector on another type
    List<Integer> topTwo = Arrays.asList(42, 7, 99, 15, 63).parallelStream()
        .collect(TopN.of(2, Comparator.<Integer>naturalOrder()));
    System.out.println("4. top 2 numbers: " + topTwo);

    // 5. A collector into an ArrayList, the way Collectors.toList() works
    Collector<String, ?, List<String>> toArrayList = Collector.of(
        ArrayList::new, List::add, (a, b) -> { a.addAll(b); return a; });
    System.out.println("5. " + Arrays.asList("Tom", "Harry").stream().collect(toArrayList));
  }
}
 OUTPUT of the above code
1. sequential: 5 employees, total 28400.00, average 5680.00, top earner Deborah Sprightly
   parallel:   5 employees, total 28400.00, average 5680.00, top earner Deborah Sprightly
2. initials: THEN, characteristics: [IDENTITY_FINISH]
3. top 3 by salary: [Deborah Sprightly, Ethan Hardy, Nancy Smith]
4. top 2 numbers: [99, 63]
5. [Tom, Harry]

Explanation of the code

  • Part 1: Collector.of() receives four method referencesClick to read: method references to PayrollStats. The sequential stream fills a single container; the parallel stream fills one per chunk and merges them with combine(). Both give the same report, which is the test that the combiner is correct.
  • PayrollStats.combine() must handle a partial container that has seen no employees yet, which is why it checks other.topEarner != null. A parallel stream may well create such an empty container.
  • Part 2: the three-argument Collector.of() has no finisher, so the StringBuilder itself is the result, and characteristics() reports IDENTITY_FINISH. StringBuilder::append works as the combiner because append(CharSequence) returns the builder.
  • Part 3: TopN implements the interface's five methods. Its container is a PriorityQueue ordered so that the smallest kept element is at the head; when the queue grows beyond n, poll() removes that smallest one. The finisher copies the queue into a list sorted from greatest to smallest.
  • TopN declares UNORDERED, because the top three salaries are the same whatever order the employees arrive in. This lets a parallel stream skip the work of preserving the encounter order.
  • Part 4: because TopN is generic, the same class finds the two largest numbers in a parallel stream of Integers.
  • Part 5: a three-line collector into an ArrayList - essentially what Collectors.toList() does in Java 8. The combiner is written as a block lambda because addAll() returns a boolean, not the list.

Rules for a correct collector

  • The supplier must return a new container every time. Returning a shared object makes parallel chunks overwrite each other.
  • The combiner must give the same result as sequential accumulation: combining the containers for "Tom, Harry" and "Ethan, Nancy" must equal accumulating all four into one container. Test it, as part 1 does, by comparing a sequential and a parallel run.
  • Empty containers must combine correctly, because a chunk can be empty.
  • Do not claim CONCURRENT unless the container is thread-safe, such as a ConcurrentHashMap. Without it, each thread gets its own container, which is the safe default.
  • Claim IDENTITY_FINISH only if the container really is the result type; the stream will then cast it without calling the finisher.

When to write a custom collector

The predefined collectorsClick to read: predefined collectors - groupingBy(), partitioningBy(), summarizingDouble(), toMap()Click to read tutorial: Java 8 Collectors.toMap Tutorial with Examples and the rest - combine to cover most needs, and collectingAndThen()Click to read: collectingAndThen() adds a finishing step to any of them. A custom collector is worth writing when -
  • you need several results from one pass over the data, as PayrollStats does;
  • the logic needs a container the built-ins do not provide, such as the bounded priority queue in TopN;
  • the same reduction is used in many places, and a named collector reads better than a chain of nested built-ins.

Summary

In this tutorial we first saw that a collector is made of a supplier, an accumulator, a combiner and a finisher, plus characteristics that tell the stream how it may use them. Next we saw the two forms of Collector.of(). We then built a payroll summary with Collector.of() and checked it gives the same result sequentially and in parallel, built a collector without a finisher, and wrote a reusable TopN collector class used on two element types. Lastly, we listed the rules a correct collector must follow and the cases where a custom collector is worth writing.