Java 8 Custom Collector with Collector.of() Tutorial
Introduction
This tutorial explains how to write your own custom Collector in Java 8, both with the factory methodCollector.of() and by implementing the Collector interface in a class. We will start with the four functions every collector is made of - supplier, accumulator, combiner and finisher - and the characteristics that tell the stream how it may use them. We will then build a payroll-summary collector with Collector.of(), a collector without a finisher, and a reusable "top N" collector class, and check that they give the same result in parallel. Lastly, we will list the rules a correct collector must follow and when a custom collector is worth writing.(This tutorial builds on the Collector basics tutorialClick to read: Collector basics tutorial, which introduces the Collector interface and the predefined collectors.)
What a collector is made of
A collector describes a mutable reduction: start with an empty container, add every element of the stream to it, and turn the container into the result. The interfaceCollector<T, A, R> has three type parameters - T, the type of the stream's elements; A, the type of the mutable container (often hidden as ?); and R, the type of the final result - and four functions -
supplier()- aSupplier<A>that creates a new, empty container.accumulator()- aBiConsumer<A, T>that adds one element to a container.combiner()- aBinaryOperator<A>that merges two partial containers, used when a parallel streamClick to read tutorial: Java 8 Parallel Streams Tutorial with Examples has filled several of them.finisher()- aFunction<A, R>that turns the container into the result.
characteristics(), returns a set of hints: IDENTITY_FINISH (the container is already the result), UNORDERED (the result does not depend on element order) and CONCURRENT (one container may be filled by several threads at once).
Collector.of() - a collector from four lambdas
Definition ofCollector.of() in java.util.stream.Collector -static <T,A,R> Collector<T,A,R> of(Supplier<A> supplier, BiConsumer<A,T> accumulator, BinaryOperator<A> combiner, Function<A,R> finisher, Characteristics... characteristics)
AND
static <T,R> Collector<T,R,R> of(Supplier<R> supplier, BiConsumer<R,T> accumulator, BinaryOperator<R> combiner, Characteristics... characteristics)- The first form takes all four functions. It is the quickest way to write a one-off collector.
- The second form has no finisher: the container is the result, and the collector automatically gets the
IDENTITY_FINISHcharacteristic. - Further characteristics can be passed at the end, such as
Collector.Characteristics.UNORDERED.
Java 8 code example of custom collectors
The examples summarise a list of employees.PayrollStats is the mutable container for the first collector: it counts employees, adds up their salaries and remembers the top earner. TopN is a reusable collector class that keeps the n greatest elements of any type.
Employee.java
//Employee.java
public class Employee {
private final String id;
private final String name;
private final String department;
private final double salary;
public Employee(String id, String name, String department, double salary) {
this.id = id;
this.name = name;
this.department = department;
this.salary = salary;
}
public String getId() { return id; }
public String getName() { return name; }
public String getDepartment() { return department; }
public double getSalary() { return salary; }
@Override
public String toString() { return name; }
}
PayrollStats.java - the mutable container
//PayrollStats.java
// A mutable container that a collector fills, one employee at a time
public class PayrollStats {
private int count;
private double total;
private Employee topEarner;
public void add(Employee e) {
count++;
total += e.getSalary();
if (topEarner == null || e.getSalary() > topEarner.getSalary()) {
topEarner = e;
}
}
public PayrollStats combine(PayrollStats other) {
count += other.count;
total += other.total;
if (other.topEarner != null
&& (topEarner == null || other.topEarner.getSalary() > topEarner.getSalary())) {
topEarner = other.topEarner;
}
return this;
}
public String report() {
return String.format("%d employees, total %.2f, average %.2f, top earner %s",
count, total, count == 0 ? 0.0 : total / count, topEarner);
}
}
TopN.java - a collector class implementing the Collector interface
//TopN.java
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.EnumSet;
import java.util.List;
import java.util.PriorityQueue;
import java.util.Set;
import java.util.function.BiConsumer;
import java.util.function.BinaryOperator;
import java.util.function.Function;
import java.util.function.Supplier;
import java.util.stream.Collector;
// A reusable collector that keeps the n greatest elements, in descending order
public class TopN<T> implements Collector<T, PriorityQueue<T>, List<T>> {
private final int n;
private final Comparator<? super T> order;
private TopN(int n, Comparator<? super T> order) {
this.n = n;
this.order = order;
}
public static <T> TopN<T> of(int n, Comparator<? super T> order) {
return new TopN<>(n, order);
}
@Override
public Supplier<PriorityQueue<T>> supplier() {
return () -> new PriorityQueue<>(order); // smallest of the kept ones at the head
}
@Override
public BiConsumer<PriorityQueue<T>, T> accumulator() {
return (queue, element) -> {
queue.add(element);
if (queue.size() > n) queue.poll(); // drop the smallest
};
}
@Override
public BinaryOperator<PriorityQueue<T>> combiner() {
return (left, right) -> {
for (T element : right) {
left.add(element);
if (left.size() > n) left.poll();
}
return left;
};
}
@Override
public Function<PriorityQueue<T>, List<T>> finisher() {
return queue -> {
List<T> result = new ArrayList<>(queue);
result.sort(Collections.reverseOrder(order));
return result;
};
}
@Override
public Set<Characteristics> characteristics() {
return EnumSet.of(Characteristics.UNORDERED);
}
}
Java 8 code using Collector.of() and the TopN collector
//CustomCollectors.java
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Comparator;
import java.util.List;
import java.util.stream.Collector;
public class CustomCollectors {
public static void main(String[] args) {
List<Employee> staff = Arrays.asList(
new Employee("E-1001", "Tom Jones", "Finance", 5200),
new Employee("E-1002", "Harry Major", "Finance", 4100),
new Employee("E-1003", "Ethan Hardy", "Sales", 6300),
new Employee("E-1004", "Nancy Smith", "Sales", 5800),
new Employee("E-1005", "Deborah Sprightly", "IT", 7000));
// 1. Collector.of() with a supplier, accumulator, combiner and finisher
Collector<Employee, PayrollStats, String> payrollReport = Collector.of(
PayrollStats::new, // supplier: a new, empty container
PayrollStats::add, // accumulator: add one element to a container
PayrollStats::combine, // combiner: merge two partial containers
PayrollStats::report); // finisher: turn the container into the result
System.out.println("1. sequential: " + staff.stream().collect(payrollReport));
System.out.println(" parallel: " + staff.parallelStream().collect(payrollReport));
// 2. Without a finisher the container itself is the result (IDENTITY_FINISH)
Collector<String, StringBuilder, StringBuilder> initials = Collector.of(
StringBuilder::new,
(sb, name) -> sb.append(name.charAt(0)),
StringBuilder::append);
StringBuilder result = Arrays.asList("Tom", "Harry", "Ethan", "Nancy").stream()
.collect(initials);
System.out.println("2. initials: " + result + ", characteristics: "
+ initials.characteristics());
// 3. A reusable collector class implementing the Collector interface
List<Employee> topThree = staff.stream()
.collect(TopN.of(3, Comparator.comparingDouble(Employee::getSalary)));
System.out.println("3. top 3 by salary: " + topThree);
// 4. The same collector on another type
List<Integer> topTwo = Arrays.asList(42, 7, 99, 15, 63).parallelStream()
.collect(TopN.of(2, Comparator.<Integer>naturalOrder()));
System.out.println("4. top 2 numbers: " + topTwo);
// 5. A collector into an ArrayList, the way Collectors.toList() works
Collector<String, ?, List<String>> toArrayList = Collector.of(
ArrayList::new, List::add, (a, b) -> { a.addAll(b); return a; });
System.out.println("5. " + Arrays.asList("Tom", "Harry").stream().collect(toArrayList));
}
}
1. sequential: 5 employees, total 28400.00, average 5680.00, top earner Deborah Sprightly parallel: 5 employees, total 28400.00, average 5680.00, top earner Deborah Sprightly 2. initials: THEN, characteristics: [IDENTITY_FINISH] 3. top 3 by salary: [Deborah Sprightly, Ethan Hardy, Nancy Smith] 4. top 2 numbers: [99, 63] 5. [Tom, Harry]
Explanation of the code
- Part 1:
Collector.of()receives four method referencesClick to read: method references toPayrollStats. The sequential stream fills a single container; the parallel stream fills one per chunk and merges them withcombine(). Both give the same report, which is the test that the combiner is correct. PayrollStats.combine()must handle a partial container that has seen no employees yet, which is why it checksother.topEarner != null. A parallel stream may well create such an empty container.- Part 2: the three-argument
Collector.of()has no finisher, so theStringBuilderitself is the result, andcharacteristics()reportsIDENTITY_FINISH.StringBuilder::appendworks as the combiner becauseappend(CharSequence)returns the builder. - Part 3:
TopNimplements the interface's five methods. Its container is aPriorityQueueordered so that the smallest kept element is at the head; when the queue grows beyond n,poll()removes that smallest one. The finisher copies the queue into a list sorted from greatest to smallest. TopNdeclaresUNORDERED, because the top three salaries are the same whatever order the employees arrive in. This lets a parallel stream skip the work of preserving the encounter order.- Part 4: because
TopNis generic, the same class finds the two largest numbers in a parallel stream ofIntegers. - Part 5: a three-line collector into an
ArrayList- essentially whatCollectors.toList()does in Java 8. The combiner is written as a block lambda becauseaddAll()returns aboolean, not the list.
Rules for a correct collector
- The supplier must return a new container every time. Returning a shared object makes parallel chunks overwrite each other.
- The combiner must give the same result as sequential accumulation: combining the containers for "Tom, Harry" and "Ethan, Nancy" must equal accumulating all four into one container. Test it, as part 1 does, by comparing a sequential and a parallel run.
- Empty containers must combine correctly, because a chunk can be empty.
- Do not claim
CONCURRENTunless the container is thread-safe, such as aConcurrentHashMap. Without it, each thread gets its own container, which is the safe default. - Claim
IDENTITY_FINISHonly if the container really is the result type; the stream will then cast it without calling the finisher.
When to write a custom collector
The predefined collectorsClick to read: predefined collectors -groupingBy(), partitioningBy(), summarizingDouble(), toMap()Click to read tutorial: Java 8 Collectors.toMap Tutorial with Examples and the rest - combine to cover most needs, and collectingAndThen()Click to read: collectingAndThen() adds a finishing step to any of them. A custom collector is worth writing when -
- you need several results from one pass over the data, as
PayrollStatsdoes; - the logic needs a container the built-ins do not provide, such as the bounded priority queue in
TopN; - the same reduction is used in many places, and a named collector reads better than a chain of nested built-ins.
Summary
In this tutorial we first saw that a collector is made of a supplier, an accumulator, a combiner and a finisher, plus characteristics that tell the stream how it may use them. Next we saw the two forms ofCollector.of(). We then built a payroll summary with Collector.of() and checked it gives the same result sequentially and in parallel, built a collector without a finisher, and wrote a reusable TopN collector class used on two element types. Lastly, we listed the rules a correct collector must follow and the cases where a custom collector is worth writing.Java 8 Tutorials
Java 8 Collectors.toMap Tutorial with ExamplesClick to read tutorial: Java 8 Collectors.toMap Tutorial with Examples Java 8 Map merge, putIfAbsent and forEach with ExamplesClick to read tutorial: Java 8 Map merge, putIfAbsent and forEach with Examples Java 8 CompletableFuture Tutorial with ExamplesClick to read tutorial: Java 8 CompletableFuture Tutorial with Examples Java 8 CompletableFuture: Combining and Error HandlingClick to read tutorial: Java 8 CompletableFuture: Combining and Error Handling Java 8 Parallel Streams Tutorial with ExamplesClick to read tutorial: Java 8 Parallel Streams Tutorial with Examples Java 8 - How to Sort a Map by Key or ValueClick to read tutorial: Java 8 - How to Sort a Map by Key or Value Java 8 BiFunction, BiConsumer and BiPredicate TutorialClick to read tutorial: Java 8 BiFunction, BiConsumer and BiPredicate Tutorial
Java 8 Collectors.toMap Tutorial with ExamplesClick to read tutorial: Java 8 Collectors.toMap Tutorial with Examples Java 8 Map merge, putIfAbsent and forEach with ExamplesClick to read tutorial: Java 8 Map merge, putIfAbsent and forEach with Examples Java 8 CompletableFuture Tutorial with ExamplesClick to read tutorial: Java 8 CompletableFuture Tutorial with Examples Java 8 CompletableFuture: Combining and Error HandlingClick to read tutorial: Java 8 CompletableFuture: Combining and Error Handling Java 8 Parallel Streams Tutorial with ExamplesClick to read tutorial: Java 8 Parallel Streams Tutorial with Examples Java 8 - How to Sort a Map by Key or ValueClick to read tutorial: Java 8 - How to Sort a Map by Key or Value Java 8 BiFunction, BiConsumer and BiPredicate TutorialClick to read tutorial: Java 8 BiFunction, BiConsumer and BiPredicate Tutorial