finish iterator draft
This commit is contained in:
parent
b4225d50e0
commit
ba11ce21df
GPG Key ID:
8282A2BEA4ADA3D2
933
resources/stream_benchmark.log
Normal file
933
resources/stream_benchmark.log
Normal file
@ -0,0 +1,933 @@
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.Java.loop
|
||||
|
||||
# Run progress: 0.00% complete, ETA 01:31:40
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 456.332 ms/op
|
||||
# Warmup Iteration 2: 445.904 ms/op
|
||||
# Warmup Iteration 3: 447.161 ms/op
|
||||
# Warmup Iteration 4: 446.954 ms/op
|
||||
# Warmup Iteration 5: 445.977 ms/op
|
||||
Iteration 1: 446.109 ms/op
|
||||
Iteration 2: 447.066 ms/op
|
||||
Iteration 3: 445.489 ms/op
|
||||
Iteration 4: 447.256 ms/op
|
||||
Iteration 5: 447.188 ms/op
|
||||
|
||||
# Run progress: 1.82% complete, ETA 01:32:24
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 452.201 ms/op
|
||||
# Warmup Iteration 2: 445.484 ms/op
|
||||
# Warmup Iteration 3: 447.031 ms/op
|
||||
# Warmup Iteration 4: 445.762 ms/op
|
||||
# Warmup Iteration 5: 445.223 ms/op
|
||||
Iteration 1: 446.364 ms/op
|
||||
Iteration 2: 445.127 ms/op
|
||||
Iteration 3: 445.872 ms/op
|
||||
Iteration 4: 447.157 ms/op
|
||||
Iteration 5: 445.903 ms/op
|
||||
|
||||
# Run progress: 3.64% complete, ETA 01:30:48
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 450.744 ms/op
|
||||
# Warmup Iteration 2: 446.860 ms/op
|
||||
# Warmup Iteration 3: 447.187 ms/op
|
||||
# Warmup Iteration 4: 445.894 ms/op
|
||||
# Warmup Iteration 5: 446.363 ms/op
|
||||
Iteration 1: 446.135 ms/op
|
||||
Iteration 2: 445.913 ms/op
|
||||
Iteration 3: 445.983 ms/op
|
||||
Iteration 4: 445.214 ms/op
|
||||
Iteration 5: 445.505 ms/op
|
||||
|
||||
# Run progress: 5.45% complete, ETA 01:29:07
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 453.909 ms/op
|
||||
# Warmup Iteration 2: 445.268 ms/op
|
||||
# Warmup Iteration 3: 447.106 ms/op
|
||||
# Warmup Iteration 4: 452.267 ms/op
|
||||
# Warmup Iteration 5: 446.321 ms/op
|
||||
Iteration 1: 447.768 ms/op
|
||||
Iteration 2: 446.519 ms/op
|
||||
Iteration 3: 445.002 ms/op
|
||||
Iteration 4: 447.406 ms/op
|
||||
Iteration 5: 444.889 ms/op
|
||||
|
||||
# Run progress: 7.27% complete, ETA 01:27:28
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 453.557 ms/op
|
||||
# Warmup Iteration 2: 446.911 ms/op
|
||||
# Warmup Iteration 3: 447.795 ms/op
|
||||
# Warmup Iteration 4: 445.091 ms/op
|
||||
# Warmup Iteration 5: 447.698 ms/op
|
||||
Iteration 1: 444.943 ms/op
|
||||
Iteration 2: 445.156 ms/op
|
||||
Iteration 3: 447.101 ms/op
|
||||
Iteration 4: 445.289 ms/op
|
||||
Iteration 5: 445.011 ms/op
|
||||
|
||||
|
||||
Result "perf.Java.loop":
|
||||
446.055 ±(99.9%) 0.677 ms/op [Average]
|
||||
(min, avg, max) = (444.889, 446.055, 447.768), stdev = 0.904
|
||||
CI (99.9%): [445.378, 446.732] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.Java.stream
|
||||
|
||||
# Run progress: 9.09% complete, ETA 01:25:45
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 683.149 ms/op
|
||||
# Warmup Iteration 2: 591.776 ms/op
|
||||
# Warmup Iteration 3: 591.627 ms/op
|
||||
# Warmup Iteration 4: 598.434 ms/op
|
||||
# Warmup Iteration 5: 602.405 ms/op
|
||||
Iteration 1: 632.719 ms/op
|
||||
Iteration 2: 594.098 ms/op
|
||||
Iteration 3: 591.569 ms/op
|
||||
Iteration 4: 592.402 ms/op
|
||||
Iteration 5: 595.107 ms/op
|
||||
|
||||
# Run progress: 10.91% complete, ETA 01:23:50
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 678.576 ms/op
|
||||
# Warmup Iteration 2: 591.063 ms/op
|
||||
# Warmup Iteration 3: 594.322 ms/op
|
||||
# Warmup Iteration 4: 591.848 ms/op
|
||||
# Warmup Iteration 5: 594.417 ms/op
|
||||
Iteration 1: 595.271 ms/op
|
||||
Iteration 2: 592.028 ms/op
|
||||
Iteration 3: 642.472 ms/op
|
||||
Iteration 4: 593.947 ms/op
|
||||
Iteration 5: 592.536 ms/op
|
||||
|
||||
# Run progress: 12.73% complete, ETA 01:21:58
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 678.283 ms/op
|
||||
# Warmup Iteration 2: 594.497 ms/op
|
||||
# Warmup Iteration 3: 592.839 ms/op
|
||||
# Warmup Iteration 4: 592.898 ms/op
|
||||
# Warmup Iteration 5: 594.148 ms/op
|
||||
Iteration 1: 629.247 ms/op
|
||||
Iteration 2: 592.774 ms/op
|
||||
Iteration 3: 593.806 ms/op
|
||||
Iteration 4: 593.348 ms/op
|
||||
Iteration 5: 593.438 ms/op
|
||||
|
||||
# Run progress: 14.55% complete, ETA 01:20:08
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 679.167 ms/op
|
||||
# Warmup Iteration 2: 591.866 ms/op
|
||||
# Warmup Iteration 3: 592.856 ms/op
|
||||
# Warmup Iteration 4: 592.900 ms/op
|
||||
# Warmup Iteration 5: 591.494 ms/op
|
||||
Iteration 1: 635.757 ms/op
|
||||
Iteration 2: 593.654 ms/op
|
||||
Iteration 3: 591.762 ms/op
|
||||
Iteration 4: 593.685 ms/op
|
||||
Iteration 5: 592.605 ms/op
|
||||
|
||||
# Run progress: 16.36% complete, ETA 01:18:20
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 679.848 ms/op
|
||||
# Warmup Iteration 2: 594.060 ms/op
|
||||
# Warmup Iteration 3: 592.618 ms/op
|
||||
# Warmup Iteration 4: 596.782 ms/op
|
||||
# Warmup Iteration 5: 593.338 ms/op
|
||||
Iteration 1: 630.190 ms/op
|
||||
Iteration 2: 594.034 ms/op
|
||||
Iteration 3: 594.066 ms/op
|
||||
Iteration 4: 592.662 ms/op
|
||||
Iteration 5: 592.430 ms/op
|
||||
|
||||
|
||||
Result "perf.Java.stream":
|
||||
601.424 ±(99.9%) 12.606 ms/op [Average]
|
||||
(min, avg, max) = (591.569, 601.424, 642.472), stdev = 16.829
|
||||
CI (99.9%): [588.818, 614.031] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.Kotlin.loop
|
||||
|
||||
# Run progress: 18.18% complete, ETA 01:16:33
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 452.772 ms/op
|
||||
# Warmup Iteration 2: 445.813 ms/op
|
||||
# Warmup Iteration 3: 445.203 ms/op
|
||||
# Warmup Iteration 4: 447.020 ms/op
|
||||
# Warmup Iteration 5: 445.761 ms/op
|
||||
Iteration 1: 445.277 ms/op
|
||||
Iteration 2: 447.734 ms/op
|
||||
Iteration 3: 445.433 ms/op
|
||||
Iteration 4: 445.653 ms/op
|
||||
Iteration 5: 446.958 ms/op
|
||||
|
||||
# Run progress: 20.00% complete, ETA 01:14:54
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 450.903 ms/op
|
||||
# Warmup Iteration 2: 445.990 ms/op
|
||||
# Warmup Iteration 3: 447.923 ms/op
|
||||
# Warmup Iteration 4: 445.835 ms/op
|
||||
# Warmup Iteration 5: 445.211 ms/op
|
||||
Iteration 1: 447.195 ms/op
|
||||
Iteration 2: 449.569 ms/op
|
||||
Iteration 3: 444.934 ms/op
|
||||
Iteration 4: 446.823 ms/op
|
||||
Iteration 5: 445.366 ms/op
|
||||
|
||||
# Run progress: 21.82% complete, ETA 01:13:15
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 456.334 ms/op
|
||||
# Warmup Iteration 2: 446.712 ms/op
|
||||
# Warmup Iteration 3: 445.541 ms/op
|
||||
# Warmup Iteration 4: 446.043 ms/op
|
||||
# Warmup Iteration 5: 446.722 ms/op
|
||||
Iteration 1: 445.405 ms/op
|
||||
Iteration 2: 447.530 ms/op
|
||||
Iteration 3: 450.772 ms/op
|
||||
Iteration 4: 445.377 ms/op
|
||||
Iteration 5: 447.160 ms/op
|
||||
|
||||
# Run progress: 23.64% complete, ETA 01:11:34
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 452.581 ms/op
|
||||
# Warmup Iteration 2: 445.681 ms/op
|
||||
# Warmup Iteration 3: 447.341 ms/op
|
||||
# Warmup Iteration 4: 446.772 ms/op
|
||||
# Warmup Iteration 5: 447.728 ms/op
|
||||
Iteration 1: 447.228 ms/op
|
||||
Iteration 2: 446.791 ms/op
|
||||
Iteration 3: 445.374 ms/op
|
||||
Iteration 4: 447.020 ms/op
|
||||
Iteration 5: 445.255 ms/op
|
||||
|
||||
# Run progress: 25.45% complete, ETA 01:09:54
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 450.730 ms/op
|
||||
# Warmup Iteration 2: 447.652 ms/op
|
||||
# Warmup Iteration 3: 445.747 ms/op
|
||||
# Warmup Iteration 4: 444.973 ms/op
|
||||
# Warmup Iteration 5: 446.593 ms/op
|
||||
Iteration 1: 445.315 ms/op
|
||||
Iteration 2: 445.712 ms/op
|
||||
Iteration 3: 449.632 ms/op
|
||||
Iteration 4: 446.242 ms/op
|
||||
Iteration 5: 445.253 ms/op
|
||||
|
||||
|
||||
Result "perf.Kotlin.loop":
|
||||
446.600 ±(99.9%) 1.164 ms/op [Average]
|
||||
(min, avg, max) = (444.934, 446.600, 450.772), stdev = 1.554
|
||||
CI (99.9%): [445.436, 447.764] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.Kotlin.sequence
|
||||
|
||||
# Run progress: 27.27% complete, ETA 01:08:14
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 2875.137 ms/op
|
||||
# Warmup Iteration 2: 2714.992 ms/op
|
||||
# Warmup Iteration 3: 2717.500 ms/op
|
||||
# Warmup Iteration 4: 2722.842 ms/op
|
||||
# Warmup Iteration 5: 2726.368 ms/op
|
||||
Iteration 1: 2718.843 ms/op
|
||||
Iteration 2: 2730.963 ms/op
|
||||
Iteration 3: 2725.803 ms/op
|
||||
Iteration 4: 2735.491 ms/op
|
||||
Iteration 5: 2720.835 ms/op
|
||||
|
||||
# Run progress: 29.09% complete, ETA 01:06:49
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 2872.714 ms/op
|
||||
# Warmup Iteration 2: 2737.939 ms/op
|
||||
# Warmup Iteration 3: 2723.591 ms/op
|
||||
# Warmup Iteration 4: 2739.880 ms/op
|
||||
# Warmup Iteration 5: 2735.399 ms/op
|
||||
Iteration 1: 2725.004 ms/op
|
||||
Iteration 2: 2739.529 ms/op
|
||||
Iteration 3: 2735.921 ms/op
|
||||
Iteration 4: 2737.889 ms/op
|
||||
Iteration 5: 2745.381 ms/op
|
||||
|
||||
# Run progress: 30.91% complete, ETA 01:05:23
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 2863.006 ms/op
|
||||
# Warmup Iteration 2: 2726.449 ms/op
|
||||
# Warmup Iteration 3: 2733.619 ms/op
|
||||
# Warmup Iteration 4: 2725.596 ms/op
|
||||
# Warmup Iteration 5: 2743.223 ms/op
|
||||
Iteration 1: 2730.597 ms/op
|
||||
Iteration 2: 2742.830 ms/op
|
||||
Iteration 3: 2737.714 ms/op
|
||||
Iteration 4: 2727.641 ms/op
|
||||
Iteration 5: 2729.501 ms/op
|
||||
|
||||
# Run progress: 32.73% complete, ETA 01:03:54
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 2874.924 ms/op
|
||||
# Warmup Iteration 2: 2727.463 ms/op
|
||||
# Warmup Iteration 3: 2792.965 ms/op
|
||||
# Warmup Iteration 4: 2722.637 ms/op
|
||||
# Warmup Iteration 5: 2724.197 ms/op
|
||||
Iteration 1: 2736.731 ms/op
|
||||
Iteration 2: 2726.640 ms/op
|
||||
Iteration 3: 2748.462 ms/op
|
||||
Iteration 4: 2747.123 ms/op
|
||||
Iteration 5: 2741.907 ms/op
|
||||
|
||||
# Run progress: 34.55% complete, ETA 01:02:23
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 2852.575 ms/op
|
||||
# Warmup Iteration 2: 2727.111 ms/op
|
||||
# Warmup Iteration 3: 2734.597 ms/op
|
||||
# Warmup Iteration 4: 2731.829 ms/op
|
||||
# Warmup Iteration 5: 2720.765 ms/op
|
||||
Iteration 1: 2724.821 ms/op
|
||||
Iteration 2: 2733.517 ms/op
|
||||
Iteration 3: 2726.031 ms/op
|
||||
Iteration 4: 2714.774 ms/op
|
||||
Iteration 5: 2731.156 ms/op
|
||||
|
||||
|
||||
Result "perf.Kotlin.sequence":
|
||||
2732.604 ±(99.9%) 6.644 ms/op [Average]
|
||||
(min, avg, max) = (2714.774, 2732.604, 2748.462), stdev = 8.870
|
||||
CI (99.9%): [2725.960, 2739.248] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.Kotlin.stream
|
||||
|
||||
# Run progress: 36.36% complete, ETA 01:00:49
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 678.345 ms/op
|
||||
# Warmup Iteration 2: 591.611 ms/op
|
||||
# Warmup Iteration 3: 593.355 ms/op
|
||||
# Warmup Iteration 4: 593.125 ms/op
|
||||
# Warmup Iteration 5: 592.035 ms/op
|
||||
Iteration 1: 594.764 ms/op
|
||||
Iteration 2: 595.813 ms/op
|
||||
Iteration 3: 592.359 ms/op
|
||||
Iteration 4: 593.853 ms/op
|
||||
Iteration 5: 593.184 ms/op
|
||||
|
||||
# Run progress: 38.18% complete, ETA 00:59:00
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 680.533 ms/op
|
||||
# Warmup Iteration 2: 644.939 ms/op
|
||||
# Warmup Iteration 3: 592.135 ms/op
|
||||
# Warmup Iteration 4: 592.386 ms/op
|
||||
# Warmup Iteration 5: 593.396 ms/op
|
||||
Iteration 1: 592.626 ms/op
|
||||
Iteration 2: 593.254 ms/op
|
||||
Iteration 3: 594.582 ms/op
|
||||
Iteration 4: 591.622 ms/op
|
||||
Iteration 5: 592.595 ms/op
|
||||
|
||||
# Run progress: 40.00% complete, ETA 00:57:11
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 680.298 ms/op
|
||||
# Warmup Iteration 2: 591.609 ms/op
|
||||
# Warmup Iteration 3: 592.565 ms/op
|
||||
# Warmup Iteration 4: 591.873 ms/op
|
||||
# Warmup Iteration 5: 591.836 ms/op
|
||||
Iteration 1: 593.007 ms/op
|
||||
Iteration 2: 592.916 ms/op
|
||||
Iteration 3: 591.434 ms/op
|
||||
Iteration 4: 594.996 ms/op
|
||||
Iteration 5: 592.853 ms/op
|
||||
|
||||
# Run progress: 41.82% complete, ETA 00:55:23
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 678.935 ms/op
|
||||
# Warmup Iteration 2: 595.014 ms/op
|
||||
# Warmup Iteration 3: 593.754 ms/op
|
||||
# Warmup Iteration 4: 592.117 ms/op
|
||||
# Warmup Iteration 5: 593.827 ms/op
|
||||
Iteration 1: 591.940 ms/op
|
||||
Iteration 2: 591.284 ms/op
|
||||
Iteration 3: 593.308 ms/op
|
||||
Iteration 4: 597.758 ms/op
|
||||
Iteration 5: 591.519 ms/op
|
||||
|
||||
# Run progress: 43.64% complete, ETA 00:53:36
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 681.321 ms/op
|
||||
# Warmup Iteration 2: 635.465 ms/op
|
||||
# Warmup Iteration 3: 591.332 ms/op
|
||||
# Warmup Iteration 4: 593.023 ms/op
|
||||
# Warmup Iteration 5: 590.999 ms/op
|
||||
Iteration 1: 590.586 ms/op
|
||||
Iteration 2: 597.205 ms/op
|
||||
Iteration 3: 591.966 ms/op
|
||||
Iteration 4: 592.190 ms/op
|
||||
Iteration 5: 596.203 ms/op
|
||||
|
||||
|
||||
Result "perf.Kotlin.stream":
|
||||
593.353 ±(99.9%) 1.408 ms/op [Average]
|
||||
(min, avg, max) = (590.586, 593.353, 597.758), stdev = 1.880
|
||||
CI (99.9%): [591.945, 594.761] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.Kotlin.streamWrappedInSequence
|
||||
|
||||
# Run progress: 45.45% complete, ETA 00:51:49
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 3989.143 ms/op
|
||||
# Warmup Iteration 2: 3808.328 ms/op
|
||||
# Warmup Iteration 3: 3851.896 ms/op
|
||||
# Warmup Iteration 4: 3821.087 ms/op
|
||||
# Warmup Iteration 5: 3842.988 ms/op
|
||||
Iteration 1: 3838.952 ms/op
|
||||
Iteration 2: 3828.666 ms/op
|
||||
Iteration 3: 3829.225 ms/op
|
||||
Iteration 4: 3804.143 ms/op
|
||||
Iteration 5: 3803.303 ms/op
|
||||
|
||||
# Run progress: 47.27% complete, ETA 00:50:19
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 3965.935 ms/op
|
||||
# Warmup Iteration 2: 3830.953 ms/op
|
||||
# Warmup Iteration 3: 3831.254 ms/op
|
||||
# Warmup Iteration 4: 3786.462 ms/op
|
||||
# Warmup Iteration 5: 3810.373 ms/op
|
||||
Iteration 1: 3820.851 ms/op
|
||||
Iteration 2: 3850.427 ms/op
|
||||
Iteration 3: 3874.084 ms/op
|
||||
Iteration 4: 3810.283 ms/op
|
||||
Iteration 5: 3794.962 ms/op
|
||||
|
||||
# Run progress: 49.09% complete, ETA 00:48:46
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 3971.302 ms/op
|
||||
# Warmup Iteration 2: 3805.030 ms/op
|
||||
# Warmup Iteration 3: 3827.273 ms/op
|
||||
# Warmup Iteration 4: 3797.691 ms/op
|
||||
# Warmup Iteration 5: 3828.704 ms/op
|
||||
Iteration 1: 3835.989 ms/op
|
||||
Iteration 2: 3807.760 ms/op
|
||||
Iteration 3: 4022.427 ms/op
|
||||
Iteration 4: 3809.359 ms/op
|
||||
Iteration 5: 3834.482 ms/op
|
||||
|
||||
# Run progress: 50.91% complete, ETA 00:47:12
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 4038.999 ms/op
|
||||
# Warmup Iteration 2: 3790.741 ms/op
|
||||
# Warmup Iteration 3: 3816.597 ms/op
|
||||
# Warmup Iteration 4: 3804.865 ms/op
|
||||
# Warmup Iteration 5: 3835.732 ms/op
|
||||
Iteration 1: 3804.229 ms/op
|
||||
Iteration 2: 3854.098 ms/op
|
||||
Iteration 3: 3822.642 ms/op
|
||||
Iteration 4: 3817.002 ms/op
|
||||
Iteration 5: 3804.801 ms/op
|
||||
|
||||
# Run progress: 52.73% complete, ETA 00:45:37
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 3975.357 ms/op
|
||||
# Warmup Iteration 2: 3848.500 ms/op
|
||||
# Warmup Iteration 3: 3794.490 ms/op
|
||||
# Warmup Iteration 4: 3823.770 ms/op
|
||||
# Warmup Iteration 5: 3810.056 ms/op
|
||||
Iteration 1: 3831.363 ms/op
|
||||
Iteration 2: 3819.838 ms/op
|
||||
Iteration 3: 3787.684 ms/op
|
||||
Iteration 4: 3808.174 ms/op
|
||||
Iteration 5: 3815.488 ms/op
|
||||
|
||||
|
||||
Result "perf.Kotlin.streamWrappedInSequence":
|
||||
3829.209 ±(99.9%) 33.569 ms/op [Average]
|
||||
(min, avg, max) = (3787.684, 3829.209, 4022.427), stdev = 44.814
|
||||
CI (99.9%): [3795.640, 3862.779] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.Kotlin.withGenerator
|
||||
|
||||
# Run progress: 54.55% complete, ETA 00:44:00
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 10059.659 ms/op
|
||||
# Warmup Iteration 2: 8877.213 ms/op
|
||||
# Warmup Iteration 3: 8124.623 ms/op
|
||||
# Warmup Iteration 4: 8131.638 ms/op
|
||||
# Warmup Iteration 5: 8148.939 ms/op
|
||||
Iteration 1: 8148.879 ms/op
|
||||
Iteration 2: 8055.587 ms/op
|
||||
Iteration 3: 8173.149 ms/op
|
||||
Iteration 4: 8169.328 ms/op
|
||||
Iteration 5: 8132.040 ms/op
|
||||
|
||||
# Run progress: 56.36% complete, ETA 00:42:55
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 10101.875 ms/op
|
||||
# Warmup Iteration 2: 8876.112 ms/op
|
||||
# Warmup Iteration 3: 8091.701 ms/op
|
||||
# Warmup Iteration 4: 8130.830 ms/op
|
||||
# Warmup Iteration 5: 8162.659 ms/op
|
||||
Iteration 1: 8112.434 ms/op
|
||||
Iteration 2: 8077.144 ms/op
|
||||
Iteration 3: 8134.760 ms/op
|
||||
Iteration 4: 8178.659 ms/op
|
||||
Iteration 5: 8122.861 ms/op
|
||||
|
||||
# Run progress: 58.18% complete, ETA 00:41:44
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 10088.192 ms/op
|
||||
# Warmup Iteration 2: 8938.462 ms/op
|
||||
# Warmup Iteration 3: 8169.118 ms/op
|
||||
# Warmup Iteration 4: 8110.177 ms/op
|
||||
# Warmup Iteration 5: 8120.017 ms/op
|
||||
Iteration 1: 8130.142 ms/op
|
||||
Iteration 2: 8146.593 ms/op
|
||||
Iteration 3: 8161.427 ms/op
|
||||
Iteration 4: 8180.565 ms/op
|
||||
Iteration 5: 8153.313 ms/op
|
||||
|
||||
# Run progress: 60.00% complete, ETA 00:40:28
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 12624.377 ms/op
|
||||
# Warmup Iteration 2: 8872.746 ms/op
|
||||
# Warmup Iteration 3: 8114.339 ms/op
|
||||
# Warmup Iteration 4: 8144.090 ms/op
|
||||
# Warmup Iteration 5: 8054.094 ms/op
|
||||
Iteration 1: 8134.096 ms/op
|
||||
Iteration 2: 8154.380 ms/op
|
||||
Iteration 3: 8130.686 ms/op
|
||||
Iteration 4: 14015.371 ms/op
|
||||
Iteration 5: 8114.715 ms/op
|
||||
|
||||
# Run progress: 61.82% complete, ETA 00:39:07
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 10102.610 ms/op
|
||||
# Warmup Iteration 2: 8880.406 ms/op
|
||||
# Warmup Iteration 3: 8116.952 ms/op
|
||||
# Warmup Iteration 4: 8223.242 ms/op
|
||||
# Warmup Iteration 5: 8149.098 ms/op
|
||||
Iteration 1: 8152.230 ms/op
|
||||
Iteration 2: 8110.804 ms/op
|
||||
Iteration 3: 8136.879 ms/op
|
||||
Iteration 4: 8178.455 ms/op
|
||||
Iteration 5: 8149.239 ms/op
|
||||
|
||||
|
||||
Result "perf.Kotlin.withGenerator":
|
||||
8374.149 ±(99.9%) 880.647 ms/op [Average]
|
||||
(min, avg, max) = (8055.587, 8374.149, 14015.371), stdev = 1175.639
|
||||
CI (99.9%): [7493.502, 9254.797] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.NonPrimitive.loop
|
||||
|
||||
# Run progress: 63.64% complete, ETA 00:37:42
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 454.126 ms/op
|
||||
# Warmup Iteration 2: 446.612 ms/op
|
||||
# Warmup Iteration 3: 445.577 ms/op
|
||||
# Warmup Iteration 4: 447.197 ms/op
|
||||
# Warmup Iteration 5: 445.174 ms/op
|
||||
Iteration 1: 444.831 ms/op
|
||||
Iteration 2: 446.933 ms/op
|
||||
Iteration 3: 445.256 ms/op
|
||||
Iteration 4: 444.518 ms/op
|
||||
Iteration 5: 447.351 ms/op
|
||||
|
||||
# Run progress: 65.45% complete, ETA 00:35:44
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 453.077 ms/op
|
||||
# Warmup Iteration 2: 445.972 ms/op
|
||||
# Warmup Iteration 3: 448.129 ms/op
|
||||
# Warmup Iteration 4: 446.268 ms/op
|
||||
# Warmup Iteration 5: 444.718 ms/op
|
||||
Iteration 1: 447.130 ms/op
|
||||
Iteration 2: 445.217 ms/op
|
||||
Iteration 3: 445.393 ms/op
|
||||
Iteration 4: 447.177 ms/op
|
||||
Iteration 5: 445.732 ms/op
|
||||
|
||||
# Run progress: 67.27% complete, ETA 00:33:46
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 451.974 ms/op
|
||||
# Warmup Iteration 2: 446.601 ms/op
|
||||
# Warmup Iteration 3: 445.653 ms/op
|
||||
# Warmup Iteration 4: 448.023 ms/op
|
||||
# Warmup Iteration 5: 458.473 ms/op
|
||||
Iteration 1: 447.401 ms/op
|
||||
Iteration 2: 445.530 ms/op
|
||||
Iteration 3: 446.670 ms/op
|
||||
Iteration 4: 444.751 ms/op
|
||||
Iteration 5: 446.407 ms/op
|
||||
|
||||
# Run progress: 69.09% complete, ETA 00:31:49
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 452.980 ms/op
|
||||
# Warmup Iteration 2: 444.762 ms/op
|
||||
# Warmup Iteration 3: 447.634 ms/op
|
||||
# Warmup Iteration 4: 446.151 ms/op
|
||||
# Warmup Iteration 5: 444.367 ms/op
|
||||
Iteration 1: 446.652 ms/op
|
||||
Iteration 2: 445.799 ms/op
|
||||
Iteration 3: 445.204 ms/op
|
||||
Iteration 4: 446.163 ms/op
|
||||
Iteration 5: 446.287 ms/op
|
||||
|
||||
# Run progress: 70.91% complete, ETA 00:29:53
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 451.592 ms/op
|
||||
# Warmup Iteration 2: 448.889 ms/op
|
||||
# Warmup Iteration 3: 445.963 ms/op
|
||||
# Warmup Iteration 4: 445.615 ms/op
|
||||
# Warmup Iteration 5: 447.028 ms/op
|
||||
Iteration 1: 445.457 ms/op
|
||||
Iteration 2: 445.110 ms/op
|
||||
Iteration 3: 446.664 ms/op
|
||||
Iteration 4: 445.987 ms/op
|
||||
Iteration 5: 446.179 ms/op
|
||||
|
||||
|
||||
Result "perf.NonPrimitive.loop":
|
||||
445.992 ±(99.9%) 0.642 ms/op [Average]
|
||||
(min, avg, max) = (444.518, 445.992, 447.401), stdev = 0.858
|
||||
CI (99.9%): [445.349, 446.634] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.NonPrimitive.parallelStream
|
||||
|
||||
# Run progress: 72.73% complete, ETA 00:27:57
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 31838.855 ms/op
|
||||
# Warmup Iteration 2: 34561.656 ms/op
|
||||
# Warmup Iteration 3: 35669.703 ms/op
|
||||
# Warmup Iteration 4: 34349.624 ms/op
|
||||
# Warmup Iteration 5: 33726.583 ms/op
|
||||
Iteration 1: 33877.354 ms/op
|
||||
Iteration 2: 33822.699 ms/op
|
||||
Iteration 3: 33778.386 ms/op
|
||||
Iteration 4: 34008.675 ms/op
|
||||
Iteration 5: 33856.945 ms/op
|
||||
|
||||
# Run progress: 74.55% complete, ETA 00:27:23
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 32494.459 ms/op
|
||||
# Warmup Iteration 2: 36153.069 ms/op
|
||||
# Warmup Iteration 3: 36796.548 ms/op
|
||||
# Warmup Iteration 4: 35025.518 ms/op
|
||||
# Warmup Iteration 5: 33258.558 ms/op
|
||||
Iteration 1: 33554.978 ms/op
|
||||
Iteration 2: 33424.136 ms/op
|
||||
Iteration 3: 33627.128 ms/op
|
||||
Iteration 4: 34313.959 ms/op
|
||||
Iteration 5: 34112.359 ms/op
|
||||
|
||||
# Run progress: 76.36% complete, ETA 00:26:36
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 31567.609 ms/op
|
||||
# Warmup Iteration 2: 34712.718 ms/op
|
||||
# Warmup Iteration 3: 33958.542 ms/op
|
||||
# Warmup Iteration 4: 34035.515 ms/op
|
||||
# Warmup Iteration 5: 34438.473 ms/op
|
||||
Iteration 1: 33670.314 ms/op
|
||||
Iteration 2: 33864.096 ms/op
|
||||
Iteration 3: 34302.070 ms/op
|
||||
Iteration 4: 33971.974 ms/op
|
||||
Iteration 5: 33560.701 ms/op
|
||||
|
||||
# Run progress: 78.18% complete, ETA 00:25:33
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 31742.222 ms/op
|
||||
# Warmup Iteration 2: 34731.781 ms/op
|
||||
# Warmup Iteration 3: 33450.090 ms/op
|
||||
# Warmup Iteration 4: 33130.877 ms/op
|
||||
# Warmup Iteration 5: 33546.489 ms/op
|
||||
Iteration 1: 33366.136 ms/op
|
||||
Iteration 2: 33895.648 ms/op
|
||||
Iteration 3: 34376.907 ms/op
|
||||
Iteration 4: 33857.023 ms/op
|
||||
Iteration 5: 33111.330 ms/op
|
||||
|
||||
# Run progress: 80.00% complete, ETA 00:24:17
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 31518.688 ms/op
|
||||
# Warmup Iteration 2: 34623.682 ms/op
|
||||
# Warmup Iteration 3: 34725.832 ms/op
|
||||
# Warmup Iteration 4: 33545.596 ms/op
|
||||
# Warmup Iteration 5: 34196.732 ms/op
|
||||
Iteration 1: 34093.607 ms/op
|
||||
Iteration 2: 33647.008 ms/op
|
||||
Iteration 3: 34429.111 ms/op
|
||||
Iteration 4: 33643.974 ms/op
|
||||
Iteration 5: 34256.452 ms/op
|
||||
|
||||
|
||||
Result "perf.NonPrimitive.parallelStream":
|
||||
33856.919 ±(99.9%) 249.911 ms/op [Average]
|
||||
(min, avg, max) = (33111.330, 33856.919, 34429.111), stdev = 333.625
|
||||
CI (99.9%): [33607.007, 34106.830] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.NonPrimitive.sequence
|
||||
|
||||
# Run progress: 81.82% complete, ETA 00:22:51
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 29376.336 ms/op
|
||||
# Warmup Iteration 2: 27208.353 ms/op
|
||||
# Warmup Iteration 3: 27221.251 ms/op
|
||||
# Warmup Iteration 4: 27217.165 ms/op
|
||||
# Warmup Iteration 5: 27146.583 ms/op
|
||||
Iteration 1: 27302.438 ms/op
|
||||
Iteration 2: 27158.834 ms/op
|
||||
Iteration 3: 29219.908 ms/op
|
||||
Iteration 4: 27236.755 ms/op
|
||||
Iteration 5: 27213.294 ms/op
|
||||
|
||||
# Run progress: 83.64% complete, ETA 00:21:01
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 27179.841 ms/op
|
||||
# Warmup Iteration 2: 27104.263 ms/op
|
||||
# Warmup Iteration 3: 27025.071 ms/op
|
||||
# Warmup Iteration 4: 27029.083 ms/op
|
||||
# Warmup Iteration 5: 27048.692 ms/op
|
||||
Iteration 1: 27151.655 ms/op
|
||||
Iteration 2: 27469.413 ms/op
|
||||
Iteration 3: 26944.107 ms/op
|
||||
Iteration 4: 27028.756 ms/op
|
||||
Iteration 5: 27077.869 ms/op
|
||||
|
||||
# Run progress: 85.45% complete, ETA 00:19:03
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 27227.611 ms/op
|
||||
# Warmup Iteration 2: 27068.450 ms/op
|
||||
# Warmup Iteration 3: 27032.028 ms/op
|
||||
# Warmup Iteration 4: 28617.949 ms/op
|
||||
# Warmup Iteration 5: 26948.167 ms/op
|
||||
Iteration 1: 27025.160 ms/op
|
||||
Iteration 2: 27070.999 ms/op
|
||||
Iteration 3: 27105.065 ms/op
|
||||
Iteration 4: 27025.640 ms/op
|
||||
Iteration 5: 26987.588 ms/op
|
||||
|
||||
# Run progress: 87.27% complete, ETA 00:16:59
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 27227.761 ms/op
|
||||
# Warmup Iteration 2: 27007.131 ms/op
|
||||
# Warmup Iteration 3: 27072.111 ms/op
|
||||
# Warmup Iteration 4: 27429.696 ms/op
|
||||
# Warmup Iteration 5: 27052.873 ms/op
|
||||
Iteration 1: 27165.902 ms/op
|
||||
Iteration 2: 27522.829 ms/op
|
||||
Iteration 3: 27791.518 ms/op
|
||||
Iteration 4: 27459.632 ms/op
|
||||
Iteration 5: 27263.962 ms/op
|
||||
|
||||
# Run progress: 89.09% complete, ETA 00:14:49
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 27245.157 ms/op
|
||||
# Warmup Iteration 2: 27099.383 ms/op
|
||||
# Warmup Iteration 3: 27043.463 ms/op
|
||||
# Warmup Iteration 4: 26944.143 ms/op
|
||||
# Warmup Iteration 5: 27185.798 ms/op
|
||||
Iteration 1: 27204.744 ms/op
|
||||
Iteration 2: 26961.133 ms/op
|
||||
Iteration 3: 26975.095 ms/op
|
||||
Iteration 4: 26976.901 ms/op
|
||||
Iteration 5: 27095.789 ms/op
|
||||
|
||||
|
||||
Result "perf.NonPrimitive.sequence":
|
||||
27257.399 ±(99.9%) 342.686 ms/op [Average]
|
||||
(min, avg, max) = (26944.107, 27257.399, 29219.908), stdev = 457.476
|
||||
CI (99.9%): [26914.713, 27600.086] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.NonPrimitive.stream
|
||||
|
||||
# Run progress: 90.91% complete, ETA 00:12:33
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 37907.647 ms/op
|
||||
# Warmup Iteration 2: 44312.088 ms/op
|
||||
# Warmup Iteration 3: 44351.894 ms/op
|
||||
# Warmup Iteration 4: 45600.829 ms/op
|
||||
# Warmup Iteration 5: 44292.840 ms/op
|
||||
Iteration 1: 48919.294 ms/op
|
||||
Iteration 2: 50229.844 ms/op
|
||||
Iteration 3: 46366.206 ms/op
|
||||
Iteration 4: 46053.252 ms/op
|
||||
Iteration 5: 44309.572 ms/op
|
||||
|
||||
# Run progress: 92.73% complete, ETA 00:10:26
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 36888.253 ms/op
|
||||
# Warmup Iteration 2: 44609.411 ms/op
|
||||
# Warmup Iteration 3: 43528.954 ms/op
|
||||
# Warmup Iteration 4: 44225.429 ms/op
|
||||
# Warmup Iteration 5: 43411.202 ms/op
|
||||
Iteration 1: 43504.814 ms/op
|
||||
Iteration 2: 43979.944 ms/op
|
||||
Iteration 3: 43733.566 ms/op
|
||||
Iteration 4: 43561.106 ms/op
|
||||
Iteration 5: 44148.817 ms/op
|
||||
|
||||
# Run progress: 94.55% complete, ETA 00:08:05
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 44621.184 ms/op
|
||||
# Warmup Iteration 2: 43440.425 ms/op
|
||||
# Warmup Iteration 3: 43381.843 ms/op
|
||||
# Warmup Iteration 4: 43383.749 ms/op
|
||||
# Warmup Iteration 5: 43329.551 ms/op
|
||||
Iteration 1: 43357.752 ms/op
|
||||
Iteration 2: 43427.313 ms/op
|
||||
Iteration 3: 43366.149 ms/op
|
||||
Iteration 4: 44638.673 ms/op
|
||||
Iteration 5: 43953.000 ms/op
|
||||
|
||||
# Run progress: 96.36% complete, ETA 00:05:34
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 37357.182 ms/op
|
||||
# Warmup Iteration 2: 43539.552 ms/op
|
||||
# Warmup Iteration 3: 43484.783 ms/op
|
||||
# Warmup Iteration 4: 44086.420 ms/op
|
||||
# Warmup Iteration 5: 43492.440 ms/op
|
||||
Iteration 1: 47195.842 ms/op
|
||||
Iteration 2: 43471.970 ms/op
|
||||
Iteration 3: 43533.048 ms/op
|
||||
Iteration 4: 44247.170 ms/op
|
||||
Iteration 5: 43474.395 ms/op
|
||||
|
||||
# Run progress: 98.18% complete, ETA 00:02:52
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 46786.069 ms/op
|
||||
# Warmup Iteration 2: 44113.438 ms/op
|
||||
# Warmup Iteration 3: 44319.157 ms/op
|
||||
# Warmup Iteration 4: 44343.082 ms/op
|
||||
# Warmup Iteration 5: 44142.656 ms/op
|
||||
Iteration 1: 44516.563 ms/op
|
||||
Iteration 2: 44090.180 ms/op
|
||||
Iteration 3: 44287.859 ms/op
|
||||
Iteration 4: 44238.998 ms/op
|
||||
Iteration 5: 44227.628 ms/op
|
||||
|
||||
|
||||
Result "perf.NonPrimitive.stream":
|
||||
44673.318 ±(99.9%) 1325.832 ms/op [Average]
|
||||
(min, avg, max) = (43357.752, 44673.318, 50229.844), stdev = 1769.948
|
||||
CI (99.9%): [43347.486, 45999.150] (assumes normal distribution)
|
||||
|
||||
|
||||
# Run complete. Total time: 02:42:16
|
||||
|
||||
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
|
||||
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
|
||||
experiments, perform baseline and negative tests that provide experimental control, make sure
|
||||
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
|
||||
Do not assume the numbers tell you what you want them to tell.
|
||||
|
||||
Benchmark Mode Cnt Score Error Units
|
||||
Java.loop avgt 25 446.055 ± 0.677 ms/op
|
||||
Java.stream avgt 25 601.424 ± 12.606 ms/op
|
||||
Kotlin.loop avgt 25 446.600 ± 1.164 ms/op
|
||||
Kotlin.sequence avgt 25 2732.604 ± 6.644 ms/op
|
||||
Kotlin.stream avgt 25 593.353 ± 1.408 ms/op
|
||||
Kotlin.streamWrappedInSequence avgt 25 3829.209 ± 33.569 ms/op
|
||||
Kotlin.withGenerator avgt 25 8374.149 ± 880.647 ms/op
|
||||
NonPrimitive.loop avgt 25 445.992 ± 0.642 ms/op
|
||||
NonPrimitive.parallelStream avgt 25 33856.919 ± 249.911 ms/op
|
||||
NonPrimitive.sequence avgt 25 27257.399 ± 342.686 ms/op
|
||||
NonPrimitive.stream avgt 25 44673.318 ± 1325.832 ms/op
|
||||
@ -1,765 +0,0 @@
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.Java.loop
|
||||
|
||||
# Run progress: 0.00% complete, ETA 01:23:20
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 503.008 ms/op
|
||||
# Warmup Iteration 2: 497.339 ms/op
|
||||
# Warmup Iteration 3: 497.504 ms/op
|
||||
# Warmup Iteration 4: 526.782 ms/op
|
||||
# Warmup Iteration 5: 515.432 ms/op
|
||||
Iteration 1: 501.527 ms/op
|
||||
Iteration 2: 499.508 ms/op
|
||||
Iteration 3: 498.462 ms/op
|
||||
Iteration 4: 492.443 ms/op
|
||||
Iteration 5: 497.703 ms/op
|
||||
|
||||
# Run progress: 2.00% complete, ETA 01:24:21
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 498.714 ms/op
|
||||
# Warmup Iteration 2: 494.639 ms/op
|
||||
# Warmup Iteration 3: 494.410 ms/op
|
||||
# Warmup Iteration 4: 496.832 ms/op
|
||||
# Warmup Iteration 5: 528.760 ms/op
|
||||
Iteration 1: 495.342 ms/op
|
||||
Iteration 2: 498.370 ms/op
|
||||
Iteration 3: 495.366 ms/op
|
||||
Iteration 4: 496.939 ms/op
|
||||
Iteration 5: 496.600 ms/op
|
||||
|
||||
# Run progress: 4.00% complete, ETA 01:22:57
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 510.115 ms/op
|
||||
# Warmup Iteration 2: 502.292 ms/op
|
||||
# Warmup Iteration 3: 503.220 ms/op
|
||||
# Warmup Iteration 4: 500.617 ms/op
|
||||
# Warmup Iteration 5: 514.586 ms/op
|
||||
Iteration 1: 511.165 ms/op
|
||||
Iteration 2: 494.087 ms/op
|
||||
Iteration 3: 494.140 ms/op
|
||||
Iteration 4: 494.629 ms/op
|
||||
Iteration 5: 497.200 ms/op
|
||||
|
||||
# Run progress: 6.00% complete, ETA 01:20:56
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 513.161 ms/op
|
||||
# Warmup Iteration 2: 495.656 ms/op
|
||||
# Warmup Iteration 3: 494.638 ms/op
|
||||
# Warmup Iteration 4: 490.464 ms/op
|
||||
# Warmup Iteration 5: 500.923 ms/op
|
||||
Iteration 1: 496.505 ms/op
|
||||
Iteration 2: 493.417 ms/op
|
||||
Iteration 3: 497.258 ms/op
|
||||
Iteration 4: 503.422 ms/op
|
||||
Iteration 5: 496.757 ms/op
|
||||
|
||||
# Run progress: 8.00% complete, ETA 01:19:13
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 508.611 ms/op
|
||||
# Warmup Iteration 2: 497.342 ms/op
|
||||
# Warmup Iteration 3: 515.634 ms/op
|
||||
# Warmup Iteration 4: 496.081 ms/op
|
||||
# Warmup Iteration 5: 501.787 ms/op
|
||||
Iteration 1: 495.476 ms/op
|
||||
Iteration 2: 511.040 ms/op
|
||||
Iteration 3: 494.175 ms/op
|
||||
Iteration 4: 493.056 ms/op
|
||||
Iteration 5: 497.224 ms/op
|
||||
|
||||
|
||||
Result "perf.Java.loop":
|
||||
497.672 ±(99.9%) 3.576 ms/op [Average]
|
||||
(min, avg, max) = (492.443, 497.672, 511.165), stdev = 4.773
|
||||
CI (99.9%): [494.097, 501.248] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.Java.stream
|
||||
|
||||
# Run progress: 10.00% complete, ETA 01:17:30
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 752.945 ms/op
|
||||
# Warmup Iteration 2: 655.457 ms/op
|
||||
# Warmup Iteration 3: 656.324 ms/op
|
||||
# Warmup Iteration 4: 659.984 ms/op
|
||||
# Warmup Iteration 5: 653.979 ms/op
|
||||
Iteration 1: 702.193 ms/op
|
||||
Iteration 2: 658.415 ms/op
|
||||
Iteration 3: 684.926 ms/op
|
||||
Iteration 4: 679.830 ms/op
|
||||
Iteration 5: 682.021 ms/op
|
||||
|
||||
# Run progress: 12.00% complete, ETA 01:15:56
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 751.638 ms/op
|
||||
# Warmup Iteration 2: 663.695 ms/op
|
||||
# Warmup Iteration 3: 663.717 ms/op
|
||||
# Warmup Iteration 4: 667.699 ms/op
|
||||
# Warmup Iteration 5: 657.080 ms/op
|
||||
Iteration 1: 657.204 ms/op
|
||||
Iteration 2: 669.600 ms/op
|
||||
Iteration 3: 661.893 ms/op
|
||||
Iteration 4: 658.053 ms/op
|
||||
Iteration 5: 677.995 ms/op
|
||||
|
||||
# Run progress: 14.00% complete, ETA 01:14:17
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 753.697 ms/op
|
||||
# Warmup Iteration 2: 662.388 ms/op
|
||||
# Warmup Iteration 3: 665.430 ms/op
|
||||
# Warmup Iteration 4: 656.421 ms/op
|
||||
# Warmup Iteration 5: 660.469 ms/op
|
||||
Iteration 1: 726.873 ms/op
|
||||
Iteration 2: 675.244 ms/op
|
||||
Iteration 3: 664.811 ms/op
|
||||
Iteration 4: 656.496 ms/op
|
||||
Iteration 5: 655.133 ms/op
|
||||
|
||||
# Run progress: 16.00% complete, ETA 01:12:41
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 758.258 ms/op
|
||||
# Warmup Iteration 2: 658.559 ms/op
|
||||
# Warmup Iteration 3: 653.749 ms/op
|
||||
# Warmup Iteration 4: 676.368 ms/op
|
||||
# Warmup Iteration 5: 653.075 ms/op
|
||||
Iteration 1: 694.018 ms/op
|
||||
Iteration 2: 663.863 ms/op
|
||||
Iteration 3: 654.269 ms/op
|
||||
Iteration 4: 657.201 ms/op
|
||||
Iteration 5: 682.597 ms/op
|
||||
|
||||
# Run progress: 18.00% complete, ETA 01:11:04
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 755.350 ms/op
|
||||
# Warmup Iteration 2: 673.888 ms/op
|
||||
# Warmup Iteration 3: 655.083 ms/op
|
||||
# Warmup Iteration 4: 658.737 ms/op
|
||||
# Warmup Iteration 5: 657.309 ms/op
|
||||
Iteration 1: 671.508 ms/op
|
||||
Iteration 2: 673.933 ms/op
|
||||
Iteration 3: 654.189 ms/op
|
||||
Iteration 4: 654.793 ms/op
|
||||
Iteration 5: 658.981 ms/op
|
||||
|
||||
|
||||
Result "perf.Java.stream":
|
||||
671.042 ±(99.9%) 13.278 ms/op [Average]
|
||||
(min, avg, max) = (654.189, 671.042, 726.873), stdev = 17.726
|
||||
CI (99.9%): [657.763, 684.320] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.Kotlin.loop
|
||||
|
||||
# Run progress: 20.00% complete, ETA 01:09:23
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 501.288 ms/op
|
||||
# Warmup Iteration 2: 519.426 ms/op
|
||||
# Warmup Iteration 3: 506.916 ms/op
|
||||
# Warmup Iteration 4: 494.905 ms/op
|
||||
# Warmup Iteration 5: 495.065 ms/op
|
||||
Iteration 1: 505.477 ms/op
|
||||
Iteration 2: 499.944 ms/op
|
||||
Iteration 3: 502.763 ms/op
|
||||
Iteration 4: 494.921 ms/op
|
||||
Iteration 5: 491.712 ms/op
|
||||
|
||||
# Run progress: 22.00% complete, ETA 01:07:35
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 519.974 ms/op
|
||||
# Warmup Iteration 2: 495.087 ms/op
|
||||
# Warmup Iteration 3: 493.488 ms/op
|
||||
# Warmup Iteration 4: 503.083 ms/op
|
||||
# Warmup Iteration 5: 494.544 ms/op
|
||||
Iteration 1: 498.104 ms/op
|
||||
Iteration 2: 494.670 ms/op
|
||||
Iteration 3: 494.472 ms/op
|
||||
Iteration 4: 496.515 ms/op
|
||||
Iteration 5: 500.076 ms/op
|
||||
|
||||
# Run progress: 24.00% complete, ETA 01:05:49
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 502.969 ms/op
|
||||
# Warmup Iteration 2: 515.297 ms/op
|
||||
# Warmup Iteration 3: 500.498 ms/op
|
||||
# Warmup Iteration 4: 504.327 ms/op
|
||||
# Warmup Iteration 5: 495.414 ms/op
|
||||
Iteration 1: 492.781 ms/op
|
||||
Iteration 2: 499.457 ms/op
|
||||
Iteration 3: 497.536 ms/op
|
||||
Iteration 4: 506.625 ms/op
|
||||
Iteration 5: 488.861 ms/op
|
||||
|
||||
# Run progress: 26.00% complete, ETA 01:04:02
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 499.845 ms/op
|
||||
# Warmup Iteration 2: 500.112 ms/op
|
||||
# Warmup Iteration 3: 500.099 ms/op
|
||||
# Warmup Iteration 4: 502.852 ms/op
|
||||
# Warmup Iteration 5: 495.422 ms/op
|
||||
Iteration 1: 520.156 ms/op
|
||||
Iteration 2: 493.320 ms/op
|
||||
Iteration 3: 510.862 ms/op
|
||||
Iteration 4: 497.780 ms/op
|
||||
Iteration 5: 480.626 ms/op
|
||||
|
||||
# Run progress: 28.00% complete, ETA 01:02:15
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 507.837 ms/op
|
||||
# Warmup Iteration 2: 501.298 ms/op
|
||||
# Warmup Iteration 3: 496.518 ms/op
|
||||
# Warmup Iteration 4: 499.621 ms/op
|
||||
# Warmup Iteration 5: 506.226 ms/op
|
||||
Iteration 1: 492.717 ms/op
|
||||
Iteration 2: 491.870 ms/op
|
||||
Iteration 3: 523.607 ms/op
|
||||
Iteration 4: 496.943 ms/op
|
||||
Iteration 5: 492.535 ms/op
|
||||
|
||||
|
||||
Result "perf.Kotlin.loop":
|
||||
498.573 ±(99.9%) 6.962 ms/op [Average]
|
||||
(min, avg, max) = (480.626, 498.573, 523.607), stdev = 9.294
|
||||
CI (99.9%): [491.611, 505.535] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.Kotlin.stream
|
||||
|
||||
# Run progress: 40.00% complete, ETA 00:51:37
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 770.439 ms/op
|
||||
# Warmup Iteration 2: 654.815 ms/op
|
||||
# Warmup Iteration 3: 659.349 ms/op
|
||||
# Warmup Iteration 4: 663.637 ms/op
|
||||
# Warmup Iteration 5: 657.116 ms/op
|
||||
Iteration 1: 658.083 ms/op
|
||||
Iteration 2: 662.591 ms/op
|
||||
Iteration 3: 674.376 ms/op
|
||||
Iteration 4: 656.337 ms/op
|
||||
Iteration 5: 657.640 ms/op
|
||||
|
||||
# Run progress: 42.00% complete, ETA 00:49:56
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 769.538 ms/op
|
||||
# Warmup Iteration 2: 654.025 ms/op
|
||||
# Warmup Iteration 3: 663.950 ms/op
|
||||
# Warmup Iteration 4: 661.340 ms/op
|
||||
# Warmup Iteration 5: 721.330 ms/op
|
||||
Iteration 1: 654.068 ms/op
|
||||
Iteration 2: 666.005 ms/op
|
||||
Iteration 3: 658.865 ms/op
|
||||
Iteration 4: 657.615 ms/op
|
||||
Iteration 5: 656.568 ms/op
|
||||
|
||||
# Run progress: 44.00% complete, ETA 00:48:14
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 773.322 ms/op
|
||||
# Warmup Iteration 2: 655.102 ms/op
|
||||
# Warmup Iteration 3: 660.459 ms/op
|
||||
# Warmup Iteration 4: 656.585 ms/op
|
||||
# Warmup Iteration 5: 715.147 ms/op
|
||||
Iteration 1: 654.845 ms/op
|
||||
Iteration 2: 655.245 ms/op
|
||||
Iteration 3: 662.457 ms/op
|
||||
Iteration 4: 664.278 ms/op
|
||||
Iteration 5: 692.333 ms/op
|
||||
|
||||
# Run progress: 46.00% complete, ETA 00:46:32
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 751.623 ms/op
|
||||
# Warmup Iteration 2: 655.612 ms/op
|
||||
# Warmup Iteration 3: 693.453 ms/op
|
||||
# Warmup Iteration 4: 652.434 ms/op
|
||||
# Warmup Iteration 5: 664.927 ms/op
|
||||
Iteration 1: 655.975 ms/op
|
||||
Iteration 2: 661.740 ms/op
|
||||
Iteration 3: 661.267 ms/op
|
||||
Iteration 4: 656.503 ms/op
|
||||
Iteration 5: 663.981 ms/op
|
||||
|
||||
# Run progress: 48.00% complete, ETA 00:44:51
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 755.101 ms/op
|
||||
# Warmup Iteration 2: 659.349 ms/op
|
||||
# Warmup Iteration 3: 659.407 ms/op
|
||||
# Warmup Iteration 4: 697.422 ms/op
|
||||
# Warmup Iteration 5: 653.759 ms/op
|
||||
Iteration 1: 654.175 ms/op
|
||||
Iteration 2: 661.572 ms/op
|
||||
Iteration 3: 653.442 ms/op
|
||||
Iteration 4: 668.452 ms/op
|
||||
Iteration 5: 681.106 ms/op
|
||||
|
||||
|
||||
Result "perf.Kotlin.stream":
|
||||
661.981 ±(99.9%) 6.825 ms/op [Average]
|
||||
(min, avg, max) = (653.442, 661.981, 692.333), stdev = 9.111
|
||||
CI (99.9%): [655.156, 668.805] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.Kotlin.streamWrappedInSequence
|
||||
|
||||
# Run progress: 50.00% complete, ETA 00:43:08
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 4771.596 ms/op
|
||||
# Warmup Iteration 2: 4469.354 ms/op
|
||||
# Warmup Iteration 3: 4503.951 ms/op
|
||||
# Warmup Iteration 4: 4642.394 ms/op
|
||||
# Warmup Iteration 5: 4570.822 ms/op
|
||||
Iteration 1: 4734.829 ms/op
|
||||
Iteration 2: 4448.201 ms/op
|
||||
Iteration 3: 4450.036 ms/op
|
||||
Iteration 4: 4649.510 ms/op
|
||||
Iteration 5: 4360.084 ms/op
|
||||
|
||||
# Run progress: 52.00% complete, ETA 00:41:56
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 4827.821 ms/op
|
||||
# Warmup Iteration 2: 4491.528 ms/op
|
||||
# Warmup Iteration 3: 4429.582 ms/op
|
||||
# Warmup Iteration 4: 4616.589 ms/op
|
||||
# Warmup Iteration 5: 4425.327 ms/op
|
||||
Iteration 1: 4536.462 ms/op
|
||||
Iteration 2: 4507.816 ms/op
|
||||
Iteration 3: 4448.700 ms/op
|
||||
Iteration 4: 4499.069 ms/op
|
||||
Iteration 5: 4481.440 ms/op
|
||||
|
||||
# Run progress: 54.00% complete, ETA 00:40:38
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 5100.392 ms/op
|
||||
# Warmup Iteration 2: 4510.386 ms/op
|
||||
# Warmup Iteration 3: 4520.391 ms/op
|
||||
# Warmup Iteration 4: 4455.398 ms/op
|
||||
# Warmup Iteration 5: 4438.041 ms/op
|
||||
Iteration 1: 4463.642 ms/op
|
||||
Iteration 2: 4513.981 ms/op
|
||||
Iteration 3: 4451.207 ms/op
|
||||
Iteration 4: 4592.493 ms/op
|
||||
Iteration 5: 4558.072 ms/op
|
||||
|
||||
# Run progress: 56.00% complete, ETA 00:39:12
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 4625.825 ms/op
|
||||
# Warmup Iteration 2: 4523.034 ms/op
|
||||
# Warmup Iteration 3: 4642.197 ms/op
|
||||
# Warmup Iteration 4: 4420.747 ms/op
|
||||
# Warmup Iteration 5: 4655.348 ms/op
|
||||
Iteration 1: 4655.329 ms/op
|
||||
Iteration 2: 4426.920 ms/op
|
||||
Iteration 3: 4483.991 ms/op
|
||||
Iteration 4: 4499.432 ms/op
|
||||
Iteration 5: 4537.659 ms/op
|
||||
|
||||
# Run progress: 58.00% complete, ETA 00:37:47
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 5112.802 ms/op
|
||||
# Warmup Iteration 2: 4446.181 ms/op
|
||||
# Warmup Iteration 3: 4580.273 ms/op
|
||||
# Warmup Iteration 4: 4410.598 ms/op
|
||||
# Warmup Iteration 5: 4403.912 ms/op
|
||||
Iteration 1: 4502.871 ms/op
|
||||
Iteration 2: 4654.358 ms/op
|
||||
Iteration 3: 4422.066 ms/op
|
||||
Iteration 4: 4356.905 ms/op
|
||||
Iteration 5: 4456.772 ms/op
|
||||
|
||||
|
||||
Result "perf.Kotlin.streamWrappedInSequence":
|
||||
4507.674 ±(99.9%) 69.124 ms/op [Average]
|
||||
(min, avg, max) = (4356.905, 4507.674, 4734.829), stdev = 92.279
|
||||
CI (99.9%): [4438.549, 4576.798] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.Kotlin.withGenerator
|
||||
|
||||
# Run progress: 60.00% complete, ETA 00:36:14
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 12972.312 ms/op
|
||||
# Warmup Iteration 2: 11150.766 ms/op
|
||||
# Warmup Iteration 3: 9243.075 ms/op
|
||||
# Warmup Iteration 4: 9264.701 ms/op
|
||||
# Warmup Iteration 5: 11170.115 ms/op
|
||||
Iteration 1: 9341.833 ms/op
|
||||
Iteration 2: 9283.496 ms/op
|
||||
Iteration 3: 9174.673 ms/op
|
||||
Iteration 4: 9202.388 ms/op
|
||||
Iteration 5: 9320.724 ms/op
|
||||
|
||||
# Run progress: 62.00% complete, ETA 00:35:07
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 11221.832 ms/op
|
||||
# Warmup Iteration 2: 14477.742 ms/op
|
||||
# Warmup Iteration 3: 9534.827 ms/op
|
||||
# Warmup Iteration 4: 9291.195 ms/op
|
||||
# Warmup Iteration 5: 9361.180 ms/op
|
||||
Iteration 1: 13425.934 ms/op
|
||||
Iteration 2: 9402.840 ms/op
|
||||
Iteration 3: 9205.030 ms/op
|
||||
Iteration 4: 9145.462 ms/op
|
||||
Iteration 5: 9412.641 ms/op
|
||||
|
||||
# Run progress: 64.00% complete, ETA 00:33:49
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 11481.680 ms/op
|
||||
# Warmup Iteration 2: 11080.843 ms/op
|
||||
# Warmup Iteration 3: 9253.032 ms/op
|
||||
# Warmup Iteration 4: 9399.855 ms/op
|
||||
# Warmup Iteration 5: 8943.780 ms/op
|
||||
Iteration 1: 9482.955 ms/op
|
||||
Iteration 2: 9124.821 ms/op
|
||||
Iteration 3: 9198.032 ms/op
|
||||
Iteration 4: 9194.677 ms/op
|
||||
Iteration 5: 9252.317 ms/op
|
||||
|
||||
# Run progress: 66.00% complete, ETA 00:32:26
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 11560.755 ms/op
|
||||
# Warmup Iteration 2: 10979.804 ms/op
|
||||
# Warmup Iteration 3: 8778.534 ms/op
|
||||
# Warmup Iteration 4: 9271.147 ms/op
|
||||
# Warmup Iteration 5: 8962.573 ms/op
|
||||
Iteration 1: 8849.864 ms/op
|
||||
Iteration 2: 8943.724 ms/op
|
||||
Iteration 3: 8956.114 ms/op
|
||||
Iteration 4: 8851.836 ms/op
|
||||
Iteration 5: 8918.313 ms/op
|
||||
|
||||
# Run progress: 68.00% complete, ETA 00:30:56
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 11657.552 ms/op
|
||||
# Warmup Iteration 2: 14300.899 ms/op
|
||||
# Warmup Iteration 3: 9345.045 ms/op
|
||||
# Warmup Iteration 4: 9167.124 ms/op
|
||||
# Warmup Iteration 5: 9481.174 ms/op
|
||||
Iteration 1: 9249.084 ms/op
|
||||
Iteration 2: 9231.042 ms/op
|
||||
Iteration 3: 9360.817 ms/op
|
||||
Iteration 4: 9328.045 ms/op
|
||||
Iteration 5: 9094.625 ms/op
|
||||
|
||||
|
||||
Result "perf.Kotlin.withGenerator":
|
||||
9358.051 ±(99.9%) 647.987 ms/op [Average]
|
||||
(min, avg, max) = (8849.864, 9358.051, 13425.934), stdev = 865.044
|
||||
CI (99.9%): [8710.065, 10006.038] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.NonPrimitive.parallelStream
|
||||
|
||||
# Run progress: 70.00% complete, ETA 00:29:25
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 45622.300 ms/op
|
||||
# Warmup Iteration 2: 51268.916 ms/op
|
||||
# Warmup Iteration 3: 54557.341 ms/op
|
||||
# Warmup Iteration 4: 53486.933 ms/op
|
||||
# Warmup Iteration 5: 49141.534 ms/op
|
||||
Iteration 1: 48819.449 ms/op
|
||||
Iteration 2: 49136.332 ms/op
|
||||
Iteration 3: 50845.214 ms/op
|
||||
Iteration 4: 49430.190 ms/op
|
||||
Iteration 5: 48870.124 ms/op
|
||||
|
||||
# Run progress: 72.00% complete, ETA 00:29:57
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 50170.272 ms/op
|
||||
# Warmup Iteration 2: 53994.209 ms/op
|
||||
# Warmup Iteration 3: 53407.886 ms/op
|
||||
# Warmup Iteration 4: 51364.240 ms/op
|
||||
# Warmup Iteration 5: 51227.612 ms/op
|
||||
Iteration 1: 51931.093 ms/op
|
||||
Iteration 2: 50769.543 ms/op
|
||||
Iteration 3: 56751.344 ms/op
|
||||
Iteration 4: 52092.542 ms/op
|
||||
Iteration 5: 53091.324 ms/op
|
||||
|
||||
# Run progress: 74.00% complete, ETA 00:30:08
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 49080.774 ms/op
|
||||
# Warmup Iteration 2: 56156.551 ms/op
|
||||
# Warmup Iteration 3: 52052.861 ms/op
|
||||
# Warmup Iteration 4: 55415.614 ms/op
|
||||
# Warmup Iteration 5: 51782.162 ms/op
|
||||
Iteration 1: 51509.481 ms/op
|
||||
Iteration 2: 55253.665 ms/op
|
||||
Iteration 3: 50771.941 ms/op
|
||||
Iteration 4: 54707.276 ms/op
|
||||
Iteration 5: 51359.288 ms/op
|
||||
|
||||
# Run progress: 76.00% complete, ETA 00:29:52
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 50272.019 ms/op
|
||||
# Warmup Iteration 2: 57595.569 ms/op
|
||||
# Warmup Iteration 3: 59850.768 ms/op
|
||||
# Warmup Iteration 4: 54712.763 ms/op
|
||||
# Warmup Iteration 5: 53759.817 ms/op
|
||||
Iteration 1: 55310.711 ms/op
|
||||
Iteration 2: 52339.983 ms/op
|
||||
Iteration 3: 53796.593 ms/op
|
||||
Iteration 4: 51265.637 ms/op
|
||||
Iteration 5: 51163.343 ms/op
|
||||
|
||||
# Run progress: 78.00% complete, ETA 00:29:13
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 46287.662 ms/op
|
||||
# Warmup Iteration 2: 56583.661 ms/op
|
||||
# Warmup Iteration 3: 54958.161 ms/op
|
||||
# Warmup Iteration 4: 52485.864 ms/op
|
||||
# Warmup Iteration 5: 51463.926 ms/op
|
||||
Iteration 1: 59061.271 ms/op
|
||||
Iteration 2: 62958.959 ms/op
|
||||
Iteration 3: 54422.187 ms/op
|
||||
Iteration 4: 51840.252 ms/op
|
||||
Iteration 5: 52454.721 ms/op
|
||||
|
||||
|
||||
Result "perf.NonPrimitive.parallelStream":
|
||||
52798.098 ±(99.9%) 2451.723 ms/op [Average]
|
||||
(min, avg, max) = (48819.449, 52798.098, 62958.959), stdev = 3272.980
|
||||
CI (99.9%): [50346.375, 55249.822] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.NonPrimitive.sequence
|
||||
|
||||
# Run progress: 80.00% complete, ETA 00:28:09
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 31539.028 ms/op
|
||||
# Warmup Iteration 2: 32167.361 ms/op
|
||||
# Warmup Iteration 3: 31438.191 ms/op
|
||||
# Warmup Iteration 4: 31771.676 ms/op
|
||||
# Warmup Iteration 5: 31476.264 ms/op
|
||||
Iteration 1: 32394.913 ms/op
|
||||
Iteration 2: 31534.739 ms/op
|
||||
Iteration 3: 32371.211 ms/op
|
||||
Iteration 4: 31587.742 ms/op
|
||||
Iteration 5: 32116.086 ms/op
|
||||
|
||||
# Run progress: 82.00% complete, ETA 00:25:53
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 32147.157 ms/op
|
||||
# Warmup Iteration 2: 31512.206 ms/op
|
||||
# Warmup Iteration 3: 31765.894 ms/op
|
||||
# Warmup Iteration 4: 31205.766 ms/op
|
||||
# Warmup Iteration 5: 32011.824 ms/op
|
||||
Iteration 1: 31948.070 ms/op
|
||||
Iteration 2: 35102.995 ms/op
|
||||
Iteration 3: 31417.171 ms/op
|
||||
Iteration 4: 31561.729 ms/op
|
||||
Iteration 5: 31358.093 ms/op
|
||||
|
||||
# Run progress: 84.00% complete, ETA 00:23:29
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 31004.566 ms/op
|
||||
# Warmup Iteration 2: 31665.419 ms/op
|
||||
# Warmup Iteration 3: 31381.758 ms/op
|
||||
# Warmup Iteration 4: 31739.301 ms/op
|
||||
# Warmup Iteration 5: 33043.687 ms/op
|
||||
Iteration 1: 31251.636 ms/op
|
||||
Iteration 2: 34870.238 ms/op
|
||||
Iteration 3: 31329.340 ms/op
|
||||
Iteration 4: 31706.639 ms/op
|
||||
Iteration 5: 31342.416 ms/op
|
||||
|
||||
# Run progress: 86.00% complete, ETA 00:20:56
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 31439.350 ms/op
|
||||
# Warmup Iteration 2: 31796.581 ms/op
|
||||
# Warmup Iteration 3: 30981.395 ms/op
|
||||
# Warmup Iteration 4: 31014.350 ms/op
|
||||
# Warmup Iteration 5: 32090.784 ms/op
|
||||
Iteration 1: 35625.972 ms/op
|
||||
Iteration 2: 31931.136 ms/op
|
||||
Iteration 3: 31184.763 ms/op
|
||||
Iteration 4: 31694.722 ms/op
|
||||
Iteration 5: 31880.194 ms/op
|
||||
|
||||
# Run progress: 88.00% complete, ETA 00:18:16
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 32086.933 ms/op
|
||||
# Warmup Iteration 2: 31691.944 ms/op
|
||||
# Warmup Iteration 3: 31720.550 ms/op
|
||||
# Warmup Iteration 4: 31687.710 ms/op
|
||||
# Warmup Iteration 5: 31665.377 ms/op
|
||||
Iteration 1: 31649.628 ms/op
|
||||
Iteration 2: 31965.411 ms/op
|
||||
Iteration 3: 31552.680 ms/op
|
||||
Iteration 4: 31191.134 ms/op
|
||||
Iteration 5: 31948.312 ms/op
|
||||
|
||||
|
||||
Result "perf.NonPrimitive.sequence":
|
||||
32100.679 ±(99.9%) 913.441 ms/op [Average]
|
||||
(min, avg, max) = (31184.763, 32100.679, 35625.972), stdev = 1219.417
|
||||
CI (99.9%): [31187.238, 33014.120] (assumes normal distribution)
|
||||
|
||||
|
||||
# JMH version: 1.23
|
||||
# VM version: JDK 15-ea, OpenJDK 64-Bit Server VM, 15-ea+17-717
|
||||
# VM invoker: /usr/lib/jvm/java-15-openjdk/bin/java
|
||||
# VM options: <none>
|
||||
# Warmup: 5 iterations, 10 s each
|
||||
# Measurement: 5 iterations, 10 s each
|
||||
# Timeout: 10 min per iteration
|
||||
# Threads: 1 thread, will synchronize iterations
|
||||
# Benchmark mode: Average time, time/op
|
||||
# Benchmark: perf.NonPrimitive.stream
|
||||
|
||||
# Run progress: 90.00% complete, ETA 00:15:28
|
||||
# Fork: 1 of 5
|
||||
# Warmup Iteration 1: 43230.332 ms/op
|
||||
# Warmup Iteration 2: 50557.338 ms/op
|
||||
# Warmup Iteration 3: 50822.458 ms/op
|
||||
# Warmup Iteration 4: 50159.518 ms/op
|
||||
# Warmup Iteration 5: 50098.131 ms/op
|
||||
Iteration 1: 50816.292 ms/op
|
||||
Iteration 2: 50245.734 ms/op
|
||||
Iteration 3: 50341.716 ms/op
|
||||
Iteration 4: 50361.851 ms/op
|
||||
Iteration 5: 50362.410 ms/op
|
||||
|
||||
# Run progress: 92.00% complete, ETA 00:12:49
|
||||
# Fork: 2 of 5
|
||||
# Warmup Iteration 1: 53187.949 ms/op
|
||||
# Warmup Iteration 2: 51120.670 ms/op
|
||||
# Warmup Iteration 3: 51116.071 ms/op
|
||||
# Warmup Iteration 4: 51517.877 ms/op
|
||||
# Warmup Iteration 5: 51755.674 ms/op
|
||||
Iteration 1: 51481.433 ms/op
|
||||
Iteration 2: 51822.715 ms/op
|
||||
Iteration 3: 51509.446 ms/op
|
||||
Iteration 4: 51537.729 ms/op
|
||||
Iteration 5: 53196.225 ms/op
|
||||
|
||||
# Run progress: 94.00% complete, ETA 00:09:58
|
||||
# Fork: 3 of 5
|
||||
# Warmup Iteration 1: 43924.846 ms/op
|
||||
# Warmup Iteration 2: 50782.107 ms/op
|
||||
# Warmup Iteration 3: 50232.693 ms/op
|
||||
# Warmup Iteration 4: 49835.131 ms/op
|
||||
# Warmup Iteration 5: 50901.686 ms/op
|
||||
Iteration 1: 50166.901 ms/op
|
||||
Iteration 2: 50617.046 ms/op
|
||||
Iteration 3: 50144.998 ms/op
|
||||
Iteration 4: 50044.519 ms/op
|
||||
Iteration 5: 50792.107 ms/op
|
||||
|
||||
# Run progress: 96.00% complete, ETA 00:06:51
|
||||
# Fork: 4 of 5
|
||||
# Warmup Iteration 1: 44810.711 ms/op
|
||||
# Warmup Iteration 2: 51014.134 ms/op
|
||||
# Warmup Iteration 3: 50839.319 ms/op
|
||||
# Warmup Iteration 4: 51794.822 ms/op
|
||||
# Warmup Iteration 5: 51088.069 ms/op
|
||||
Iteration 1: 51650.679 ms/op
|
||||
Iteration 2: 51178.894 ms/op
|
||||
Iteration 3: 51247.988 ms/op
|
||||
Iteration 4: 50767.387 ms/op
|
||||
Iteration 5: 50682.972 ms/op
|
||||
|
||||
# Run progress: 98.00% complete, ETA 00:03:31
|
||||
# Fork: 5 of 5
|
||||
# Warmup Iteration 1: 42961.998 ms/op
|
||||
# Warmup Iteration 2: 50554.667 ms/op
|
||||
# Warmup Iteration 3: 49991.236 ms/op
|
||||
# Warmup Iteration 4: 49157.238 ms/op
|
||||
# Warmup Iteration 5: 50339.484 ms/op
|
||||
Iteration 1: 50025.853 ms/op
|
||||
Iteration 2: 50515.643 ms/op
|
||||
Iteration 3: 50206.781 ms/op
|
||||
Iteration 4: 49584.429 ms/op
|
||||
Iteration 5: 52719.464 ms/op
|
||||
|
||||
|
||||
Result "perf.NonPrimitive.stream":
|
||||
50880.849 ±(99.9%) 644.167 ms/op [Average]
|
||||
(min, avg, max) = (49584.429, 50880.849, 53196.225), stdev = 859.945
|
||||
CI (99.9%): [50236.681, 51525.016] (assumes normal distribution)
|
||||
|
||||
|
||||
# Run complete. Total time: 03:01:10
|
||||
|
||||
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
|
||||
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
|
||||
experiments, perform baseline and negative tests that provide experimental control, make sure
|
||||
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
|
||||
Do not assume the numbers tell you what you want them to tell.
|
||||
|
||||
Benchmark Mode Cnt Score Error Units
|
||||
Java.loop avgt 25 497.672 ± 3.576 ms/op
|
||||
Java.stream avgt 25 671.042 ± 13.278 ms/op
|
||||
Kotlin.loop avgt 25 498.573 ± 6.962 ms/op
|
||||
Kotlin.stream avgt 25 661.981 ± 6.825 ms/op
|
||||
Kotlin.streamWrappedInSequence avgt 25 4507.674 ± 69.124 ms/op
|
||||
Kotlin.withGenerator avgt 25 9358.051 ± 647.987 ms/op
|
||||
NonPrimitive.parallelStream avgt 25 52798.098 ± 2451.723 ms/op
|
||||
NonPrimitive.sequence avgt 25 32100.679 ± 913.441 ms/op
|
||||
NonPrimitive.stream avgt 25 50880.849 ± 644.167 ms/op
|
||||
360
wip/iterators.md
360
wip/iterators.md
@ -1,4 +1,4 @@
|
||||
# Stream, Sequence, Iterator – a story of laziness and sad benchmarking noises
|
||||
# Stream, Sequence, Iterator – a story of laziness and sad JVM benchmarking noises
|
||||
Many programming languages have started to include more functional features in their standard libraries.
|
||||
One of those features is lazy collections, for lack of a better term,
|
||||
which seem to have a different name in each language
|
||||
@ -12,39 +12,359 @@ For almost every language out there that offers lazy iterators,
|
||||
|
||||
I was personally interested in this because, being a Java/Kotlin developer,
|
||||
I use Java’s Streams and Kotlin’s Sequences almost every day
|
||||
with relatively little regard for performance.
|
||||
with relatively little regard for potential performance implications.
|
||||
They are intuitive to write and are easy to reason about,
|
||||
which is usually much more important than the results of a thousand microbenchmarks,
|
||||
so please don’t stop using your favorite language feature because it’s 2.8% slower than the primitive alternative.
|
||||
so please don’t stop using your favorite language feature because it’s 2.8% slower than the alternative.
|
||||
Most code is already bad enough as is without desperate optimization attempts.
|
||||
|
||||
Still, I wanted to know how they compare to imperative code.
|
||||
There are some resources on this for Java 8’s Stream API,
|
||||
but Kotlin’s Sequences seem to just be accepted as
|
||||
more convenient Streams[^convenience].
|
||||
Rust is here as a baseline for comparisons
|
||||
because it is generally regarded as having very optimized iterators.
|
||||
|
||||
[^convenience]: If you’ve ever used them, you’ll know what I mean.
|
||||
Java’s Streams are built in a way that allows for easy parallelism,
|
||||
but brings its own problems and limitations for the usage.
|
||||
We’ll see some of that in the code examples later.
|
||||
|
||||
## What *is* an iterator?
|
||||
You can think of an iterator as a pipeline of data.
|
||||
It’s not a list, so it doesn’t support indexing,
|
||||
because it doesn’t actually hold any data.
|
||||
It just contains information on how to get or make that data.
|
||||
You can make it produce data and use that
|
||||
(which is often called ‘consuming’ the iterator
|
||||
because if you read data from the pipeline, it’s usually gone),
|
||||
or you can add a new step to the pipeline and hand the result to someone else,
|
||||
who can then consume it or add even more operations.
|
||||
You also don’t know when (or if at all) an iterator will end.
|
||||
Someone could sit at the other end and constantly put new data into your pipeline.
|
||||
You can think of an iterator as a pipeline.
|
||||
It lets you write code as a sequence of instructions to be applied to all elements of a container.
|
||||
|
||||
An important aspect to note is: adding an operation to the pipeline does nothing
|
||||
Let’s use a simple example to demonstrate this.
|
||||
We want to take all numbers from 1 to 100,000,
|
||||
multiply each of them by 2,
|
||||
and then add all of them.[^sum]
|
||||
|
||||
[^sum]: You could also just compute the sum and take that \* 2, but we specifically want that intermediate step for the example.
|
||||
|
||||
First, the imperative solution:
|
||||
|
||||
```kotlin
|
||||
var sum = 0
|
||||
for (i in 1..100_000) {
|
||||
sum += i * 2
|
||||
}
|
||||
return sum
|
||||
```
|
||||
|
||||
and now the functional version using a Sequence (Kotlin’s name for Streams/iterators):
|
||||
|
||||
```kotlin
|
||||
return (1..100_000).asSequence()
|
||||
.map { it * 2 }
|
||||
.sum()
|
||||
```
|
||||
|
||||
An iterator not a list, and it doesn’t support indexing,[^index]
|
||||
because it doesn’t actually contain any data.
|
||||
It just knows how to get or compute it for you,
|
||||
but you don’t know how it does that.
|
||||
You don’t even always know when (or if at all) an iterator will end
|
||||
(in this case, we do, because we create the Sequence from the range `1..100_000`,
|
||||
meaning it will produce the numbers from 1 to 100,00 before it ends).
|
||||
You can tell an iterator to produce or emit data if you want to use it
|
||||
(which is often called ‘consuming’
|
||||
because if you read data from the pipeline,
|
||||
it’s usually gone),
|
||||
or you can add a new step to it and hand the new pipeline to someone else,
|
||||
who can then consume it or add even more steps.
|
||||
|
||||
[^index]: Or any other operation like it.
|
||||
No `iterator[0]`, no `iterator.get(0)` or whatever your favorite language uses.
|
||||
An operation like `iterator.last()` might exist,
|
||||
but it will consume the entire iterator instead of just accessing the last element.
|
||||
|
||||
An important aspect to note is:
|
||||
adding an operation to the pipeline does nothing
|
||||
until someone actually starts reading from it,
|
||||
and even then, only the elements that are consumed are computed.[^inf]
|
||||
and even then, only the elements that are consumed are computed.[^inf]
|
||||
This makes it possible to operate on huge data sets while keeping memory usage low,
|
||||
because only the currently active element has to be held in memory.
|
||||
|
||||
[^inf]: This is what makes infinite iterators possible to begin with.
|
||||
[^inf]: This is what makes infinite iterators possible.
|
||||
They can be very useful and are used a lot in functional languages,
|
||||
but they’re not today’s topic.
|
||||
|
||||
## Cold, hard numbers
|
||||
We’ll use that small example from the last section as our first example:
|
||||
take a range of numbers, double each number, and compute the sum –
|
||||
except this time, we’ll do the numbers from 1 to 1 billion.
|
||||
Since everything we’re doing is lazy, memory usage shouldn’t be an issue.
|
||||
|
||||
I will use different implementations to solve them and benchmark all of them.
|
||||
Here are the different approaches I came up with:
|
||||
|
||||
- a simple for loop in Java
|
||||
- Java’s LongStream
|
||||
- a for each loop with a range in Kotlin
|
||||
- Java’s LongStream called from Kotlin[^ktjava]
|
||||
- Java’s Stream wrapped in a Kotlin Sequence
|
||||
- a Kotlin range wrapped in a Sequence
|
||||
- Kotlin’s Sequence with a generator to create the range
|
||||
|
||||
[^ktjava]: Mainly to make sure there is no performance difference between the two.
|
||||
|
||||
The benchmarks were executed on an Intel Xeon E3-1271 v3 with 32 GB of RAM,
|
||||
running Arch Linux with kernel 5.4.20-1-lts,
|
||||
using the (at the time of writing) latest OpenJDK preview build (`15-ea+17-717`),
|
||||
Kotlin 1.4-M1, and [jmh](https://openjdk.java.net/projects/code-tools/jmh/) version 1.23.
|
||||
The bytecode target was set to Java 15 for the Java code and Java 13 for Kotlin
|
||||
(newer versions are currently unsupported).
|
||||
|
||||
Source code for the Java tests:
|
||||
|
||||
```java
|
||||
public long stream() {
|
||||
return LongStream.range(1, upper)
|
||||
.map(l -> l * 2)
|
||||
.sum();
|
||||
}
|
||||
|
||||
public long loop() {
|
||||
long sum = 0;
|
||||
for (long i = 0; i < upper; i++) {
|
||||
sum += i * 2;
|
||||
}
|
||||
return sum;
|
||||
}
|
||||
```
|
||||
|
||||
and for Kotlin:
|
||||
|
||||
```kotlin
|
||||
fun stream() =
|
||||
LongStream.range(1, upper)
|
||||
.map { it * 2 }
|
||||
.sum()
|
||||
|
||||
fun loop(): Long {
|
||||
var sum = 0L
|
||||
for (l in 1L until upper) {
|
||||
sum += l * 2
|
||||
}
|
||||
return sum
|
||||
}
|
||||
|
||||
fun streamWrappedInSequence() =
|
||||
LongStream.range(1L, upper)
|
||||
.asSequence()
|
||||
.map { it * 2 }
|
||||
.sum()
|
||||
|
||||
fun sequence() =
|
||||
(1 until upper).asSequence()
|
||||
.map { it * 2 }
|
||||
.sum()
|
||||
|
||||
fun withGenerator() =
|
||||
generateSequence(0L, { it + 1L })
|
||||
.take(upper.toInt())
|
||||
.map { it * 2 }
|
||||
.sum()
|
||||
```
|
||||
|
||||
with `const val upper = 1_000_000_000L`.[^`1 until upper` is used in these examples because unlike `lower..upper`, `until` is end-inclusive like Java’s LongStream.range().]
|
||||
|
||||
Without wasting any more of your time, here are the results:
|
||||
|
||||
```plaintext
|
||||
Benchmark Mode Cnt Score Error Units
|
||||
Java.loop avgt 25 446.055 ± 0.677 ms/op
|
||||
Java.stream avgt 25 601.424 ± 12.606 ms/op
|
||||
Kotlin.loop avgt 25 446.600 ± 1.164 ms/op
|
||||
Kotlin.sequence avgt 25 2732.604 ± 6.644 ms/op
|
||||
Kotlin.stream avgt 25 593.353 ± 1.408 ms/op
|
||||
Kotlin.streamWrappedInSequence avgt 25 3829.209 ± 33.569 ms/op
|
||||
Kotlin.withGenerator avgt 25 8374.149 ± 880.647 ms/op
|
||||
```
|
||||
|
||||
(full JMH output)[https://ruru.moe/pSK13p8].
|
||||
Unsurprisingly, using Streams from Java and Kotlin is almost identical in terms of performance.
|
||||
The same is true for imperative loops,
|
||||
meaning Kotlin ranges introduce no overhead compared to incrementing for loops.
|
||||
|
||||
More surprisingly, using Sequences is an order of magnitude slower.
|
||||
That was not at all according to my expectations, so I investigated.
|
||||
|
||||
As it turns out, Java’s `LongStream` exists because Stream<Long> is *much* slower.
|
||||
The JVM has to use `Long` rather than `long` when the type is used for generics,
|
||||
which involves an additional boxing step and the allocation for the `Long` object.[^primitives]
|
||||
Still, we now know that Streams have about 25% overhead compared to the simple loop for this example,
|
||||
that generating sequences is a comparatively slow process,
|
||||
and that wrapping Streams comes at a considerable cost (compared to a sequence created from a range).
|
||||
|
||||
[^primitives]: The JVM has a few primitive types, such as `int`, `char`, or array types.
|
||||
They are different from any other type because they cannot be `null`.
|
||||
Every regular type on the JVM extends `java.lang.Object` and is just a reference that is being passed around.
|
||||
The primitives are values, not references, so there’s a lot less overhead involved.
|
||||
Unfortunately, primitives can’t be used as generic types,
|
||||
so a list of longs will always convert the `long` to `Long` before adding it.
|
||||
|
||||
That last point seemed odd, so I attached a profiler to see where the CPU time is lost.
|
||||
|
||||
We can see that the `LongStream` can produce a `PrimitiveIterator.OfLong` that is used as a source for the Sequence.
|
||||
The operation of boxing a primitive `long` into an object `Long`
|
||||
(that’s the `Long.valueOf()` step) takes almost as long as advancing the underlying iterator itself.
|
||||
7.7% of the CPU time is spent in `Sequence.hasNext()`.
|
||||
The exact breakdown of that looks as follows:
|
||||
|
||||
The Sequence introduces very little overhead here, as it just delegates to `hasNext()` of the underlying iterator.
|
||||
Worth noting is that the iterator calls `accept` as part of `hasNext()`,
|
||||
which will already advance the underlying iterator.
|
||||
The value returned by that will be stored temporarily until `nextLong()` is called.
|
||||
|
||||
```java
|
||||
public boolean tryAdvance(LongConsumer consumer) {
|
||||
final long i = from;
|
||||
if (i < upTo) {
|
||||
from++;
|
||||
consumer.accept(i);
|
||||
return true;
|
||||
}
|
||||
// more stuff down here
|
||||
}
|
||||
```
|
||||
|
||||
where `consumer.accept()` is
|
||||
|
||||
```java
|
||||
public void accept(T t) {
|
||||
valueReady = true;
|
||||
nextElement = t;
|
||||
}
|
||||
```
|
||||
|
||||
Knowing this, I have to wonder why `nextLong()` takes as long as it does.
|
||||
Looking at [the implementation](https://github.com/openjdk/jdk/blob/6bab0f539fba8fb441697846347597b4a0ade428/src/java.base/share/classes/java/util/Spliterators.java#L756),
|
||||
I don’t understand where all that time is going.
|
||||
`hasNext()` should always be called before `next()`,
|
||||
so `next()` just has to return a precomputed value.
|
||||
|
||||
Nevertheless, we can now explain the performance difference with the additional boxing step.
|
||||
Primitives good; everything else bad.
|
||||
|
||||
With that in mind, I wrote a second test that avoids the unboxing issue to compare Streams and Sequences.
|
||||
The next snippet uses a simple wrapper class that guarantees that we have no primitives
|
||||
to execute a few operations on a Stream/Sequence.
|
||||
I’ll use this opportunity to also compare parallel and sequential streams.
|
||||
|
||||
The steps are simple:
|
||||
Take a long -> create a LongWrapper from it, double the contained value (which creates a new LongWrapper), extract the value, compute the sum.
|
||||
That may sound overcomplicated,
|
||||
but it’s sadly close to the reality of enterprise code.
|
||||
Wrapper types are everywhere.
|
||||
|
||||
```kotlin
|
||||
inner class LongWrapper(val value: Long) {
|
||||
fun double() = LongWrapper(value * 2)
|
||||
}
|
||||
|
||||
fun sequence(): Long =
|
||||
(1 until upper).asSequence()
|
||||
.map(::LongWrapper)
|
||||
.map(LongWrapper::double)
|
||||
.map(LongWrapper::value)
|
||||
.sum()
|
||||
|
||||
fun stream(): Optional<Long> =
|
||||
StreamSupport.stream((1 until upper).spliterator(), false)
|
||||
.map(::LongWrapper)
|
||||
.map(LongWrapper::double)
|
||||
.map(LongWrapper::value)
|
||||
.reduce(Long::plus)
|
||||
|
||||
fun parallelStream(): Optional<Long> =
|
||||
StreamSupport.stream((1 until upper).spliterator(), true)
|
||||
.map(::LongWrapper)
|
||||
.map(LongWrapper::double)
|
||||
.map(LongWrapper::value)
|
||||
.reduce(Long::plus)
|
||||
|
||||
|
||||
fun loop(): Long {
|
||||
var sum = 0L
|
||||
for (l in 1 until upper) {
|
||||
val wrapper = LongWrapper(l)
|
||||
val doubled = wrapper.double()
|
||||
sum += doubled.value
|
||||
}
|
||||
return sum
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
The results here paint a different picture:
|
||||
|
||||
```
|
||||
NonPrimitive.loop avgt 25 445.992 ± 0.642 ms/op
|
||||
NonPrimitive.sequence avgt 25 27257.399 ± 342.686 ms/op
|
||||
NonPrimitive.stream avgt 25 44673.318 ± 1325.832 ms/op
|
||||
NonPrimitive.parallelStream avgt 25 33856.919 ± 249.911 ms/op
|
||||
```
|
||||
|
||||
Full results are in the (JMH log from earlier)[https://ruru.moe/pSK13p8].
|
||||
|
||||
The overhead of Java streams is much higher than that of Kotlin Sequences,
|
||||
and even a parallel Stream is slower than using a Sequence.
|
||||
even though Sequences only use a single thread,
|
||||
but both are miles behind the simple for loop.
|
||||
My first assumption was that the compiler optimized away the wrapper type and just added the longs,
|
||||
but looking at [the bytecode](https://p.kageru.moe/AUJKiG),
|
||||
the constructor invocation and the `double()` method calls are still there.
|
||||
It’s hard to know what the JIT does at runtime,
|
||||
but the numbers certainly suggest that the wrapper is simply optimized away.
|
||||
The profiler report wasn’t helpful either,
|
||||
which further leads me to believe that the JIT just deletes the method and inlines the calculations.
|
||||
|
||||
This tells us that not only do Streams/Sequences have a very measurable overhead,
|
||||
but they severely limit the optimizer’s
|
||||
(be it compile-time or JIT)
|
||||
ability to understand the code,
|
||||
which can lead to significant slowdowns in code that can be optimized.
|
||||
Obviously, code that doesn’t rely on the optimizer as much won’t be affected to the same degree.
|
||||
|
||||
## Conclusion
|
||||
|
||||
Overall, I think that Kotlin’s Sequences are a good addition to the language.
|
||||
They fall behind Streams when working with primitives
|
||||
because the Java standard library has subtypes for many generic constructs to more efficiently handle primitive types,
|
||||
but in most real-world JVM applications (that being enterprise-level bloatware),
|
||||
primitives are the exception rather than the rule.
|
||||
Still, Kotlin has some types that optimize for this, such as `LongIterator`,
|
||||
but without a `LongSequence` to go with it,
|
||||
the boxing will still happen eventually,
|
||||
and all the performance gains are void.
|
||||
|
||||
I hope that we can get a few more types like it in the future,
|
||||
which will be especially useful once Kotlin/Native reaches maturity
|
||||
and starts being used for small/embedded hardware.
|
||||
|
||||
Apart from the performance, Sequences are also a lot easier to understand and even extend than Streams.
|
||||
Implementing your own Sequence requires barely more than an implementation of the underlying iterator,
|
||||
as can be seen in [CoalescingSequence](https://git.kageru.moe/kageru/Sekwences/src/branch/master/src/main/kotlin/moe/kageru/sekwences/CoalescingSequence.kt)
|
||||
which I implemented last year to get a feeling for how all of this works.
|
||||
Streams on the other hand are a lot more complex. They actually extend `Consumer<T>`,
|
||||
so a `Stream<T>` is just a `void consume(T input)` that can be called repeatedly.
|
||||
That makes it a lot harder to grasp where data is coming from and how it is requested, at least to me.
|
||||
|
||||
Simplicity is often underrated in software, but I consider it a huge plus for Sequences.
|
||||
|
||||
I will continue to use them liberally,
|
||||
unless I find myself in a situation where I need to process a huge number of primitives.
|
||||
And even then, I now know that Java’s Streams are a good choice.
|
||||
25% might sound like a lot,
|
||||
but it’s more than worth it if it means leaving code that is much easier to understand and modify for the next person.
|
||||
Unless you’re actually in a very performance-critical part of your application,
|
||||
but if you ever find yourself in that situation,
|
||||
you should switch to a different language.
|
||||
|
||||
|
||||
|
||||
|
||||
On that note: I was originally going to include Rust’s iterators here for comparison,
|
||||
but rustc optimized away all of my benchmarks with [constant time solutions](https://godbolt.org/z/iJaWVP).
|
||||
It’s a fascinating topic,
|
||||
and I might write a separate blog post
|
||||
where I dissect some of the assembly that rustc/LLVM produced,
|
||||
but I need to properly understand it myself first.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user