benchmark压测Pulsar遇到error: java.lang.ArrayIndexOutOfBoundsException: Index 1310745 out of bounds for length 1310720

Pulsar的配置使用这里的配置:benchmark/driver-pulsar/deploy/ssd/templates at master · openmessaging/benchmark · GitHub
benchmark按照这里的文档 - The OpenMessaging Benchmark Framework

遇到error:

17:49:14.954 [pulsar-client-io-1-5] WARN - Write error on message
java.util.concurrent.CompletionException: java.lang.ArrayIndexOutOfBoundsException: value outside of histogram covered range. 
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1383806 out of bounds for length 1310720
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314) ~[?:?]
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319) ~[?:?]
	at java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:787) ~[?:?]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) [?:?]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) [?:?]
	at org.apache.pulsar.client.impl.ProducerImpl$1.sendComplete(ProducerImpl.java:352) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.client.impl.ProducerImpl$OpSendMsg.sendComplete(ProducerImpl.java:1383) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.client.impl.ProducerImpl.ackReceived(ProducerImpl.java:1111) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.client.impl.ClientCnx.handleSendReceipt(ClientCnx.java:411) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:221) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:327) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:314) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:435) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:487) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:385) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [pulsar-client-all-2.10.1.jar:2.10.1]
	at org.apache.pulsar.shade.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [pulsar-client-all-2.10.1.jar:2.10.1]
	at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.ArrayIndexOutOfBoundsException: value outside of histogram covered range. 
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1383806 out of bounds for length 1310720
	at org.HdrHistogram.AbstractHistogram.handleRecordException(AbstractHistogram.java:532) ~[HdrHistogram-2.1.10.jar:2.1.10]
	at org.HdrHistogram.AbstractHistogram.recordSingleValue(AbstractHistogram.java:522) ~[HdrHistogram-2.1.10.jar:2.1.10]
	at org.HdrHistogram.AbstractHistogram.recordValue(AbstractHistogram.java:426) ~[HdrHistogram-2.1.10.jar:2.1.10]
	at org.HdrHistogram.Recorder.recordValue(Recorder.java:99) ~[HdrHistogram-2.1.10.jar:2.1.10]
	at io.openmessaging.benchmark.worker.LocalWorker.lambda$null$8(LocalWorker.java:232) ~[classes/:?]
	at java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:783) ~[?:?]
	... 28 more

refer to: 如何对 Pulsar 压测呢?使用什么工具?

这个是 OpenMessaging的一个问题,在绘制图的时候,某些数值超出范围了,这里有个之前参考

可以给OpenMessaging提个issues

临时的解决方法为把时间调大.
在测试的过程中,以kafka, pulsar的metrics进行监控,忽略benchmark的输出结果。

TimeUnit.SECONDS.toMicros(1200000)

1 Like

通常端到端超时 60 秒应该也足够了。
建议也可以先降低发送的速率,逐渐增加。