Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Parameter --noCapture triggers java.lang.UnsupportedOperationException and interrupted traffic forwarding #540

Closed
ParvelAWS opened this issue Mar 28, 2024 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@ParvelAWS
Copy link

What is the bug?

When the proxy needs to bypass or pause offloading to Kafka topic while still keeping forwarding traffic towards OpenSearch, the parameter --noCapture is used. But it's triggering exception reported for every message and traffic is not forwarded to OpenSearch.
Pod Logs:
{"@timestamp":"2024-03-27T23:38:33.372Z","ecs.version":"1.2.0","log.level":"WARN","message":"An exception 'java.lang.UnsupportedOperationException: spaceLeft() can only be called on CodedOutputStreams that are writing to a flat array or ByteBuffer.' [enable DEBUG level for full stacktrace] was thrown by a user handler's exceptionCaught() method while handling the following exception:","[process.thread.name](http://process.thread.name/)":"captureProxyPoolWorker-3-1","log.logger":"io.netty.channel.AbstractChannelHandlerContext","error.type":"java.lang.UnsupportedOperationException","error.message":"spaceLeft() can only be called on CodedOutputStreams that are writing to a flat array or ByteBuffer.","error.stack_trace":"java.lang.UnsupportedOperationException: spaceLeft() can only be called on CodedOutputStreams that are writing to a flat array or ByteBuffer.\n\tat com.google.protobuf.CodedOutputStream$AbstractBufferedEncoder.spaceLeft(CodedOutputStream.java:2233)\n\tat org.opensearch.migrations.trafficcapture.StreamChannelConnectionCaptureSerializer.beginSubstreamObservation(StreamChannelConnectionCaptureSerializer.java:150)\n\tat org.opensearch.migrations.trafficcapture.StreamChannelConnectionCaptureSerializer.addCloseEvent(StreamChannelConnectionCaptureSerializer.java:240)\n\tat com.verizon.opensearch.proxy.offload.MultiTargetOffloader.addCloseEvent(MultiTargetOffloader.java:45)\n\tat org.opensearch.migrations.trafficcapture.netty.LoggingHttpHandler.channelUnregistered(LoggingHttpHandler.java:160)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:217)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:195)\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelUnregistered(AbstractChannelHandlerContext.java:188)\n\tat io.netty.channel.DefaultChannelPipeline$HeadContext.channelUnregistered(DefaultChannelPipeline.java:1388)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:215)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:195)\n\tat io.netty.channel.DefaultChannelPipeline.fireChannelUnregistered(DefaultChannelPipeline.java:821)\n\tat io.netty.channel.AbstractChannel$AbstractUnsafe$7.run(AbstractChannel.java:821)\n\tat io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)\n\tat io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)\n\tat io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)\n\tat io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:566)\n\tat io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)\n\tat io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\tat io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat java.base/java.lang.Thread.run(Thread.java:833)\n","time":"2024-03-27T23:38:33.372Z"}

How can one reproduce the bug?

With the following parameters supplied
"--listenPort", "80", "--noCapture", "--destinationUri", "https://vpc-xxx-opensearch.us-east-1.es.amazonaws.com:443", "--insecureDestination", "--suppressCaptureForHeaderMatch", "User-Agent", "^ELB-HealthChecker/2\\.0"]

What is the expected behavior?

The proxy should stop writing to Kafka queue but still forwarding traffic to OpenSearch

What is your host/environment?

EKS 1.28
AWS Linux Min 2023

Do you have any screenshots?

If applicable, add screenshots to help explain your problem.

Do you have any additional context?

Add any other context about the problem.

@ParvelAWS ParvelAWS added bug Something isn't working untriaged labels Mar 28, 2024
@AndreKurait AndreKurait self-assigned this Mar 28, 2024
@AndreKurait
Copy link
Member

Hi Parvel, i'll be taking this up.
https://opensearch.atlassian.net/browse/MIGRATIONS-1616

@AndreKurait
Copy link
Member

#543 has been merged. Closing this bug

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants