You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It would be nice if the replayer can have a built-in submodule that can read and stream S3 objects (in similar protobuf format) like the messages in Kafka topic and feed it into replay
What solution would you like?
A sub module, like S3InputStream similar to a Kafka stream reader, to feed protobuf messages into main replayer executor.
What alternatives have you considered?
An external Python or other shell scripts to feed S3 into stdin of the replayer pipe
Do you have any additional context?
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered:
Understood initially it may compromise the capability to pause/interrupt/resume replay offset as in Kafka topic stream. May have some workarounds like a journal tag file in S3 bucket.
ParvelAWS
changed the title
[FEATURE] Consume S3 persisted object as an input stream
[FEATURE] Consume S3 persisted objects as an input stream
Feb 16, 2024
Is your feature request related to a problem?
It would be nice if the replayer can have a built-in submodule that can read and stream S3 objects (in similar protobuf format) like the messages in Kafka topic and feed it into replay
What solution would you like?
A sub module, like S3InputStream similar to a Kafka stream reader, to feed protobuf messages into main replayer executor.
What alternatives have you considered?
An external Python or other shell scripts to feed S3 into stdin of the replayer pipe
Do you have any additional context?
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: