You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Usage: zoe topics produce [OPTIONS]
produce messages into topics
Options:
--dry-run Do not actually produce records
-t, --topic VALUE Topic to write to
--subject TEXT Avro subject name to use
-k, --key-path TEXT Jmespath (or jq) expression to extract the key
-v, --value-path TEXT Jmespath (or jq) expression to extract the value
--ts-path TEXT Jmespath (or jq) expression to extract the timestamp
--streaming Read data line by line continuously
--timeout INT Timeout in millis
--from-stdin Consume data from stdin
--from-file PATH Consume data from a json file
-h, --help Show this message and exit
Examples:
Produce a record into the 'input' topic from stdin (requires input to be a json array):
> echo '[{"id": "1", "msg": "hello"}]' | zoe -c local topics produce -t input --from-stdin
Use streaming mode to accept records one by one (does not require input to be a json array):
> echo '{"id": "1", "msg": "hello"}' | zoe -c local topics produce -t input --from-stdin --streaming
Use the id field of the input messages to determine the Kafka record key:
> echo '[{"id": "1", "msg": "hello"}]' | zoe -c local topics produce -t input --from-stdin --key-path 'id'
Write the data from a json file (requires the content in the file to be a json array):
> zoe -c local topics produce -t input --from-file data.json
Pipe data from another topic:
> zoe -c remote topics consume input --continuously | zoe -c local topics produce -t output --from-stdin --streaming
Which suggests that I can use either jmespath or jq for -k and -v options.
However, when I try to use jq syntax, I am getting the following error:
2022-05-18 14:01:46 INFO zoe: producing '1' records to topic 'my-topic'
failure: runner 'local' failed
cause:
error: Invalid jmespath expression '.__metadata__.key'
The option --dialect (by analogy with zoe topics consume) also doesn't work:
Error: no such option: "--dialect".
The reason I am trying to use jq and not jmespath is because I need to pipe data between topics in different clusters. And my topic has a key, which I also need to pipe.
I haven't found another way to do this easily other than to use --expose-metadata option for the consumer, and then remove the __metadata__ attribute on the producer side for the value, which is not possible with jmespath (at least I don't know the way), but can be easily done with jq.
zoe version print output:
{"projectVersion":"0.28.0","buildTimestamp":"2021-11-04T23:39:18.809435","createdBy":"Gradle 7.2","buildJdk":"14.0.2 (Azul Systems, Inc. 14.0.2+12)","buildOS":"Linux amd64 5.11.0-1020-azure"}
The text was updated successfully, but these errors were encountered:
When I run such a command:
I am getting this feedback:
Which suggests that I can use either
jmespath
orjq
for-k
and-v
options.However, when I try to use
jq
syntax, I am getting the following error:Error:
The option
--dialect
(by analogy withzoe topics consume
) also doesn't work:The reason I am trying to use
jq
and notjmespath
is because I need to pipe data between topics in different clusters. And my topic has a key, which I also need to pipe.I haven't found another way to do this easily other than to use
--expose-metadata
option for the consumer, and then remove the__metadata__
attribute on the producer side for the value, which is not possible withjmespath
(at least I don't know the way), but can be easily done withjq
.zoe version print
output:The text was updated successfully, but these errors were encountered: