avro

The title might seem a bit vague but I don't know how to describe it any better tbh :-)

Anyway this is what happened: I got some 500 responses from the schema registry and all I could see in the logs was :

[2020-04-02 16:03:35,048] INFO 100.96.14.58 - - [02/Apr/2020:16:03:34 +0000] "PUT /config/some-topic-value HTTP/1.1" 500 69  502 (io.confluent.rest-utils.requests)

The logs di

Currently, there isn't a way to get the table properties in the SparkOrcWriter via the WriterFactory.

Converting timestamp to topic/partition offset is not easy without coding.... a nice new feature would be to be able to consume messages by providing from-timestamp, and let the kafkactl code compute the corresponding start offset value.

The following command succeeds with 0.3.21, but fails after upgrade to 0.3.25:

ratatool bigDiffy --input-mode=bigquery --output-mode=bigquery --key=id --lhs="dataset1.bigdiffy_input" --rhs="bar_artist_snapshot.data_20220130" --output=dataset2.artist-snapshot-2-diffy --project=bar --runner=DataflowRunner --tempLocation=gs://foo/christhomas/staging/ --serviceAccount=foo --region=europe-west1