DataOps

We are using the protobuf-git configuration as described at https://cloudhut.dev/docs/features/protobuf#git-repository
In our repository the proto-files live within a proto directory, which seems to be very common, and contains 5 levels of nested folders.

Currently KOWL searches only the first 5 levels of the checkout for .proto files, so our last level is not considered.

• Please

We need to support writing to and reading from TFRecord format.

Reference doc: https://www.tensorflow.org/tutorials/load_data/tfrecord

More on TypeTransfomers can be found here.

Related PR that adds PyTorch tensor and module as Flyte types: flyteorg/flytekit#1032

The default RubrixLogHTTPMiddleware record mapper for token classification expect a structured including a text field for inputs. This could make prediction model inputs a bit cumbersome. Default mapper could accepts also flat strings as inputs:

def token_classification_mapper(inputs, outputs):
    i

Sending a rest call to delete a job specification throws 404 where as grpc call works fine. Steps to reproduce

curl -X DELETE "http://localhost:9100/v1/project/my-project/namespace/kush/helloworld" -H  "accept: application/json"

Task Overview

• Currently timestamp_column is the only configuration that is needed to be configured globally in the model config section (usually it's being configured in the properties.yml under elementary in the config tag).
• Passing the timestamp_column as a test param will enable running multiple tests with different timestamp columns. For example running a test with updated_at colum

Currently, both Kafka and Influx sink logs only the data(Row) that is being sent.

Add support for logging column names as well along with data points similar to the implementation in log sink.

This will enable users to correlate the data points with column names.

Zap configurations should be pushed to grpc middleware here: cmd/setup.go#L47

In golang client, consumers get dynamic message instance after parsing. Add an example in the docs on how to use dynamic message instance to get values of different types in consumer code.

List of protobuf types to cover

• timestamp
• duration
• bytes
• message type
• struct
• map

Siren creates alertmanager config and sycns with alertmanger. The alert manager config can change for the same subscriptions if their order changes. We should follow some sorting conventions and stick to those conventions to create an alert manager config.