Build software better, together

dflemstr / rq

1.2k

Record Query - A tool for doing record analysis and transformation

rust protobuf command-line-tool json avro messagepack yaml toml javascript lodash

JavaScript Updated Feb 11, 2019

confluentinc / schema-registry

884

Confluent Schema registry for Kafka

schema-registry kafka schema schemas avro avro-schema rest-api confluent

Java Updated May 1, 2019 16 issues need help

miguno / kafka-storm-starter

716

Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while …

apache-kafka kafka apache-storm storm spark apache-spark scala integration avro apache-avro

Scala Updated Jan 24, 2017

mtth / avsc

704

Avro for JavaScript ⚡️

avro serialization rpc javascript

JavaScript Updated Apr 26, 2019 1 issue needs help

sksamuel / avro4s

418

Avro schema generation and serialization / deserialization for Scala

scala avro avro-schema schema-generation scala-macros coproduct serialization

Scala Updated Apr 21, 2019

thekvs / cpp-serializers

393

Benchmark comparing various data serialization libraries (thrift, protobuf etc.) for C++

cpp serialization protobuf capn-proto thrift flatbuffers cereal performance-testing boost msgpack avro apache-avro c-plus-plus yas

C++ Updated Apr 28, 2019

Netflix / iceberg

266

Iceberg is a table format for large, slow-moving tabular data

spark hadoop parquet avro

Java Updated Feb 2, 2019 1 issue needs help

Landoop / schema-registry-ui

252

Web tool for Avro Schema Registry |

schema-registry kafka avro

JavaScript Updated Apr 25, 2019

spotify / ratatool

197

A tool for data sampling, data generation, and data diffing

scala scalacheck avro parquet bigquery protobuf

Scala Updated Apr 30, 2019 2 issues need help

divolte / divolte-collector

189

Divolte Collector

divolte-collector hdfs kafka gcs java avro pubsub clickstream analytics analytics-tracking

Java Updated May 1, 2019

HariSekhon / DevOps-Python-tools

178

DevOps CLI Tools - Hadoop, Spark, HBase, Log Anonymizer, Ambari Blueprints, AWS CloudFormation, Linux, Docker, Spark …

Python Updated Apr 14, 2019

confluentinc / examples

168

Apache Kafka and Confluent Platform examples and demos

Java Updated Apr 30, 2019

pennsignals / aptos

135

☀️ A tool for validating data using JSON Schema and converting JSON Schema documents into different data-interchange…

json-schema avro-schema command-line-tool validation python3 data-interchange avro schema-conversion cli

Python Updated Mar 15, 2019 3 issues need help

uber / storagetapper

117

StorageTapper is a scalable realtime MySQL change data streaming and transformation service

mysql kafka avro cdc etl

Go Updated Jan 7, 2019

miguno / avro-hadoop-starter

109

Example MapReduce jobs in Java, Hive, Pig, and Hadoop Streaming that work on Avro data.

avro hadoop java mapreduce hive pig

Java Updated Nov 12, 2015

Chabane / bigdata-playground

106

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache F…

TypeScript Updated Feb 1, 2019

hpgrahsl / kafka-connect-mongodb

97

A Kafka Connect MongoDB Sink Connector

kafka kafka-connect mongodb sink-connector sink connector change-data-capture cdc avro json bson debezium confluent-hub azure-cosmosdb cosmosdb

Java Updated Apr 11, 2019

theiterators / kebs

84

Scala library to eliminate boilerplate

scala slick play akka akka-http boilerplate play-framework restful hack avro spray

Scala Updated Feb 11, 2019 2 issues need help

commercehub-oss / gradle-avro-plugin

82

A Gradle plugin to allow easily performing Java code generation for Apache Avro. It supports JSON schema declaration …

gradle-plugin java groovy avro

Java Updated Apr 28, 2019

ovotech / kafka-serialization

70

Lego bricks to build Apache Kafka serializers and deserializers

apache-kafka serialization json circe avro4s avro json4s spray-json

Scala Updated Apr 15, 2019

Eneco / kafka-connect-twitter

69

Kafka Connect Sink/Source for Twitter

kafka twitter tweets avro kafka-connect twitter4j stream connector

Scala Updated Jun 17, 2018

Landoop / schema-registry

68

A CLI and Go client for Kafka Schema Registry

kafka schema-registry cli api go avro-schema avro schema confluent

Go Updated Mar 27, 2019

indix / schemer

66

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

spark schema-registry schema-inference parquet avro graphql-api json tsv

Scala Updated Sep 20, 2018

sevenmind / kaufmann_ex

63

Kafka backed service library.

kafka elixir avro schema microservice concurrency

Elixir Updated Apr 26, 2019

dasch / avro_turf

61

A library that makes it easier to use the Avro serialization format from Ruby.

ruby schema-registry avro avro-data schema

Ruby Updated Apr 30, 2019

dcos / dcos-metrics

58

The metrics pipeline for DC/OS 1.9-1.11

metrics avro statsd dogstatsd prometheus

C++ Updated Feb 27, 2019

salsify / avro-schema-registry

52

Implementation of the Confluent Schema Registry API as a Rails application

avro schema-registry

Ruby Updated Apr 25, 2019

salsify / avro-builder

41

Ruby DSL to create Avro schemas

avro

Ruby Updated Oct 17, 2017

salsify / avromatic

40

Generate Ruby models from Avro schemas

avro

Ruby Updated Feb 15, 2019

yamrcraft / etl-light

35

A light Kafka to HDFS/S3 ETL library based on Apache Spark

etl kafka spark hdfs s3 job batch parquet avro scala docker protobuf

Scala Updated Jun 29, 2017

avro

Repositories 294