Apache Avro is a data serialization system.

Record Query - A tool for doing record analysis and transformation

Confluent Schema Registry for Kafka

Apache Kafka and Confluent Platform examples and demos

What's in your data? Extract schema, statistics and entities from datasets

Avro for JavaScript ⚡

pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].

ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.

[PROJECT IS NO LONGER MAINTAINED] Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

Avro schema generation and serialization / deserialization for Scala

ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Benchmark comparing various data serialization libraries (thrift, protobuf etc.) for C++

Command Line Tool for managing Apache Kafka

More than 2000+ Data engineer interview questions.

Data Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files

Iceberg is a table format for large, slow-moving tabular data

Web tool for Avro Schema Registry |

Lightweight message bus interface for .NET (pub/sub and request-response) with transport plugins for popular message brokers.

Flexible, Fast & Compact Serialization with RPC