Logstash - transport and process your logs, events, or other data

SeaTunnel is a distributed, high-performance data integration platform for the synchronization and transformation of massive data (offline & real-time).

The open source high performance data integration platform built for developers.

Flow-based programming for JavaScript

This repository is a getting started guide to Singer.

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Open source SQL engine in Python

A scalable general purpose micro-framework for defining dataflows. THIS REPOSITORY HAS BEEN MOVED TO www.github.com/dagworks-inc/hamilton

A scalable general purpose micro-orchestration framework for defining dataflows. You can use it to build dataframes, numpy matrices, python objects, ML models, LLM workflows, etc. Embed Hamilton anywhere python runs, e.g. spark, airflow, jupyter, fastapi, python scripts, etc. Comes with lineage out of the box.

ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

A simplified, lightweight ETL Framework based on Apache Spark

Extract, Transform, Load: Any SQL Database in 4 lines of Code.

Knowledge Graph Toolkit

A tool for building feature stores.

A lightweight ETL (extract, transform, load) library and data integration toolbox for .NET.

SeaTunnel is a distributed, high-performance data integration platform for the synchronization and transformation of massive data (offline & real-time).

Bender - Serverless ETL Framework

A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.

mito ETL tool

A visual ETL development and debugging tool for big data