etl-job

Star

Here are 21 public repositories matching this topic...

san089 / goodreads_etl_pipeline

Star

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Updated Mar 9, 2020
Python

AlexIoannides / pyspark-example-project

Star

Example project implementing best practices for PySpark ETL jobs and applications.

python data-science spark etl pyspark data-engineering etl-pipeline etl-job

Updated Jul 9, 2020
Python

HeadspringLabs / bulk-writer

Star

Provides guidance for fast ETL jobs, an IDataReader implementation for SqlBulkCopy (or the MySql or Oracle equivalents) that wraps an IEnumerable, and libraries for mapping entites to table columns.

sql pipeline etl sqlbulkcopy stream-data pipeline-stage etl-job bulk-writer

Updated May 19, 2020
C#

paillave / Etl.Net

Star

Mass processing data with a complete ETL for .net developers

csv etl dotnet extract sftp entity-framework business-intelligence dotnet-core load transform dotnet-standard csv-parser csv-reader csv-writer etl-job

Updated Feb 22, 2020
C#

visiologyofficial / vixtract

Star

etl etl-framework etl-pipeline etl-components etl-job etl-automation

Updated Sep 7, 2020
HTML

felipefrizzo / terraform-aws-kinesis-firehose

Star

This code creates a Kinesis Firehose in AWS to send CloudWatch log data to S3.

big-data analytics terraform kinesis-firehose cloudwatch-logs parquet terraform-provider etl-job terraform-aws big-data-processing

Updated Mar 30, 2020
HCL

michaelbironneau / analyst

Star

A declarative, SQL-like DSL for data integration tasks.

sql etl data-integration etl-job

Updated Jul 4, 2018
Go

yennanliu / XJob

Star

Airflow POC demo : 1) env set up 2) airflow DAG 3) Spark/ML pipeline | #DE

Updated Aug 9, 2020
Python

2298-Software / Mambo

Star

A simple in-memory, configuration driven, data processing pipeline for Apache Spark.

spark stream hadoop pipeline turbine etl-framework etl-job

Updated Sep 24, 2019
Scala

doriel / Excel-to-Mysql-ETL

Star

World Cup Data ETL

mysql sqlalchemy etl cup exploratory-data-analysis eda pandas seaborn pyhton worldcup pymysql fifa exploratory-analysis etl-job discover-insights cup-data

Updated Aug 7, 2019
Jupyter Notebook

TheCocoTeam / SourceWatcher

Star

A PHP project which allows extracting, transforming, loading and watching different sources of data.

php etl php7 php-library php-framework etl-framework etl-pipeline etl-components etl-job etl-jobs etl-automation

Updated Aug 31, 2020
JavaScript

amantewary / Sentiment-Analysis-of-Tweets-Using-ETL-process-and-Elastic-Search

Star

Sentiment Analysis of Tweets Using ETL process and Elastic Search

elasticsearch sentiment-analysis azure etl-job

Updated Jun 7, 2018
Python

orkhanbaghirli / PyETL

Star

python 3.5 package for ETL jobs

python python3 pure-python etl-job

Updated Dec 3, 2018
Python

ronfroy / ETL

Star

PHP ETL Implementation

etl php7 etl-framework etl-pipeline etl-job

Updated Sep 13, 2018
PHP

san089 / airflow-training

Star

Introduction to the data pipeline management with Airflow. Airflow schedule and maintain numerous ETL processes running on a large scale Enterprise Data Warehouse.

airflow scheduler scheduling apache etl-framework etl-pipeline etl-job etl-automation

Updated Nov 26, 2018
Python

cdvx / etl-python

Star

This is a command line application to demonstrate a sample ETL pipeline in python. It takes a PostgreSQL dataset that is provided by https://raw.githubusercontent.com/cdvx/etl-python/movies-sql/movielens.sql and transfers the data to MongoDB.

sql python-3 etl-job postgresql-dataset