Apache Spark is an open source distributed general-purpose cluster-computing framework. It provides an interface for programming entire clusters with implicit data parallelism and fault tolerance.

Here are 971 public repositories matching this topic...

mlflow / mlflow

Star 8k

Open

[FR] Check and truncate string length

8

PhilipMay commented Jun 12, 2020

MLflow seems to have a length limit of 5000 when setting tags (see below).

[...]
  File "/home/smay/miniconda3/envs/py38/lib/python3.8/site-packages/mlflow/utils/validation.py", line 136, in _validate_length_limit
    raise MlflowException(
mlflow.exceptions.MlflowException: Tag value '[0.8562690322984875, 0.8544098885636596, 0.8544098885636596, 0.8544098885636596, 0.85440988856365

Read more

area/tracking enhancement good first issue help wanted

Open

[BUG] Run name not set when using start_run inside a MLproject execution

5

Open

[BUG] uploading artifacts to FTP server doesn't work

2

Find more good first issues →

dotnet / spark

Star 1.6k

Open

[FEATURE REQUEST]: Implement ML Features

22

GoEddie commented Dec 30, 2019

This is to track implementation of the ML-Features: https://spark.apache.org/docs/latest/ml-features

Bucketizer has been implemented in dotnet/spark#378 but there are more features that should be implemented.

• Feature Extractors
  • TF-IDF
  • Word2Vec (dotnet/spark#491)
  • CountVectorizer (https://github.com/dotnet/spark/p

Read more

enhancement good first issue help wanted

Open

Implement accumulator in TaskRunner.cs

Open

Write unit tests for SimpleWorker.

2

Find more good first issues →

sparklyr / sparklyr

Star 763

R interface for Apache Spark

machine-learning r spark apache-spark dplyr ide distributed rstats sparklyr livy remote-clusters

• Updated Dec 15, 2020
• R

san089 / goodreads_etl_pipeline

Star 747

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

python airflow spark apache-spark scheduler s3 data-engineering data-lake warehouse redshift data-migration livy etl-framework apache-airflow emr-cluster etl-pipeline etl-job data-engineering-pipeline airflow-dag goodreads-data-pipeline

• Updated Mar 9, 2020
• Python

Created by Matei Zaharia

Released May 26, 2014

Repository

apache/spark

Website

spark.apache.org

Wikipedia

Wikipedia

Related Topics