apache / iceberg

20 Pull requests merged by 13 people

• ORC: Upgrade to 1.6.5

  #1546 merged Oct 2, 2020

• Add names to parameterized tests and simplify the parameters list

  #1539 merged Oct 2, 2020

• Spark: Implement equals and hashCode in SparkTable

  #1530 merged Oct 1, 2020

• MR: use encryption manager for input files

  #1532 merged Oct 1, 2020

• API: Add name to Table and Catalog APIs

  #1537 merged Oct 1, 2020

• Flink: Add flink sink document

  #1464 merged Oct 1, 2020

• Hive read docs

  #1490 merged Oct 1, 2020

• Hive: fix temporary folder location and cleanup in TestCatalogs

  #1535 merged Oct 1, 2020

• Add null check when calling catalog.close in SparkTestBase

  #1529 merged Sep 30, 2020

• ORC: Fix predicate pushdown for Not In and Not Equals

  #1536 merged Sep 30, 2020

• Spark: Implement equals and hashCode in SparkBatchScan

  #1512 merged Sep 30, 2020

• Replace Literal Integer with Constant Reference

  #1528 merged Sep 29, 2020

• ISSUE-1520 Document writing against partitioned table in Spark

  #1523 merged Sep 29, 2020

• Flink: Integrate Flink reader to SQL

  #1509 merged Sep 29, 2020

• [API][Core - Types/Avro][Parquet] Add a precondition null check for the constructor arg of PruneColumns

  #1491 merged Sep 29, 2020

• [MINOR] Switch to FileIO for InputFormat

  #1526 merged Sep 29, 2020

• Suppress UnnecessaryAnonymousClass warnings for classes with defaultDelete as a Consumer

  #1493 merged Sep 28, 2020

• Flink: Flink SQL only supports TimeType with default precision

  #1521 merged Sep 28, 2020

• Fix javadoc encoding

  #1518 merged Sep 28, 2020

• Suppress the last remaining invalid DangerousStringInternUsage warning

  #1519 merged Sep 28, 2020

3 Pull requests opened by 3 people

• Provide API and Implementation for Creating Iceberg Tables from Spark

  #1525 opened Sep 28, 2020

• Allow impl for LocationProvider for to be created during runtime

  #1531 opened Sep 30, 2020

• Spark3: Make views that reference Iceberg tables show current data

  #1545 opened Oct 2, 2020

6 Issues closed by 5 people

• TestCatalogs temporary folder created in wrong location and not cleaned up

  #1534 closed Oct 5, 2020

• [InputFormat Followup] InputFiles should be created using FileIO and Encryption manager

  #865 closed Oct 1, 2020

• Consider add friendly name() method in Iceberg Table Interface

  #658 closed Oct 1, 2020

• CI does not trigger automatically.

  #1090 closed Sep 30, 2020

• Necessary information on writing data against partitioned table with Spark is missing

  #1520 closed Sep 29, 2020

• Spark reads the table created by flink throws an exception

  #1504 closed Sep 29, 2020

10 Issues opened by 5 people

• Create an action to clean up tables

  #1549 opened Oct 2, 2020

• Consider adding extra validation for metrics config when creating a table

  #1548 opened Oct 2, 2020

• Ignore rewrite commits while validating copy-on-write deletes in OverwriteFiles

  #1547 opened Oct 2, 2020

• Switch to using table.name() instead of toString() where it makes sense

  #1544 opened Oct 1, 2020

• numBuckets can cause division by zero

  #1543 opened Oct 1, 2020

• Capture the template parameter in the tests for TestIcebergInputFormat#Factory<T>

  #1542 opened Oct 1, 2020

• Update FlinkCatalog to use Namespace for baseNamespace field for cleaner output

  #1541 opened Oct 1, 2020

• Add in support for vectorized parquet reads on data files with row position (_pos) in the metadata column

  #1540 opened Oct 1, 2020

• Dynamically load LocationProvider implementation

  #1538 opened Oct 1, 2020

• Error when creating a table `InvalidObjectException`

  #1533 opened Sep 30, 2020

28 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• HiveMetaHook implementation to enable CREATE TABLE and DROP TABLE from Hive queries

  #1495 commented on Oct 1, 2020 • 49 new comments

• Enable Hive3 builds for iceberg-mr and iceberg-hive-metastore

  #1478 commented on Oct 1, 2020 • 47 new comments

• MR: apply row-level delete files when reading

  #1497 commented on Oct 3, 2020 • 35 new comments

• Use schema at the time of the snapshot when reading a snapshot.

  #1508 commented on Oct 1, 2020 • 25 new comments

• Update the Iceberg spec for row-level deletes

  #1499 commented on Oct 3, 2020 • 21 new comments

• Flink: maintain the complete data files into manifest before checkpoint finished.

  #1477 commented on Sep 29, 2020 • 10 new comments

• [WIP] Spark3: SQL extensions for rollback_to_snapshot

  #1473 commented on Sep 30, 2020 • 9 new comments

• Flink: Support sink when disable flink checkpoint disable

  #1515 commented on Sep 30, 2020 • 9 new comments

• Adding support for time-based partitioning on long column type

  #417 commented on Oct 2, 2020 • 7 new comments

• Hive: Make HiveCatalog based tables automatically readable from Hive

  #1505 commented on Sep 30, 2020 • 6 new comments

• Reconsider caching behavior in Spark 3

  #1485 commented on Sep 29, 2020 • 5 new comments

• Issue-1511: Read errors out with multiple DeleteFile entries after UPSERT

  #1514 commented on Oct 5, 2020 • 5 new comments

• any plan for Iceberg Table on S3?

  #1468 commented on Oct 1, 2020 • 4 new comments

• reuse container when reading parquet records

  #1522 commented on Sep 30, 2020 • 4 new comments

• Distributing Job Planning on Spark

  #1422 commented on Sep 29, 2020 • 3 new comments

• Parquet: Add vectorized position reader

  #1356 commented on Sep 30, 2020 • 2 new comments

• Flink: apply row-level delete when reading

  #1517 commented on Sep 29, 2020 • 2 new comments

• inconsistent handling when TableOperations#current() returns null

  #1513 commented on Sep 28, 2020 • 1 new comment

• make Flink iceberg sink work without checkpoint enabled.

  #1442 commented on Sep 29, 2020 • 1 new comment

• Spark DataFrame write fails if input dataframe has columns in different order than iceberg schema

  #741 commented on Sep 29, 2020 • 1 new comment

• The document can't perfectly display on mac 13 inch

  #1524 commented on Sep 30, 2020 • 1 new comment

• Should we consider adding a github action to lint YAML files for correctness / proper style?

  #1300 commented on Sep 30, 2020 • 1 new comment

• Spark: use newHadoopConf() in SessionState to get Hadoop conf

  #1142 commented on Sep 30, 2020 • 1 new comment

• Refactor ReusableMapData in Spark and Flink parquet readers

  #1331 commented on Oct 1, 2020 • 1 new comment

• WIP - Add a Parallelized Spark Job Planning Path

  #1421 commented on Oct 5, 2020 • 1 new comment

• Add position delete validation that data files have not been deleted

  #1469 commented on Oct 3, 2020 • 1 new comment

• Add partition summaries in SnapshotSummary builder

  #1367 commented on Oct 2, 2020 • 0 new comments

• Core: Parallelize manifest list reads in metadata tables

  #1440 commented on Sep 29, 2020 • 0 new comments