21 Pull requests opened by 16 people

• [SPARK-46547][SS] Fix deadlock between maintenance thread and streaming aggregation operator

  #44542 opened Dec 30, 2023

• [WIP][SPARK-46549][INFRA] Cache the Python dependencies for SQL tests

  #44546 opened Dec 31, 2023

• [WIP][SPARK-37039][PS] Fix `Series.astype` to work properly with missing value

  #44570 opened Jan 3, 2024

• [SPARK-46382][SQL] XML: Refactor inference and parsing

  #44571 opened Jan 3, 2024

• [SPARK-46566][SQL] Session level config was not loaded when isolation is enable.

  #44572 opened Jan 3, 2024

• [SPARK-46575][SQL][HIVE] Make HiveThriftServer2.startWithContext DevelopApi retriable and fix flakiness of ThriftServerWithSparkContextInHttpSuite

  #44575 opened Jan 3, 2024

• [SPARK-46589][CONNECT] Handle a non-existent error class

  #44581 opened Jan 3, 2024

• [SPARK-46581][CORE] Update comment on isZero in AccumulatorV2

  #44583 opened Jan 3, 2024

• [SPARK-46594][DOCS] The description of `spark.yarn.shuffle.server.recovery.disabled` should be in chapter `Configuring the External Shuffle Service`

  #44593 opened Jan 4, 2024

• [SPARK-46598][SQL] OrcColumnarBatchReader should respect the memory mode when creating column vectors for the missing column

  #44598 opened Jan 4, 2024

• Generate subqueries for correctness testing

  #44599 opened Jan 4, 2024

• [SPARK-46599][SQL] XML: Use TypeCoercion.findTightestCommonType for compatibility check

  #44601 opened Jan 5, 2024

• [SPARK-46600][SQL] Move shared code between SqlConf and SqlApiConf to SqlApiConfHelper

  #44602 opened Jan 5, 2024

• [SPARK-46603][PYTHON][DOCS] Refine docstring of `parse_url/url_encode/url_decode`

  #44604 opened Jan 5, 2024

• [SPARK-46601] [CORE] Fix log error in handleStatusMessage

  #44606 opened Jan 5, 2024

• [SPARK-46604][SQL] Make `Literal.apply` support `s.c.immuable.ArraySeq`

  #44607 opened Jan 5, 2024

• [SPARK-46605][CONNECT] Make `lit/typedLit` function in connect module support `s.c.immutable.ArraySeq`

  #44608 opened Jan 5, 2024

• [SPARK-46606][PYTHON][DOCS] Refine docstring of `convert_timezone/make_dt_interval/make_interval`

  #44610 opened Jan 5, 2024

• [SPARK-46607][PYTHON][TESTS] Check the testing mode

  #44611 opened Jan 5, 2024

• [SPARK-46525][DOCKER][TESTS][FOLLOWUP] Fix docker-integration-tests on Apple Silicon for db2 and oracle with colima docker environment

  #44612 opened Jan 5, 2024

• [WIP][CORE] Remove ThreadLocal by replace SimpleDateFormat with DateTimeFormatter

  #44613 opened Jan 5, 2024

33 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [SPARK-46536][SQL] Support GROUP BY calendar_interval_type

  #44538 commented on Jan 5, 2024 • 31 new comments

• [SPARK-46541][SQL][CONNECT] Fix the ambiguous column reference in self join

  #44532 commented on Jan 5, 2024 • 22 new comments

• [SPARK-46408][SQL] Support date_sub on V2ExpressionBuilder

  #44357 commented on Jan 5, 2024 • 21 new comments

• [SPIP-IN-PROGRESS][DO-NOT-MERGE][SS] Add base support for new arbitrary state management operator, single valueState type, multiple state variables and underlying support for column families for RocksDBStateStoreProvider with/without changelog checkpointing

  #43961 commented on Jan 3, 2024 • 19 new comments

• [SPARK-46383] Reduce Driver Heap Usage by Reducing the Lifespan of `TaskInfo.accumulables()`

  #44321 commented on Jan 5, 2024 • 10 new comments

• [SPARK-45352][SQL] Eliminate foldable window partitions

  #43144 commented on Jan 4, 2024 • 7 new comments

• [Don't Merge] upgrade sbt 1.9.8

  #44516 commented on Jan 4, 2024 • 7 new comments

• [SPARK-46512][CORE] Optimize shuffle reading when both sort and combine are used.

  #44512 commented on Jan 4, 2024 • 6 new comments

• [SPARK-46522][PYTHON] Block Python data source registration with name conflicts

  #44507 commented on Jan 5, 2024 • 4 new comments

• Collations proof of concept

  #44537 commented on Jan 5, 2024 • 3 new comments

• [SPARK-39800][SQL][WIP] DataSourceV2: View Support

  #44197 commented on Jan 2, 2024 • 3 new comments

• [SPARK-46331][SQL] Removing CodegenFallback from subset of DateTime expressions and version() expression

  #44261 commented on Jan 4, 2024 • 3 new comments

• [SPARK-46442][SQL] DS V2 supports push down PERCENTILE_CONT and PERCENTILE_DISC

  #44397 commented on Jan 5, 2024 • 3 new comments

• [WIP][SPARK-45720] Upgrade AWS SDK to v2 for Spark Kinesis connector module

  #44211 commented on Jan 5, 2024 • 2 new comments

• [SPARK-40876][SQL] Widening type promotion for decimals with larger scale in Parquet readers

  #44513 commented on Jan 4, 2024 • 1 new comment

• [WIP][SS] Python streaming source

  #44416 commented on Dec 31, 2023 • 1 new comment

• [SPARK-46400][CORE][SQL] When there are corrupted files in the local maven repo, skip this cache and try again

  #44343 commented on Jan 2, 2024 • 1 new comment

• [SPARK-42307][SQL][BUILD][DOCS] Adding in a better name for `_LEGACY_ERROR_TEMP_2232`

  #44337 commented on Jan 2, 2024 • 1 new comment

• [DOCS][MINOR] Fix typos

  #44276 commented on Dec 30, 2023 • 1 new comment

• [SPARK-45862][PYTHON][DOCS] Add user guide for basic dataframe operations

  #43972 commented on Dec 30, 2023 • 1 new comment

• [SPARK-41954][CORE] Add isDecommissioned in ExecutorDeadException

  #43475 commented on Jan 4, 2024 • 1 new comment

• [SPARK-43152][SS] Parametrisable output metadata path (_spark_metadata)

  #43147 commented on Jan 6, 2024 • 1 new comment

• [SPARK-42332][SQL][DOCS] Changing the require to a SparkException in ComplexTypeMergingExpression

  #44336 commented on Jan 3, 2024 • 0 new comments

• [SPARK-44639][SS][YARN] Use Java tmp dir for local RocksDB state storage on Yarn

  #42301 commented on Jan 1, 2024 • 0 new comments

• [SPARK-42492][SQL] Add new function filter_value

  #40085 commented on Jan 1, 2024 • 0 new comments

• [SPARK-46473][SQL] Reuse `getPartitionedFile` method

  #44437 commented on Jan 3, 2024 • 0 new comments

• [SPARK-46477][SQL] Add bucket info to SD in toHivePartition

  #44441 commented on Jan 2, 2024 • 0 new comments

• [WIP][SPARK-46608][SQL] Restore backward compatibility of `JdbcDialect.classifyException`

  #44449 commented on Jan 5, 2024 • 0 new comments

• [SPARK-22876][YARN] Respect YARN AM failure validity interval

  #42570 commented on Jan 1, 2024 • 0 new comments

• [SPARK-46574][BUILD] Upgrade maven plugin to latest version

  #43908 commented on Jan 3, 2024 • 0 new comments

• [SPARK-45959][SQL] Improving performance when addition of 1 column at a time causes increase in the LogicalPlan tree depth

  #43854 commented on Jan 5, 2024 • 0 new comments

• [SPARK-37019][SQL] Add codegen support to array higher-order functions

  #34558 commented on Jan 1, 2024 • 0 new comments

• [SPARK-35564][SQL] Support subexpression elimination for conditionally evaluated expressions

  #32987 commented on Jan 1, 2024 • 0 new comments