28 Pull requests opened by 23 people

• [SPARK-51646][SQL] Fix propagating collation in views with default collation

  #50436 opened Mar 27, 2025

• [SPARK-51644][BUILD][CONNECT] fix for spark shells

  #50437 opened Mar 27, 2025

• [SPARK-51638][CORE] Fix fetching the remote disk stored RDD blocks via the external shuffle service

  #50439 opened Mar 28, 2025

• [MINOR][DOC] grammatical corrections, spellings and typos

  #50450 opened Mar 28, 2025

• [SPARK-51660][CORE] Gracefully handle when MDC is not supported

  #50452 opened Mar 28, 2025

• [SPARK-51653][CONNECT][TESTS] Migrate the tests related to `RemoteSparkSession` from the `connect-client-jvm` module to `connect-client-integration-tests`

  #50463 opened Mar 31, 2025

• [SPARK-36600] [PYTHON] Minor fix to avoid list creation for generators with schema

  #50464 opened Mar 31, 2025

• [SPARK-51688][PYTHON] Use Unix Domain Socket between Python and JVM communication

  #50466 opened Mar 31, 2025

• [SPARK-51673][SQL] Apply default collation to alter view query

  #50468 opened Mar 31, 2025

• [SPARK-50131][SQL] Add IN Subquery DataFrame API

  #50470 opened Mar 31, 2025

• [WIP] Add column pruning API to Python Data Sources

  #50472 opened Apr 1, 2025

• Enable -Xsource:3-cross compiler flag

  #50474 opened Apr 1, 2025

• [SPARK-51562][SQL] Add the time() function

  #50479 opened Apr 1, 2025

• [WIP] Lock Hardening - RocksDB State Machine

  #50481 opened Apr 1, 2025

• [SPARK-51685][SS] Excessive Info logging from RocksDb operations causing too big executor stderr files

  #50483 opened Apr 1, 2025

• [SPARK-50953][FOLLOW-UP] Allow whitespace/tab in variantGet paths

  #50484 opened Apr 1, 2025

• [SPARK-51496][SQL][FOLLOW-UP] Preserve the case of DataSourceV2Relation option keys

  #50487 opened Apr 2, 2025

• [SPARK-51690][SS] Change the protocol of ListState.put()/get()/appendList() from Arrow to simple custom protocol

  #50488 opened Apr 2, 2025

• [SPARK-51691][CORE] SerializationDebugger should swallow exception when try to find the reason of serialization problem

  #50489 opened Apr 2, 2025

• [SPARK-51692][SQL]Fix bug while replace cte ref which contains dynamicpruning

  #50490 opened Apr 2, 2025

• fix typo in pyspark documentation (minor edit)

  #50492 opened Apr 2, 2025

• [SPARK-51704][SQL] Eliminate unnecessary collect operation

  #50494 opened Apr 2, 2025

• [SPARK-51694][SQL] Fix load v2 function with non-buildin V2 session catalog

  #50495 opened Apr 2, 2025

• [SPARK-51695][SQL] Introduce Parser Changes for Table Constraints (CHECK, UNIQUE, PK, FK)

  #50496 opened Apr 2, 2025

• [WIP] State machine hardening

  #50497 opened Apr 2, 2025

• [SPARK-51699][BUILD] Upgrade to Apache parent pom 34

  #50500 opened Apr 2, 2025

• [SPARK-51696] Fix the VariantGet .sql method to also output the target data type

  #50504 opened Apr 3, 2025

• [SPARK-51707][CONNECT][TESTS] Handling of `IllegalStateException` with `Memory leaked` message in the `afterAll` of `RemoteSparkSession`

  #50506 opened Apr 3, 2025

22 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [SPARK-51439][SQL] Support SQL UDF with DEFAULT argument

  #50408 commented on Apr 2, 2025 • 21 new comments

• [SPARK-51358] [SS] Introduce snapshot upload lag detection through StateStoreCoordinator

  #50123 commented on Apr 2, 2025 • 13 new comments

• [SPARK-51423][SQL] Add the current_time() function for TIME datatype

  #50336 commented on Apr 2, 2025 • 12 new comments

• [SPARK-51592][SQL] Avoid logging makes Driver OOM in AdaptiveSparkPlanExec

  #50368 commented on Mar 28, 2025 • 7 new comments

• [SPARK-51272][CORE]. Fix for the race condition in Scheduler causing failure in retrying all partitions in case of indeterministic shuffle keys

  #50033 commented on Apr 3, 2025 • 7 new comments

• [SPARK-51647][INFRA] Add a job to guard REPLs: spark-sql and spark-shell

  #50423 commented on Mar 31, 2025 • 6 new comments

• [WIP][SPARK-51395][SQL] Refine handling of default values in procedures

  #50197 commented on Apr 1, 2025 • 6 new comments

• [SPARK-44856][PYTHON] Improve Python UDTF arrow serializer performance

  #50099 commented on Apr 3, 2025 • 3 new comments

• [SPARK-51270] Support nanosecond precision timestamp in Variant

  #50270 commented on Mar 27, 2025 • 1 new comment

• [SPARK-51414][SQL] Add the make_time() function

  #50269 commented on Apr 1, 2025 • 1 new comment

• [SPARK-51338][INFRA] Add automated CI build for `connect-examples`

  #50187 commented on Mar 27, 2025 • 1 new comment

• [SPARK-51411][SS][DOCS] Add documentation for the transformWithState operator

  #50177 commented on Apr 2, 2025 • 1 new comment

• [SPARK-51503][SQL] Support Variant type in XML scan

  #50300 commented on Apr 3, 2025 • 0 new comments

• [SPARK-50838][SQL] Add Cross Join as legal in recursion of Recursive CTE

  #50308 commented on Apr 2, 2025 • 0 new comments

• [WIP][SPARK-44662][SQL] Perf improvement in BroadcastHashJoin queries with stream side join key on non partition columns

  #49209 commented on Mar 27, 2025 • 0 new comments

• Canonicalized CTEs to avoid cached result not being used and recomputed

  #50360 commented on Apr 2, 2025 • 0 new comments

• [SPARK-49872][CORE] allow unlimited json size again

  #49163 commented on Mar 28, 2025 • 0 new comments

• [SPARK-51611][SQL] New iteration of single-pass Analyzer functionality

  #50406 commented on Apr 2, 2025 • 0 new comments

• [WIP][SPARK-24815] [CORE] Trigger Interval based DRA for Structured Streaming

  #42352 commented on Mar 29, 2025 • 0 new comments

• [SPARK-51637][PYTHON] Implement createColumnarReader for Python Data Source

  #50414 commented on Mar 31, 2025 • 0 new comments

• [Core]Convert shuffleWriteTime from Nanoseconds to Milliseconds for Consistency with Other Metrics

  #50418 commented on Apr 1, 2025 • 0 new comments

• [SPARK-37019][SQL] Add codegen support to array higher-order functions

  #34558 commented on Apr 1, 2025 • 0 new comments