Share via


Databricks Runtime 17.1 (Beta)

The following release notes provide information about Databricks Runtime 17.1 (Beta), powered by Apache Spark 4.0.0.

Azure Databricks released this version in July 2025.

Tip

To see release notes for Databricks Runtime versions that have reached end-of-support (EoS), see End-of-support Databricks Runtime release notes. The EoS Databricks Runtime versions have been retired and might not be updated.

New features and improvements

Support for schema and catalog level default collation

You can now set a default collation for schemas and catalogs in Databricks Runtime 17.1. This allows you to define a collation that applies to all objects created within the schema or catalog, ensuring consistent collation behavior across your data.

Better handling of JSON options with VARIANT

The from_json and to_json functions now correctly apply JSON options when working with top-level VARIANT schemas. This ensures consistent behavior with other supported data types.

display() supports Streaming Real-Time Mode

You can use the display() function with streaming queries that use a real-time trigger. In this mode, display() automatically uses update output mode and avoids adding a LIMIT clause, which is unsupported with update. The display refresh interval is also decoupled from batch duration to better support real-time responsiveness.

Rate source supports Streaming Real-Time Mode

The rate source in Structured Streaming now works with real-time triggers, enabling continuous data generation for streaming pipelines using Streaming Real-Time Mode.

Event-time timers supported for time-based windows

Azure Databricks supports event-time-based triggers for time-based window operations in Streaming Real-Time Mode. This enables more accurate handling of late data and watermarking in real-time streaming queries.

Improved schema listing

The Snowflake connector uses the INFORMATION_SCHEMA table instead of the SHOW SCHEMAS command to list schemas. This change removes the 10,000-schema limit of the previous approach and improves support for databases with a large number of schemas.

Improved string column mapping in Redshift writes

When writing tables to Redshift using spark.write, string columns without an explicit length are mapped to VARCHAR(MAX) instead of TEXT. This avoids Redshift’s default behavior of interpreting TEXT as VARCHAR(256), which could cause errors when writing longer strings.

Support for DDL, SHOW, and DESCRIBE on access-controlled tables

SQL commands like DDL, SHOW, and DESCRIBE now run on fine-grained access-controlled tables using dedicated clusters. When needed to enforce access controls, Azure Databricks routes these commands through serverless compute, just like it does for reads, merges, and appends.

Support for TIMESTAMP WITHOUT TIME ZONE syntax

You can now specify TIMESTAMP WITHOUT TIME ZONE instead of TIMESTAMP_NTZ. This change improves compatibility with the SQL Standard.

Behavioral changes

Resolved subquery correlation issue

Azure Databricks no longer incorrectly correlates semantically equal aggregate expressions between a subquery and its outer query. Previously, this could lead to incorrect query results.

Timestamp NTZ literals no longer pushed down

For both the Snowflake connector and Snowflake Lakehouse Federation, TIMESTAMP_NTZ (timestamp without time zone) literals are no longer pushed down to Snowflake. This change prevents query failures caused by incompatible timestamp handling and improves reliability for affected queries.

Error thrown for invalid CHECK constraints

Azure Databricks now throws an AnalysisException if a CHECK constraint expression cannot be resolved during constraint validation.

Pulsar connector no longer exposes Bouncy Castle

The Bouncy Castle library is now shaded in the Pulsar connector to prevent classpath conflicts. As a result, Spark jobs can no longer access org.bouncycastle.* classes from the connector. If your application code depends on Bouncy Castle, install the library manually on your cluster.

Library upgrades

  • Upgraded Python libraries:

    • google-auth from 2.40.1 to 2.40.0
    • mlflow-skinny from 3.0.0 to 2.22.0
    • opentelemetry-api from 1.33.0 to 1.32.1
    • opentelemetry-sdk from 1.33.0 to 1.32.1
    • opentelemetry-semantic-conventions from 0.54b0 to 0.53b1
    • packaging from 24.1 to 24.2
  • Upgraded R libraries:

    No R libraries were upgraded in this version.

  • Upgraded Java libraries:

    • software.amazon.cryptools.AmazonCorrettoCryptoProvider from 2.4.1-linux-x86_64 to 2.5.0-linux-x86_64

Apache Spark

Databricks Runtime 17.1 includes Apache Spark 4.0.0. This release includes all Spark fixes and improvements included in version 17.0, as well as the following additional bug fixes and improvements made to Spark:

  • SPARK-52721 Fix message parameter for CANNOT_PARSE_DATATYPE
  • SPARK-51919 Allow overwriting statically registered Python Data Source
  • SPARK-52632 Revert "Pretty display V2 write plan nodes"
  • SPARK-52715 Pretty SQL string of TIME - DAY-TIME INTERVAL
  • SPARK-52632 Pretty display V2 write plan nodes
  • SPARK-51268 Revert "Make TreeNode lock-free"
  • SPARK-52701 Fix float32 type widening in mod with bool under ANSI
  • SPARK-52563 Fix var naming bug in _assert_pandas_almost_equal
  • SPARK-51268 Make TreeNode lock-free
  • SPARK-52541 Add programming guide for Declarative Pipelines
  • SPARK-51695 Fix test failures caused by CHECK CONSTRAINT syntax change
  • SPARK-52666 Map User Defined Type to correct MutableValue in SpecificInternalRow
  • SPARK-52660 Add time type to CodeGenerator#javaClass
  • SPARK-52693 Support +/- ANSI day-time intervals to/from TIME
  • SPARK-52705 Refactor deterministic check for grouping expressions
  • SPARK-52699 Support aggregating TIME type in interpreted mode
  • SPARK-52503 Fix drop when the input column is not existent
  • SPARK-52535 Improve code readability of rule ApplyColumnarRulesAndInsertTransitions
  • SPARK-52551 Add a new v2 Predicate BOOLEAN_EXPRESSION
  • SPARK-52698 Revert "Improve type hints for datasource module"
  • SPARK-52684 Make CACHE TABLE Commands atomic while encountering execution errors
  • SPARK-52698 Improve type hints for datasource module
  • SPARK-52671 RowEncoder shall not lookup a resolved UDT
  • SPARK-52670 Make HiveResult work with UserDefinedType#stringifyValue
  • SPARK-51695 Sync check constraint parser syntax
  • SPARK-51695 Align CREATE/REPLACE TABLE column-and-constraint syntax with ANSI SQL
  • SPARK-52692 Support caching the TIME type
  • SPARK-51516 Support TIME by the Thrift server
  • SPARK-52674 Clean up the usage of deprecated APIs related to RandomStringUtils
  • SPARK-51428 Assign Aliases for collated expression trees deterministically
  • SPARK-52672 Don't replace Sort/Having expressions with aliases if expression exists in Aggregate
  • SPARK-52618 Casting TIME(n) to TIME(m)
  • SPARK-51695 Revert "Align CREATE/REPLACE TABLE column-and-constraint syntax with ANSI SQL"
  • SPARK-52656 Fix current_time()
  • SPARK-52630 Reorganize streaming operator and state mgmt code and dirs
  • SPARK-51695 Align CREATE/REPLACE TABLE column-and-constraint syntax with ANSI SQL
  • SPARK-52649 Trim aliases before matching Sort/Having/Filter expressions in buildAggExprList
  • SPARK-52578 Add metrics for rows to track case and action in MergeRowsExec
  • SPARK-49428 Move Connect Scala Client from Connector to SQL
  • SPARK-51695 Revert "Align CREATE/REPLACE TABLE column-and-constraint syntax with ANSI SQL"
  • SPARK-52494 Support colon-sign operator syntax to access Variant fields
  • SPARK-51695 Align CREATE/REPLACE TABLE column-and-constraint syntax with ANSI SQL
  • SPARK-52642 Use daemonWorkers.get(worker) to avoid unexpected out-of-sync between idleWorkers and daemonWorkers
  • SPARK-52570 Enable divide-by-zero for numeric rmod with ANSI enabled
  • SPARK-52633 Deduplicate single Union child output before DeduplicateRelations
  • SPARK-52570 Revert "Enable divide-by-zero for numeric rmod with ANSI enabled"
  • SPARK-52600 Move CompletionIterator to common/utils
  • SPARK-52626 Allow grouping by the time type
  • SPARK-52570 Enable divide-by-zero for numeric rmod with ANSI enabled
  • SPARK-52395 Fail reducer task immediately when Prism fetch request fails
  • SPARK-52596 Try make TIMESTAMP_NTZ from DATE and TIME
  • SPARK-52599 Support periodical traceback dump in Driver side workers
  • SPARK-52595 Rename the TimeAdd expression to TimestampAddInterval
  • SPARK-52586 Introduce AnyTimeType
  • SPARK-52583 Add an Developer API for stringifying values in UserDefinedType
  • SPARK-52586 Revert "Introduce AnyTimeType"
  • SPARK-52586 Introduce AnyTimeType
  • SPARK-52354 Add type coercion to UnionLoop
  • SPARK-52579 Set periodical traceback dump for Python workers
  • SPARK-52456 Lower the minimum limit of spark.eventLog.rolling.maxFileSize
  • SPARK-52536 Set AsyncProfilerLoader extractionDir to spark local dir
  • SPARK-52405 Extend V2JDBCTest to test multiple partition reading
  • SPARK-52303 Promote ExternalCommandRunner to Stable
  • SPARK-52395 Fast fail when shuffle fetch failure happens
  • SPARK-52572 Avoid CAST_INVALID_INPUT of DataFrame.isin in ANSI mode
  • SPARK-52349 Enable boolean division tests with ANSI enabled
  • SPARK-52402 Fix divide-by-zero errors in Kendall and Pearson correlation under ANSI mode
  • SPARK-52356 Enable divide-by-zero for boolean mod/rmod with ANSI enabled
  • SPARK-52288 Avoid INVALID_ARRAY_INDEX in split/rsplit when ANSI mode is on
  • SPARK-52557 Avoid CAST_INVALID_INPUT of to_numeric(errors='coerce') in ANSI mode
  • SPARK-52440 Add python install extra for declarative pipelines
  • SPARK-52569 Fix the class cast exception in SecondsOfTimeWithFraction
  • SPARK-52540 Make TIMESTAMP_NTZ from DATE and TIME
  • SPARK-52539 Introduce session hooks
  • SPARK-52430 Resolve bug with nullability of Union in rCTEs
  • SPARK-52554 Avoid multiple roundtrips for config check in Spark Connect
  • SPARK-52553 Fix NumberFormatException when reading v1 changelog
  • SPARK-52534 Make MLCache and MLHandler thread-safe
  • SPARK-52460 Store internal TIME values as nanoseconds
  • SPARK-52482 ZStandard support for file data source reader
  • SPARK-52531 OuterReference in subquery aggregate is incorrectly tied to outer query aggregate
  • SPARK-52339 Fix comparison of InMemoryFileIndex instances
  • SPARK-52418 Add a noElements state variable to the PercentileHeap to avoid repetitive calculations of isEmpty()
  • SPARK-52537 Print stacktrace on creating temp dir failure
  • SPARK-52462 Enforce type coercion before children output deduplication in Union
  • SPARK-52528 Enable divide-by-zero for numeric mod with ANSI enabled
  • SPARK-52458 Support spark.eventLog.excludedPatterns
  • SPARK-52493 Support TIMESTAMP WITHOUT TIME ZONE
  • SPARK-51563 Support the fully qualified type name TIME(n) WITHOUT TIME ZONE
  • SPARK-52450 Improve performance of schema deepcopy
  • SPARK-52519 Enable divide-by-zero for numeric floordiv with ANSI enabled
  • SPARK-52489 Forbid duplicate SQLEXCEPTION and NOT FOUND handlers inside SQL Script
  • SPARK-52488 Strip alias before wrapping outer references under HAVING
  • SPARK-52383 Improve errors in SparkConnectPlanner
  • SPARK-52335 Unify the 'invalid bucket count' error for both Connect and Classic
  • SPARK-52337 Make InvalidPlanInput a user-facing error
  • SPARK-52312 Ignore V2WriteCommand when caching DataFrame
  • SPARK-52487 Add Stage Submitted Time and Duration to StagePage Detail
  • SPARK-52433 Unify the string coercion in createDataFrame
  • SPARK-52405 Revert "Extend V2JDBCTest to test multiple partition reading"
  • SPARK-52405 Extend V2JDBCTest to test multiple partition reading
  • SPARK-52427 Normalize aggregate expression list covered by a Project
  • SPARK-51745 Enforce State Machine for RocksDBStateStore
  • SPARK-52447 Move GetViewColumnByNameAndOrdinal normalization to SessionCatalogSuite
  • SPARK-52397 Idempotent ExecutePlan: the second ExecutePlan with same operationId and plan reattaches
  • SPARK-52355 Infer VariantVal object type as VariantType when creating a DataFrame
  • SPARK-52425 Refactor ExtractValue so it can be reused in single-pass implementation
  • SPARK-52249 Enable divide-by-zero for numeric truediv with ANSI enabled
  • SPARK-52228 Integrate state interaction microbenchmark into Quicksilver (TCP)
  • SPARK-48356 Improve FOR statement's column schema inference
  • SPARK-48353 Exception handling improvements
  • SPARK-52060 Make OneRowRelationExec node
  • SPARK-52391 Refactor TransformWithStateExec to extract shared functions and variables into an abstract base class for Scala and Python
  • SPARK-52218 Make current datetime functions evaluable again
  • SPARK-52350 Fix link for SS programming guide in 4.0
  • SPARK-52385 Remove TempResolvedColumns from InheritAnalysisRules name
  • SPARK-52392 New single-pass Analyzer functionality
  • SPARK-52243 Add NERF support for schema-related InvalidPlanInput errors
  • SPARK-52338 Support for inheriting default collation from schema to View
  • SPARK-52307 Support Scalar Arrow Iterator UDF
  • SPARK-52300 Make SQL UDTVF resolution use consistent configurations with view resolution
  • SPARK-52219 Schema level collation support for tables
  • SPARK-52228 Construct the benchmark purposed TWS state server with in-memory state impls and the benchmark code in python
  • SPARK-52333 Squeeze the protocol of retrieving timers for transformWithState in PySpark
  • SPARK-52272 V2SessionCatalog does not alter schema on Hive Catalog
  • SPARK-51890 Add New Parameter options to Logical Plan View.
  • SPARK-51936 Revert "ReplaceTableAsSelect should overwrite the new table instead of append"
  • SPARK-52060 Revert "Make OneRowRelationExec node"
  • SPARK-51936 ReplaceTableAsSelect should overwrite the new table instead of append
  • SPARK-51302 Spark Connect supports JDBC should use the DataFrameReader API
  • SPARK-52060 Make OneRowRelationExec node
  • SPARK-50137 Avoid fallback to Hive-incompatible ways when table creation fails by thrift exception
  • SPARK-52184 Wrap external engine JDBC syntax errors with a unified exception
  • SPARK-47618 Use Magic Committer for all S3 buckets by default
  • SPARK-52305 Refine the docstring for isnotnull, equal_null, nullif, nullifzero, nvl, nvl2, zeroifnull
  • SPARK-52174 Enable spark.checkpoint.compress by default
  • SPARK-52224 Introduce pyyaml as a dependency for the Python client
  • SPARK-52233 Fix map_zip_with for Floating Point Types
  • SPARK-52036 Add SQLConfHelper trait to v2.FileScan
  • SPARK-52159 Remove too wide condition for MariaDB dialect
  • SPARK-50104 Support SparkSession.executeCommand in Connect
  • SPARK-52159 Enable by default not found granular check
  • SPARK-52240 Corrected row index usage when exploding packed arrays in vectorized reader
  • SPARK-52153 Fix from_json and to_json with variant
  • SPARK-48618 Improve not exists test to check for both 3part and 2part names
  • SPARK-48618 Add table existence override for Redshift
  • SPARK-52216 Make InvalidCommandInput a user-facing error
  • SPARK-52299 replace deprecated JsonNode.fields
  • SPARK-52287 Improve SparkContext not to populate o.a.s.internal.io.cloud.*-related setting if not exist
  • SPARK-51430 Stop PySpark context logger from propagating logs to stdout
  • SPARK-52278 Scalar Arrow UDF support named arguments
  • SPARK-52026 Block pandas API on Spark on ANSI mode by default
  • SPARK-52171 StateDataSource join implementation for state v3
  • SPARK-52159 Properly handle table existence check for jdbc dialects
  • SPARK-52158 Add InvalidInputErrors object to centralize errors in SparkConnectPlanner
  • SPARK-52026 Revert "Block pandas API on Spark on ANSI mode by default"
  • SPARK-52026 Block pandas API on Spark on ANSI mode by default
  • SPARK-52195 Fix initial state column dropping issue for Python TWS
  • SPARK-52211 Strip $ suffix from SparkConnectServer INFO log
  • SPARK-52239 Support register an arrow UDF
  • SPARK-52215 Implement Scalar Arrow UDF
  • SPARK-52188 Fix for StateDataSource where StreamExecution.RUN_ID_KEY is not set
  • SPARK-52189 New golden file tests for NATURAL JOIN with GROUP BY, HAVING, ORDER BY and LCAs
  • SPARK-52079 Normalize internal projections under Aggregate as well
  • SPARK-52234 Fix error on non-string input to schema_of_csv/xml
  • SPARK-52221 Refactor SqlScriptingLocalVariableManager into more generic context manager
  • SPARK-52181 Increase variant size limit to 128MiB
  • SPARK-52217 Skip outer reference validation under Filter in single-pass resolver
  • SPARK-51260 Move V2ExpressionBuilder and PushableExpression to Catalyst module
  • SPARK-52156 Put legacy CREATE TEMPORARY TABLE ... USING provider under the flag
  • SPARK-51012 Remove SparkStrategy from Connect Shims.
  • SPARK-52109 Add listTableSummaries API to Data Source V2 Table Catalog API
  • SPARK-50915 Add getCondition and deprecate getErrorClass in PySparkException
  • SPARK-52022 Add additional check in default method for SparkThrowable.getQueryContext
  • SPARK-51147 Refactor streaming related classes to a dedicated streaming directory
  • SPARK-52120 Pass the session to ConnectHelper
  • SPARK-52021 Separate operator and expression patterns
  • SPARK-51722 Remove "stop" origin from ParseException
  • SPARK-52096 Reclassify kafka source offset assertion error
  • SPARK-51985 Remove Experimental from (Long|Double)AccumulatorSource
  • SPARK-51156 Static token authentication support in Spark Connect
  • SPARK-52157 Use earlier computed name for OuterReference in single-pass resolver
  • SPARK-52100 Enable overriding the recursion level limit in syntax
  • SPARK-51605 Try to create the parent directory before touching the logFile
  • SPARK-52127 Squeeze the protocol of MapState KEYS / VALUES / ITERATOR for transformWithState in PySpark
  • SPARK-52040 ResolveLateralColumnAliasReference should retain the plan id
  • SPARK-51638 Fix fetching the remote disk stored RDD blocks via the external shuffle service
  • SPARK-51983 Prepare the test environment for pandas API on Spark with ANSI mode enabled
  • SPARK-52087 Add copying of tags and origin to AliasHelper.trimNonTopLevelAliases
  • SPARK-51210 Add --enable-native-access=ALL-UNNAMED to Java options for Java 24+
  • SPARK-52052 Add .broadcast in the companion object of SerializableConfiguration
  • SPARK-51823 Add config to not persist state store on executors
  • SPARK-51291 Reclassify validation errors thrown from state store loading
  • SPARK-52006 Exclude CollectMetricsExec accumulator from Spark UI + event logs + metric heartbeats
  • SPARK-52076 Explicitly closes ExecutePlanResponseReattachableIterator after usage
  • SPARK-51428 Revert "Assign Aliases for collated expression trees deterministically"
  • SPARK-52079 Normalize order of attributes in inner project lists
  • SPARK-52033 Fix Generate node bug fix where the output of the child node can have multiple copies of the same Attribute
  • SPARK-51428 Assign Aliases for collated expression trees deterministically
  • SPARK-52055 Suppress the exception in ExecutePlanResponseReattachableIterator.del
  • SPARK-51921 Use long type for TTL duration in millisecond in transformWithState
  • SPARK-50763 Add Analyzer rule for resolving SQL table functions
  • SPARK-51884 Part 1.a Add outer scope attributes for SubqueryExpression
  • SPARK-50762 Add Analyzer rule for resolving SQL scalar UDFs
  • SPARK-51814 Uses list(self) instead of StructType.fields for old version compat
  • SPARK-51537 Construct the session-specific classloader based on the default session classloader on executor
  • SPARK-50777 Remove redundant no-op init/destroy methods from Filter classes
  • SPARK-51691 SerializationDebugger should swallow exception when try to find the reason of serialization problem
  • SPARK-51386 Assign name to error conditions _LEGACY_ERROR_TEMP_3300-3302
  • SPARK-50734 Add catalog API for creating and registering SQL UDFs
  • SPARK-51989 Add missing Filter subclasses to all list in datasource
  • SPARK-52037 Order in LCA inner project lists should respect order in original project list
  • SPARK-52007 Expression IDs shouldn't be present in grouping expressions when using grouping sets
  • SPARK-51860 Disable spark.connect.grpc.debug.enabled by default
  • SPARK-47952 Support retrieving the real SparkConnectService GRPC address and port programmatically when running on Yarn
  • SPARK-50906 Add nullability check for if inputs of to_avro align with schema
  • SPARK-50581 Fix support for UDAF in Dataset.observe()
  • SPARK-49946 Require an error class in SparkOutOfMemoryError
  • SPARK-49748 Add getCondition and deprecate getErrorClass in SparkThrowable
  • SPARK-51959 Make functions not import ParentDataFrame
  • SPARK-51964 Correctly resolve attributes from hidden output in ORDER BY and HAVING on top of an Aggregate in single-pass Analyzer
  • SPARK-49942 Rename errorClass to condition in classifyException()
  • SPARK-51963 Simplify IndexToString.transform
  • SPARK-51946 Eagerly fail for creating hive-incompatible datasource table with 'col' a partition name
  • SPARK-51553 Modify EXTRACT to support TIME data type
  • SPARK-51891 Squeeze the protocol of ListState GET / PUT / APPENDLIST for transformWithState in PySpark
  • SPARK-51931 Add maxBytesPerOutputBatch to limit the number of bytes of Arrow output batch
  • SPARK-51441 Add DSv2 APIs for constraints
  • SPARK-51814 Use RecordBatch.schema.names instead of column_names for old pyarrow compatibility
  • SPARK-51913 JDBCTableCatalog#loadTable should throw no such table error correctly
  • SPARK-51840 Restore Partition columns in HiveExternalCatalog#alterTable
  • SPARK-51414 Add the make_time() function
  • SPARK-49700 Unified Scala Interface for Connect and Classic
  • SPARK-51914 Add com.mysql.cj to spark.sql.hive.metastore.sharedPrefixes
  • SPARK-51827 Support Spark Connect on transformWithState in PySpark
  • SPARK-51901 Disallow generator functions in grouping sets
  • SPARK-51423 Add the current_time() function for TIME datatype
  • SPARK-51680 Set the logical type for TIME in the parquet writer
  • SPARK-51889 Fix a bug for MapState clear() in Python TWS
  • SPARK-51869 Create classification for user errors within UDFs for Scala TransformWithState
  • SPARK-51878 Improve fillDefaultValue by exec the foldable default expression.
  • SPARK-51757 Fix LEAD/LAG Function Offset Exceeds Window Group Size
  • SPARK-51805 Get function with improper argument should throw proper exception instead of an internal one
  • SPARK-51900 Properly throw datatype mismatch in single-pass Analyzer
  • SPARK-51421 Get seconds of TIME datatype
  • SPARK-51661 Partitions discovery of TIME column values
  • SPARK-51687 Pushdown filters with TIME values to parquet
  • SPARK-51419 Get hours of TIME datatype
  • SPARK-51881 Make AvroOptions comparable
  • SPARK-51861 Remove duplicated/unnecessary info of InMemoryRelation Plan Detail
  • SPARK-51814 Introduce a new API transformWithState in PySpark
  • SPARK-51779 Use virtual column families for stream-stream joins
  • SPARK-51649 Dynamic writes/reads of TIME partitions
  • SPARK-51634 Support TIME in off-heap column vectors
  • SPARK-51877 Add functions 'chr', 'random' and 'uuid'
  • SPARK-51610 Support the TIME data type in the parquet datasource
  • SPARK-51420 Get minutes of TIME datatype
  • SPARK-51822 Throwing classified error when disallowed functions are called during StatefulProcessor.init()
  • SPARK-51868 Move type coercion validation to a separate object
  • SPARK-49747 Migrate connect/ files to structured logging

Azure Databricks ODBC/JDBC driver support

Azure Databricks supports ODBC/JDBC drivers released in the past 2 years. Please download the recently released drivers and upgrade (download ODBC, download JDBC).

System environment

  • Operating System: Ubuntu 24.04.2 LTS
  • Java: Zulu17.58+21-CA
  • Scala: 2.13.16
  • Python: 3.12.3
  • R: 4.4.2
  • Delta Lake: 4.0.0

Installed Python libraries

Library Version Library Version Library Version
annotated-types 0.7.0 anyio 4.6.2 argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0 arrow 1.3.0 asttokens 2.0.5
astunparse 1.6.3 async-lru 2.0.4 attrs 24.3.0
autocommand 2.2.2 azure-common 1.1.28 azure-core 1.34.0
azure-identity 1.20.0 azure-mgmt-core 1.5.0 azure-mgmt-web 8.0.0
azure-storage-blob 12.23.0 azure-storage-file-datalake 12.17.0 babel 2.16.0
backports.tarfile 1.2.0 beautifulsoup4 4.12.3 black 24.10.0
bleach 6.2.0 blinker 1.7.0 boto3 1.36.2
botocore 1.36.3 cachetools 5.5.1 certifi 2025.1.31
cffi 1.17.1 chardet 4.0.0 charset-normalizer 3.3.2
click 8.1.7 cloudpickle 3.0.0 comm 0.2.1
contourpy 1.3.1 cryptography 43.0.3 cycler 0.11.0
Cython 3.0.12 databricks-sdk 0.49.0 dbus-python 1.3.2
debugpy 1.8.11 decorator 5.1.1 defusedxml 0.7.1
Deprecated 1.2.13 distlib 0.3.9 docstring-to-markdown 0.11
executing 0.8.3 facets-overview 1.1.1 fastapi 0.115.12
fastjsonschema 2.21.1 filelock 3.18.0 fonttools 4.55.3
fqdn 1.5.1 fsspec 2023.5.0 gitdb 4.0.11
GitPython 3.1.43 google-api-core 2.20.0 google-auth 2.40.0
google-cloud-core 2.4.3 google-cloud-storage 3.1.0 google-crc32c 1.7.1
google-resumable-media 2.7.2 googleapis-common-protos 1.65.0 grpcio 1.67.0
grpcio-status 1.67.0 h11 0.14.0 httpcore 1.0.2
httplib2 0.20.4 httpx 0.27.0 idna 3.7
importlib-metadata 6.6.0 importlib_resources 6.4.0 inflect 7.3.1
iniconfig 1.1.1 ipyflow-core 0.0.209 ipykernel 6.29.5
ipython 8.30.0 ipython-genutils 0.2.0 ipywidgets 7.8.1
isodate 0.6.1 isoduration 20.11.0 jaraco.context 5.3.0
jaraco.functools 4.0.1 jaraco.text 3.12.1 jedi 0.19.2
Jinja2 3.1.5 jmespath 1.0.1 joblib 1.4.2
json5 0.9.25 jsonpointer 3.0.0 jsonschema 4.23.0
jsonschema-specifications 2023.7.1 jupyter-events 0.10.0 jupyter-lsp 2.2.0
jupyter_client 8.6.3 jupyter_core 5.7.2 jupyter_server 2.14.1
jupyter_server_terminals 0.4.4 jupyterlab 4.3.4 jupyterlab-pygments 0.1.2
jupyterlab-widgets 1.0.0 jupyterlab_server 2.27.3 kiwisolver 1.4.8
launchpadlib 1.11.0 lazr.restfulclient 0.14.6 lazr.uri 1.0.6
markdown-it-py 2.2.0 MarkupSafe 3.0.2 matplotlib 3.10.0
matplotlib-inline 0.1.7 mccabe 0.7.0 mdurl 0.1.0
mistune 2.0.4 mlflow-skinny 2.22.0 mmh3 5.1.0
more-itertools 10.3.0 msal 1.32.3 msal-extensions 1.3.1
mypy-extensions 1.0.0 nbclient 0.8.0 nbconvert 7.16.4
nbformat 5.10.4 nest-asyncio 1.6.0 nodeenv 1.9.1
notebook 7.3.2 notebook_shim 0.2.3 numpy 2.1.3
oauthlib 3.2.2 opentelemetry-api 1.32.1 opentelemetry-sdk 1.32.1
opentelemetry-semantic-conventions 0.53b1 overrides 7.4.0 packaging 24.2
pandas 2.2.3 pandocfilters 1.5.0 parso 0.8.4
pathspec 0.10.3 patsy 1.0.1 pexpect 4.8.0
pillow 11.1.0 pip 25.0.1 platformdirs 3.10.0
plotly 5.24.1 pluggy 1.5.0 prometheus_client 0.21.0
prompt-toolkit 3.0.43 proto-plus 1.26.1 protobuf 5.29.4
psutil 5.9.0 psycopg2 2.9.3 ptyprocess 0.7.0
pure-eval 0.2.2 pyarrow 19.0.1 pyasn1 0.4.8
pyasn1-modules 0.2.8 pyccolo 0.0.71 pycparser 2.21
pydantic 2.10.6 pydantic_core 2.27.2 pyflakes 3.2.0
Pygments 2.15.1 PyGObject 3.48.2 pyiceberg 0.9.0
PyJWT 2.10.1 pyodbc 5.2.0 pyparsing 3.2.0
pyright 1.1.394 pytest 8.3.5 python-dateutil 2.9.0.post0
python-json-logger 3.2.1 python-lsp-jsonrpc 1.1.2 python-lsp-server 1.12.0
pytoolconfig 1.2.6 pytz 2024.1 PyYAML 6.0.2
pyzmq 26.2.0 referencing 0.30.2 requests 2.32.3
rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rich 13.9.4
rope 1.12.0 rpds-py 0.22.3 rsa 4.9.1
s3transfer 0.11.3 scikit-learn 1.6.1 scipy 1.15.1
seaborn 0.13.2 Send2Trash 1.8.2 setuptools 74.0.0
six 1.16.0 smmap 5.0.0 sniffio 1.3.0
sortedcontainers 2.4.0 soupsieve 2.5 sqlparse 0.5.3
ssh-import-id 5.11 stack-data 0.2.0 starlette 0.46.2
statsmodels 0.14.4 strictyaml 1.7.3 tenacity 9.0.0
terminado 0.17.1 threadpoolctl 3.5.0 tinycss2 1.4.0
tokenize_rt 6.1.0 tomli 2.0.1 tornado 6.4.2
traitlets 5.14.3 typeguard 4.3.0 types-python-dateutil 2.9.0.20241206
typing_extensions 4.12.2 tzdata 2024.1 ujson 5.10.0
unattended-upgrades 0.1 uri-template 1.3.0 urllib3 2.3.0
uvicorn 0.34.2 virtualenv 20.29.3 wadllib 1.3.6
wcwidth 0.2.5 webcolors 24.11.1 webencodings 0.5.1
websocket-client 1.8.0 whatthepatch 1.0.2 wheel 0.45.1
widgetsnbextension 3.6.6 wrapt 1.17.0 yapf 0.40.2
zipp 3.21.0

Installed R libraries

R libraries are installed from the Posit Package Manager CRAN snapshot on 2025-03-20.

Library Version Library Version Library Version
arrow 19.0.1 askpass 1.2.1 assertthat 0.2.1
backports 1.5.0 base 4.4.2 base64enc 0.1-3
bigD 0.3.0 bit 4.6.0 bit64 4.6.0-1
bitops 1.0-9 blob 1.2.4 boot 1.3-30
brew 1.0-10 brio 1.1.5 broom 1.0.7
bslib 0.9.0 cachem 1.1.0 callr 3.7.6
caret 7.0-1 cellranger 1.1.0 chron 2.3-62
class 7.3-22 cli 3.6.4 clipr 0.8.0
clock 0.7.2 cluster 2.1.6 codetools 0.2-20
colorspace 2.1-1 commonmark 1.9.5 compiler 4.4.2
config 0.3.2 conflicted 1.2.0 cpp11 0.5.2
crayon 1.5.3 credentials 2.0.2 curl 6.2.1
data.table 1.17.0 datasets 4.4.2 DBI 1.2.3
dbplyr 2.5.0 desc 1.4.3 devtools 2.4.5
diagram 1.6.5 diffobj 0.3.5 digest 0.6.37
downlit 0.4.4 dplyr 1.1.4 dtplyr 1.3.1
e1071 1.7-16 ellipsis 0.3.2 evaluate 1.0.3
fansi 1.0.6 farver 2.1.2 fastmap 1.2.0
fontawesome 0.5.3 forcats 1.0.0 foreach 1.5.2
foreign 0.8-86 forge 0.2.0 fs 1.6.5
future 1.34.0 future.apply 1.11.3 gargle 1.5.2
generics 0.1.3 gert 2.1.4 ggplot2 3.5.1
gh 1.4.1 git2r 0.35.0 gitcreds 0.1.2
glmnet 4.1-8 globals 0.16.3 glue 1.8.0
googledrive 2.1.1 googlesheets4 1.1.1 gower 1.0.2
graphics 4.4.2 grDevices 4.4.2 grid 4.4.2
gridExtra 2.3 gsubfn 0.7 gt 0.11.1
gtable 0.3.6 hardhat 1.4.1 haven 2.5.4
highr 0.11 hms 1.1.3 htmltools 0.5.8.1
htmlwidgets 1.6.4 httpuv 1.6.15 httr 1.4.7
httr2 1.1.1 ids 1.0.1 ini 0.3.1
ipred 0.9-15 isoband 0.2.7 iterators 1.0.14
jquerylib 0.1.4 jsonlite 1.9.1 juicyjuice 0.1.0
KernSmooth 2.23-22 knitr 1.50 labeling 0.4.3
later 1.4.1 lattice 0.22-5 lava 1.8.1
lifecycle 1.0.4 listenv 0.9.1 lubridate 1.9.4
magrittr 2.0.3 markdown 1.13 MASS 7.3-60.0.1
Matrix 1.6-5 memoise 2.0.1 methods 4.4.2
mgcv 1.9-1 mime 0.13 miniUI 0.1.1.1
mlflow 2.20.4 ModelMetrics 1.2.2.2 modelr 0.1.11
munsell 0.5.1 nlme 3.1-164 nnet 7.3-19
numDeriv 2016.8-1.1 openssl 2.3.2 parallel 4.4.2
parallelly 1.42.0 pillar 1.10.1 pkgbuild 1.4.6
pkgconfig 2.0.3 pkgdown 2.1.1 pkgload 1.4.0
plogr 0.2.0 plyr 1.8.9 praise 1.0.0
prettyunits 1.2.0 pROC 1.18.5 processx 3.8.6
prodlim 2024.06.25 profvis 0.4.0 progress 1.2.3
progressr 0.15.1 promises 1.3.2 proto 1.0.0
proxy 0.4-27 ps 1.9.0 purrr 1.0.4
R6 2.6.1 ragg 1.3.3 randomForest 4.7-1.2
rappdirs 0.3.3 rcmdcheck 1.4.0 RColorBrewer 1.1-3
Rcpp 1.0.14 RcppEigen 0.3.4.0.2 reactable 0.4.4
reactR 0.6.1 readr 2.1.5 readxl 1.4.5
recipes 1.2.0 rematch 2.0.0 rematch2 2.1.2
remotes 2.5.0 reprex 2.1.1 reshape2 1.4.4
rlang 1.1.5 rmarkdown 2.29 RODBC 1.3-26
roxygen2 7.3.2 rpart 4.1.23 rprojroot 2.0.4
Rserve 1.8-15 RSQLite 2.3.9 rstudioapi 0.17.1
rversions 2.1.2 rvest 1.0.4 sass 0.4.9
scales 1.3.0 selectr 0.4-2 sessioninfo 1.2.3
shape 1.4.6.1 shiny 1.10.0 sourcetools 0.1.7-1
sparklyr 1.9.0 SparkR 4.0.0 sparsevctrs 0.3.1
spatial 7.3-17 splines 4.4.2 sqldf 0.4-11
SQUAREM 2021.1 stats 4.4.2 stats4 4.4.2
stringi 1.8.4 stringr 1.5.1 survival 3.5-8
swagger 5.17.14.1 sys 3.4.3 systemfonts 1.2.1
tcltk 4.4.2 testthat 3.2.3 textshaping 1.0.0
tibble 3.2.1 tidyr 1.3.1 tidyselect 1.2.1
tidyverse 2.0.0 timechange 0.3.0 timeDate 4041.110
tinytex 0.56 tools 4.4.2 tzdb 0.5.0
urlchecker 1.0.1 usethis 3.1.0 utf8 1.2.4
utils 4.4.2 uuid 1.2-1 V8 6.0.2
vctrs 0.6.5 viridisLite 0.4.2 vroom 1.6.5
waldo 0.6.1 whisker 0.4.1 withr 3.0.2
xfun 0.51 xml2 1.3.8 xopen 1.0.1
xtable 1.8-4 yaml 2.3.10 zeallot 0.1.0
zip 2.3.2

Installed Java and Scala libraries (Scala 2.13 cluster version)

Group ID Artifact ID Version
antlr antlr 2.7.7
com.amazonaws amazon-kinesis-client 1.12.0
com.amazonaws aws-java-sdk-autoscaling 1.12.638
com.amazonaws aws-java-sdk-cloudformation 1.12.638
com.amazonaws aws-java-sdk-cloudfront 1.12.638
com.amazonaws aws-java-sdk-cloudhsm 1.12.638
com.amazonaws aws-java-sdk-cloudsearch 1.12.638
com.amazonaws aws-java-sdk-cloudtrail 1.12.638
com.amazonaws aws-java-sdk-cloudwatch 1.12.638
com.amazonaws aws-java-sdk-cloudwatchmetrics 1.12.638
com.amazonaws aws-java-sdk-codedeploy 1.12.638
com.amazonaws aws-java-sdk-cognitoidentity 1.12.638
com.amazonaws aws-java-sdk-cognitosync 1.12.638
com.amazonaws aws-java-sdk-config 1.12.638
com.amazonaws aws-java-sdk-core 1.12.638
com.amazonaws aws-java-sdk-datapipeline 1.12.638
com.amazonaws aws-java-sdk-directconnect 1.12.638
com.amazonaws aws-java-sdk-directory 1.12.638
com.amazonaws aws-java-sdk-dynamodb 1.12.638
com.amazonaws aws-java-sdk-ec2 1.12.638
com.amazonaws aws-java-sdk-ecs 1.12.638
com.amazonaws aws-java-sdk-efs 1.12.638
com.amazonaws aws-java-sdk-elasticache 1.12.638
com.amazonaws aws-java-sdk-elasticbeanstalk 1.12.638
com.amazonaws aws-java-sdk-elasticloadbalancing 1.12.638
com.amazonaws aws-java-sdk-elastictranscoder 1.12.638
com.amazonaws aws-java-sdk-emr 1.12.638
com.amazonaws aws-java-sdk-glacier 1.12.638
com.amazonaws aws-java-sdk-glue 1.12.638
com.amazonaws aws-java-sdk-iam 1.12.638
com.amazonaws aws-java-sdk-importexport 1.12.638
com.amazonaws aws-java-sdk-kinesis 1.12.638
com.amazonaws aws-java-sdk-kms 1.12.638
com.amazonaws aws-java-sdk-lambda 1.12.638
com.amazonaws aws-java-sdk-logs 1.12.638
com.amazonaws aws-java-sdk-machinelearning 1.12.638
com.amazonaws aws-java-sdk-opsworks 1.12.638
com.amazonaws aws-java-sdk-rds 1.12.638
com.amazonaws aws-java-sdk-redshift 1.12.638
com.amazonaws aws-java-sdk-route53 1.12.638
com.amazonaws aws-java-sdk-s3 1.12.638
com.amazonaws aws-java-sdk-ses 1.12.638
com.amazonaws aws-java-sdk-simpledb 1.12.638
com.amazonaws aws-java-sdk-simpleworkflow 1.12.638
com.amazonaws aws-java-sdk-sns 1.12.638
com.amazonaws aws-java-sdk-sqs 1.12.638
com.amazonaws aws-java-sdk-ssm 1.12.638
com.amazonaws aws-java-sdk-storagegateway 1.12.638
com.amazonaws aws-java-sdk-sts 1.12.638
com.amazonaws aws-java-sdk-support 1.12.638
com.amazonaws aws-java-sdk-swf-libraries 1.11.22
com.amazonaws aws-java-sdk-workspaces 1.12.638
com.amazonaws jmespath-java 1.12.638
com.clearspring.analytics stream 2.9.8
com.databricks Rserve 1.8-3
com.databricks databricks-sdk-java 0.27.0
com.databricks.scalapb scalapb-runtime_2.13 0.4.15-11
com.esotericsoftware kryo-shaded 4.0.3
com.esotericsoftware minlog 1.3.0
com.fasterxml classmate 1.5.1
com.fasterxml.jackson.core jackson-annotations 2.18.2
com.fasterxml.jackson.core jackson-core 2.18.2
com.fasterxml.jackson.core jackson-databind 2.18.2
com.fasterxml.jackson.dataformat jackson-dataformat-cbor 2.18.2
com.fasterxml.jackson.dataformat jackson-dataformat-yaml 2.15.2
com.fasterxml.jackson.datatype jackson-datatype-joda 2.18.2
com.fasterxml.jackson.datatype jackson-datatype-jsr310 2.18.2
com.fasterxml.jackson.module jackson-module-paranamer 2.18.2
com.fasterxml.jackson.module jackson-module-scala_2.13 2.18.2
com.github.ben-manes.caffeine caffeine 2.9.3
com.github.blemale scaffeine_2.13 4.1.0
com.github.fommil jniloader 1.1
com.github.fommil.netlib native_ref-java 1.1
com.github.fommil.netlib native_ref-java 1.1-natives
com.github.fommil.netlib native_system-java 1.1
com.github.fommil.netlib native_system-java 1.1-natives
com.github.fommil.netlib netlib-native_ref-linux-x86_64 1.1-natives
com.github.fommil.netlib netlib-native_system-linux-x86_64 1.1-natives
com.github.luben zstd-jni 1.5.6-10
com.github.virtuald curvesapi 1.08
com.github.wendykierp JTransforms 3.1
com.google.api.grpc proto-google-common-protos 2.5.1
com.google.code.findbugs jsr305 3.0.0
com.google.code.gson gson 2.11.0
com.google.crypto.tink tink 1.16.0
com.google.errorprone error_prone_annotations 2.36.0
com.google.flatbuffers flatbuffers-java 24.3.25
com.google.guava failureaccess 1.0.2
com.google.guava guava 33.4.0-jre
com.google.guava listenablefuture 9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc j2objc-annotations 3.0.0
com.google.protobuf protobuf-java 3.25.5
com.google.protobuf protobuf-java-util 3.25.5
com.helger profiler 1.1.1
com.ibm.icu icu4j 75.1
com.jcraft jsch 0.1.55
com.lihaoyi fansi_2.13 0.5.0
com.lihaoyi sourcecode_2.13 0.1.9
com.microsoft.azure azure-data-lake-store-sdk 2.3.10
com.microsoft.sqlserver mssql-jdbc 11.2.2.jre8
com.microsoft.sqlserver mssql-jdbc 12.8.0.jre8
com.ning compress-lzf 1.1.2
com.sun.mail javax.mail 1.5.2
com.sun.xml.bind jaxb-core 2.2.11
com.sun.xml.bind jaxb-impl 2.2.11
com.tdunning json 1.8
com.thoughtworks.paranamer paranamer 2.8
com.trueaccord.lenses lenses_2.13 0.4.13
com.twitter chill-java 0.10.0
com.twitter chill_2.13 0.10.0
com.twitter util-app_2.13 19.8.1
com.twitter util-core_2.13 19.8.1
com.twitter util-function_2.13 19.8.1
com.twitter util-jvm_2.13 19.8.1
com.twitter util-lint_2.13 19.8.1
com.twitter util-registry_2.13 19.8.1
com.twitter util-stats_2.13 19.8.1
com.typesafe config 1.4.3
com.typesafe.scala-logging scala-logging_2.13 3.9.2
com.uber h3 3.7.3
com.univocity univocity-parsers 2.9.1
com.zaxxer HikariCP 4.0.3
com.zaxxer SparseBitSet 1.3
commons-cli commons-cli 1.9.0
commons-codec commons-codec 1.17.2
commons-collections commons-collections 3.2.2
commons-dbcp commons-dbcp 1.4
commons-fileupload commons-fileupload 1.5
commons-httpclient commons-httpclient 3.1
commons-io commons-io 2.18.0
commons-lang commons-lang 2.6
commons-logging commons-logging 1.1.3
commons-pool commons-pool 1.5.4
dev.ludovic.netlib arpack 3.0.3
dev.ludovic.netlib blas 3.0.3
dev.ludovic.netlib lapack 3.0.3
info.ganglia.gmetric4j gmetric4j 1.0.10
io.airlift aircompressor 2.0.2
io.delta delta-sharing-client_2.13 1.3.3
io.dropwizard.metrics metrics-annotation 4.2.30
io.dropwizard.metrics metrics-core 4.2.30
io.dropwizard.metrics metrics-graphite 4.2.30
io.dropwizard.metrics metrics-healthchecks 4.2.30
io.dropwizard.metrics metrics-jetty9 4.2.30
io.dropwizard.metrics metrics-jmx 4.2.30
io.dropwizard.metrics metrics-json 4.2.30
io.dropwizard.metrics metrics-jvm 4.2.30
io.dropwizard.metrics metrics-servlets 4.2.30
io.github.java-diff-utils java-diff-utils 4.15
io.netty netty-all 4.1.118.Final
io.netty netty-buffer 4.1.118.Final
io.netty netty-codec 4.1.118.Final
io.netty netty-codec-http 4.1.118.Final
io.netty netty-codec-http2 4.1.118.Final
io.netty netty-codec-socks 4.1.118.Final
io.netty netty-common 4.1.118.Final
io.netty netty-handler 4.1.118.Final
io.netty netty-handler-proxy 4.1.118.Final
io.netty netty-resolver 4.1.118.Final
io.netty netty-tcnative-boringssl-static 2.0.70.Final-db-r0-linux-aarch_64
io.netty netty-tcnative-boringssl-static 2.0.70.Final-db-r0-linux-x86_64
io.netty netty-tcnative-boringssl-static 2.0.70.Final-db-r0-osx-aarch_64
io.netty netty-tcnative-boringssl-static 2.0.70.Final-db-r0-osx-x86_64
io.netty netty-tcnative-boringssl-static 2.0.70.Final-db-r0-windows-x86_64
io.netty netty-tcnative-classes 2.0.70.Final
io.netty netty-transport 4.1.118.Final
io.netty netty-transport-classes-epoll 4.1.118.Final
io.netty netty-transport-classes-kqueue 4.1.118.Final
io.netty netty-transport-native-epoll 4.1.118.Final
io.netty netty-transport-native-epoll 4.1.118.Final-linux-aarch_64
io.netty netty-transport-native-epoll 4.1.118.Final-linux-riscv64
io.netty netty-transport-native-epoll 4.1.118.Final-linux-x86_64
io.netty netty-transport-native-kqueue 4.1.118.Final-osx-aarch_64
io.netty netty-transport-native-kqueue 4.1.118.Final-osx-x86_64
io.netty netty-transport-native-unix-common 4.1.118.Final
io.prometheus simpleclient 0.16.1-databricks
io.prometheus simpleclient_common 0.16.1-databricks
io.prometheus simpleclient_dropwizard 0.16.1-databricks
io.prometheus simpleclient_pushgateway 0.16.1-databricks
io.prometheus simpleclient_servlet 0.16.1-databricks
io.prometheus simpleclient_servlet_common 0.16.1-databricks
io.prometheus simpleclient_tracer_common 0.16.1-databricks
io.prometheus simpleclient_tracer_otel 0.16.1-databricks
io.prometheus simpleclient_tracer_otel_agent 0.16.1-databricks
io.prometheus.jmx collector 0.18.0
jakarta.annotation jakarta.annotation-api 1.3.5
jakarta.servlet jakarta.servlet-api 4.0.3
jakarta.validation jakarta.validation-api 2.0.2
jakarta.ws.rs jakarta.ws.rs-api 2.1.6
javax.activation activation 1.1.1
javax.annotation javax.annotation-api 1.3.2
javax.el javax.el-api 2.2.4
javax.jdo jdo-api 3.0.1
javax.media jai_core jai_core_dummy
javax.transaction jta 1.1
javax.transaction transaction-api 1.1
javax.xml.bind jaxb-api 2.2.11
javolution javolution 5.5.1
jline jline 2.14.6
joda-time joda-time 2.13.0
net.java.dev.jna jna 5.8.0
net.razorvine pickle 1.5
net.sf.jpam jpam 1.1
net.sf.opencsv opencsv 2.3
net.sf.supercsv super-csv 2.2.0
net.snowflake snowflake-ingest-sdk 0.9.6
net.sourceforge.f2j arpack_combined_all 0.1
org.acplt.remotetea remotetea-oncrpc 1.1.2
org.antlr ST4 4.0.4
org.antlr antlr-runtime 3.5.2
org.antlr antlr4-runtime 4.13.1
org.antlr stringtemplate 3.2.1
org.apache.ant ant 1.10.11
org.apache.ant ant-jsch 1.10.11
org.apache.ant ant-launcher 1.10.11
org.apache.arrow arrow-format 18.2.0
org.apache.arrow arrow-memory-core 18.2.0
org.apache.arrow arrow-memory-netty 18.2.0
org.apache.arrow arrow-memory-netty-buffer-patch 18.2.0
org.apache.arrow arrow-vector 18.2.0
org.apache.avro avro 1.12.0
org.apache.avro avro-ipc 1.12.0
org.apache.avro avro-mapred 1.12.0
org.apache.commons commons-collections4 4.4
org.apache.commons commons-compress 1.27.1
org.apache.commons commons-crypto 1.1.0
org.apache.commons commons-lang3 3.17.0
org.apache.commons commons-math3 3.6.1
org.apache.commons commons-text 1.13.0
org.apache.curator curator-client 5.7.1
org.apache.curator curator-framework 5.7.1
org.apache.curator curator-recipes 5.7.1
org.apache.datasketches datasketches-java 6.1.1
org.apache.datasketches datasketches-memory 3.0.2
org.apache.derby derby 10.14.2.0
org.apache.hadoop hadoop-client-runtime 3.4.1
org.apache.hive hive-beeline 2.3.10
org.apache.hive hive-cli 2.3.10
org.apache.hive hive-jdbc 2.3.10
org.apache.hive hive-llap-client 2.3.10
org.apache.hive hive-llap-common 2.3.10
org.apache.hive hive-serde 2.3.10
org.apache.hive hive-shims 2.3.10
org.apache.hive hive-storage-api 2.8.1
org.apache.hive.shims hive-shims-0.23 2.3.10
org.apache.hive.shims hive-shims-common 2.3.10
org.apache.hive.shims hive-shims-scheduler 2.3.10
org.apache.httpcomponents httpclient 4.5.14
org.apache.httpcomponents httpcore 4.4.16
org.apache.ivy ivy 2.5.3
org.apache.logging.log4j log4j-1.2-api 2.24.3
org.apache.logging.log4j log4j-api 2.24.3
org.apache.logging.log4j log4j-core 2.24.3
org.apache.logging.log4j log4j-layout-template-json 2.24.3
org.apache.logging.log4j log4j-slf4j2-impl 2.24.3
org.apache.orc orc-core 2.1.1-shaded-protobuf
org.apache.orc orc-format 1.1.0-shaded-protobuf
org.apache.orc orc-mapreduce 2.1.1-shaded-protobuf
org.apache.orc orc-shims 2.1.1
org.apache.poi poi 5.4.1
org.apache.poi poi-ooxml 5.4.1
org.apache.poi poi-ooxml-full 5.4.1
org.apache.poi poi-ooxml-lite 5.4.1
org.apache.thrift libfb303 0.9.3
org.apache.thrift libthrift 0.16.0
org.apache.ws.xmlschema xmlschema-core 2.3.1
org.apache.xbean xbean-asm9-shaded 4.26
org.apache.xmlbeans xmlbeans 5.3.0
org.apache.yetus audience-annotations 0.13.0
org.apache.zookeeper zookeeper 3.9.3
org.apache.zookeeper zookeeper-jute 3.9.3
org.checkerframework checker-qual 3.43.0
org.codehaus.janino commons-compiler 3.0.16
org.codehaus.janino janino 3.0.16
org.datanucleus datanucleus-api-jdo 4.2.4
org.datanucleus datanucleus-core 4.1.17
org.datanucleus datanucleus-rdbms 4.1.19
org.datanucleus javax.jdo 3.2.0-m3
org.eclipse.jetty jetty-client 9.4.53.v20231009
org.eclipse.jetty jetty-continuation 9.4.53.v20231009
org.eclipse.jetty jetty-http 9.4.53.v20231009
org.eclipse.jetty jetty-io 9.4.53.v20231009
org.eclipse.jetty jetty-jndi 9.4.53.v20231009
org.eclipse.jetty jetty-plus 9.4.53.v20231009
org.eclipse.jetty jetty-proxy 9.4.53.v20231009
org.eclipse.jetty jetty-security 9.4.53.v20231009
org.eclipse.jetty jetty-server 9.4.53.v20231009
org.eclipse.jetty jetty-servlet 9.4.53.v20231009
org.eclipse.jetty jetty-servlets 9.4.53.v20231009
org.eclipse.jetty jetty-util 9.4.53.v20231009
org.eclipse.jetty jetty-util-ajax 9.4.53.v20231009
org.eclipse.jetty jetty-webapp 9.4.53.v20231009
org.eclipse.jetty jetty-xml 9.4.53.v20231009
org.eclipse.jetty.websocket websocket-api 9.4.53.v20231009
org.eclipse.jetty.websocket websocket-client 9.4.53.v20231009
org.eclipse.jetty.websocket websocket-common 9.4.53.v20231009
org.eclipse.jetty.websocket websocket-server 9.4.53.v20231009
org.eclipse.jetty.websocket websocket-servlet 9.4.53.v20231009
org.fusesource.leveldbjni leveldbjni-all 1.8
org.glassfish.hk2 hk2-api 2.6.1
org.glassfish.hk2 hk2-locator 2.6.1
org.glassfish.hk2 hk2-utils 2.6.1
org.glassfish.hk2 osgi-resource-locator 1.0.3
org.glassfish.hk2.external aopalliance-repackaged 2.6.1
org.glassfish.hk2.external jakarta.inject 2.6.1
org.glassfish.jersey.containers jersey-container-servlet 2.41
org.glassfish.jersey.containers jersey-container-servlet-core 2.41
org.glassfish.jersey.core jersey-client 2.41
org.glassfish.jersey.core jersey-common 2.41
org.glassfish.jersey.core jersey-server 2.41
org.glassfish.jersey.inject jersey-hk2 2.41
org.hibernate.validator hibernate-validator 6.2.5.Final
org.ini4j ini4j 0.5.4
org.javassist javassist 3.29.2-GA
org.jboss.logging jboss-logging 3.4.1.Final
org.jdbi jdbi 2.63.1
org.jetbrains annotations 17.0.0
org.jline jline 3.27.1-jdk8
org.joda joda-convert 1.7
org.jodd jodd-core 3.5.2
org.json4s json4s-ast_2.13 4.0.7
org.json4s json4s-core_2.13 4.0.7
org.json4s json4s-jackson-core_2.13 4.0.7
org.json4s json4s-jackson_2.13 4.0.7
org.json4s json4s-scalap_2.13 4.0.7
org.lz4 lz4-java 1.8.0-databricks-1
org.mlflow mlflow-spark_2.13 2.9.1
org.objenesis objenesis 3.3
org.postgresql postgresql 42.6.1
org.roaringbitmap RoaringBitmap 1.2.1
org.rocksdb rocksdbjni 9.8.4
org.rosuda.REngine REngine 2.1.0
org.scala-lang scala-compiler_2.13 2.13.16
org.scala-lang scala-library_2.13 2.13.16
org.scala-lang scala-reflect_2.13 2.13.16
org.scala-lang.modules scala-collection-compat_2.13 2.11.0
org.scala-lang.modules scala-java8-compat_2.13 0.9.1
org.scala-lang.modules scala-parallel-collections_2.13 1.2.0
org.scala-lang.modules scala-parser-combinators_2.13 2.4.0
org.scala-lang.modules scala-xml_2.13 2.3.0
org.scala-sbt test-interface 1.0
org.scalacheck scalacheck_2.13 1.18.0
org.scalactic scalactic_2.13 3.2.19
org.scalameta common_2.13 4.13.5
org.scalameta io_2.13 4.13.5
org.scalameta mdoc-parser_2.13 2.6.4
org.scalameta metaconfig-core_2.13 0.15.0
org.scalameta metaconfig-pprint_2.13 0.15.0
org.scalameta metaconfig-typesafe-config_2.13 0.15.0
org.scalameta parsers_2.13 4.13.5
org.scalameta scalafmt-config_2.13 3.9.6
org.scalameta scalafmt-core_2.13 3.9.6
org.scalameta scalafmt-macros_2.13 3.9.6
org.scalameta scalafmt-sysops_2.13 3.9.6
org.scalameta scalameta_2.13 4.13.5
org.scalameta trees_2.13 4.13.5
org.scalanlp breeze-macros_2.13 2.1.0
org.scalanlp breeze_2.13 2.1.0
org.scalatest scalatest-compatible 3.2.19
org.scalatest scalatest-core_2.13 3.2.19
org.scalatest scalatest-diagrams_2.13 3.2.19
org.scalatest scalatest-featurespec_2.13 3.2.19
org.scalatest scalatest-flatspec_2.13 3.2.19
org.scalatest scalatest-freespec_2.13 3.2.19
org.scalatest scalatest-funspec_2.13 3.2.19
org.scalatest scalatest-funsuite_2.13 3.2.19
org.scalatest scalatest-matchers-core_2.13 3.2.19
org.scalatest scalatest-mustmatchers_2.13 3.2.19
org.scalatest scalatest-propspec_2.13 3.2.19
org.scalatest scalatest-refspec_2.13 3.2.19
org.scalatest scalatest-shouldmatchers_2.13 3.2.19
org.scalatest scalatest-wordspec_2.13 3.2.19
org.scalatest scalatest_2.13 3.2.19
org.slf4j jcl-over-slf4j 2.0.16
org.slf4j jul-to-slf4j 2.0.16
org.slf4j slf4j-api 2.0.16
org.slf4j slf4j-simple 1.7.25
org.threeten threeten-extra 1.8.0
org.tukaani xz 1.10
org.typelevel algebra_2.13 2.8.0
org.typelevel cats-kernel_2.13 2.8.0
org.typelevel paiges-core_2.13 0.4.4
org.typelevel spire-macros_2.13 0.18.0
org.typelevel spire-platform_2.13 0.18.0
org.typelevel spire-util_2.13 0.18.0
org.typelevel spire_2.13 0.18.0
org.wildfly.openssl wildfly-openssl 1.1.3.Final
org.xerial sqlite-jdbc 3.42.0.0
org.xerial.snappy snappy-java 1.1.10.3
org.yaml snakeyaml 2.0
oro oro 2.0.8
pl.edu.icm JLargeArrays 1.5
software.amazon.cryptools AmazonCorrettoCryptoProvider 2.5.0-linux-x86_64
stax stax-api 1.0.1

Tip

To see release notes for Databricks Runtime versions that have reached end-of-support (EoS), see End-of-support Databricks Runtime release notes. The EoS Databricks Runtime versions have been retired and might not be updated.