Skip to content

cockroachdb/cockroach

master
Switch branches/tags
Code

Latest commit

84875: backupccl: handle range keys in BACKUP r=erikgrinaker a=msbutler

Previously BACKUP would not back up range tombstones. With this patch, BACKUPs
with revision_history will backup range tombstones. Non-revision history backups
are not affected by this diff because MVCCExportToSST filters all tombstones
out of the backup already.

Specifically, this patch replaces the iterators used in the backup_processor
with the pebbleIterator, which has baked in range key support. This refactor
introduces a 5% regression in backup runtime, even when the backup has no range
keys, though #83051 hopes to address this gap. See details below on the
benchmark experiment.

At this point a backup with range keys is restorable, thanks to #84214. Note
that the restore codebase still touches iterators that are not range key aware.
This is not a problem because restored data does not have range keys, nor do
the empty ranges restore dumps data into. These iterators (e.g. in SSTBatcher
and in CheckSSTConflicts) will be updated when #70428 gets fixed.

Fixes #71155

Release note: none

To benchmark this diff, the following commands were used on the following sha
a5ccdc3, with and without this commit, over
three trials:
```
roachprod create -n 5 --gce-machine-type=n2-standard-16 $CLUSTER
roachprod put $CLUSTER [build] cockroach

roachprod wipe $CLUSTER; roachprod start $CLUSTER;
roachprod run $CLUSTER:1 -- "./cockroach workload init bank --rows 1000000000"
roachprod sql $CLUSTER:1 -- -e "BACKUP INTO 'gs://somebucket/michael-rangkey?AUTH=implicit'"
```

The backup on the control binary took on average 478 seconds with a stdev of 13
seconds, while the backup with the treatment binary took on average 499 seconds
with stddev of 8 seconds.

84883: kvserver: add server-side transaction retry metrics r=arulajmani a=arulajmani

This patch adds a few new metrics to track successful/failed
server-side transaction retries. Specifically, whenever we attempt
to retry a read or write batch or run into a read within uncertainty
interval error, we increment specific counters indicating if the
retry was successful or not.

Release note: None

85074: upgrades: add checkpointing for `raftAppliedIndexTermMigration` r=irfansharif a=erikgrinaker

Forward-port of #84909, for posterity.

----

The `raftAppliedIndexTermMigration` upgrade migration could be
unreliable. It iterates over all ranges and runs a `Migrate` request
which must be applied on all replicas. However, if any ranges merge or
replicas are unavailable, the migration fails and starts over from the
beginning. In large clusters with many ranges, this meant that it might
never complete.

This patch makes the upgrade more robust, by retrying each `Migrate`
request 5 times, and checkpointing the progress after every fifth batch
(1000 ranges), allowing resumption on failure. At some point this should
be made part of the migration infrastructure.

NB: This fix was initially submitted for 22.1, and even though the
migration will be removed for 22.2, it is forward-ported for posterity.

Release note: None

85086: eval: stop ignoring all ResolveOIDFromOID errors r=ajwerner a=rafiss

fixes #84448

The decision about whether an error is safe to ignore is made at the
place where the error is created/returned. This way, the callers don't
need to be aware of any new error codes that the implementation may
start returning in the future.

Release note (bug fix): Fixed incorrect error handling that could cause
casts to OID types to fail in some cases.

Co-authored-by: Michael Butler <butler@cockroachlabs.com>
Co-authored-by: Arul Ajmani <arulajmani@gmail.com>
Co-authored-by: Erik Grinaker <grinaker@cockroachlabs.com>
Co-authored-by: Rafi Shamim <rafi@cockroachlabs.com>
811ca1f

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
Jul 26, 2022
dev
Jul 22, 2022


CockroachDB is a cloud-native distributed SQL database designed to build, scale, and manage modern, data-intensive applications.

What is CockroachDB?

CockroachDB is a distributed SQL database built on a transactional and strongly-consistent key-value store. It scales horizontally; survives disk, machine, rack, and even datacenter failures with minimal latency disruption and no manual intervention; supports strongly-consistent ACID transactions; and provides a familiar SQL API for structuring, manipulating, and querying data.

For more details, see our FAQ or architecture document.

Docs

For guidance on installation, development, deployment, and administration, see our User Documentation.

Starting with CockroachCloud

We can run CockroachDB for you, so you don't have to run your own cluster.

See our online documentation: Quickstart with CockroachCloud

Starting with CockroachDB

  1. Install CockroachDB: using a pre-built executable or build it from source.
  2. Start a local cluster and connect to it via the built-in SQL client.
  3. Learn more about CockroachDB SQL.
  4. Use a PostgreSQL-compatible driver or ORM to build an app with CockroachDB.
  5. Explore core features, such as data replication, automatic rebalancing, and fault tolerance and recovery.

Client Drivers

CockroachDB supports the PostgreSQL wire protocol, so you can use any available PostgreSQL client drivers to connect from various languages.

Deployment

  • CockroachCloud - Steps to create a free CockroachCloud cluster on your preferred Cloud platform.
  • Manual - Steps to deploy a CockroachDB cluster manually on multiple machines.
  • Cloud - Guides for deploying CockroachDB on various cloud platforms.
  • Orchestration - Guides for running CockroachDB with popular open-source orchestration systems.

Need Help?

Building from source

See our wiki for more details.

Contributing

We welcome your contributions! If you're looking for issues to work on, try looking at the good first issue list. We do our best to tag issues suitable for new external contributors with that label, so it's a great way to find something you can help with!

See our wiki for more details.

Engineering discussions take place on our public mailing list, cockroach-db@googlegroups.com. Also please join our Community Slack (there's a dedicated #contributors channel!) to ask questions, discuss your ideas, and connect with other contributors.

Design

For an in-depth discussion of the CockroachDB architecture, see our Architecture Guide. For the original design motivation, see our design doc.

Licensing

Current CockroachDB code is released under a combination of two licenses, the Business Source License (BSL) and the Cockroach Community License (CCL).

When contributing to a CockroachDB feature, you can find the relevant license in the comments at the top of each file.

For more information, see the Licensing FAQs.

Comparison with Other Databases

To see how key features of CockroachDB stack up against other databases, check out CockroachDB in Comparison.

See Also