API for table changes that produce snapshots. More...

#include <snapshot_update.h>

Inheritance diagram for iceberg::SnapshotUpdate:

Classes
struct	ApplyResult
	Result of applying a snapshot update. More...

struct	ContentFileWithSequenceNumber

Public Member Functions
Kind	kind () const override
	Return the kind of this pending update.

bool	IsRetryable () const override
	Whether this update can be retried after a commit conflict.

auto &	DeleteWith (this auto &self, std::function< Status(const std::string &)> delete_func)
	Set a callback to delete files instead of the table's default.

auto &	StageOnly (this auto &self)
	Stage a snapshot in table metadata, but do not make it current.

auto &	ScanManifestsWith (this auto &self, Executor &executor)
	Configure an executor for manifest planning work.

auto &	ToBranch (this auto &self, const std::string &branch)
	Perform operations on a particular branch.

auto &	Set (this auto &self, const std::string &property, const std::string &value)
	Set a summary property in the snapshot produced by this update.

auto &	WriteManifestsWith (this auto &self, Executor &executor, int32_t parallelism)
	Configure an executor and max writer count for writing new manifests.

Result< ApplyResult >	Apply ()
	Apply the update's changes to create a new snapshot.

Status	Finalize (Result< const TableMetadata * > commit_result) override
	Finalize the snapshot update, cleaning up any uncommitted files.

Public Member Functions inherited from iceberg::PendingUpdate
virtual Status	Commit ()
	Apply the pending changes and commit.

	PendingUpdate (const PendingUpdate &)=delete

PendingUpdate &	operator= (const PendingUpdate &)=delete

	PendingUpdate (PendingUpdate &&) noexcept=default

PendingUpdate &	operator= (PendingUpdate &&) noexcept=default

Public Member Functions inherited from iceberg::ErrorCollector
	ErrorCollector (ErrorCollector &&)=default

ErrorCollector &	operator= (ErrorCollector &&)=default

	ErrorCollector (const ErrorCollector &)=default

ErrorCollector &	operator= (const ErrorCollector &)=default

template<typename... Args>
auto &	AddError (this auto &self, ErrorKind kind, const std::format_string< Args... > fmt, Args &&... args)
	Add a specific error and return reference to derived class.

auto &	AddError (this auto &self, Error err)
	Add an existing error object and return reference to derived class.

auto &	AddError (this auto &self, std::unexpected< Error > err)
	Add an unexpected result's error and return reference to derived class.

bool	has_errors () const
	Check if any errors have been collected.

size_t	error_count () const
	Get the number of errors collected.

Status	CheckErrors () const
	Check for accumulated errors and return them if any exist.

void	ClearErrors ()
	Clear all accumulated errors.

const std::vector< Error > &	errors () const
	Get read-only access to all collected errors.

Protected Member Functions
	SnapshotUpdate (std::shared_ptr< TransactionContext > ctx)

Result< std::vector< ManifestFile > >	WriteDataManifests (std::span< const std::shared_ptr< DataFile > > files, const std::shared_ptr< PartitionSpec > &spec, std::optional< int64_t > data_sequence_number=std::nullopt)
	Write data manifests for the given data files.

Result< std::vector< ManifestFile > >	WriteDeleteManifests (std::span< const ContentFileWithSequenceNumber > files, const std::shared_ptr< PartitionSpec > &spec)

const std::string &	target_branch () const

bool	can_inherit_snapshot_id () const

const std::string &	commit_uuid () const

int32_t	manifest_count () const

int32_t	attempt () const

int64_t	target_manifest_size_bytes () const

virtual Status	CleanUncommitted (const std::unordered_set< std::string > &committed)=0
	Clean up any uncommitted manifests that were created.

virtual std::string	operation ()=0
	A string that describes the action that produced the new snapshot.

virtual Status	Validate (const TableMetadata &current_metadata, const std::shared_ptr< Snapshot > &snapshot)
	Validate the current metadata.

virtual Result< std::vector< ManifestFile > >	Apply (const TableMetadata &metadata_to_update, const std::shared_ptr< Snapshot > &snapshot)=0
	Apply the update's changes to the given metadata and snapshot.

virtual std::unordered_map< std::string, std::string >	Summary ()=0
	Get the summary map for this operation.

virtual void	SetSummaryProperty (const std::string &property, const std::string &value)
	Set a summary property.

virtual bool	CleanupAfterCommit () const
	Check if cleanup should happen after commit.

int64_t	SnapshotId ()
	Get or generate the snapshot ID for the new snapshot.

Status	DeleteFile (const std::string &path)
	Delete a file at the given path.

std::string	ManifestPath ()

std::string	ManifestListPath ()

SnapshotSummaryBuilder &	summary_builder ()

SnapshotSummaryBuilder	BuildManifestCountSummary (std::span< const ManifestFile > manifests, int32_t replaced_manifests_count)

Protected Member Functions inherited from iceberg::PendingUpdate
	PendingUpdate (std::shared_ptr< TransactionContext > ctx)

const TableMetadata &	base () const

Protected Attributes
SnapshotSummaryBuilder	summary_

Protected Attributes inherited from iceberg::PendingUpdate
std::shared_ptr< TransactionContext >	ctx_

Protected Attributes inherited from iceberg::ErrorCollector
std::vector< Error >	errors_

Additional Inherited Members
Public Types inherited from iceberg::PendingUpdate
enum class	Kind : uint8_t { kExpireSnapshots , kSetSnapshot , kUpdateLocation , kUpdatePartitionSpec , kUpdatePartitionStatistics , kUpdateProperties , kUpdateSchema , kUpdateSnapshot , kUpdateSnapshotReference , kUpdateSortOrder , kUpdateStatistics }

Detailed Description

API for table changes that produce snapshots.

This class contains common methods for all updates that create a new table Snapshot. It also provides the shared implementation for snapshot ID assignment, manifest list writing, retry-safe apply, and cleanup.

Member Function Documentation

◆ Apply() [1/2]

Result< SnapshotUpdate::ApplyResult > iceberg::SnapshotUpdate::Apply ( )

Apply the update's changes to create a new snapshot.

This method validates the changes, applies them to the current base metadata, and creates a new snapshot without committing it. Commit retries call Apply() again with refreshed metadata so the same changes can be applied to the new latest snapshot.

Returns: A result containing the new snapshot, or an error.

◆ Apply() [2/2]

virtual Result< std::vector< ManifestFile > > iceberg::SnapshotUpdate::Apply	(	const TableMetadata &	metadata_to_update,
		const std::shared_ptr< Snapshot > &	snapshot
	)

protectedpure virtual

Apply the update's changes to the given metadata and snapshot.

Parameters

metadata_to_update	The base table metadata to apply changes to
snapshot	Snapshot to apply the changes to

Returns: A vector of manifest files for the new snapshot

Implemented in iceberg::FastAppend, and iceberg::MergingSnapshotUpdate.

◆ CleanUncommitted()

virtual Status iceberg::SnapshotUpdate::CleanUncommitted ( const std::unordered_set< std::string > & committed )

protectedpure virtual

Clean up any uncommitted manifests that were created.

Manifests may not be committed if Apply is called multiple times because a commit conflict has occurred. Implementations may keep around manifests because the same changes will be made by both Apply calls. This method instructs the implementation to clean up those manifests and passes the paths of the manifests that were actually committed.

Parameters

committed A set of manifest paths that were actually committed

Implemented in iceberg::FastAppend, and iceberg::MergingSnapshotUpdate.

◆ CleanupAfterCommit()

virtual bool iceberg::SnapshotUpdate::CleanupAfterCommit ( ) const

inlineprotectedvirtual

Check if cleanup should happen after commit.

Returns: True if cleanup should happen after commit

Reimplemented in iceberg::FastAppend.

◆ DeleteFile()

Status iceberg::SnapshotUpdate::DeleteFile ( const std::string & path )

protected

Delete a file at the given path.

Parameters

path	The path of the file to delete

Returns: A status indicating the result of the deletion

◆ DeleteWith()

auto & iceberg::SnapshotUpdate::DeleteWith	(	this auto &	self,
		std::function< Status(const std::string &)>	delete_func
	)

inline

Set a callback to delete files instead of the table's default.

Parameters

delete_func A function used to delete file locations.

Returns: This update for method chaining.

Note: Cannot be called more than once.

◆ Finalize()

Status iceberg::SnapshotUpdate::Finalize ( Result< const TableMetadata * > commit_result )

overridevirtual

Finalize the snapshot update, cleaning up any uncommitted files.

Reimplemented from iceberg::PendingUpdate.

◆ IsRetryable()

bool iceberg::SnapshotUpdate::IsRetryable ( ) const

inlineoverridevirtual

Whether this update can be retried after a commit conflict.

Implements iceberg::PendingUpdate.

◆ kind()

Kind iceberg::SnapshotUpdate::kind ( ) const

inlineoverridevirtual

Return the kind of this pending update.

Implements iceberg::PendingUpdate.

◆ operation()

virtual std::string iceberg::SnapshotUpdate::operation ( )

protectedpure virtual

A string that describes the action that produced the new snapshot.

Returns: A string operation name

Implemented in iceberg::TestMergeAppend, iceberg::TestOverwriteUpdate, iceberg::DeleteFiles, iceberg::FastAppend, iceberg::MergeAppend, iceberg::OverwriteFiles, iceberg::RewriteFiles, and iceberg::RowDelta.

◆ ScanManifestsWith()

auto & iceberg::SnapshotUpdate::ScanManifestsWith	(	this auto &	self,
		Executor &	executor
	)

inline

Configure an executor for manifest planning work.

Parameters

executor Executor to use while planning manifests.

Returns: Reference to this for method chaining.

◆ Set()

auto & iceberg::SnapshotUpdate::Set	(	this auto &	self,
		const std::string &	property,
		const std::string &	value
	)

inline

Set a summary property in the snapshot produced by this update.

Parameters

property	A String property name.
value	A String property value.

Returns: This update for method chaining.

◆ SetSummaryProperty()

void iceberg::SnapshotUpdate::SetSummaryProperty	(	const std::string &	property,
		const std::string &	value
	)

protectedvirtual

Set a summary property.

Implementations may override this to retain custom properties across retry-safe summary rebuilds.

Reimplemented in iceberg::FastAppend.

◆ StageOnly()

auto & iceberg::SnapshotUpdate::StageOnly ( this auto & self )

inline

Stage a snapshot in table metadata, but do not make it current.

The snapshot is assigned an ID and added to table metadata. The table's current snapshot ID is not updated.

Returns: This update for method chaining.

◆ Summary()

virtual std::unordered_map< std::string, std::string > iceberg::SnapshotUpdate::Summary ( )

protectedpure virtual

Get the summary map for this operation.

Returns: A map of summary properties

Implemented in iceberg::FastAppend, and iceberg::MergingSnapshotUpdate.

◆ ToBranch()

auto & iceberg::SnapshotUpdate::ToBranch	(	this auto &	self,
		const std::string &	branch
	)

inline

Perform operations on a particular branch.

Parameters

branch The name of a SnapshotRef of type branch.

Returns: This update for method chaining.

◆ Validate()

virtual Status iceberg::SnapshotUpdate::Validate	(	const TableMetadata &	current_metadata,
		const std::shared_ptr< Snapshot > &	snapshot
	)

inlineprotectedvirtual

Validate the current metadata.

Child operations can override this to add custom validation.

Parameters

current_metadata	Current table metadata to validate
snapshot	Ending snapshot on the lineage which is being validated

Reimplemented in iceberg::DeleteFiles, iceberg::OverwriteFiles, iceberg::RewriteFiles, and iceberg::RowDelta.

◆ WriteDataManifests()

Result< std::vector< ManifestFile > > iceberg::SnapshotUpdate::WriteDataManifests	(	std::span< const std::shared_ptr< DataFile > >	files,
		const std::shared_ptr< PartitionSpec > &	spec,
		std::optional< int64_t >	data_sequence_number = `std::nullopt`
	)

protected

Write data manifests for the given data files.

Parameters

files	Data files to write
spec	The partition spec to use
data_sequence_number	Optional data sequence number for the files

Returns: A vector of manifest files

◆ WriteManifestsWith()

auto & iceberg::SnapshotUpdate::WriteManifestsWith	(	this auto &	self,
		Executor &	executor,
		int32_t	parallelism
	)

inline

Configure an executor and max writer count for writing new manifests.

If this method is not called, manifest writes remain serial. When configured, files may be split into independent rolling-writer groups.

Note: Custom FileIO implementations and registered writer factories used for manifest writes must support concurrent calls when an executor is configured.

The documentation for this class was generated from the following files:

iceberg/update/snapshot_update.h
iceberg/update/snapshot_update.cc

Classes

Public Member Functions

Protected Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Member Function Documentation

◆ Apply() [1/2]

◆ Apply() [2/2]

◆ CleanUncommitted()

◆ CleanupAfterCommit()

◆ DeleteFile()

◆ DeleteWith()

◆ Finalize()

◆ IsRetryable()

◆ kind()

◆ operation()

◆ ScanManifestsWith()

◆ Set()

◆ SetSummaryProperty()

◆ StageOnly()

◆ Summary()

◆ ToBranch()

◆ Validate()

◆ WriteDataManifests()

◆ WriteManifestsWith()