iceberg::ManifestMergeManager Class Reference

Merges small manifests into larger ones using greedy bin-packing. More...

#include <manifest_merge_manager.h>

Public Types
using	SnapshotIdSupplier = std::function< int64_t()>

Public Member Functions
	ManifestMergeManager (const ManifestMergeManager &)=delete
ManifestMergeManager &	operator= (const ManifestMergeManager &)=delete
Result< std::vector< ManifestFile > >	MergeManifests (const std::vector< ManifestFile > &existing_manifests, const std::vector< ManifestFile > &new_manifests, const TableMetadata &metadata, const ManifestWriterFactory &writer_factory)
	Merge existing and new manifests according to configured thresholds.
int32_t	ReplacedManifestsCount () const
	Returns the number of manifests replaced by cached merged outputs.
Status	CleanUncommitted (const std::unordered_set< std::string > &committed)
	Delete cached merged manifests whose paths were not committed and roll back replaced-manifest accounting.

Static Public Member Functions
static Result< std::unique_ptr< ManifestMergeManager > >	Make (ManifestContent content, int64_t target_size_bytes, int32_t min_count_to_merge, bool merge_enabled, std::shared_ptr< FileIO > file_io, SnapshotIdSupplier snapshot_id_supplier, std::function< Status(const std::string &)> delete_file={})
	Construct a merge manager with the given configuration.

Detailed Description

Merges small manifests into larger ones using greedy bin-packing.

Manifests are grouped by partition_spec_id before merging; manifests with different spec IDs are never merged together. Within a group, manifests are accumulated into bins until a bin would exceed target_size_bytes, at which point the bin is flushed (written) and a new one started. Manifests already larger than target_size_bytes pass through unchanged.

Note

This class is non-copyable and non-movable.

Member Function Documentation

Make()

Result< std::unique_ptr< ManifestMergeManager > > iceberg::ManifestMergeManager::Make ( ManifestContent  content, int64_t  target_size_bytes, int32_t  min_count_to_merge, bool  merge_enabled, std::shared_ptr< FileIO >  file_io, SnapshotIdSupplier  snapshot_id_supplier, std::function< Status(const std::string &)>  delete_file = {}  )

static

Construct a merge manager with the given configuration.

Parameters

content	Manifest content this manager accepts
target_size_bytes	Target output manifest size in bytes
min_count_to_merge	Minimum number of manifests before any merging occurs
merge_enabled	Whether merging is enabled at all
file_io	File IO used to open manifests for reading
snapshot_id_supplier	Supplies the snapshot id being committed
delete_file	Callback that deletes uncommitted merged manifest files

MergeManifests()

Result< std::vector< ManifestFile > > iceberg::ManifestMergeManager::MergeManifests	(	const std::vector< ManifestFile > &	existing_manifests,
		const std::vector< ManifestFile > &	new_manifests,
		const TableMetadata &	metadata,
		const ManifestWriterFactory &	writer_factory
	)

Merge existing and new manifests according to configured thresholds.

Manifests are grouped by partition_spec_id. Within each group, a greedy bin-packing algorithm combines manifests up to target_size_bytes. The bin that contains the newest manifest is protected by min_count_to_merge: if it has fewer than that many items it is passed through unchanged.

Parameters

existing_manifests	Manifests already in the base snapshot
new_manifests	Newly written manifests to incorporate
metadata	Table metadata (provides specs and schema for readers)
writer_factory	Factory to create new ManifestWriter instances

Returns

The merged manifest list, or an error

---

The documentation for this class was generated from the following files:

• iceberg/manifest/manifest_merge_manager.h
• iceberg/manifest/manifest_merge_manager.cc