Coordinates reading manifest files and producing scan tasks. More...

#include <manifest_group.h>

Inheritance diagram for iceberg::ManifestGroup:

Public Types
using	CreateTasksFunction = std::function< Result< std::vector< std::shared_ptr< ScanTask > > >(std::vector< ManifestEntry > &&, const TaskContext &)>

Public Member Functions
	ManifestGroup (ManifestGroup &&) noexcept

ManifestGroup &	operator= (ManifestGroup &&) noexcept

	ManifestGroup (const ManifestGroup &)=delete

ManifestGroup &	operator= (const ManifestGroup &)=delete

ManifestGroup &	FilterData (std::shared_ptr< Expression > filter)
	Set a row-level data filter.

ManifestGroup &	FilterFiles (std::shared_ptr< Expression > filter)
	Set a filter that is evaluated against each DataFile's metadata.

ManifestGroup &	FilterPartitions (std::shared_ptr< Expression > filter)
	Set a partition filter expression.

ManifestGroup &	FilterManifestEntries (std::function< bool(const ManifestEntry &)> predicate)
	Set a custom manifest entry filter predicate.

ManifestGroup &	IgnoreDeleted ()
	Ignore deleted entries in manifests.

ManifestGroup &	IgnoreExisting ()
	Ignore existing entries in manifests.

ManifestGroup &	IgnoreResiduals ()
	Ignore residual filter computation.

ManifestGroup &	Select (std::vector< std::string > columns)
	Select specific columns from manifest entries.

ManifestGroup &	CaseSensitive (bool case_sensitive)
	Set case sensitivity for column name matching.

ManifestGroup &	ColumnsToKeepStats (std::unordered_set< int32_t > column_ids)
	Specify columns that should retain their statistics.

ManifestGroup &	PlanWith (OptionalExecutor executor)
	Configure an optional executor for manifest planning.

ManifestGroup &	WithScanMetrics (std::shared_ptr< ScanMetrics > scan_metrics)
	Attach scan metrics to receive per-manifest and per-file counters.

Result< std::vector< std::shared_ptr< FileScanTask > > >	PlanFiles () &&
	Plan scan tasks for all matching data files.

Result< FileScanTaskStreamPtr >	PlanFilesStream () &&
	Lazily plan scan tasks for matching data files.

Result< std::vector< ManifestEntry > >	Entries ()
	Get all matching manifest entries.

Result< std::vector< std::shared_ptr< ScanTask > > >	Plan (const CreateTasksFunction &create_tasks)
	Plan tasks using a custom task creation function.

Public Member Functions inherited from iceberg::ErrorCollector
	ErrorCollector (ErrorCollector &&)=default

ErrorCollector &	operator= (ErrorCollector &&)=default

	ErrorCollector (const ErrorCollector &)=default

ErrorCollector &	operator= (const ErrorCollector &)=default

template<typename... Args>
auto &	AddError (this auto &self, ErrorKind kind, const std::format_string< Args... > fmt, Args &&... args)
	Add a specific error.

auto &	AddError (this auto &self, Error err)
	Add an existing error object.

auto &	AddError (this auto &self, ::iceberg::unexpected< Error > err)
	Add an unexpected result's error.

auto &	AddError (this auto &self, std::unexpected< Error > err)

bool	has_errors () const
	Check if any errors have been collected.

size_t	error_count () const
	Get the number of errors collected.

Status	CheckErrors () const
	Check for accumulated errors and return them if any exist.

void	ClearErrors ()
	Clear all accumulated errors.

const std::vector< Error > &	errors () const
	Get read-only access to all collected errors.

Static Public Member Functions
static Result< std::unique_ptr< ManifestGroup > >	Make (std::shared_ptr< FileIO > io, std::shared_ptr< Schema > schema, std::unordered_map< int32_t, std::shared_ptr< PartitionSpec > > specs_by_id_, std::vector< ManifestFile > manifests)
	Construct a ManifestGroup with a list of manifests.

static Result< std::unique_ptr< ManifestGroup > >	Make (std::shared_ptr< FileIO > io, std::shared_ptr< Schema > schema, std::unordered_map< int32_t, std::shared_ptr< PartitionSpec > > specs_by_id, std::vector< ManifestFile > data_manifests, std::vector< ManifestFile > delete_manifests)
	Construct a ManifestGroup with pre-separated manifests.

Additional Inherited Members
Protected Attributes inherited from iceberg::ErrorCollector
std::vector< Error >	errors_

Detailed Description

Coordinates reading manifest files and producing scan tasks.

Member Function Documentation

◆ ColumnsToKeepStats()

ManifestGroup & iceberg::ManifestGroup::ColumnsToKeepStats ( std::unordered_set< int32_t > column_ids )

Specify columns that should retain their statistics.

Parameters

column_ids Field IDs of columns whose statistics should be preserved.

◆ FilterManifestEntries()

ManifestGroup & iceberg::ManifestGroup::FilterManifestEntries ( std::function< bool(const ManifestEntry &)> predicate )

Set a custom manifest entry filter predicate.

When an executor is configured with PlanWith(), this predicate may be called concurrently. Callers must synchronize any captured mutable state.

Parameters

predicate A function that returns true if the entry should be included.

◆ Make() [1/2]

static Result< std::unique_ptr< ManifestGroup > > iceberg::ManifestGroup::Make	(	std::shared_ptr< FileIO >	io,
		std::shared_ptr< Schema >	schema,
		std::unordered_map< int32_t, std::shared_ptr< PartitionSpec > >	specs_by_id,
		std::vector< ManifestFile >	data_manifests,
		std::vector< ManifestFile >	delete_manifests
	)

static

Construct a ManifestGroup with pre-separated manifests.

Parameters

io	FileIO for reading manifest files.
schema	Current table schema.
specs_by_id	Mapping of partition spec ID to PartitionSpec.
data_manifests	List of data manifest files.
delete_manifests	List of delete manifest files.

◆ Make() [2/2]

static Result< std::unique_ptr< ManifestGroup > > iceberg::ManifestGroup::Make	(	std::shared_ptr< FileIO >	io,
		std::shared_ptr< Schema >	schema,
		std::unordered_map< int32_t, std::shared_ptr< PartitionSpec > >	specs_by_id_,
		std::vector< ManifestFile >	manifests
	)

static

Construct a ManifestGroup with a list of manifests.

Parameters

io	FileIO for reading manifest files.
schema	Current table schema.
specs_by_id	Mapping of partition spec ID to PartitionSpec.
manifests	List of manifest files to process.

◆ Plan()

Result< std::vector< std::shared_ptr< ScanTask > > > iceberg::ManifestGroup::Plan ( const CreateTasksFunction & create_tasks )

Plan tasks using a custom task creation function.

Parameters

create_tasks A function that creates ScanTasks from entries and context.

Returns: A list of ScanTask objects, or error on failure.

◆ PlanFiles()

Result< std::vector< std::shared_ptr< FileScanTask > > > iceberg::ManifestGroup::PlanFiles ( ) &&

Plan scan tasks for all matching data files.

Consumes this group and collects PlanFilesStream() into a vector.

◆ PlanFilesStream()

Result< FileScanTaskStreamPtr > iceberg::ManifestGroup::PlanFilesStream ( ) &&

Lazily plan scan tasks for matching data files.

The returned stream owns the planning state and may outlive this ManifestGroup. An executor configured through PlanWith() is borrowed and must remain alive until the stream is destroyed, as later Next() calls may submit work to it.

It reads one bounded manifest batch at a time instead of materializing all manifest entries and scan tasks. When PlanWith() configures an executor, entry streams for manifests in each batch are opened in parallel, while entries are consumed one manifest at a time. Delete manifests are still read eagerly when creating the stream because delete files must be indexed before data-file planning can begin. Creating the stream consumes this group's configuration, so this method may only be called on an rvalue.

◆ PlanWith()

ManifestGroup & iceberg::ManifestGroup::PlanWith ( OptionalExecutor executor )

Configure an optional executor for manifest planning.

The executor is borrowed and must remain alive throughout planning and until any stream returned by PlanFilesStream() is destroyed.

Parameters

executor Executor to use, or std::nullopt to plan manifests serially.

Returns: Reference to this for method chaining.

◆ Select()

ManifestGroup & iceberg::ManifestGroup::Select ( std::vector< std::string > columns )

Select specific columns from manifest entries.

Task planning also reads partition values for delete matching and residuals, and may temporarily read statistics needed for equality-delete matching.

An empty list selects no user columns. A list containing * selects all columns. If this method is not called, all columns are selected.

Parameters

columns Column names to select from manifest entries.

The documentation for this class was generated from the following file:

iceberg/manifest/manifest_group.h

Public Types

Public Member Functions

Static Public Member Functions

Additional Inherited Members

Detailed Description

Member Function Documentation

◆ ColumnsToKeepStats()

◆ FilterManifestEntries()

◆ Make() [1/2]

◆ Make() [2/2]

◆ Plan()

◆ PlanFiles()

◆ PlanFilesStream()

◆ PlanWith()

◆ Select()