iceberg::arrow Namespace Reference

Arrow-backed IO and data conversion APIs. More...

Classes
struct	S3Properties
	S3 configuration property keys for ArrowS3FileIO. More...

Functions
ICEBERG_BUNDLE_EXPORT std::unique_ptr< FileIO >	MakeMockFileIO ()
ICEBERG_BUNDLE_EXPORT std::unique_ptr< FileIO >	MakeLocalFileIO ()
ICEBERG_BUNDLE_EXPORT Result< std::unique_ptr< FileIO > >	MakeS3FileIO (const std::unordered_map< std::string, std::string > &properties={})
	Create an S3 FileIO backed by Arrow's S3FileSystem.
ICEBERG_BUNDLE_EXPORT Status	InitializeS3FileIODeleteExecutor (Executor *executor)
	Initialize the shared executor for S3 bulk deletes.
ICEBERG_BUNDLE_EXPORT Status	FinalizeS3 ()
	Finalize (clean up) the Arrow S3 subsystem.
ICEBERG_BUNDLE_EXPORT void	RegisterAll ()
	Register Arrow FileIOs and Arrow-backed C Data utilities.
constexpr bool	EqualsIgnoreAsciiCase (std::string_view left, std::string_view right)
	ASCII-only case-insensitive comparison: schemes are ASCII (RFC 3986), and <cctype> is locale-sensitive.
constexpr bool	IsS3Scheme (std::string_view scheme)
	Return whether a URI scheme is S3-compatible; case-insensitive, because scheme routing is.
constexpr bool	IsS3CredentialPrefix (std::string_view prefix)
	Return whether a storage credential prefix belongs to S3.

Variables
constexpr std::array< std::string_view, 4 >	kS3Schemes = {"s3", "s3a", "s3n", "oss"}
	URI schemes served by the Arrow S3 FileIO, lower-case. These schemes share the S3 client implementation but remain distinct when routing credentials.

Detailed Description

Arrow-backed IO and data conversion APIs.

Function Documentation

FinalizeS3()

ICEBERG_BUNDLE_EXPORT Status iceberg::arrow::FinalizeS3

(

)

Finalize (clean up) the Arrow S3 subsystem.

Must be called before process exit if S3 was initialized, otherwise Arrow's static destructors may cause a non-zero exit.

InitializeS3FileIODeleteExecutor()

ICEBERG_BUNDLE_EXPORT Status iceberg::arrow::InitializeS3FileIODeleteExecutor

(

Executor *

executor

)

Initialize the shared executor for S3 bulk deletes.

nullptr runs sequentially. The first delete freezes this default if not initialized. Repeating the same executor is allowed; replacing it is not. The caller owns it and must keep it alive until the last delete completes. Includes registry-created FileIOs. See Executor for blocking requirements.

IsS3CredentialPrefix()

constexpr bool iceberg::arrow::IsS3CredentialPrefix ( std::string_view  prefix )

inlineconstexpr

Return whether a storage credential prefix belongs to S3.

Accepts the bare s3 prefix (exactly: a case variant would be stored as a key no normalized location can match) or a URI prefix such as s3a://bucket. Bare aliases such as oss are deliberately rejected, matching Java S3FileIO's credential filter.

MakeS3FileIO()

ICEBERG_BUNDLE_EXPORT Result< std::unique_ptr< FileIO > > iceberg::arrow::MakeS3FileIO

(

const std::unordered_map< std::string, std::string > &

properties = {}

)

Create an S3 FileIO backed by Arrow's S3FileSystem.

This function initializes the S3 subsystem if not already initialized (thread-safe). The S3 initialization is cached once per process.

Parameters

properties

Configuration properties for S3 access. See S3Properties for available keys (credentials, region, endpoint, timeouts, etc.).

Returns

A FileIO instance for S3 operations, or an error if S3 is not supported.

RegisterAll()

ICEBERG_BUNDLE_EXPORT void iceberg::arrow::RegisterAll

(

)

Register Arrow FileIOs and Arrow-backed C Data utilities.

This operation is idempotent and safe to call multiple times.

Variable Documentation

kS3Schemes

constexpr std::array<std::string_view, 4> iceberg::arrow::kS3Schemes = {"s3", "s3a", "s3n", "oss"}

inlineconstexpr

URI schemes served by the Arrow S3 FileIO, lower-case. These schemes share the S3 client implementation but remain distinct when routing credentials.

Single source of truth: registration, IsS3Scheme and alias normalization all derive from this list, so a new alias only has to be added here.

oss is served because the store is S3-compatible; see the FileIO docs.