iceberg::FileIO Class Reference

Pluggable module for reading, writing, and deleting files. More...

#include <file_io.h>

Inheritance diagram for iceberg::FileIO:

Public Member Functions
virtual Result< std::unique_ptr< InputFile > >	NewInputFile (std::string file_location)
	Create an input file handle for the given location.
virtual Result< std::unique_ptr< InputFile > >	NewInputFile (std::string file_location, size_t length)
	Create an input file handle for the given location with a known length.
virtual Result< std::unique_ptr< OutputFile > >	NewOutputFile (std::string file_location)
	Create an output file handle for the given location.
virtual Result< std::string >	ReadFile (const std::string &file_location, std::optional< size_t > length)
	Read the content of the file at the given location.
virtual Status	WriteFile (const std::string &file_location, std::string_view content)
	Write the given content to the file at the given location.
virtual Status	DeleteFile (const std::string &file_location)
	Delete a file at the given location.
virtual Status	DeleteFiles (const std::vector< std::string > &file_locations)
	Delete files at the given locations.
virtual SupportsStorageCredentials *	AsSupportsStorageCredentials ()
	Return storage-credential support when implemented by this FileIO.

Detailed Description

Pluggable module for reading, writing, and deleting files.

This module handles metadata and data file bytes for table IO.

Note that these functions are not atomic. For example, if a write fails, the file may be partially written. Implementations should be careful to avoid corrupting metadata files.

Member Function Documentation

DeleteFile()

virtual Status iceberg::FileIO::DeleteFile ( const std::string &  file_location )

inlinevirtual

Delete a file at the given location.

Parameters

file_location

The location of the file to delete.

Returns

void if the delete succeeded, an error code if the delete failed.

Reimplemented in iceberg::arrow::ArrowFileSystemFileIO, and iceberg::test::StdFileIO.

DeleteFiles()

Status iceberg::FileIO::DeleteFiles ( const std::vector< std::string > &  file_locations )

virtual

Delete files at the given locations.

Implementations that can delete multiple files efficiently should override this method. The default implementation deletes files sequentially using DeleteFile and returns the first error encountered.

Parameters

file_locations

The locations of the files to delete.

Returns

void if all deletes succeed, or an error code if any delete fails.

Reimplemented in iceberg::arrow::ArrowFileSystemFileIO.

NewInputFile() [1/2]

Result< std::unique_ptr< InputFile > > iceberg::FileIO::NewInputFile ( std::string  file_location )

virtual

Create an input file handle for the given location.

Reimplemented in iceberg::arrow::ArrowFileSystemFileIO, iceberg::MockFileIO, and iceberg::test::StdFileIO.

NewInputFile() [2/2]

Result< std::unique_ptr< InputFile > > iceberg::FileIO::NewInputFile ( std::string  file_location, size_t  length  )

virtual

Create an input file handle for the given location with a known length.

The length is a caller-provided content length hint. Implementations may use it to avoid an extra metadata lookup.

Reimplemented in iceberg::arrow::ArrowFileSystemFileIO, iceberg::MockFileIO, and iceberg::test::StdFileIO.

NewOutputFile()

Result< std::unique_ptr< OutputFile > > iceberg::FileIO::NewOutputFile ( std::string  file_location )

virtual

Create an output file handle for the given location.

Reimplemented in iceberg::arrow::ArrowFileSystemFileIO, iceberg::MockFileIO, and iceberg::test::StdFileIO.

ReadFile()

Result< std::string > iceberg::FileIO::ReadFile ( const std::string &  file_location, std::optional< size_t >  length  )

virtual

Read the content of the file at the given location.

Parameters

file_location	The location of the file to read.
length	The number of bytes to read. Some object storage need to specify the length to read, e.g. S3 GetObject has a Range parameter.

Returns

The content of the file if the read succeeded, an error code if the read failed.

WriteFile()

Status iceberg::FileIO::WriteFile ( const std::string &  file_location, std::string_view  content  )

virtual

Write the given content to the file at the given location.

Parameters

file_location	The location of the file to write.
content	The content to write to the file.

Returns

void if the write succeeded, an error code if the write failed.

---

The documentation for this class was generated from the following files:

• iceberg/file_io.h
• iceberg/file_io.cc