writer.h

class ChunksWriter

Chunks writing strategy that decides when and how exactly write chunks to a file. See RFC 0018 for Standard and Streaming Layout description.

Subclassed by ouster::osf::StreamingLayoutCW

Public Functions

virtual void save_message(const uint32_t stream_id, const ts_t ts, const std::vector<uint8_t> &buf) = 0

Save a message to a specified stream.

Parameters:

  • stream_id – [in] The stream id to associate with the message.

  • ts – [in] The timestamp for the messages.

  • buf – [in] A vector of message buffers to record.

virtual void finish() = 0

Finish the process of saving messages and write out the stream stats.

virtual uint32_t chunk_size() const = 0

Get the chunksize

Returns:

the chunk size

virtual ~ChunksWriter() = default

Default deconstructor.

class Writer

OSF Writer provides the base universal interface to store the collection of metadata entries, streams and corresponding objects.

Public Functions

Writer(const std::string &file_name, uint32_t chunk_size = 0)
Throws:

std::runtime_error – Exception on file writing issues.

Parameters:

  • file_name – [in] The filename of the output OSF file.

  • chunk_size – [in] The chunk size in bytes to use for the OSF file. This argument is optional, and if not provided the default value of 2MB is used. If the current chunk being written exceeds the chunk_size, a new chunk will be started on the next call to save. This allows an application to tune the number of messages (e.g. lidar scans) per chunk, which affects the granularity of the message index stored in the StreamingInfo in the file metadata. A smaller chunk_size means more messages are indexed and a larger number of index entries. A more granular index allows for more precise seeking at the slight expense of a larger file.

Writer(const std::string &filename, const ouster::sensor::sensor_info &info, const std::vector<std::string> &fields_to_write = std::vector<std::string>(), uint32_t chunk_size = 0)
Parameters:

  • filename – [in] The filename to output to.

  • info – [in] The sensor info to use for a single stream OSF file.

  • chunk_size – [in] The chunksize to use for the OSF file, this parameter is optional.

  • fields_to_write – [in] The fields from scans to actually save into the OSF. If not provided uses the fields from the first saved lidar scan for this sensor. This parameter is optional.

Writer(const std::string &filename, const std::vector<ouster::sensor::sensor_info> &info, const std::vector<std::string> &fields_to_write = std::vector<std::string>(), uint32_t chunk_size = 0)
Parameters:

  • filename – [in] The filename to output to.

  • info – [in] The sensor info vector to use for a multi stream OSF file.

  • chunk_size – [in] The chunksize to use for the OSF file, this parameter is optional.

  • fields_to_write – [in] The fields from scans to actually save into the OSF. If not provided uses the fields from the first saved lidar scan for this sensor. This parameter is optional.

template<typename MetaType, typename ...MetaParams>
inline uint32_t add_metadata(MetaParams&&... params)

Add metadata to the OSF file.

Template Parameters:

  • MetaType – The type of metadata to add.

  • MetaParams – The type of meta parameters to add.

Parameters:

params – [in] The parameters to add.

uint32_t add_metadata(MetadataEntry &&entry)

Adds a MetadataEntry to the OSF file.

Parameters:

entry – [in] The metadata entry to add to the OSF file.

uint32_t add_metadata(MetadataEntry &entry)

Adds a MetadataEntry to the OSF file.

Parameters:

entry – [in] The metadata entry to add to the OSF file.

std::shared_ptr<MetadataEntry> get_metadata(const uint32_t metadata_id) const

Get and return a metadata entry.

Parameters:

metadata_id – [in] The id of the metadata to get and return.

Returns:

The correct MetadataEntry.

template<class MetadataEntryClass>
inline std::shared_ptr<MetadataEntryClass> get_metadata(const uint32_t metadata_id) const

Get and return a metadata entry.

Parameters:

metadata_id – [in] The id of the metadata to get and return.

Template Parameters:

MetadataEntryClass – The type of metadata to get and return.

Returns:

The correct MetadataEntry.

template<typename Stream, typename ...StreamParams>
inline Stream create_stream(StreamParams&&... params)

Creating streams by passing itself as first argument of the ctor and following the all other parameters.

Template Parameters:

  • Stream – The specified stream object type.

  • StreamParams – The specified stream parameter types.

Parameters:

params – [in] The parameters to use when creating a stream.

void save_message(const uint32_t stream_id, const ts_t ts, const std::vector<uint8_t> &buf)

Writer accepts messages in the form of bytes buffers with linked meta_id and timestamp.

Todo:

[pb]: It should be hidden into private/protected, but I don’t see yet how to do it and give an access to every derived Stream objects.

Throws:

std::logic_error – Exception on non existent stream id.

Parameters:

  • stream_id – [in] The stream to save the message to.

  • ts – [in] The timestamp to use for the message.

  • buf – [in] The message to save in the form of a byte vector.

uint32_t add_sensor(const ouster::sensor::sensor_info &info, const std::vector<std::string> &fields_to_write = std::vector<std::string>())

Adds info about a sensor to the OSF and returns the stream index to to write scans to it’s stream.

Parameters:

  • info – [in] The info of the sensor to add to the file.

  • fields_to_write – [in] The fields from scans to actually save into the OSF. If not provided uses the fields from the first saved lidar scan for this sensor. This parameter is optional.

Returns:

The stream index for the newly added sensor.

void save(uint32_t stream_index, const LidarScan &scan)

Save a single scan to the specified stream_index in an OSF file. The concept of the stream_index is related to the sensor_info vector. Consider the following: 

sensor_info info1; // The first sensor in this OSF file
sensor_info info2; // The second sensor in this OSF file
sensor_info info3; // The third sensor in this OSF file

Writer output = Writer(filename, {info1, info2, info3});

LidarScan scan = RANDOM_SCAN_HERE;

// To save the LidarScan of scan to the first sensor, you would do the
// following
output.save(0, scan);

// To save the LidarScan of scan to the second sensor, you would do the
// following
output.save(1, scan);

// To save the LidarScan of scan to the third sensor, you would do the
// following
output.save(2, scan);

Throws:

  • std::logic_error – Will throw exception on writer being closed.

  • std::logic_error – ///< Will throw exception on ///< out of bound stream_index.

Parameters:

  • stream_index – [in] The index of the corrosponding sensor_info to use.

  • scan – [in] The scan to save.

void save(uint32_t stream_index, const LidarScan &scan, const ouster::osf::ts_t timestamp)

Save a single scan to the specified stream_index in an OSF file indexed with the provided timestamp.

Throws:

  • std::logic_error – Will throw exception on writer being closed.

  • std::logic_error – ///< Will throw exception on ///< out of bound stream_index.

Parameters:

  • stream_index – [in] The index of the corrosponding sensor_info to use.

  • scan – [in] The scan to save.

  • timestamp – [in] Timestamp to index this scan with.

void save(const std::vector<LidarScan> &scans)

Save multiple scans in an OSF file. The concept of the stream_index is related to the sensor_info vector. Consider the following: 

sensor_info info1; // The first sensor in this OSF file
sensor_info info2; // The second sensor in this OSF file
sensor_info info3; // The third sensor in this OSF file

Writer output = Writer(filename, {info1, info2, info3});

LidarScan sensor1_scan = RANDOM_SCAN_HERE;
LidarScan sensor2_scan = RANDOM_SCAN_HERE;
LidarScan sensor3_scan = RANDOM_SCAN_HERE;

// To save the scans matched appropriately to their sensors, you would do
// the following
output.save({sensor1_scan, sensor2_scan, sensor3_scan});

Throws:

std::logic_error – Will throw exception on writer being closed

Parameters:

scans – [in] The vector of scans to save.

const MetadataStore &meta_store() const

Returns the metadata store. This is used for getting the entire set of flatbuffer metadata entries.

Returns:

The flatbuffer metadata entries.

const std::string &metadata_id() const

Returns the metadata id label.

Returns:

The metadata id label.

void set_metadata_id(const std::string &id)

Sets the metadata id label.

Parameters:

id – [in] The metadata id label to set.

const std::string &filename() const

Return the filename for the OSF file.

Returns:

The filename for the OSF file.

uint32_t chunk_size() const

Get the chunk size used for the OSF file.

Returns:

The chunk size for the OSF file.

const std::vector<ouster::sensor::sensor_info> &sensor_info() const

Return the sensor info vector. Consider the following: 

sensor_info info1; // The first sensor in this OSF file
sensor_info info2; // The second sensor in this OSF file
sensor_info info3; // The third sensor in this OSF file

Writer output = Writer(filename, {info1, info2, info3});

// The following will be true
output.sensor_info() == std::vector<sensor_info>{info1, info2, info3};

Returns:

The sensor info vector.

const ouster::sensor::sensor_info sensor_info(int stream_index) const

Get the specified sensor info Consider the following: 

sensor_info info1; // The first sensor in this OSF file
sensor_info info2; // The second sensor in this OSF file
sensor_info info3; // The third sensor in this OSF file

Writer output = Writer(filename, {info1, info2, info3});

// The following will be true
output.sensor_info(0) == info1;
output.sensor_info(1) == info2;
output.sensor_info(2) == info3;

Parameters:

stream_index – [in] The sensor info to return.

Returns:

The correct sensor info.

uint32_t sensor_info_count() const

Get the number of sensor_info objects.

Returns:

The sensor_info count.

void close()

Finish file with a proper metadata object, and header.

inline bool is_closed() const

Returns if the writer is closed or not.

Returns:

If the writer is closed or not.

Writer(const Writer&) = delete

Disallow copying and moving.

Writer &operator=(const Writer&) = delete

Disallow copying and moving.

Writer(Writer&&) = delete

Disallow copying and moving.

Writer &operator=(Writer&&) = delete

Disallow copying and moving.

ChunksLayout chunks_layout() const

Get the specific chunks layout of the OSF file.

Returns:

The chunks layout of the OSF file.

class ChunkBuilder

Encapsulate chunk seriualization operations.

Public Functions

void save_message(const uint32_t stream_id, const ts_t ts, const std::vector<uint8_t> &msg_buf)

Save messages to the serialized chunks.

Throws:

std::logic_error – Exception on a size mismatch

Parameters:

  • stream_id – [in] The stream to save the message to.

  • ts – [in] The timestamp to use for the message.

  • msg_buf – [in] The message to save in the form of a byte vector.

void reset()

Completely wipe all data and start the chunk anew.

std::vector<uint8_t> finish()

Finish out the serialization of the chunk and return the raw flatbuffer output.

Returns:

The serialized chunk in a raw flatbuffer byte vector.

uint32_t size() const

Returns the flatbufferbuilder size.

Returns:

The flatbufferbuilder size.

uint32_t messages_count() const

Returns the number of messages saved so far.

Returns:

The number of messages saved so far.

ts_t start_ts() const

The lowest timestamp in the chunk.

ts_t end_ts() const

The highest timestamp in the chunk.