roboto.experimental.topics.decode.mcap#

Module Contents#

class roboto.experimental.topics.decode.mcap.McapFileDecoder(group, partition, window, params)#

Bases: roboto.experimental.topics.decode.common.FileDecoder

Decodes one MCAP file of a partition through a cursor of the mcap_codec Rust extension.

The cursor reads the MCAP channel on the group’s topic, or the file’s only channel when the group names no topic. The codec reads the fields the group supplies, shifts each timestamp by the partition’s time_offset_ns, and keeps the rows in the window. The cursor reads the file’s summary when it opens, then only the chunks that hold the topic’s messages (for a log-time timestamp, only those whose log times can fall in the window), all downloaded before the first batch is decoded.

Iterating batches() raises RobotoReadPlanExecutionException with kind invalid-timestamp at the first row whose timestamp the codec cannot read as signed 64-bit nanoseconds once shifted (a value of the wrong type, NaN or infinite, or out of range), and RobotoInternalException when the codec cannot read a message.

Parameters:
batches()#

The window’s rows, in the file’s stored row order; iterate it once.

Each batch has the columns of topic_data_schema() over value_fields: the row number, the timestamp, then the value columns. A row’s number is its 0-based position among the file’s rows of the topic, counting every stored row, including rows outside the window and rows with a null timestamp, so a row has the same number in every file of its partition. The timestamp is absolute: the stored value in nanoseconds plus the partition’s time_offset_ns. Batch boundaries carry no meaning.

Return type:

collections.abc.Iterator[pyarrow.RecordBatch]

close()#

Release the file. Safe to call more than once.

Return type:

None

struct_field_names(path)#

The names of the fields of the struct at path in the file, in the file’s order.

None when the file has no struct at path.

Parameters:

path (roboto.domain.topics.record.FieldPath)

Return type:

Optional[list[str]]

property value_fields: list[pyarrow.Field]#

The value columns, one per top-level field the file supplies, sorted by name, comparing Unicode code points.

Each struct keeps the fields the file supplies, in the order the file stores them.

Return type:

list[pyarrow.Field]

On this page