oasislmf.pytools.converters.bintocsv.utils.fm ============================================= .. py:module:: oasislmf.pytools.converters.bintocsv.utils.fm Attributes ---------- .. autoapisummary:: oasislmf.pytools.converters.bintocsv.utils.fm.logger Classes ------- .. autoapisummary:: oasislmf.pytools.converters.bintocsv.utils.fm.FMReader Functions --------- .. autoapisummary:: oasislmf.pytools.converters.bintocsv.utils.fm.read_buffer oasislmf.pytools.converters.bintocsv.utils.fm.fm_tocsv Module Contents --------------- .. py:data:: logger .. py:class:: FMReader(len_sample, data_dtype) Bases: :py:obj:`oasislmf.pytools.common.event_stream.EventReader` Abstract class to read event stream This class provide a generic interface to read multiple event stream using: - selector : handle back pressure, the program is paused and don't use resource if nothing is in the stream buffer - memoryview : read a chuck (PIPE_CAPACITY) of data at a time then work on it using a numpy byte view of this buffer To use those methods need to be implemented: - __init__(self, ...) the constructor with all data structure needed to read and store the event stream - read_buffer(self, byte_mv, cursor, valid_buff, event_id, item_id) simply point to a local numba.jit function name read_buffer (a template is provided bellow) this function should implement the specific logic of where and how to store the event information. Those to method may be overwritten - item_exit(self): specific logic to do when an item is finished (only executed once the stream is finished but no 0,0 closure was present) - event_read_log(self): what kpi to log when a full event is read usage snippet: with ExitStack() as stack: streams_in, (stream_type, stream_agg_type, len_sample) = init_streams_in(files_in, stack) reader = CustomReader() for event_id in reader.read_streams(streams_in): .. py:attribute:: logger .. py:attribute:: data .. py:attribute:: idx .. py:attribute:: state .. py:method:: read_buffer(byte_mv, cursor, valid_buff, event_id, item_id, **kwargs) .. py:function:: read_buffer(byte_mv, cursor, valid_buff, event_id, item_id, data, idxs, state) .. py:function:: fm_tocsv(stack, file_in, file_out, file_type, noheader)