oasislmf.pytools.plt.manager

Attributes

logger
SPLT_output
MPLT_output
QPLT_output
SPLT_headers
MPLT_headers
QPLT_headers
SPLT_dtype
MPLT_dtype
QPLT_dtype
SPLT_fmt
MPLT_fmt
QPLT_fmt

Classes

PLTReader

Abstract class to read event stream

Functions

read_buffer(byte_mv, cursor, valid_buff, event_id, ...)
read_input_files(run_dir, compute_qplt, sample_size)	Reads all input files and returns a dict of relevant data
run(run_dir, files_in[, splt_output_file, ...])	Runs PLT calculations
main([run_dir, files_in, splt, mplt, qplt, noheader])

Module Contents

oasislmf.pytools.plt.manager.logger

oasislmf.pytools.plt.manager.SPLT_output

oasislmf.pytools.plt.manager.MPLT_output

oasislmf.pytools.plt.manager.QPLT_output

oasislmf.pytools.plt.manager.SPLT_headers

oasislmf.pytools.plt.manager.MPLT_headers

oasislmf.pytools.plt.manager.QPLT_headers

oasislmf.pytools.plt.manager.SPLT_dtype

oasislmf.pytools.plt.manager.MPLT_dtype

oasislmf.pytools.plt.manager.QPLT_dtype

oasislmf.pytools.plt.manager.SPLT_fmt = ''

oasislmf.pytools.plt.manager.MPLT_fmt = ''

oasislmf.pytools.plt.manager.QPLT_fmt = ''

class oasislmf.pytools.plt.manager.PLTReader(len_sample, compute_splt, compute_mplt, compute_qplt, occ_map, period_weights, granular_date, intervals)

Bases: oasislmf.pytools.common.event_stream.EventReader

Abstract class to read event stream

This class provide a generic interface to read multiple event stream using: - selector : handle back pressure, the program is paused and don’t use resource if nothing is in the stream buffer - memoryview : read a chuck (PIPE_CAPACITY) of data at a time then work on it using a numpy byte view of this buffer

To use those methods need to be implemented: - __init__(self, …) the constructor with all data structure needed to read and store the event stream - read_buffer(self, byte_mv, cursor, valid_buff, event_id, item_id)

simply point to a local numba.jit function name read_buffer (a template is provided bellow) this function should implement the specific logic of where and how to store the event information.

Those to method may be overwritten - item_exit(self):

specific logic to do when an item is finished (only executed once the stream is finished but no 0,0 closure was present)

• event_read_log(self):

  what kpi to log when a full event is read

usage snippet:

with ExitStack() as stack:

streams_in, (stream_type, stream_agg_type, len_sample) = init_streams_in(files_in, stack) reader = CustomReader(<read relevant attributes>) for event_id in reader.read_streams(streams_in):

<event logic>

logger

splt_data

splt_idx

mplt_data

mplt_idx

qplt_data

qplt_idx

state

occ_map

period_weights

granular_date

intervals

curr_file_idx = None

read_buffer(byte_mv, cursor, valid_buff, event_id, item_id, file_idx)

oasislmf.pytools.plt.manager.read_buffer(byte_mv, cursor, valid_buff, event_id, item_id, state, splt_data, splt_idx, mplt_data, mplt_idx, qplt_data, qplt_idx, occ_map, period_weights, granular_date, intervals)

oasislmf.pytools.plt.manager.read_input_files(run_dir, compute_qplt, sample_size)

Reads all input files and returns a dict of relevant data

Args:

run_dir (str | os.PathLike): Path to directory containing required files structure compute_qplt (bool): Compute QPLT bool sample_size (int): Sample size

Returns:

file_data (Dict[str, Any]): A dict of relevent data extracted from files

oasislmf.pytools.plt.manager.run(run_dir, files_in, splt_output_file=None, mplt_output_file=None, qplt_output_file=None, noheader=False)

Runs PLT calculations

Args:

run_dir (str | os.PathLike): Path to directory containing required files structure files_in (str | os.PathLike): Path to summary binary input file splt_output_file (str, optional): Path to SPLT output file. Defaults to None. mplt_output_file (str, optional): Path to MPLT output file. Defaults to None. qplt_output_file (str, optional): Path to QPLT output file. Defaults to None. noheader (bool): Boolean value to skip header in output file

oasislmf.pytools.plt.manager.main(run_dir='.', files_in=None, splt=None, mplt=None, qplt=None, noheader=False, **kwargs)