oasislmf.pytools.pla.streams

Module Contents

Classes

PlaReader

Abstract class to read event stream

Functions

read_buffer(byte_mv, cursor, valid_buff, event_id, ...)

read the gul loss stream, apply the post loss amplification factor and load it into out_byte_mv buffer

read_and_write_streams(stream_in, stream_out, ...)

Read input stream from gulpy or gulcalc, determine amplification ID from

Attributes

logger

oasislmf.pytools.pla.streams.logger[source]
oasislmf.pytools.pla.streams.read_buffer(byte_mv, cursor, valid_buff, event_id, item_id, items_amps, plafactors, default_factor, out_byte_mv, out_cursor)[source]

read the gul loss stream, apply the post loss amplification factor and load it into out_byte_mv buffer This modified version of the read_buffer template return result when the whole input buffer is read and not when an event is read. therefore it cannot be used to read multiple stream at a time because events would be mixed up.

Args:

byte_mv: input byte array cursor: read cursor valid_buff: valid part of input array event_id: last event id item_id: last item_id items_amps (numpy array): amplification IDs where indexes correspond to item IDs plafactors (dict): event ID and amplification ID pairs mapped to loss factors default_factor (float): post loss reduction/amplification factor to be used if loss factor not found in plafactors out_byte_mv: output byte arrau out_cursor: single value array to store valid part of out_byte_mv

class oasislmf.pytools.pla.streams.PlaReader(items_amps, plafactors, default_factor)[source]

Bases: oasislmf.pytools.common.event_stream.EventReader

Abstract class to read event stream

This class provide a generic interface to read multiple event stream using: - selector : handle back pressure, the program is paused and don’t use resource if nothing is in the stream buffer - memoryview : read a chuck (PIPE_CAPACITY) of data at a time then work on it using a numpy byte view of this buffer

To use those methods need to be implemented: - __init__(self, …) the constructor with all data structure needed to read and store the event stream - read_buffer(self, byte_mv, cursor, valid_buff, event_id, item_id)

simply point to a local numba.jit function name read_buffer (a template is provided bellow) this function should implement the specific logic of where and how to store the event information.

Those to method may be overwritten - item_exit(self):

specific logic to do when an item is finished (only executed once the stream is finished but no 0,0 closure was present)

  • event_read_log(self):

    what kpi to log when a full event is read

usage snippet:
with ExitStack() as stack:

streams_in, (stream_type, stream_agg_type, len_sample) = init_streams_in(files_in, stack) reader = CustomReader(<read relevant attributes>) for event_id in reader.read_streams(streams_in):

<event logic>

read_buffer(byte_mv, cursor, valid_buff, event_id, item_id)[source]
oasislmf.pytools.pla.streams.read_and_write_streams(stream_in, stream_out, items_amps, plafactors, default_factor)[source]

Read input stream from gulpy or gulcalc, determine amplification ID from item ID, determine loss factor from event ID and amplification ID pair, multiply losses by relevant factors, and write to output stream.

Input stream is binary file with layout:

stream type (oasis_int), maximum sidx value (oasis_int), event ID 1 (oasis_int), item ID 1 (oasis_int), sample ID/sidx 1 (oasis_int), loss for sidx 1 (oasis_float), … sample ID/sidx n (oasis_int), loss for sidx n (oasis_float), 0 (oasis_int), 0.0 (4-byte float), event ID 1 (oasis_int), item ID 2 (oasis_int), … event ID M (oasis_int), item ID N (oasis_int), sample ID/sidx 1 (oasis_int), loss for sidx 1 (oasis_float), … sample ID/sidx n (oasis_int), loss for sidx n (oasis_float)

Sample ID/sidx of 0 indicates start of next event ID-item ID pair. Output stream has same format as input stream.

Args:

stream_in (buffer): input stream stream_out (buffer): output stream items_amps (numpy array): amplification IDs where indexes correspond to item IDs plafactors (dict): event ID and amplification ID pairs mapped to loss factors default_factor (float): post loss reduction/amplification factor to be used if loss factor not found in plafactors