oasislmf.execution.load_balancer

Module Contents

Functions

get_next_event_index(read_buffer, last_event_index, ...)

try to get the index of the end of the event

produce(even_queue, event, stopper)

producer(in_stream, pipeline, read_size, stopper)

consumer(out_stream, pipeline, write_size, sentinel, ...)

balance(pipe_in, pipe_out, read_size, write_size, ...)

Load balance events for a list of input fil_path to a list of output fil_path

main()

Attributes

logger

last_event_padding

number_size

CHECK_STOPPER_DURATION

parser

oasislmf.execution.load_balancer.logger[source]
oasislmf.execution.load_balancer.last_event_padding = b'\x00\x00\x00\x00\x00\x00\x00\x00'[source]
oasislmf.execution.load_balancer.number_size = 4[source]
oasislmf.execution.load_balancer.CHECK_STOPPER_DURATION = 1[source]
oasislmf.execution.load_balancer.parser[source]
exception oasislmf.execution.load_balancer.ProducerStopped[source]

Bases: RuntimeError

Unspecified run-time error.

oasislmf.execution.load_balancer.get_next_event_index(read_buffer, last_event_index, last_event_id, max_cursor)[source]

try to get the index of the end of the event if found return the index and 0 to indicate it is found if not found return the index of the last item parsed and the last event id

Parameters:

  • sub – byte array to parse

  • last_item_index – last index parsed

  • last_event_id – last event idea parsed (0 means no event)

Returns:

last index parsed, last event idea parsed (0 means the chunk sub[:last_item_index] is a full event

oasislmf.execution.load_balancer.produce(even_queue, event, stopper)[source]
oasislmf.execution.load_balancer.producer(in_stream, pipeline, read_size, stopper)[source]
oasislmf.execution.load_balancer.consumer(out_stream, pipeline, write_size, sentinel, stopper)[source]
oasislmf.execution.load_balancer.balance(pipe_in, pipe_out, read_size, write_size, queue_size)[source]

Load balance events for a list of input fil_path to a list of output fil_path

Parameters:

  • pipe_in – list of fil_path fil_path to take as input

  • pipe_out – list of fil_path fil_path to take as input

  • read_size – int size of the maximum amount of Byte read from one input at a time

  • queue_size – int maximum size ofthe buffer queue

oasislmf.execution.load_balancer.main()[source]