oasislmf.execution.load_balancer

Attributes

logger
last_event_padding
number_size
CHECK_STOPPER_DURATION
parser

Exceptions

ProducerStopped

Unspecified run-time error.

Functions

get_next_event_index(read_buffer, last_event_index, ...)	try to get the index of the end of the event
produce(even_queue, event, stopper)
producer(in_stream, pipeline, read_size, stopper)
consumer(out_stream, pipeline, write_size, sentinel, ...)
balance(pipe_in, pipe_out, read_size, write_size, ...)	Load balance events for a list of input fil_path to a list of output fil_path
main()

Module Contents

oasislmf.execution.load_balancer.logger

oasislmf.execution.load_balancer.last_event_padding = b'\x00\x00\x00\x00\x00\x00\x00\x00'

oasislmf.execution.load_balancer.number_size = 4

oasislmf.execution.load_balancer.CHECK_STOPPER_DURATION = 1

oasislmf.execution.load_balancer.parser

exception oasislmf.execution.load_balancer.ProducerStopped

Bases: RuntimeError

Unspecified run-time error.

oasislmf.execution.load_balancer.get_next_event_index(read_buffer, last_event_index, last_event_id, max_cursor)

try to get the index of the end of the event if found return the index and 0 to indicate it is found if not found return the index of the last item parsed and the last event id

Parameters:

• sub – byte array to parse

• last_item_index – last index parsed

• last_event_id – last event idea parsed (0 means no event)

Returns:

last index parsed, last event idea parsed (0 means the chunk sub[:last_item_index] is a full event

oasislmf.execution.load_balancer.produce(even_queue, event, stopper)

oasislmf.execution.load_balancer.producer(in_stream, pipeline, read_size, stopper)

oasislmf.execution.load_balancer.consumer(out_stream, pipeline, write_size, sentinel, stopper)

oasislmf.execution.load_balancer.balance(pipe_in, pipe_out, read_size, write_size, queue_size)

Load balance events for a list of input fil_path to a list of output fil_path

Parameters:

• pipe_in – list of fil_path fil_path to take as input

• pipe_out – list of fil_path fil_path to take as input

• read_size – int size of the maximum amount of Byte read from one input at a time

• queue_size – int maximum size ofthe buffer queue

oasislmf.execution.load_balancer.main()