oasislmf.execution.load_balancer

Attributes

logger
last_event_padding
number_size
CHECK_STOPPER_DURATION
parser

Exceptions

ProducerStopped

Unspecified run-time error.

Functions

get_next_event_index(read_buffer, last_event_index, ...)	Try to get the index of the end of the event
produce(even_queue, event, stopper)
producer(in_stream, pipeline, read_size, stopper)
consumer(out_stream, pipeline, write_size, sentinel, ...)
balance(pipe_in, pipe_out, read_size, write_size, ...)	Load balance events for a list of input fil_path to a list of output fil_path
main()

Module Contents

oasislmf.execution.load_balancer.logger

oasislmf.execution.load_balancer.last_event_padding = b'\x00\x00\x00\x00\x00\x00\x00\x00'

oasislmf.execution.load_balancer.number_size = 4

oasislmf.execution.load_balancer.CHECK_STOPPER_DURATION = 1

oasislmf.execution.load_balancer.parser

exception oasislmf.execution.load_balancer.ProducerStopped

Bases: RuntimeError

Unspecified run-time error.

oasislmf.execution.load_balancer.get_next_event_index(read_buffer, last_event_index, last_event_id, max_cursor)

Try to get the index of the end of the event if found return the index and 0 to indicate it is found if not found return the index of the last item parsed and the last event id

Parameters:

• read_buffer – int32 view of the input stream to parse

• last_event_index – index (in int32 words) of the end of the last complete item parsed, where parsing resumes

• last_event_id – id of the event currently being parsed (0 means none seen yet)

• max_cursor – number of valid int32 words in read_buffer, parsing stops before it

Returns:

last_item_index] is a full event

Return type:

last index parsed, last event idea parsed (0 means the chunk sub[

oasislmf.execution.load_balancer.produce(even_queue, event, stopper)

oasislmf.execution.load_balancer.producer(in_stream, pipeline, read_size, stopper)

oasislmf.execution.load_balancer.consumer(out_stream, pipeline, write_size, sentinel, stopper)

oasislmf.execution.load_balancer.balance(pipe_in, pipe_out, read_size, write_size, queue_size)

Load balance events for a list of input fil_path to a list of output fil_path

Parameters:

• pipe_in (list) – fil_path to take as input

• pipe_out (list) – fil_path to write the balanced events to

• read_size (int) – size of the maximum amount of Byte read from one input at a time

• write_size (int) – size of the maximum amount of Byte written to one output at a time

• queue_size (int) – maximum size ofthe buffer queue

oasislmf.execution.load_balancer.main()