oasislmf.pytools.fm.compute_sparse

This module implements the core loss computation for the Oasis Financial Module using sparse array storage. It processes insurance/reinsurance losses through a hierarchical node structure, applying financial terms (deductibles, limits, etc.) at each level.

Architecture Overview

The FM computation uses a bottom-up traversal of a tree structure where: - Leaf nodes (items) contain ground-up losses from the GUL stream - Internal nodes aggregate losses from children and apply financial profiles - The root represents the final insured/reinsured loss output

Storage Model

Data is stored using a CSR (Compressed Sparse Row) inspired format: - sidx_indptr[i:i+1] gives the range of sample indices for node i - sidx_val contains the actual sample index values (sidx) - loss_indptr[i:i+1] gives the range of loss values for loss pointer i - loss_val contains the actual loss values aligned with sidx_val - extras_indptr/extras_val similarly store deductible/overlimit/underlimit values

Computation Flow

For each event: 1. Read losses from input stream into sparse arrays 2. For each level (bottom to top):

1. Aggregate children losses into parent nodes

2. Apply financial profiles (calc rules) to compute insured loss

3. Back-allocate results to base children (for allocation rules 1 & 2)

4. Queue parent nodes for the next level

3. Output final losses to stream

Key Concepts

• profile_len: Number of profiles for a node (may differ from layer_len for step policies)

• layer_len: Number of layers in output

• cross_layer_profile: When True, one profile applies to merged loss across all layers

• base_children: The leaf-level descendants of a node (used for back allocation)

• allocation_rule: 0=no allocation, 1=proportional to input, 2=proportional to output

Attributes

logger

Functions

get_base_children(node, children, nodes_array, ...)	Find all leaf-level (base) descendants of a node using breadth-first traversal.
first_time_layer(profile_count, base_children_count, ...)	Initialize multi-layer loss storage for base children when first encountering layered computation.
first_time_layer_extra(profile_count, ...)	Initialize multi-layer loss AND extras storage for base children.
aggregate_children_extras(node, children_count, ...)	Aggregate losses AND extras from multiple children into a parent node.
aggregate_children(node, children_count, nodes_array, ...)	Aggregate losses from multiple children into a parent node (without extras tracking).
set_parent_next_compute(parent_id, child_id, ...)	Register a parent node for computation at the next level.
load_net_value(computes, compute_idx, nodes_array, ...)	Convert gross losses to net losses for output streaming.
compute_event(compute_info, keep_input_loss, ...)	Compute insured losses for a single event through the entire financial structure.
init_variable(compute_info, max_sidx_val, temp_dir, ...)	Initialize all arrays needed for FM computation.
reset_variable(children, compute_idx, computes)	reset the per event array

Module Contents

oasislmf.pytools.fm.compute_sparse.logger

oasislmf.pytools.fm.compute_sparse.get_base_children(node, children, nodes_array, temp_children_queue)

Find all leaf-level (base) descendants of a node using breadth-first traversal.

Base children are the nodes at the lowest level that have no children themselves. These are needed for back allocation - when we apply financial terms at a higher level, we need to distribute the resulting losses back to the original items.

Algorithm: 1. Start with the direct children of the node 2. For each child, check if it has its own children 3. If yes, add those grandchildren to the queue and continue 4. If no, this is a base child - store it at the front of the array 5. Continue until all descendants are processed

The result is temp_children_queue[0:base_child_i] containing all base children.

Args:

node: The parent node whose base children we want to find children: Array tracking children for each node (children[node[‘children’]] = count,

followed by child node IDs)

nodes_array: Array of all node information temp_children_queue: Working array to store the traversal queue and final results

Returns:

int: Number of base children found

oasislmf.pytools.fm.compute_sparse.first_time_layer(profile_count, base_children_count, temp_children_queue, compute_idx, nodes_array, sidx_indptr, sidx_indexes, loss_indptr, loss_val)

Initialize multi-layer loss storage for base children when first encountering layered computation.

When a node has multiple layers but its children were computed with only one layer, we need to create separate loss arrays for each layer. This copies the layer 0 loss to layers 1..N-1 so that back allocation can work independently on each layer.

This is a lazy initialization - we only create the extra layer storage when we actually need it, which saves memory for nodes that never reach multi-layer parents.

Args:

profile_count: Number of profiles/layers to create base_children_count: Number of base children to process temp_children_queue: Array containing base children node IDs compute_idx: Computation state tracking pointers nodes_array: Array of node information sidx_indptr: Sample index pointers sidx_indexes: Node to sample index mapping loss_indptr: Loss value pointers loss_val: Loss values array

oasislmf.pytools.fm.compute_sparse.first_time_layer_extra(profile_count, base_children_count, temp_children_queue, compute_idx, nodes_array, sidx_indptr, sidx_indexes, loss_indptr, loss_val, extras_indptr, extras_val)

Initialize multi-layer loss AND extras storage for base children.

Same as first_time_layer but also handles the extras array (deductible, overlimit, underlimit). For aggregation cases (single base child), extras are copied from layer 0. For back allocation cases (multiple base children), extras are zeroed for new layers since each layer will compute its own extras through back allocation.

Args:

profile_count: Number of profiles/layers to create base_children_count: Number of base children to process temp_children_queue: Array containing base children node IDs compute_idx: Computation state tracking pointers nodes_array: Array of node information sidx_indptr: Sample index pointers sidx_indexes: Node to sample index mapping loss_indptr: Loss value pointers loss_val: Loss values array extras_indptr: Extras value pointers extras_val: Extras values array (deductible, overlimit, underlimit)

oasislmf.pytools.fm.compute_sparse.aggregate_children_extras(node, children_count, nodes_array, children, temp_children_queue, compute_idx, temp_node_sidx, sidx_indexes, sidx_indptr, sidx_val, all_sidx, temp_node_loss, loss_indptr, loss_val, temp_node_extras, extras_indptr, extras_val)

Aggregate losses AND extras from multiple children into a parent node.

Similar to aggregate_children but also tracks the “extras” - deductible amount, overlimit, and underlimit values that are needed for back allocation when financial terms modify losses.

The extras represent: - DEDUCTIBLE: Amount deducted from loss (policy deductible applied) - OVERLIMIT: Amount exceeding the policy limit - UNDERLIMIT: Remaining capacity under the limit (limit - loss)

These extras must be aggregated alongside losses so that when back allocation occurs, we know how to distribute the deductible/limit effects back to individual items proportionally.

Args:

node: Parent node being computed children_count: Number of direct children nodes_array: Array of all node information children: Children tracking array temp_children_queue: Working array for base children lookup compute_idx: Computation state pointers temp_node_sidx: Dense boolean array marking active sidx values sidx_indexes: Maps node_id to sidx array position sidx_indptr: Pointers into sidx_val sidx_val: Sample index values all_sidx: All possible sidx values temp_node_loss: Dense array for accumulating losses loss_indptr: Pointers into loss_val loss_val: Loss values temp_node_extras: Dense array [profile, sidx, 3] for extras accumulation extras_indptr: Pointers into extras_val extras_val: Extras values [deductible, overlimit, underlimit]

Returns:

int: Number of sidx values for this node

oasislmf.pytools.fm.compute_sparse.aggregate_children(node, children_count, nodes_array, children, temp_children_queue, compute_idx, temp_node_sidx, sidx_indexes, sidx_indptr, sidx_val, all_sidx, temp_node_loss, loss_indptr, loss_val)

Aggregate losses from multiple children into a parent node (without extras tracking).

This function sums the losses from all children for each sample index (sidx). It handles the sparse-to-dense-to-sparse conversion needed for aggregation: 1. For each child, read its sparse loss values 2. Accumulate into a dense temporary array (temp_node_loss) indexed by sidx 3. Convert back to sparse storage for the parent node

Multi-layer handling: - If the parent has multiple profiles but children only have one layer,

triggers first_time_layer to create layer storage for children

• Each profile/layer is processed separately

The function also creates the sidx array for the parent node (union of all children’s sidx values) on the first profile iteration.

Args:

node: Parent node being computed children_count: Number of direct children nodes_array: Array of all node information children: Children tracking array (count + child IDs per node) temp_children_queue: Working array for base children lookup compute_idx: Computation state pointers (sidx_i, sidx_ptr_i, loss_ptr_i, etc.) temp_node_sidx: Dense boolean array marking which sidx values have data sidx_indexes: Maps node_id to its sidx array position sidx_indptr: Pointers into sidx_val for each node sidx_val: Sample index values all_sidx: Ordered array of all possible sidx values for iteration temp_node_loss: Dense array [profile, sidx] for accumulating child losses loss_indptr: Pointers into loss_val loss_val: Loss values aligned with sidx_val

Returns:

int: Number of sidx values (node_val_count) for this node

oasislmf.pytools.fm.compute_sparse.set_parent_next_compute(parent_id, child_id, nodes_array, children, computes, compute_idx)

Register a parent node for computation at the next level.

As we process nodes at the current level, we track which parent nodes will need to be computed next. This function: 1. Adds the child to the parent’s children list 2. If this is the first child seen for this parent, adds the parent to the

compute queue for the next level

The children array uses a count-then-values format: - children[parent[‘children’]] = count of children - children[parent[‘children’] + 1..count] = child node IDs

Args:

parent_id: Node ID of the parent to queue child_id: Node ID of the child being linked nodes_array: Array of all node information children: Children tracking array computes: Queue of nodes to compute (current level followed by next level) compute_idx: Contains next_compute_i pointing to end of queue

oasislmf.pytools.fm.compute_sparse.load_net_value(computes, compute_idx, nodes_array, sidx_indptr, sidx_indexes, loss_indptr, loss_val)

Convert gross losses to net losses for output streaming.

Net loss = input loss - insured loss (what remains after insurance pays)

For multi-layer policies, net loss at layer i = net loss at layer i-1 minus the insured loss paid by layer i. This creates a waterfall where each layer pays from what remains after previous layers.

This function iterates through the output nodes and replaces the gross loss values with net loss values in-place, then resets the compute index so the stream writer outputs from the beginning.

Called when net_loss output is requested instead of or in addition to gross loss.

oasislmf.pytools.fm.compute_sparse.compute_event(compute_info, keep_input_loss, nodes_array, node_parents_array, node_profiles_array, len_array, max_sidx_val, sidx_indexes, sidx_indptr, sidx_val, loss_indptr, loss_val, extras_indptr, extras_val, children, computes, compute_idx, item_parent_i, fm_profile, stepped)

Compute insured losses for a single event through the entire financial structure.

This is the main computation function that processes one event’s losses through all levels of the insurance/reinsurance hierarchy. Results are stored in-place in loss_val.

Algorithm Overview

The computation proceeds bottom-up through the financial structure levels:

For each level (starting from items, going up to final output):

For each node to compute at this level:

1. AGGREGATE: Sum losses from children nodes - Multiple children: aggregate into temp arrays, create parent sidx - Single child: reuse child’s storage (optimization) - No children (item level): use input losses directly

2. APPLY PROFILE: Apply financial terms to the aggregated loss - For each profile (may be 1 per layer or 1 cross-layer):

   • Run calc/calc_extra with the profile’s calc rules

   • Handles deductibles, limits, shares, etc.

3. BACK ALLOCATE: Distribute results back to base children - Rule 0: No allocation (output at aggregate level) - Rule 1: Proportional to original input loss - Rule 2: Proportional to computed loss (pro-rata)

4. QUEUE PARENTS: Register parent nodes for next level computation

Cross-Layer Profiles

When cross_layer_profile=True, losses from all layers are first merged, the profile is applied to the total, then results are back-allocated proportionally to each layer.

Args:

compute_info: Computation metadata (levels, allocation rule, etc.) keep_input_loss: If True, preserve input loss for net loss calculation nodes_array: Static node information (agg_id, level_id, pointers) node_parents_array: Parent node IDs for each node node_profiles_array: Profile metadata (i_start, i_end into fm_profile) len_array: Size for dense temporary arrays max_sidx_val: Maximum sample index value sidx_indexes: Node to sidx array position mapping sidx_indptr: CSR-style pointers into sidx_val sidx_val: Sample index values loss_indptr: CSR-style pointers into loss_val loss_val: Loss values (modified in place) extras_indptr: CSR-style pointers into extras_val extras_val: Extras [deductible, overlimit, underlimit] children: Dynamic children tracking per node computes: Queue of nodes to process compute_idx: Computation state (current position, pointers) item_parent_i: Tracks which parent index for multi-parent items fm_profile: Array of financial profile terms stepped: True/None flag for stepped policies (None for JIT compatibility)

oasislmf.pytools.fm.compute_sparse.init_variable(compute_info, max_sidx_val, temp_dir, low_memory)

Initialize all arrays needed for FM computation.

Creates the sparse storage arrays for sample indices, losses, and extras. These use a CSR-like format where: - *_indptr arrays point to the start of each node’s data - *_val arrays contain the actual values

The loss and extras arrays share the same indexing as sidx - each node’s loss[i] corresponds to sidx[i]. This allows using sidx_indexes to track the length of values for any node.

Args:

compute_info: Metadata with array size requirements max_sidx_val: Maximum sample index (determines array sizing) temp_dir: Directory for memory-mapped files (low_memory mode) low_memory: If True, use memory-mapped files instead of RAM

Returns:

Tuple of all initialized arrays needed by compute_event

oasislmf.pytools.fm.compute_sparse.reset_variable(children, compute_idx, computes)

reset the per event array Args:

children: array of all the children with loss value for each node compute_idx: single element named array containing all the pointer needed to tract the computation (compute_idx_dtype) computes: array of node to compute