oasislmf.pytools.getmodel.vulnerability

This file houses a series of functions that load, convert, and save vulnerability data with Parquet files.

Attributes

logger
vulnerability_filename
vulnerability_dataset
parquetvulnerability_meta_filename
vulnerability_parquet_filename
vulnerability_bloc_size
parser

Functions

get_vuln_info(→ Tuple[int, int, int])	Extracts meta data from the vulnerability data.
get_array(→ Tuple[List[int], List[List[List[float]]]])	Flattens the vulnerability data into a one-dimensional array for each vulnerability ID.
iter_table(vulns_bin, num_intensity_bins, ...)	Loops through the vulnerability data, converting it into one-dimensional arrays, in-turn converting this into
vulnerability_to_parquet(→ None)	Converts the vulnerability data to parquet file and saves it.
main()

Module Contents

oasislmf.pytools.getmodel.vulnerability.logger

oasislmf.pytools.getmodel.vulnerability.vulnerability_filename = 'vulnerability.bin'

oasislmf.pytools.getmodel.vulnerability.vulnerability_dataset = 'vulnerability_dataset.parquet'

oasislmf.pytools.getmodel.vulnerability.parquetvulnerability_meta_filename = 'vulnerability_parquet_meta.json'

oasislmf.pytools.getmodel.vulnerability.vulnerability_parquet_filename = 'part_{}.parquet'

oasislmf.pytools.getmodel.vulnerability.vulnerability_bloc_size

oasislmf.pytools.getmodel.vulnerability.get_vuln_info(vulns_bin) → Tuple[int, int, int]

Extracts meta data from the vulnerability data.

Parameters:

vulns_bin – (List[vulnerability_dtype]) vulnerability data from the file

Returns: (Tuple[int, int, int]) number of vulnerability IDs, number of intensity bins, number of damage bins

oasislmf.pytools.getmodel.vulnerability.get_array(vulns_bin, num_intensity_bins, num_damage_bins, max_vulnerability_id_size) → Tuple[List[int], List[List[List[float]]]]

Flattens the vulnerability data into a one-dimensional array for each vulnerability ID.

NOTE => Numba: cannot cache generator for the moment to work properlly, data on the same vulnerability_id must

all be in one block

Parameters:

• vulns_bin – (List[vulnerability_dtype]) vulnerability data from file

• num_intensity_bins – (int) the number of intensity bins in the data

• num_damage_bins – (int) the number of damage bins in the data

• max_vulnerability_id_size – (int) the size of the vulnerability_ids array inside the function, this will be the size of the vulnerability IDs array that is returned

Yields:

Tuple[List[int], List[List[List[float]]]] –

array of vulnerability IDs, vulnerability data.

One batch is yielded each time max_vulnerability_id_size IDs have been filled, and a final partial batch at the end.

oasislmf.pytools.getmodel.vulnerability.iter_table(vulns_bin, num_intensity_bins, num_damage_bins, info, max_vulnerability_id_size)

Loops through the vulnerability data, converting it into one-dimensional arrays, in-turn converting this into PyArrow arrays and then converting into Tables.

Parameters:

• vulns_bin – (List[vulnerability_dtype]) vulnerability data from file

• num_intensity_bins – (int) the number of intensity bins in the data

• num_damage_bins – (int) the number of damage bins in the data

• info – (dict) meta data around the vulnerability data. It has to store: num_vulnerability_id => number of vulnerability IDs num_intensity_bins => number of intensity bins num_damage_bins => number of damage bins

• max_vulnerability_id_size – (int) the size of the vulnerability_ids array

Yields:

pyarrow.Table –

one table per batch of vulnerability IDs, with the vulnerability_id and

vuln_array columns and info attached as metadata

oasislmf.pytools.getmodel.vulnerability.vulnerability_to_parquet(run_dir) → None

Converts the vulnerability data to parquet file and saves it.

Parameters:

run_dir – (str) the directory of the data where this process is to take place

Returns: None

oasislmf.pytools.getmodel.vulnerability.parser

oasislmf.pytools.getmodel.vulnerability.main()