oasislmf.pytools.common.input_files

Attributes

logger
AMPLIFICATIONS_FILE
CORRELATIONS_FILENAME
COVERAGES_FILE
EVENTRATES_FILE
FMPOLICYTC_FILE
FMPROGRAMME_FILE
FMPROFILE_FILE
FMSUMMARYXREF_FILE
FMXREF_FILE
GULSUMMARYXREF_FILE
ITEMS_FILE
OCCURRENCE_FILE
PERIODS_FILE
QUANTILE_FILE
RETURNPERIODS_FILE

Classes

OccurrenceCSR

Functions

read_amplifications([run_dir, filename, use_stdin, raw])	Get array of amplification IDs from amplifications.bin, where index
read_correlations(run_dir[, ignore_file_type, filename])	Load the correlations from the correlations file.
read_coverages([run_dir, ignore_file_type, filename, ...])	Load the coverages from the coverages file.
read_event_rates(run_dir[, filename])	Reads event rates from a CSV file
read_quantile(sample_size, run_dir[, filename, ...])	Generate a quantile interval Dictionary based on sample size and quantile binary file
read_occurrence_bin([run_dir, filename, use_stdin])	Read the occurrence binary file and returns the occurrence records with their header options
read_occurrence(run_dir[, filename])	Read the occurrence binary file and return an id_index-backed CSR map.
occ_get(occ_csr, event_id)	Return the occ_flat slice for event_id, or an empty slice on miss.
occ_get_date(occ_date_id, granular_date)	Returns date as year, month, day, hour, minute from occ_date_id
occ_get_date_id(granular_date, occ_year, occ_month, ...)	Returns the occ_date_id from year, month, day, hour, minute and whether it is a granular date
read_periods(no_of_periods, run_dir[, filename])	Returns an array of period weights for each period between 1 and no_of_periods inclusive (with no gaps).
read_returnperiods(use_return_period_file, run_dir[, ...])	Returns an array of return periods decreasing order with no duplicates.

Module Contents

oasislmf.pytools.common.input_files.logger

class oasislmf.pytools.common.input_files.OccurrenceCSR

Bases: tuple

event_id_index

occ_offsets

occ_flat

oasislmf.pytools.common.input_files.AMPLIFICATIONS_FILE = 'amplifications.bin'

oasislmf.pytools.common.input_files.CORRELATIONS_FILENAME = 'correlations.bin'

oasislmf.pytools.common.input_files.COVERAGES_FILE = 'coverages.bin'

oasislmf.pytools.common.input_files.EVENTRATES_FILE = 'event_rates.csv'

oasislmf.pytools.common.input_files.FMPOLICYTC_FILE = 'fm_policytc.bin'

oasislmf.pytools.common.input_files.FMPROGRAMME_FILE = 'fm_programme.bin'

oasislmf.pytools.common.input_files.FMPROFILE_FILE = 'fm_profile.bin'

oasislmf.pytools.common.input_files.FMSUMMARYXREF_FILE = 'fmsummaryxref.bin'

oasislmf.pytools.common.input_files.FMXREF_FILE = 'fmxref.bin'

oasislmf.pytools.common.input_files.GULSUMMARYXREF_FILE = 'gulsummaryxref.bin'

oasislmf.pytools.common.input_files.ITEMS_FILE = 'items.bin'

oasislmf.pytools.common.input_files.OCCURRENCE_FILE = 'occurrence.bin'

oasislmf.pytools.common.input_files.PERIODS_FILE = 'periods.bin'

oasislmf.pytools.common.input_files.QUANTILE_FILE = 'quantile.bin'

oasislmf.pytools.common.input_files.RETURNPERIODS_FILE = 'returnperiods.bin'

oasislmf.pytools.common.input_files.read_amplifications(run_dir='', filename=AMPLIFICATIONS_FILE, use_stdin=False, raw=False)

Get array of amplification IDs from amplifications.bin, where index corresponds to item ID.

amplifications.bin is binary file with layout:

reserved header (4-byte int), item ID 1 (4-byte int), amplification ID a_1 (4-byte int), … item ID n (4-byte int), amplification ID a_n (4-byte int)

Parameters:

• run_dir (str) – path to amplifications.bin file

• filename (str | os.PathLike) – amplifications file name

• use_stdin (bool) – Use standard input for file data, ignores run_dir/filename. Defaults to False.

• raw (bool) – If True, return the validated flat int32 array (zero-copy memmap for file inputs) instead of building the 1-based lookup copy. Intended for sequential read paths (e.g. bintocsv) where random access by item_id is not needed.

Returns:

If raw=False (default), a 1-based lookup array of

amplification IDs where index corresponds to item ID. If raw=True, the flat validated int32 array of interleaved (item_id, amplification_id) pairs.

Return type:

items_amps (numpy.ndarray)

oasislmf.pytools.common.input_files.read_correlations(run_dir, ignore_file_type=set(), filename=CORRELATIONS_FILENAME)

Load the correlations from the correlations file.

Parameters:

• run_dir (str) – path to correlations file

• ignore_file_type (Set[str]) – file extension to ignore when loading.

• filename (str | os.PathLike) – correlations file name

Returns:

one row per item, holding item_id,

peril_correlation_group, damage_correlation_value, hazard_group_id and hazard_correlation_value. A memmap when read from the binary file.

Return type:

numpy.array[correlations_dtype]

Raises:

FileNotFoundError – if no correlations file is found with a non-ignored extension

oasislmf.pytools.common.input_files.read_coverages(run_dir='', ignore_file_type=set(), filename=COVERAGES_FILE, use_stdin=False)

Load the coverages from the coverages file.

Parameters:

• run_dir (str) – path to coverages file

• ignore_file_type (Set[str]) – file extension to ignore when loading.

• filename (str | os.PathLike) – coverages file name

• use_stdin (bool) – Use standard input for file data, ignores run_dir/filename. Defaults to False.

Returns:

array with the coverage values for each coverage_id.

Return type:

numpy.array[oasis_float]

oasislmf.pytools.common.input_files.read_event_rates(run_dir, filename=EVENTRATES_FILE)

Reads event rates from a CSV file

Parameters:

• run_dir (str | os.PathLike) – Path to input files dir

• filename (str | os.PathLike) – event rates csv file name

Returns:

unique event ids event_rates (ndarray[oasis_float]): event rates

Return type:

unique_event_ids (ndarray[oasis_int])

oasislmf.pytools.common.input_files.read_quantile(sample_size, run_dir, filename=QUANTILE_FILE, return_empty=False)

Generate a quantile interval Dictionary based on sample size and quantile binary file

Parameters:

• sample_size (int) – Sample size

• run_dir (str | os.PathLike) – Path to input files dir

• filename (str | os.PathLike) – quantile binary file name

• return_empty (bool) – return an empty intervals array regardless of the existence of the quantile binary

Returns:

Numpy array emulating a dictionary for numba

Return type:

intervals (quantile_interval_dtype)

oasislmf.pytools.common.input_files.read_occurrence_bin(run_dir='', filename=OCCURRENCE_FILE, use_stdin=False)

Read the occurrence binary file and returns the occurrence records with their header options

Parameters:

• run_dir (str | os.PathLike) – Path to input files dir

• filename (str | os.PathLike) – occurrence binary file name

• use_stdin (bool) – Use standard input for file data, ignores run_dir/filename. Defaults to False.

Returns:

• occ_arr: the occurrence records, of occurrence_dtype or, when the dates are granular, occurrence_granular_dtype

• date_algorithm: date algorithm flag read from the file header

• granular_date: granular date flag read from the file header

• no_of_periods: number of periods read from the file header

Return type:

Tuple[numpy.array, int, int, int]

Raises:

RuntimeError – if the file is truncated or the date algorithm is unknown

oasislmf.pytools.common.input_files.read_occurrence(run_dir, filename=OCCURRENCE_FILE)

Read the occurrence binary file and return an id_index-backed CSR map.

Returns:

id_index-backed CSR occurrence map date_algorithm (int): date algorithm flag granular_date (int): granular date flag no_of_periods (int): number of periods

Return type:

occ_csr (OccurrenceCSR)

oasislmf.pytools.common.input_files.occ_get(occ_csr, event_id)

Return the occ_flat slice for event_id, or an empty slice on miss.

The caller iterates the result directly; the loop body is skipped if the event is not present in the occurrence file.

oasislmf.pytools.common.input_files.occ_get_date(occ_date_id, granular_date)

Returns date as year, month, day, hour, minute from occ_date_id

Parameters:

• occ_date_id (np.int32 | np.int64) – occurrence file date id (int64 for granular dates)

• granular_date (bool) – boolean for whether granular date should be extracted or not

Returns:

Returns year, month, date, hour, minute

Return type:

(oasis_int, oasis_int, oasis_int, oasis_int, oasis_int)

oasislmf.pytools.common.input_files.occ_get_date_id(granular_date, occ_year, occ_month, occ_day, occ_hour=0, occ_minute=0)

Returns the occ_date_id from year, month, day, hour, minute and whether it is a granular date

Parameters:

• granular_date (bool) – boolean for whether granular date should be extracted or not

• occ_year (int) – Occurrence Year.

• occ_month (int) – Occurrence Month.

• occ_day (int) – Occurrence Day.

• occ_hour (int) – Occurrence Hour. Defaults to 0.

• occ_minute (int) – Occurrence Minute. Defaults to 0.

Returns:

occurrence file date id (int64 for granular dates)

Return type:

occ_date_id (np.int64)

oasislmf.pytools.common.input_files.read_periods(no_of_periods, run_dir, filename=PERIODS_FILE)

Returns an array of period weights for each period between 1 and no_of_periods inclusive (with no gaps).

Parameters:

• no_of_periods (int) – Number of periods

• run_dir (str | os.PathLike) – Path to input files dir

• filename (str | os.PathLike) – periods binary file name

Returns:

Period weights

Return type:

period_weights (ndarray[periods_dtype])

oasislmf.pytools.common.input_files.read_returnperiods(use_return_period_file, run_dir, filename=RETURNPERIODS_FILE)

Returns an array of return periods decreasing order with no duplicates.

Parameters:

• use_return_period_file (bool) – Bool to use Return Period File

• run_dir (str | os.PathLike) – Path to input files dir

• filename (str | os.PathLike) – return periods binary file name

Returns:

Return Periods use_return_period_file (bool): Bool to use Return Period File

Return type:

return_periods (ndarray[np.int32])