oasislmf.pytools.getmodel.footprint

This file houses the classes that load the footprint data from compressed, binary, and CSV files.

Attributes

logger
uncompressedMask
intensityMask
CURRENT_DIRECTORY

Exceptions

OasisFootPrintError

Raises exceptions when loading footprints.

Classes

Footprint	This class is the base class for the footprint loaders.
FootprintCsv	This class is responsible for loading footprint data from CSV.
FootprintBin	This class is responsible loading the event data from the footprint binary files.
FootprintBinZ	This class is responsible for loading event data from compressed event data.
FootprintParquet	This class is responsible for loading event data from parquet event data.
FootprintParquetDynamic	This class is responsible for loading event data from parquet dynamic event sets and maps

Functions

stitch_data(areaperil_id, intensity_bin_id, ...)

Creates a list of tuples from three np.arrays (all inputs must be the same length).

Module Contents

oasislmf.pytools.getmodel.footprint.logger

oasislmf.pytools.getmodel.footprint.uncompressedMask = 2

oasislmf.pytools.getmodel.footprint.intensityMask = 1

oasislmf.pytools.getmodel.footprint.CURRENT_DIRECTORY = ''

exception oasislmf.pytools.getmodel.footprint.OasisFootPrintError(message: str)

Bases: Exception

Raises exceptions when loading footprints.

class oasislmf.pytools.getmodel.footprint.Footprint(storage: oasis_data_manager.filestore.backends.base.BaseStorage, df_engine='oasis_data_manager.df_reader.reader.OasisPandasReader')

This class is the base class for the footprint loaders.

Attributes:

storage (BaseStorage): the storage object used to lookup files stack (ExitStack): the context manager that combines other context managers and cleanup functions

storage

stack

df_engine = 'oasis_data_manager.df_reader.reader.OasisPandasReader'

static get_footprint_fmt_priorities()

Get list of footprint file format classes in order of priority.

Returns: (list) footprint file format classes

classmethod load(storage: oasis_data_manager.filestore.backends.base.BaseStorage, ignore_file_type=set(), df_engine='oasis_data_manager.df_reader.reader.OasisPandasReader')

Loads the loading classes defined in this file checking to see if the files are in the static path whilst doing so. The loading goes through the hierarchy with the following order:

-> parquet -> compressed binary file -> binary file -> CSV file -> parquet (with dynamic generation)

If the compressed binary file is present, this will be loaded. If it is not, then the binary file will be loaded and so on.

Args:

storage (BaseStorage): the storage object used to lookup files ignore_file_type (Set[str]): type of file to be skipped in the hierarchy. This can be a choice of:

parquet json z bin idx

Returns: (Union[FootprintBinZ, FootprintBin, FootprintCsv]) the loaded class

abstract get_event(event_id)

get_df_reader(filepath, **kwargs) → oasis_data_manager.df_reader.reader.OasisReader

static prepare_df_data(data_frame: pandas.DataFrame) → numpy.array

Reads footprint data from a parquet file.

Returns: (np.array) footprint data loaded from the parquet file

class oasislmf.pytools.getmodel.footprint.FootprintCsv(storage: oasis_data_manager.filestore.backends.base.BaseStorage, df_engine='oasis_data_manager.df_reader.reader.OasisPandasReader')

Bases: Footprint

This class is responsible for loading footprint data from CSV.

Attributes (when in context):

footprint (np.array[EventCSV]): event data loaded from the CSV file num_intensity_bins (int): number of intensity bins in the data has_intensity_uncertainty (bool): if the data has uncertainty footprint_index (dict): map of footprint IDs with the index in the data

footprint_filenames

get_event(event_id)

Gets the event from self.footprint based off the event ID passed in.

Args:

event_id: (int) the ID belonging to the Event being extracted

Returns: (np.array[EventCSV]) the event that was extracted

class oasislmf.pytools.getmodel.footprint.FootprintBin(storage: oasis_data_manager.filestore.backends.base.BaseStorage, df_engine='oasis_data_manager.df_reader.reader.OasisPandasReader')

Bases: Footprint

This class is responsible loading the event data from the footprint binary files.

Attributes (when in context):

footprint (mmap.mmap): loaded data from the binary file which has header and then Event data num_intensity_bins (int): number of intensity bins in the data has_intensity_uncertainty (bool): if the data has uncertainty footprint_index (dict): map of footprint IDs with the index in the data

footprint_filenames

get_event(event_id)

Gets the event from self.footprint based off the event ID passed in.

Args:

event_id: (int) the ID belonging to the Event being extracted

Returns: (np.array(Event)) the event that was extracted

class oasislmf.pytools.getmodel.footprint.FootprintBinZ(storage: oasis_data_manager.filestore.backends.base.BaseStorage, df_engine='oasis_data_manager.df_reader.reader.OasisPandasReader')

Bases: Footprint

This class is responsible for loading event data from compressed event data.

Attributes (when in context):

zfootprint (mmap.mmap): loaded data from the compressed binary file which has header and then Event data num_intensity_bins (int): number of intensity bins in the data has_intensity_uncertainty (bool): if the data has uncertainty uncompressed_size (int): the size in which the data is when it is decompressed index_dtype (Union[EventIndexBinZ, EventIndexBin]) the data type footprint_index (dict): map of footprint IDs with the index in the data

footprint_filenames

get_event(event_id)

Gets the event from self.zfootprint based off the event ID passed in.

Args:

event_id: (int) the ID belonging to the Event being extracted

Returns: (np.array[Event]) the event that was extracted

class oasislmf.pytools.getmodel.footprint.FootprintParquet(storage: oasis_data_manager.filestore.backends.base.BaseStorage, df_engine='oasis_data_manager.df_reader.reader.OasisPandasReader')

Bases: Footprint

This class is responsible for loading event data from parquet event data.

Attributes (when in context):

num_intensity_bins (int): number of intensity bins in the data has_intensity_uncertainty (bool): if the data has uncertainty footprint_index (dict): map of footprint IDs with the index in the data

footprint_filenames: List[str]

get_event(event_id: int)

Gets the event data from the partitioned parquet data file.

Args:

event_id: (int) the ID belonging to the Event being extracted

Returns: (np.array[Event]) the event that was extracted

class oasislmf.pytools.getmodel.footprint.FootprintParquetDynamic(storage: oasis_data_manager.filestore.backends.base.BaseStorage, df_engine='oasis_data_manager.df_reader.reader.OasisPandasReader')

Bases: Footprint

This class is responsible for loading event data from parquet dynamic event sets and maps It will build the footprint from the underlying event defintion and hazard case files

Attributes (when in context):

num_intensity_bins (int): number of intensity bins in the data has_intensity_uncertainty (bool): if the data has uncertainty. Only “no” is supported return periods (list): the list of return periods in the model. Not currently used

footprint_filenames: List[str]

get_event(event_id: int)

Gets the event data from the partitioned parquet data file.

Args:

event_id: (int) the ID belonging to the Event being extracted

Returns: (np.array[Event]) the event that was extracted

oasislmf.pytools.getmodel.footprint.stitch_data(areaperil_id, intensity_bin_id, probability, buffer)

Creates a list of tuples from three np.arrays (all inputs must be the same length). Args:

areaperil_id: (np.array) list of areaperil IDs intensity_bin_id: (np.array) list of probability bin IDs probability: (np.array) list of probabilities buffer: (np.array[Event]) list of zeros to be populated with the previous lists

Returns: