oasislmf.lookup.factory

Module Contents

Classes

KeyServerFactory	A factory class to create the Keys Server that will be use to generate the keys files
BasicKeyServer	A basic implementation of the KeyServerInterface

class oasislmf.lookup.factory.KeyServerFactory

Bases: object

A factory class to create the Keys Server that will be use to generate the keys files All Key Server must implement the interface defined in lookup.interface.KeyServerInterface

Oasis provides a built-in Key Server that manage the generation of the key files from the key provided by a built-in or a custom Key Lookup.

The factory now return a KeyServer object and not a KeyLookup. The parameter to pass has also been simplified usage of all the below parameter are now deprecated

• complex_lookup_config_fp => pass the path to your complex lookup config directly in lookup_config_fg

• lookup_module_path => set as key ‘lookup_module_path’ in the lookup config

• model_keys_data_path => set as key ‘keys_data_path’ in the lookup config

• model_version_file_path => set the model information (‘supplier_id’, ‘model_id’, ‘model_version’) directly into the config

classmethod get_config(config_fp)

classmethod get_model_info(model_version_file_path)

Get model information from the model version file.

classmethod update_deprecated_args(config_dir, config, complex_lookup_config_fp, model_keys_data_path, model_version_file_path, lookup_module_path)

classmethod create(model_keys_data_path=None, model_version_file_path=None, lookup_module_path=None, lookup_config=None, lookup_config_json=None, lookup_config_fp=None, complex_lookup_config_fp=None, user_data_dir=None, output_directory=None)

Creates a keys lookup class instance for the given model and supplier - local file paths are required for the model keys data folder, the model version file and the Git repository for the model keys server. Returns a pair (model_info, klc), where model_info is a dictionary holding model information from the model version file and klc is the lookup service class instance for the model.

class oasislmf.lookup.factory.BasicKeyServer(config, config_dir=None, user_data_dir=None, output_dir=None)

A basic implementation of the KeyServerInterface will load the KeyLookup class from config[‘lookup_module_path’] if present or used the built-in KeyLookup KeyLookup must implement the KeyLookupInterface

will provide a multiprocess solution if KeyLoopup implement the process_locations_multiproc method

both single and multiprocess solutions will use low amount of memory as they process the key by chunk of limited size.

This class implement all the file writing method that were previously handled by the lookup factory

interface_version = '1'

valid_format = ['oasis', 'json']

error_heading_row

model_data_heading_row

key_success_heading_row

key_success_with_amplification_heading_row

key_success_with_message_heading_row

key_success_with_amplification_and_message_heading_row

min_bloc_size = 1000

max_bloc_size = 10000

get_lookup_cls()

static create_lookup(lookup_cls, config, config_dir, user_data_dir, output_dir, lookup_id)

abstract get_locations(location_fp)

load exposure data from location_fp and return the exposure dataframe

static location_producer(error_queue, loc_df, part_count, loc_queue)

static lookup_multiproc_worker(error_queue, lookup_cls, config, config_dir, user_data_dir, output_dir, lookup_id, loc_queue, key_queue)

static key_producer(key_queue, error_queue, worker_count)

get_success_heading_row(keys, keys_success_msg)

write_json_keys_file(results, keys_success_msg, successes_fp, errors_fp)

write_oasis_keys_file(results, keys_success_msg, successes_fp, errors_fp)

write_keys_file(results, successes_fp, errors_fp, output_format, keys_success_msg)

generate_key_files_singleproc(loc_df, successes_fp, errors_fp, output_format, keys_success_msg, **kwargs)

generate_key_files_multiproc(loc_df, successes_fp, errors_fp, output_format, keys_success_msg, num_cores, num_partitions, **kwargs)

Process and return the lookup results a location row Used in multiprocessing based query

location_row is of type <class ‘pandas.core.series.Series’>

generate_key_files(location_fp=None, successes_fp=None, errors_fp=None, output_format='oasis', keys_success_msg=False, multiproc_enabled=True, multiproc_num_cores=-1, multiproc_num_partitions=-1, location_df=None, **kwargs)

generate key files by calling: 1. get_locations to get a location object from the location_fp 2. process_locations or process_locations_multiproc to get results object from the locations object 3. write_keys_file to writes the relevant files from the results object