Events

reels.Events

EventTuples(ev_id)

Iterator of (emitter, description, weight, code) tuples.

Source code in reels/Events.py

def __init__(self, ev_id):
    self.ev_id = ev_id
    self.prev_tuple = ''

Events(max_num_events=1000, binary_image=None)

Interface to the c++ container object to hold events.

The purpose of this object is to be filled (either via successive insert_row() or define_event() calls) and passed to a Clips (and indirectly to Targets objects). The content can just be serialized as a pickle to make it pythonic or checked for size. No other introspection methods exist.

Parameters:

Name	Type	Description	Default
max_num_events	int	The maximum number of events to limit the discovery via insert_row() to the max_num_events more frequent/recent.	1000
binary_image	bool	An optional binary image (returned by save_as_binary_image()) to initialize the object with data copied from another Events object.	None

Source code in reels/Events.py

def __init__(self, max_num_events=1000, binary_image=None):
    self.ev_id = new_events()

    events_set_max_num_events(self.ev_id, max_num_events)
    events_set_store_strings(self.ev_id, True)

    self.max_num_events = max_num_events

    if binary_image is not None:
        self.load_from_binary_image(binary_image)

__getstate__()

Used by pickle.dump() (See https://docs.python.org/3/library/pickle.html)

Source code in reels/Events.py

def __getstate__(self):
    """Used by pickle.dump() (See https://docs.python.org/3/library/pickle.html)"""
    return self.save_as_binary_image()

__setstate__(state)

Used by pickle.load() (See https://docs.python.org/3/library/pickle.html)

Source code in reels/Events.py

def __setstate__(self, state):
    """Used by pickle.load() (See https://docs.python.org/3/library/pickle.html)"""
    self.ev_id = new_events()
    self.load_from_binary_image(state)

copy(dictionary=None)

Creates a new Events object that is either a copy of the current one or a copy with the codes transformed by a dictionary.

The typical use of this method is in combination with optimize_events() typically to create a copy of an Events object and optimize the copy rather than the original object to leave the latter unmodified. Also, after optimize_events() the dictionary returned completely defines the optimization and can be used to transform the original object by applying it during the copy.

NOTE: The copy is identical in terms of recorded events and their codes, but does not have usage frequency statistics since it is created via define_event() calls, not by insert_row() over a complete dataset. It is intended to operate in combination with either optimize_events() or a Targets object, not to continue populating it with events via insert_row().

Parameters:

Name	Type	Description	Default
dictionary	dict	A dictionary to be applied during the copy. The dictionary must be returned by a previous optimize_events() of an identical object in order to have the same codes defined. Otherwise, the codes not present in the dictionary will not be translated into events.	None

Returns:

Type	Description
Events	A new Events object

Source code in reels/Events.py

def copy(self, dictionary = None):
    """
    Creates a new Events object that is either a copy of the current one or a copy with the codes transformed by a dictionary.

    The typical use of this method is in combination with optimize_events() typically to create a copy of an Events object
    and optimize the copy rather than the original object to leave the latter unmodified. Also, after optimize_events()
    the dictionary returned completely defines the optimization and can be used to transform the original object by applying it
    during the copy.

    NOTE: The copy is identical in terms of recorded events and their codes, but does not have usage frequency statistics since
    it is created via define_event() calls, not by insert_row() over a complete dataset. It is intended to operate in combination
    with either optimize_events() or a Targets object, not to continue populating it with events via insert_row().

    Args:
        dictionary (dict): A dictionary to be applied during the copy. The dictionary must be returned by a previous optimize_events()
            of an identical object in order to have the same codes defined. Otherwise, the codes not present in the dictionary
            will not be translated into events.

    Returns:
        (Events): A new Events object
    """

    ret = Events(max_num_events = self.max_num_events)

    if dictionary is not None:
        for emitter, description, weight, code in self.describe_events():
            if code in dictionary:
                code = dictionary[code]
                ret.define_event(emitter, description, weight, code)

    else:
        for emitter, description, weight, code in self.describe_events():
            ret.define_event(emitter, description, weight, code)

    return ret

define_event(emitter, description, weight, code)

Define events explicitly.

Caveat: insert_row() and define_event() should not be mixed. The former is for event discovery and the latter for explicit definition. A set of events is build either one way or the other.

Parameters:

Name	Type	Description	Default
emitter	str	The "emitter". A C/Python string representing "owner of event".	required
description	str	The "description". A C/Python string representing "the event".	required
weight	float	The "weight". A double representing a weight of the event.	required
code	int	A unique code number identifying the event.	required

Returns:

Type	Description
bool	True on success.

Source code in reels/Events.py

def define_event(self, emitter, description, weight, code):
    """Define events explicitly.

    **Caveat**: insert_row() and define_event() should not be mixed. The former is for
    event discovery and the latter for explicit definition. A set of events is build
    either one way or the other.

    Args:
        emitter (str):     The "emitter". A C/Python string representing "owner of event".
        description (str): The "description". A C/Python string representing "the event".
        weight (float):    The "weight". A double representing a weight of the event.
        code (int):        A unique code number identifying the event.

    Returns:
        (bool): True on success.
    """
    return events_define_event(self.ev_id, emitter, description, weight, code)

describe_events()

Return an iterator of (emitter, description, weight, code) tuples describing all events.

Returns:

Type	Description
EventTuples	An iterator of (emitter, description, weight, code) tuple on success or None on failure.

Source code in reels/Events.py

def describe_events(self):
    """Return an iterator of (emitter, description, weight, code) tuples describing all events.

    Returns:
        (EventTuples): An iterator of (emitter, description, weight, code) tuple on success or None on failure.
    """
    return EventTuples(self.ev_id)

insert_row(emitter, description, weight)

Process a row from a transaction file.

Caveat: insert_row() and define_event() should not be mixed. The former is for event discovery and the latter for explicit definition. A set of events is build either one way or the other.

Parameters:

Name	Type	Description	Default
emitter	str	The "emitter". A C/Python string representing "owner of event".	required
description	str	The "description". A C/Python string representing "the event".	required
weight	float	The "weight". A double representing a weight of the event.	required

Returns:

Type	Description
bool	True on success.

Source code in reels/Events.py

def insert_row(self, emitter, description, weight):
    """Process a row from a transaction file.

    **Caveat**: insert_row() and define_event() should not be mixed. The former is for
    event discovery and the latter for explicit definition. A set of events is build
    either one way or the other.

    Args:
        emitter (str):     The "emitter". A C/Python string representing "owner of event".
        description (str): The "description". A C/Python string representing "the event".
        weight (float):    The "weight". A double representing a weight of the event.

    Returns:
        (bool): True on success.
    """
    return events_insert_row(self.ev_id, emitter, description, weight)

load_from_binary_image(binary_image)

Load the state of the c++ Events object from a binary_image returned by a previous save_as_binary_image() call.

Parameters:

Name	Type	Description	Default
binary_image	list	A list of strings returned by save_as_binary_image()	required

Returns:

Type	Description
bool	True on success, destroys, initializes and returns false on failure.

Source code in reels/Events.py

def load_from_binary_image(self, binary_image):
    """Load the state of the c++ Events object from a binary_image
        returned by a previous save_as_binary_image() call.

    Args:
        binary_image (list): A list of strings returned by save_as_binary_image()

    Returns:
        (bool): True on success, destroys, initializes and returns false on failure.
    """
    failed = False

    for binary_image_block in binary_image:
        if not events_load_block(self.ev_id, binary_image_block):
            failed = True
            break

    if not failed:
        failed = not events_load_block(self.ev_id, "")

    if failed:
        destroy_events(self.ev_id)
        self.ev_id = new_events()
        return False

    return True

num_events()

Return the number of events in the object.

Returns:

Type	Description
int	The number of events stored in the object.

Source code in reels/Events.py

def num_events(self):
    """Return the number of events in the object.

    Returns:
        (int): The number of events stored in the object.
    """
    return events_num_events(self.ev_id)

optimize_events(clips, targets, num_steps=10, codes_per_step=5, threshold=0.0001, force_include='', force_exclude='', x_form='linear', agg='longest', p=0.5, depth=1000, as_states=True, exp_decay=0.00693, lower_bound_p=0.95, log_lift=True)

Events optimizer.

Optimizes the events to maximize prediction signal. (F1 score over same number of positives.)
It converts code values many-to-one trying to group event codes into categories that represent similar events.

Before starting, a non-optimized Events object must be populated with an initial set of codes we want to reduce by assigning
new many-to-one codes to them.

The algorithm initially removes all codes not found in the clips object. This completely removes them.

The algorithm builds a list of most promising (not already used) codes at the beginning of each step by full tree search.
From that list, each code is tried downwards as {noise, new_code, last_code} for score improvement above threshold
up to codes_per_step steps. And assigned a new code accordingly.
The codes assigned become part of the internal EventCodeMap and in the next step they will replace their old values.

When the algorithm finishes, the internal EventCodeMap is used to rename the object codes and the whole process is reported.

Parameters:

Name	Type	Description	Default
clips	int	The id of a clips object with the same codes and clips for a set of clients whose prediction we optimize.	required
targets	int	The id of a Targets object whose internal TargetMap defines the targets. (Internally a new Targets object will be used to make the predictions we want to optimize.)	required
num_steps	int	The number of steps to iterate. The method will stop early if no codes are found at a step.	10
codes_per_step	int	The number of codes to be tried from the top of the priority list at each step.	5
threshold	float	A minimum threshold, below which a score change is not considered improvement.	0.0001
force_include	str	An optional pointer to a set of codes that must be included before starting.	''
force_exclude	str	An optional pointer to a set of codes that will excluded and set to the base code.	''
x_form	str	The x_form argument to fit the internal Targets object prediction model.	'linear'
agg	str	The agg argument to fit the internal Targets object prediction model.	'longest'
p	float	The p argument to fit the internal Targets object prediction model.	0.5
depth	int	The depth argument to fit the internal Targets object prediction model.	1000
as_states	bool	The as_states argument to fit the internal Targets object prediction model.	True
exp_decay	float	Exponential Decay Factor applied to the internal score in terms of depth. That score selects what codes enter the model. The decay is applied to the average tree depth. 0 is no decay, default value = 0.00693 decays to 0.5 in 100 steps.	0.00693
lower_bound_p	float	Another p for lower bound, but applied to the scoring process rather than the model.	0.95
log_lift	bool	A boolean to set if lift (= LB(included)/LB(after inclusion)) is log() transformed or not.	True

Returns:

Type	Description
tuple	A tuple (success, dictionary, top_codes, log)

Source code in reels/Events.py

def optimize_events(self, clips: int, targets: int, num_steps: int=10, codes_per_step: int=5, threshold: float=0.0001,
                    force_include: str='', force_exclude: str='', x_form: str='linear', agg: str='longest', p: float=0.5,
                    depth: int=1000, as_states: bool=True, exp_decay: float=0.00693, lower_bound_p: float=0.95, log_lift: bool=True):
    """Events optimizer.

        Optimizes the events to maximize prediction signal. (F1 score over same number of positives.)
        It converts code values many-to-one trying to group event codes into categories that represent similar events.

        Before starting, a non-optimized Events object must be populated with an initial set of codes we want to reduce by assigning
        new many-to-one codes to them.

        The algorithm initially removes all codes not found in the clips object. This completely removes them.

        The algorithm builds a list of most promising (not already used) codes at the beginning of each step by full tree search.
        From that list, each code is tried downwards as {noise, new_code, last_code} for score improvement above threshold
        up to codes_per_step steps. And assigned a new code accordingly.
        The codes assigned become part of the internal EventCodeMap and in the next step they will replace their old values.

        When the algorithm finishes, the internal EventCodeMap is used to rename the object codes and the whole process is reported.

    Args:
        clips:          The id of a clips object with the same codes and clips for a set of clients whose prediction we optimize.
        targets:        The id of a Targets object whose internal TargetMap defines the targets. (Internally a new Targets object
                        will be used to make the predictions we want to optimize.)
        num_steps:      The number of steps to iterate. The method will stop early if no codes are found at a step.
        codes_per_step: The number of codes to be tried from the top of the priority list at each step.
        threshold:      A minimum threshold, below which a score change is not considered improvement.
        force_include:  An optional pointer to a set of codes that must be included before starting.
        force_exclude:  An optional pointer to a set of codes that will excluded and set to the base code.
        x_form:         The x_form argument to fit the internal Targets object prediction model.
        agg:            The agg argument to fit the internal Targets object prediction model.
        p:              The p argument to fit the internal Targets object prediction model.
        depth:          The depth argument to fit the internal Targets object prediction model.
        as_states:      The as_states argument to fit the internal Targets object prediction model.
        exp_decay:      Exponential Decay Factor applied to the internal score in terms of depth. That score selects what
                        codes enter the model. The decay is applied to the average tree depth. 0 is no decay, default
                        value = 0.00693 decays to 0.5 in 100 steps.
        lower_bound_p:  Another p for lower bound, but applied to the scoring process rather than the model.
        log_lift:       A boolean to set if lift (= LB(included)/LB(after inclusion)) is log() transformed or not.

    Returns:
        (tuple): A tuple (success, dictionary, top_codes, log)
    """

    ret = events_optimize_events(self.ev_id, clips.cp_id, targets.tr_id, num_steps, codes_per_step, threshold, force_include,
                                 force_exclude, x_form, agg, p, depth, as_states, exp_decay, lower_bound_p, log_lift)

    success = ret.startswith('SUCCESS')

    if not success:
        return success, None, None, ret

    txt = ret.split('\n')

    i = next(i for i, s in enumerate(txt) if s.startswith('  Final dictionary ='))
    log = '\n'.join(txt[1:i])
    dictionary = eval(txt[i][20:])

    i = next(i for i, s in enumerate(txt) if s.startswith('code_performance'))
    top_codes = '\n'.join(txt[(i + 1):])

    return success, dictionary, top_codes, log

save_as_binary_image()

Saves the state of the c++ Events object as a Python list of strings referred to a binary_image.

Returns:

Type	Description
list	The binary_image containing the state of the Events. There is not much you can do with it except serializing it as a Python (e.g., pickle) object and loading it into another Events object. Pass it to the constructor to create an initialized object,

Source code in reels/Events.py

def save_as_binary_image(self):
    """Saves the state of the c++ Events object as a Python
        list of strings referred to a binary_image.

    Returns:
        (list): The binary_image containing the state of the Events. There is
            not much you can do with it except serializing it as a Python
            (e.g., pickle) object and loading it into another Events object.
            Pass it to the constructor to create an initialized object,
    """
    bi_idx = events_save(self.ev_id)
    if bi_idx == 0:
        return None

    bi = []
    bi_size = size_binary_image_iterator(bi_idx)
    for t in range(bi_size):
        bi.append(next_binary_image_iterator(bi_idx))

    destroy_binary_image_iterator(bi_idx)

    return bi