Skip to content

Events

reels.Events

EventTuples(ev_id)

Iterator of (emitter, description, weight, code) tuples.

Source code in reels/Events.py
41
42
43
def __init__(self, ev_id):
    self.ev_id = ev_id
    self.prev_tuple = ''

Events(max_num_events=1000, binary_image=None)

Interface to the c++ container object to hold events.

The purpose of this object is to be filled (either via successive insert_row() or define_event() calls) and passed to a Clips (and indirectly to Targets objects). The content can just be serialized as a pickle to make it pythonic or checked for size. No other introspection methods exist.

Parameters:

Name Type Description Default
max_num_events

The maximum number of events to limit the discovery via insert_row() to the max_num_events more frequent/recent.

1000
binary_image

An optional binary image (returned by save_as_binary_image()) to initialize the object with data copied from another Events object.

None
Source code in reels/Events.py
77
78
79
80
81
82
83
84
85
86
def __init__(self, max_num_events=1000, binary_image=None):
    self.ev_id = new_events()

    events_set_max_num_events(self.ev_id, max_num_events)
    events_set_store_strings(self.ev_id, True)

    self.max_num_events = max_num_events

    if binary_image is not None:
        self.load_from_binary_image(binary_image)

__getstate__()

Used by pickle.dump() (See https://docs.python.org/3/library/pickle.html)

Source code in reels/Events.py
97
98
99
def __getstate__(self):
    """Used by pickle.dump() (See https://docs.python.org/3/library/pickle.html)"""
    return self.save_as_binary_image()

__setstate__(state)

Used by pickle.load() (See https://docs.python.org/3/library/pickle.html)

Source code in reels/Events.py
101
102
103
104
def __setstate__(self, state):
    """Used by pickle.load() (See https://docs.python.org/3/library/pickle.html)"""
    self.ev_id = new_events()
    self.load_from_binary_image(state)

copy(dictionary=None)

Creates a new Events object that is either a copy of the current one or a copy with the codes transformed by a dictionary.

The typical use of this method is in combination with optimize_events() typically to create a copy of an Events object and optimize the copy rather than the original object to leave the latter unmodified. Also, after optimize_events() the dictionary returned completely defines the optimization and can be used to transform the original object by applying it during the copy.

NOTE: The copy is identical in terms of recorded events and their codes, but does not have usage frequency statistics since it is created via define_event() calls, not by insert_row() over a complete dataset. It is intended to operate in combination with either optimize_events() or a Targets object, not to continue populating it with events via insert_row().

Parameters:

Name Type Description Default
dictionary

A dictionary to be applied during the copy. The dictionary must be returned by a previous optimize_events() of an identical object in order to have the same codes defined. Otherwise, the codes not present in the dictionary will not be translated into events.

None

Returns:

Type Description

A new Events object

Source code in reels/Events.py
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
def copy(self, dictionary = None):
    """
    Creates a new Events object that is either a copy of the current one or a copy with the codes transformed by a dictionary.

    The typical use of this method is in combination with optimize_events() typically to create a copy of an Events object
    and optimize the copy rather than the original object to leave the latter unmodified. Also, after optimize_events()
    the dictionary returned completely defines the optimization and can be used to transform the original object by applying it
    during the copy.

    NOTE: The copy is identical in terms of recorded events and their codes, but does not have usage frequency statistics since
    it is created via define_event() calls, not by insert_row() over a complete dataset. It is intended to operate in combination
    with either optimize_events() or a Targets object, not to continue populating it with events via insert_row().

    Args:
        dictionary: A dictionary to be applied during the copy. The dictionary must be returned by a previous optimize_events()
                    of an identical object in order to have the same codes defined. Otherwise, the codes not present in the dictionary
                    will not be translated into events.

    Returns:
        A new Events object
    """

    ret = Events(max_num_events = self.max_num_events)

    if dictionary is not None:
        for emitter, description, weight, code in self.describe_events():
            if code in dictionary:
                code = dictionary[code]
                ret.define_event(emitter, description, weight, code)

    else:
        for emitter, description, weight, code in self.describe_events():
            ret.define_event(emitter, description, weight, code)

    return ret

define_event(emitter, description, weight, code)

Define events explicitly.

Caveat: insert_row() and define_event() should not be mixed. The former is for event discovery and the latter for explicit definition. A set of events is build either one way or the other.

Parameters:

Name Type Description Default
emitter

The "emitter". A C/Python string representing "owner of event".

required
description

The "description". A C/Python string representing "the event".

required
weight

The "weight". A double representing a weight of the event.

required
code

A unique code number identifying the event.

required

Returns:

Type Description

True on success.

Source code in reels/Events.py
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
def define_event(self, emitter, description, weight, code):
    """Define events explicitly.

    **Caveat**: insert_row() and define_event() should not be mixed. The former is for
    event discovery and the latter for explicit definition. A set of events is build
    either one way or the other.

    Args:
        emitter:     The "emitter". A C/Python string representing "owner of event".
        description: The "description". A C/Python string representing "the event".
        weight:      The "weight". A double representing a weight of the event.
        code:        A unique code number identifying the event.

    Returns:
        True on success.
    """
    return events_define_event(self.ev_id, emitter, description, weight, code)

describe_events()

Return an iterator of (emitter, description, weight, code) tuples describing all events.

Returns:

Type Description

An iterator of (emitter, description, weight, code) tuple on success or None on failure.

Source code in reels/Events.py
149
150
151
152
153
154
155
def describe_events(self):
    """Return an iterator of (emitter, description, weight, code) tuples describing all events.

    Returns:
        An iterator of (emitter, description, weight, code) tuple on success or None on failure.
    """
    return EventTuples(self.ev_id)

insert_row(emitter, description, weight)

Process a row from a transaction file.

Caveat: insert_row() and define_event() should not be mixed. The former is for event discovery and the latter for explicit definition. A set of events is build either one way or the other.

Parameters:

Name Type Description Default
emitter

The "emitter". A C/Python string representing "owner of event".

required
description

The "description". A C/Python string representing "the event".

required
weight

The "weight". A double representing a weight of the event.

required

Returns:

Type Description

True on success.

Source code in reels/Events.py
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
def insert_row(self, emitter, description, weight):
    """Process a row from a transaction file.

    **Caveat**: insert_row() and define_event() should not be mixed. The former is for
    event discovery and the latter for explicit definition. A set of events is build
    either one way or the other.

    Args:
        emitter:     The "emitter". A C/Python string representing "owner of event".
        description: The "description". A C/Python string representing "the event".
        weight:      The "weight". A double representing a weight of the event.

    Returns:
        True on success.
    """
    return events_insert_row(self.ev_id, emitter, description, weight)

load_from_binary_image(binary_image)

Load the state of the c++ Events object from a binary_image returned by a previous save_as_binary_image() call.

Parameters:

Name Type Description Default
binary_image

A list of strings returned by save_as_binary_image()

required

Returns:

Type Description

True on success, destroys, initializes and returns false on failure.

Source code in reels/Events.py
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def load_from_binary_image(self, binary_image):
    """Load the state of the c++ Events object from a binary_image
        returned by a previous save_as_binary_image() call.

    Args:
        binary_image: A list of strings returned by save_as_binary_image()

    Returns:
        True on success, destroys, initializes and returns false on failure.
    """
    failed = False

    for binary_image_block in binary_image:
        if not events_load_block(self.ev_id, binary_image_block):
            failed = True
            break

    if not failed:
        failed = not events_load_block(self.ev_id, "")

    if failed:
        destroy_events(self.ev_id)
        self.ev_id = new_events()
        return False

    return True

num_events()

Return the number of events in the object.

Returns:

Type Description

The number of events stored in the object.

Source code in reels/Events.py
141
142
143
144
145
146
147
def num_events(self):
    """Return the number of events in the object.

    Returns:
        The number of events stored in the object.
    """
    return events_num_events(self.ev_id)

optimize_events(clips, targets, num_steps=10, codes_per_step=5, threshold=0.0001, force_include='', force_exclude='', x_form='linear', agg='longest', p=0.5, depth=1000, as_states=True, exp_decay=0.00693, lower_bound_p=0.95, log_lift=True)

Events optimizer.

Optimizes the events to maximize prediction signal. (F1 score over same number of positives.)
It converts code values many-to-one trying to group event codes into categories that represent similar events.

Before starting, a non-optimized Events object must be populated with an initial set of codes we want to reduce by assigning
new many-to-one codes to them.

The algorithm initially removes all codes not found in the clips object. This completely removes them.

The algorithm builds a list of most promising (not already used) codes at the beginning of each step by full tree search.
From that list, each code is tried downwards as {noise, new_code, last_code} for score improvement above threshold
up to codes_per_step steps. And assigned a new code accordingly.
The codes assigned become part of the internal EventCodeMap and in the next step they will replace their old values.

When the algorithm finishes, the internal EventCodeMap is used to rename the object codes and the whole process is reported.

Parameters:

Name Type Description Default
clips

The id of a clips object with the same codes and clips for a set of clients whose prediction we optimize.

required
targets

The id of a Targets object whose internal TargetMap defines the targets. (Internally a new Targets object will be used to make the predictions we want to optimize.)

required
num_steps

The number of steps to iterate. The method will stop early if no codes are found at a step.

10
codes_per_step

The number of codes to be tried from the top of the priority list at each step.

5
threshold

A minimum threshold, below which a score change is not considered improvement.

0.0001
force_include

An optional pointer to a set of codes that must be included before starting.

''
force_exclude

An optional pointer to a set of codes that will excluded and set to the base code.

''
x_form

The x_form argument to fit the internal Targets object prediction model.

'linear'
agg

The agg argument to fit the internal Targets object prediction model.

'longest'
p

The p argument to fit the internal Targets object prediction model.

0.5
depth

The depth argument to fit the internal Targets object prediction model.

1000
as_states

The as_states argument to fit the internal Targets object prediction model.

True
exp_decay

Exponential Decay Factor applied to the internal score in terms of depth. That score selects what codes enter the model. The decay is applied to the average tree depth. 0 is no decay, default value = 0.00693 decays to 0.5 in 100 steps.

0.00693
lower_bound_p

Another p for lower bound, but applied to the scoring process rather than the model.

0.95
log_lift

A boolean to set if lift (= LB(included)/LB(after inclusion)) is log() transformed or not.

True

Returns:

Type Description

A tuple (success, dictionary, top_codes, log)

Source code in reels/Events.py
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
def optimize_events(self, clips, targets, num_steps=10, codes_per_step=5, threshold=0.0001, force_include='',
                    force_exclude='', x_form='linear', agg='longest', p=0.5, depth=1000, as_states=True,
                    exp_decay=0.00693, lower_bound_p=0.95, log_lift=True):
    """Events optimizer.

        Optimizes the events to maximize prediction signal. (F1 score over same number of positives.)
        It converts code values many-to-one trying to group event codes into categories that represent similar events.

        Before starting, a non-optimized Events object must be populated with an initial set of codes we want to reduce by assigning
        new many-to-one codes to them.

        The algorithm initially removes all codes not found in the clips object. This completely removes them.

        The algorithm builds a list of most promising (not already used) codes at the beginning of each step by full tree search.
        From that list, each code is tried downwards as {noise, new_code, last_code} for score improvement above threshold
        up to codes_per_step steps. And assigned a new code accordingly.
        The codes assigned become part of the internal EventCodeMap and in the next step they will replace their old values.

        When the algorithm finishes, the internal EventCodeMap is used to rename the object codes and the whole process is reported.

    Args:
        clips:          The id of a clips object with the same codes and clips for a set of clients whose prediction we optimize.
        targets:        The id of a Targets object whose internal TargetMap defines the targets. (Internally a new Targets object
                        will be used to make the predictions we want to optimize.)
        num_steps:      The number of steps to iterate. The method will stop early if no codes are found at a step.
        codes_per_step: The number of codes to be tried from the top of the priority list at each step.
        threshold:      A minimum threshold, below which a score change is not considered improvement.
        force_include:  An optional pointer to a set of codes that must be included before starting.
        force_exclude:  An optional pointer to a set of codes that will excluded and set to the base code.
        x_form:         The x_form argument to fit the internal Targets object prediction model.
        agg:            The agg argument to fit the internal Targets object prediction model.
        p:              The p argument to fit the internal Targets object prediction model.
        depth:          The depth argument to fit the internal Targets object prediction model.
        as_states:      The as_states argument to fit the internal Targets object prediction model.
        exp_decay:      Exponential Decay Factor applied to the internal score in terms of depth. That score selects what
                        codes enter the model. The decay is applied to the average tree depth. 0 is no decay, default
                        value = 0.00693 decays to 0.5 in 100 steps.
        lower_bound_p:  Another p for lower bound, but applied to the scoring process rather than the model.
        log_lift:       A boolean to set if lift (= LB(included)/LB(after inclusion)) is log() transformed or not.

    Returns:
        A tuple (success, dictionary, top_codes, log)
    """

    ret = events_optimize_events(self.ev_id, clips.cp_id, targets.tr_id, num_steps, codes_per_step, threshold, force_include,
                                 force_exclude, x_form, agg, p, depth, as_states, exp_decay, lower_bound_p, log_lift)

    success = ret.startswith('SUCCESS')

    if not success:
        return success, None, None, ret

    txt = ret.split('\n')

    i = next(i for i, s in enumerate(txt) if s.startswith('  Final dictionary ='))
    log = '\n'.join(txt[1:i])
    dictionary = eval(txt[i][20:])

    i = next(i for i, s in enumerate(txt) if s.startswith('code_performance'))
    top_codes = '\n'.join(txt[(i + 1):])

    return success, dictionary, top_codes, log

save_as_binary_image()

Saves the state of the c++ Events object as a Python list of strings referred to a binary_image.

Returns:

Type Description

The binary_image containing the state of the Events. There is

not much you can do with it except serializing it as a Python

(e.g., pickle) object and loading it into another Events object.

Pass it to the constructor to create an initialized object,

Source code in reels/Events.py
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
def save_as_binary_image(self):
    """Saves the state of the c++ Events object as a Python
        list of strings referred to a binary_image.

    Returns:
        The binary_image containing the state of the Events. There is
        not much you can do with it except serializing it as a Python
        (e.g., pickle) object and loading it into another Events object.
        Pass it to the constructor to create an initialized object,
    """
    bi_idx = events_save(self.ev_id)
    if bi_idx == 0:
        return None

    bi = []
    bi_size = size_binary_image_iterator(bi_idx)
    for t in range(bi_size):
        bi.append(next_binary_image_iterator(bi_idx))

    destroy_binary_image_iterator(bi_idx)

    return bi