Components of metadata

Target definition

Target is defined as an aggregate in the aggregates table by having window_type=0. Id of the aggregate must be specified in the models table.

Target example

Aggregates table:

id	variable_id	aggregate_type	window_type	name
1	1	1	0	D_count_all

Models table:

id	used	target_aggregate_id	target_start	target_length	target_length_exact	...
1	1	1	0	0	0	...

In example above, the target window starts directly after an event setting trigger true (because of target_start=0) and is counted to the end of data (target_length=0 and target_length_exact=0). The target is an D_count_all aggregate, which counts the number of occurences of the "D" event.

Models

Information about models is stored in metadata in the models table. Active models (used in on-line scoring) must have "used" flag set on "true". The models table also contains a scoring code of a model, saved in a string format.

Deployment of a new model

After computation of a new model, it is immediately implemented - new model owerwrites an old model with the same id.

If a model uses other variables than the previous one, every used aggregate must be updated. Aggregates are calculated iteratively in order to minimize the delay of processing a new event by the new model. At first, the engine calculate necessary aggregates using saved retail events, then it updates the values of aggregates if a new event appears. The process is repeated several times.

Scoring

Scoring runs when a high-level event occurs (see High-level events section below). Events are defined in metadata in the triggers table. Different events can induce scoring by different models (assignment of high-level events to models is described in metadata). Also, different groups of users can be scored by different models. Definitions of groups of users are also stored in the triggers table. Given user can be scored by multiple models at once (every event inducing scoring can be related to many models in the models_triggers table). Scoring code is kept as a string in the models table. If the Event Engine is integrated with Scoring Engine, a reference to a model stored in Scoring Engine database is kept in the models table in place of the scoring code.

In programmatic, the modeling table is build basing on unique user id and impression id. Therefore, scoring is based on impressions as well. Events inducing scoring (bid requests) may include several bid requests (impression list). In this case, at the beginning of processing the event is divided into several events (one for each impression) passed to scoring.

High-level events

High-level events are events inducing scoring or target calculating. They are defined in metadata (the triggers table) as Java expressions. Expressions will be defined by analytics. All variables, that can be used in expressions, will be listed in a separate dictionary.

Dictionaries

Dictionaries can be used in order to faciliate the use of aggregates. Dictionaries are defined in metadata in the dictionary table. When defining an aggregate, a dictionary_id of a suitable entry must be included.

Example of a dictionary

Dictionary:

variable1	cat1	cat2
1	D	AA
2	F	AA
3	D	BB
4	F	BB
5	D	AA

The dictionary table:

id	categorical	value	start	start_inclusive	end	end_inclusive	mapped_value
1	1	1	null	null	null	null	D
1	1	3	null	null	null	null	D
1	1	5	null	null	null	null	D
2	1	2	null	null	null	null	F
2	1	2	null	null	null	null	F
3	1	1	null	null	null	null	AA
3	1	2	null	null	null	null	AA
3	1	5	null	null	null	null	AA
4	1	3	null	null	null	null	BB
4	1	4	null	null	null	null	BB
5	1	1	null	null	null	null	D_AA
5	1	5	null	null	null	null	D_AA
6	1	3	null	null	null	null	D_BB
7	1	2	null	null	null	null	F_AA
8	1	4	null	null	null	null	F_BB

The aggregates table:

variable_id	aggregate_type	dictionary_id	name
1	1	1	D_count_all
1	1	2	F_count_all
1	1	3	AA_count_all
1	1	4	BB_count_all
1	1	5	D_AA_count_all
1	1	6	D_BB_count_all
1	1	7	F_AA_count_all
1	1	8	F_BB_count_all
1	2	8	F_BB_sum_all