DSL Functors
At the most fundamental level, all DSL run-time activity is performed by objects we call functors. The class declaration for new functors follows a simple pattern. The class must derive from class functor::parent<T> and the class must override the virtual evaluate() and input_data_is_ok() member. Once allocated the object must be handed to the strategy class which assumes ownership and eventually destroys it.
At every strategy interval (bar), all evaluate() members are called on all functors owned by the strategy. During evaluation, the object performs whatever logic it is intended for, and then pushes the return value onto one of the series it owns. When strategy::on_prepare_dsl() is invoked, any number of functors are created via user defined DSL. These objects pass data amongst each other in such a way that the intended DSL logic results. Functor objects are linked to each other by exchanging series pointers. Each functor usually has one output series, sometimes more, which other functors can read data from, either a single value, or a range of values.
Let's have a look at a simple functor that adds two numbers:
class add_two_numbers_class : public functor::parent <double> {
public:
sref<double> m_data_a, m_data_b;
add_two_numbers_class(void) : functor::parent<double>(1) { ; }
virtual void evaluate(void)override{
parent::push(m_data_a[0] + m_data_b[0]);
}
virtual bool input_data_is_ok(void)override{
return m_data_a.size_ok(1) && m_data_b.size_ok(1);
}
};
Since functors are specialized in one task only, they are usually implemented as 'local' class definitions inside DSL function definitions. So there is no harm in making all members public. This way class members can be initialized more easily and the constructor remains simple. The evaluate() member simply performs the required calculation - in this case adding two numbers. The input_data_is_ok() member's job is to check that the series the object is bound to actually have sufficient data. In this case we only access one value in each incoming series. This input_data_is_ok() check is very important as it allows the underlying DSL mechanism to avoid series-bounds-errors entirely.
Now let's have a look how a complete DSL function using the above functor would be defined - Tutorial 206:
sref<double> ADD_TWO_NUMBERS(sref<double> _data_a, sref<double> _data_b)
{
// Defining a local functor class. Local scope is ok since this class
// will not be used anywhere else.
class local_class : public functor::parent <double> {
public:
sref<double> m_data_a, m_data_b;
// constructor. Need to initialize parent with type 'double' and 1 output series
local_class(void) : functor::parent<double>(1) { ; }
// evaluate the functor logic - adds two numbers.
virtual void evaluate(void)override{
parent::push(0, m_data_a[0] + m_data_b[0]); // push value to series at position 0
}
// This checks the incoming data series has sufficient data
// to perform the intended calculation. Doing this properly
// avoids series-bounds-error exceptions.
virtual bool input_data_is_ok(void)override{
return m_data_a.size_ok(1) && m_data_b.size_ok(1);
}
};
// allocating and construct the functor object
auto ftor_ptr = new local_class();
ftor_ptr->m_data_a = _data_a; // initializing the series and period members
ftor_ptr->m_data_b = _data_b;
// Passing ownership to the strategy itself. This happens via thread local storage (TLS)
// The strategy will take care of destroying object
strategy_context::strategy_ptr()->assume_functor_ownership(ftor_ptr);
// we return a reference to the functor's output series, at position 0. A functor
// can have any number of output series, but usually only has one
return ftor_ptr->output(0);
}
Essentially we stuck our class definition inside a DSL function, and then added a few lines of code to create a class instance, initialize the series reference members, and pass ownership to the strategy object. The last line simply returns a series reference to the series with the calculated output values.
Performing Range Calculations
Let's have a look at a functor that does its calculation on a range of values to calculate a sum. This is the RANGE_SUM() function - also from tutorial 206:
sref<double> RANGE_SUM(sref<double> _data, size_t _period)
{
verify_non_zero(_period);
class local_class : public functor::parent <double> {
public:
sref<double> m_data;
size_t m_period;
local_class(void) : functor::parent<double>(1) { ; }
virtual void evaluate(void)override{
double sum = 0.0;
const double* data_ptr = m_data.data(); // get data pointer for faster access
for (size_t c(0); c < m_period; c++) {
sum += data_ptr[c];
}
parent::push(0, sum);
}
// Here we need to ensure that the size of the series is at least
// equivalent to 'period'
virtual bool input_data_is_ok(void)override
{
return m_data.size_ok( m_period);
}
};
auto ftor_ptr = new local_class();
ftor_ptr->m_data = _data; // initializing the series and period members
ftor_ptr->m_period = _period;
strategy_context::strategy_ptr()->assume_functor_ownership(ftor_ptr);
return ftor_ptr->output(0);
}
The overall functor is the very similar to the earlier one, except that the algorithm now looks back in time, and so we need to ensure that there is sufficient data in the incoming series. But implementing a simple range-sum with a functor with overkill. We could have done this with a simple lambda function as we saw in the earlier section on lambdas.
Function objects have the advantage of being able to retain state between strategy intervals. This makes it possible to use optimized algorithms.
Optimized Algorithms
To calculate a range sum, we really only need to iterate over all data items once. On subsequent intervals all we need to do is add the most recent number, and drop the oldest. The functor remembers the sum between calls and is thus 'stateful'. Class functor::parent has a some members that can assist with this like parent::first_bar_completed() and parent::set_first_bar_completed(). These are useful since the calculation on the first bar is different from subsequent bars.
The following function implemented the optimized sum algorithm - also from tutorial 206:
sref<double> FAST_RANGE_SUM(sref<double> _data, size_t _period)
{
verify_non_zero(_period);
class local_class : public functor::parent <double> {
public:
sref<double> m_data;
size_t m_period;
double m_sum;
local_class(void) : functor::parent<double>(1) { ; }
virtual void evaluate(void)override
{
// On the first bar, we need to iterate through all
// data items to calculate the sum
if (!parent::first_bar_completed()) {
tsa_assert(m_period > 0);
m_data.verify_size(m_period);
m_sum = 0.0;
const double* data = m_data.data();
for (size_t c = 0; c < m_period; c++) {
m_sum += data[c];
}
parent::push(0, m_sum);
parent::set_first_bar_completed(); // move on to fast algorithm
}
else {
// on subsequent bars we only add the latest
// data item and subtract the oldest. Note that the
// pointer returned by series_cref<T>::data() changes
// at every bar. You have to get a fresh copy every time!
const double* data = m_data.data();
m_sum += data[0];
m_sum -= data[m_period];
parent::push(0, m_sum);
}
}
virtual bool input_data_is_ok(void)override
{
return m_data.size_ok(m_period);
}
};
auto ftor_ptr = new local_class();
ftor_ptr->m_data = _data;
ftor_ptr->m_period = _period;
strategy_context::strategy_ptr()->assume_functor_ownership(ftor_ptr);
return ftor_ptr->output(0);
}
Mixing DSL with Functors
Functors can be very fast, they are not necessarily easy to implement. DSL is so useful because it makes working with nested functions easy! But DSL also has overhead that may need to be avoided where performance matters. Basic DSL functions are however very fast, since they are based on functors, especially in situations where an optimized algorithm is possible. You wouldn't want to re-invent the wheel inside your functors if a fast DSL function already exists. This is why it makes sense to sometimes mix and match existing DSL with new functors. This works as follows:
sref<double> CUSTOM_DSL_FUNCTION(sref<double> _data, size_t _period)
{
verify_non_zero(_period);
class local_class : public functor::parent <double> {
public:
sref<double> m_data, m_data_1, m_data_2, m_data_3, m_data_4;
size_t m_period;
local_class(void) : functor::parent<double>(1) { ; }
virtual void evaluate(void)override {
// fancy algorithm
}
virtual bool input_data_is_ok(void)override {
return m_data_1.size_ok(100);
}
};
auto ftor_ptr = new local_class();
ftor_ptr->m_period = _period;
ftor_ptr->m_data = _data;
//********************************************************************
// Use existing DSL functions instead of duplicating code in functor
ftor_ptr->m_data_1 = AVERAGE(_data, _period);
ftor_ptr->m_data_2 = VOLATILITY(_data, _period,255);
ftor_ptr->m_data_3 = STDEVP(_data, _period);
ftor_ptr->m_data_4 = REGRESSION_SLOPE(_data, _period);
//********************************************************************
strategy_context::strategy_ptr()->assume_functor_ownership(ftor_ptr);
return ftor_ptr->output(0);
}
Just after you allocate the function object, you can set the value of various input series references to series calculated by existing DSL functions. This way you benefit from the optimized implementation of core library functions and you won't need to duplicate logic inside your functors.