Callbacks are a staple concept in software engineering, to the point that sophisticated languages as C++ provide multiple constructs to implement them.
With such flexibility, it’s easy to abuse the mechanism and stretching it beyond its scope. In the first part of this article, I’ll show the C++ constructs used to implement them, with particular attention to the quirks of and specificity of each solution. In the second part I will look into the pitfalls and the abuse of callbacks, which can make a program less readable and more brittle.
Callback Types
Callbacks can be divided in four types, depending on how they are meant to be used by the calling code. Correctly identifying the callback type required by (and offered to) the user code is crucial, as the expected behaviour varies greatly.
- Completion callbacks: meant to be called back as some operation is completed, typically, to notify a certain result. Typical example is the notification of the result of a connection request. They can be synchronous if their user will call them before returning control to its own caller, or asynchronous if they are meant to be invoked at a later time.
- Behavioural callbacks: used by higher order functions to configure their own behaviour. For example, the function passed to
std::filter, which is used to decide which elements to copy into a new container. - Progression callbacks: called multiple times during the execution of the user code, to notify progression, and typically, also intermediate and final results of a long operation. A typical example is the callback used by database queries to notify each selected line. As completion callbacks, they can be synchronous or asynchronous.
- Notification callbacks: invoked to notify events happened asynchronously with respect to the main flow of the program. They differ from asynchronous progression callbacks for the fact that they are not bound to a specific operation, and so they can be indefinitely called.
In case of progression and notification callbacks, it’s common for the caller to offer them the ability to stop the current operation, for example abort a database query or disconnect a network peer, by simply returning an agreed termination code.
Callback Devices in C++
Modern C++ provides five abstractions to implement the callback models:
- Plain function pointers
- Closures (lambda with captures)
- Functors (function objects)
- Method pointers
- Interfaces (callback sets)
I’ll briefly illustrate them later on, but before dwelling in their specifics, it’s necessary to introduce the C++ abstraction capturing the concept of any callable code.
The std::function Wrapper
The std::function template class wraps any C++ code that can be “called”. This includes function pointers, lambdas, function objects and method pointers. As interfaces are sets of functions, they cannot be represented as a single “callable” objects, and so std::function can’t be used for them.
Through this abstraction, is possible to pass different callback devices to the same code, as the following example shows:
#include <functional>
#include <iostream>
#include <vector>
using Callback = std::function<int(int)>;
void processor(const std::string& prefix, const std::vector<int>& v, Callback cb)
{
std::cout << prefix << ": ";
for(const auto& element: v) {
std::cout << cb(element) << ' ';
}
std::cout << '\n';
}
// A plain function.
int multiply_by_two(int value) {
return value * 2;
}
// A functor.
struct Multiplier
{
Multiplier(int multiplier): multiplier_{multiplier} {}
// The operator() makes instances of this class callable entities.
int operator()(int value) {return value * multiplier_;}
private:
int multiplier_;
};
// A non-functor class.
struct MultiplierClass
{
MultiplierClass(int multiplier): multiplier_{multiplier} {}
// Using a standard method as callback.
int multiply(int value) {return value * multiplier_;}
private:
int multiplier_;
};
int main()
{
std::vector<int> data{1,2,3,4,5};
processor("Callback using a function pointer", data, &multiply_by_two);
processor("Callback using a functor", data, Multiplier{2});
processor("Callback using a lambda", data, [](int value){return value * 2;});
MultiplierClass doubler{2};
processor("Callback using a method from a lambda", data,
[&doubler](int value){return doubler.multiply(value);});
auto method = std::bind(&MultiplierClass::multiply, &doubler, std::placeholders::_1);
processor("Callback using a (bound) method pointer", data, method);
return 0;
}The Callback alias declared on line 7 accepts all the callable thrown at it in main.
Pitifully, std::function doesn’t come for free: with respect to a naked function pointer, it requires at least one extra dereference. This is the code generated by gcc (version 11) for the call on line 11:
mov rsi, r13
call [QWORD PTR [r12+24]]
mov esi, eaxChanging Callback to int(*)(int), the code changes into:
mov edi, DWORD PTR [rbx]
call rbp
mov esi, eaxThere are also other overheads that include various move constructors when passing a std::function object around, but they are payed only when actually using functors or closures are used.
However, std::function offers a good balance of flexibility versus performance, and in case more performance and less flexibility is needed down the line, refactoring the code to use function pointers instead is usually a simple operation.
Function Pointers
When it comes to callbacks, function pointers offer the fastest alternative, with just a minimal overhead with respect to a direct call hard-coded in the source.
Being pointers to code segments, function pointers can’t carry any additional context or data with them. When both absolute performance and status persistence are needed, it’s possible to pass both the callback function and a context to user data, which will be provided to the callback function as an additional parameter.
#include <chrono>
#include <iostream>
#include <thread>
#include <vector>
struct Context {
Context(int multiplier): multiplier_{multiplier} {}
int opCount_{0};
int multiplier_{0};
};
using Callback = int(*)(Context&, int, int);
void processor(const std::vector<int>& v, Callback cb, Context ctx)
{
int result = 0;
for(const auto& element: v) {
result += cb(ctx, element, static_cast<int>(v.size()));
}
std::cout << "Result: " << result << '\n';
}
int main()
{
using namespace std::literals;
std::vector<int> data{1,2,3,4,5};
auto lambda = [](Context& ctx, int value, int count){
if(++ctx.opCount_>= count) {
std::cout << "100%\n";
}
else {
std::cout << static_cast<int>(ctx.opCount_ * 100.0 / count )
<< "%\r" << std::flush;
std::this_thread::sleep_for(500ms);
}
return ctx.multiplier_ * value;
};
processor(data, lambda, Context{2});
return 0;
}Here, an instance of the class Context (line 6) is part of the prototype of the callback, and obviously it’s necessary to pass it along with the actual parameter (line 40). As the example shows on line 27, naked function pointer parameters can accept also lambdas, provided they don’t capture any variable.
Closures
Closures are locally declared functions that store values from the surrounding scope. C++ use the lambda operator to create closures storing values by copy or by reference.
C++ lambdas don’t extend the lifetime of the captured symbols; if an object they capture by reference is destroyed before they are called, this will cause an undefined behaviour (likely a crash), so particular care must be taken when using this model for asynchronous callbacks, especially for notification ones.
One simple solution is that of capturing a shared pointer to the necessary data.
#include <chrono>
#include <future>
#include <iostream>
#include <memory>
#include <thread>
#include <vector>
struct Context {
Context(int multiplier): multiplier_{multiplier} {}
~Context(){ std::cout << "Context cleanly destroyed\n";}
int opCount_{0};
int multiplier_{0};
};
using Callback = std::function<int(int, int)>;
int processor(const std::vector<int>& v, Callback cb)
{
int result = 0;
for(const auto& element: v) {
result += cb(element, static_cast<int>(v.size()));
}
return result;
}
int main()
{
using namespace std::literals;
std::vector<int> data{1,2,3,4,5};
std::future<int> expected_result;
{
auto ctx = std::make_shared<Context>(2);
auto lambda = [ctx](int value, int count){
if(++ctx->opCount_>= count) {
std::cout << "100%\n";
}
else {
std::cout << static_cast<int>(ctx->opCount_ * 100.0 / count )
<< "%\r" << std::flush;
std::this_thread::sleep_for(500ms);
}
return ctx->multiplier_ * value;
};
expected_result = std::move(
std::async(std::launch::async, &processor, data, std::move(lambda)));
// Here ctx exits our context.
}
auto result = expected_result.get();
std::cout << "Result: " << result << '\n';
return 0;
}The context created on line 33 as a shared pointer goes out of scope on line 48, but processor stills needs it. As it’s captured by copy on line 34, the shared_ptr holds an additional reference. The destructor for the context on line 10 is cleanly called when async completes the execution of processor.
It is still necessary to be mindful of the fact that line 50 may be executed before or after the context is destroyed, as that happens asynchronously right after the value of expected_result is set.
Also, notice the std::move given as parameter on line 46: the lambda is actually a closure, so it holds a copy of all the data it has captured; passing it as the parameter to std::async would copy it, and all the data it closed. Using std::move there will prevent this from happening.
Functors
Closures are a very flexible device, but they have the drawback of being completely opaque: once created, there is no way to inspect or interact with their internal status.
Another problem of closures is that they trivialise the importance of callback processing. This may be desired if the callback behaviour is trivial as well, but if it requires significant logic and/or data structures, it’s better to create an entity visible at module level.
The following example shows how a functor can be used both to offer callback functionalities and store relevant data for later inspection.
#include <functional>
#include <iostream>
#include <memory>
#include <vector>
using Callback = std::function<int(int)>;
void processor(const std::vector<int>& v, Callback cb)
{
std::cout << "Process results: ";
for(const auto& element: v) {
std::cout << cb(element) << ' ';
}
std::cout << '\n';
}
struct Multiplier
{
std::shared_ptr<std::vector<int>> saved_data_;
Multiplier(int multiplier):
saved_data_{std::make_shared<std::vector<int>>()},
multiplier_{multiplier} {}
int operator()(int value) {
saved_data_->push_back(value);
return value * multiplier_;
}
private:
int multiplier_;
};
int main()
{
std::vector<int> data{1,2,3,4,5};
auto multiplier = Multiplier{2};
// Actually, passing a std::function with a copy of multiplier.
processor(data, multiplier);
std::cout << "Original data: ";
for(const auto& element: *multiplier.saved_data_) {
std::cout << element << ' ';
}
std::cout << '\n';
return 0;
}Notice that since we’re still using a std::function as the callback signature (line 8), when we pass multiplier to processor on line 40, the std::function constructor actually makes a copy of the functor, and thus, to share the results with the calling code, the state on line 19 needs to be a shared pointer.
This could be avoided by renouncing to flexibility, and having an explicit Multiplier& parameter in place of the generic `std::function.
Method Pointers
Structurally, method pointers are analogous to functors; the only difference is that a specific method of the class is used as a callback, instead of the generic operator(). Since method pointers require a pointer to the actual instance to be saved as well, they offer the additional utility of not requiring to take care of sharing their internal status across entities. See the following example:
#include <functional>
#include <iostream>
#include <memory>
#include <vector>
using Callback = std::function<int(int)>;
void processor(const std::vector<int>& v, Callback cb)
{
std::cout << "Process results: ";
for(const auto& element: v) {
std::cout << cb(element) << ' ';
}
std::cout << '\n';
}
struct Multiplier
{
std::vector<int> saved_data_;
Multiplier(int multiplier):
multiplier_{multiplier} {}
int callMeBack(int value) {
saved_data_.push_back(value);
return value * multiplier_;
}
private:
int multiplier_;
};
int main()
{
std::vector<int> data{1,2,3,4,5};
auto multiplier = Multiplier{2};
processor(data, std::bind(&Multiplier::callMeBack, &multiplier, std::placeholders::_1));
std::cout << "Original data: ";
for(const auto& element: multiplier.saved_data_) {
std::cout << element << ' ';
}
std::cout << '\n';
return 0;
}Notice that thanks to std::bind on line 39 creating the std::function that will be used as callback, saved_data_ doesn’t need to be a shared pointer anymore.
However, the value of method pointers is mainly that of expressing a different semantic. Functors are objects meant to be a stateful callback object, a closure with transparent data. Method pointers are meant to just report the result back to the class where they belong.
Interfaces
In some cases, a single program entity may need to signal multiple events to whoever is interested. C++ polymorphism can be used to call arbitrary methods from any instance through a pointer to a base class; however, when implementing a callback strategy, it’s preferable to use the adapter idiom.
This idiom consists in writing an abstract base class with only virtual const methods. From that, an adapter class is derived; it’s task is that of taking one or more target instances, where the actual callbacks reside.
Here follows an example.
#include <chrono>
#include <functional>
#include <iostream>
#include <thread>
#include <vector>
class CallbackInterface
{
public:
virtual void onStart(int count) const = 0;
virtual int processElement(int element) const = 0;
virtual void onComplete(int total) const = 0;
};
struct Multiplier
{
class CallbackAdapter: public CallbackInterface
{
public:
CallbackAdapter(Multiplier* owner): owner_{owner} {}
virtual void onStart(int count) const override {
owner_->setSize(count);
}
virtual int processElement(int element) const override {
return owner_->multiply(element);
}
virtual void onComplete(int total) const override {
std::cout << "Final result: " << total << '\n';
}
private:
Multiplier* owner_;
};
Multiplier(int multiplier):
multiplier_{multiplier} {}
void setSize(int size) {size_ = size;}
int multiply(int value) {
using namespace std::chrono_literals;
std::cout << "Progress: " << (++count_ * 100.0 / size_) << "%\r" << std::flush;
if(count_ < size_) {
std::this_thread::sleep_for(500ms);
} else {
std::cout << '\n';
}
return value * multiplier_;
}
private:
int multiplier_;
int size_;
int count_{0};
};
void processor(const std::vector<int>& v, const CallbackInterface& cb)
{
cb.onStart(static_cast<int>(v.size()));
int result = 0;
for(const auto& element: v) {
result += cb.processElement(element);
}
cb.onComplete(result);
}
int main()
{
std::vector<int> data{1,2,3,4,5};
Multiplier multiplier{2};
processor(data, Multiplier::CallbackAdapter(&multiplier));
return 0;
}CallbackInterface is known by processor as the base class for the callback mechanism. Multiplier is agnostic with respect to how processor wants to call it back; all the logic is delegated to CallbackAdapter, which decouples Multiplier from the callback logic used byprocessor.
The adapter is created on line 73. In this example, it accepts a naked pointer to Multiplier; more sophisticated strategies may be needed if the adapted instance may go out of scope before processor is done.
An adapter may adapt different instances at the same time; different entities may be interested in different callbacks invoked byprocessor, and this model allows for an adapter to decide what entity need to be notified.
It is crucial to keep the adapter implementation as trivial as possible: it should limit itself to pass the callback to the adapted object with minimal interference. If the adapter methods grow complex enough to require their own testing, they should be moved into the target class.
In this first part of my review of callbacks in C++, I have shown the basic techniques, and highlighted some of the specific characteristic and requirements each solution entrails.
The next part will focus on the correct usage of callbacks, and on the consequences of abusing them.