The API of NEBLO
The API presented by NEBLO offers simple primitives and powerful mechanisms for programming generic peer-to-peer applications, in a way that is independent from the underlying overlay.
The primitives allow the peers to exchange messages with one another in two different patterns, namely, unidirectional and request-response; the latter takes place in a split-phase non-blocking way, so that the application can be made latency-tolerant and thus more performing.
The semantics of messages is not defined by NEBLO itself. Rather, a mechanism is offered to the application programmer so as to allow him to write and set up application-level handlers, which are to run upon message arrivals at each peer. The overall behaviour of the application is thus shaped by the handlers, as they define the actions to be carried out by a generic peer when receiving a message (receiver handler) or when forwarding a message to another peer (forwarder handler).
Another mechanism offered by NEBLO is ports, similar to Internet ports. Each peer can get message through different NEBLO ports, and each port can be bound to a distinct receiver handler and forwarder handler. This makes it possible to build different distributed services in the same peer-to-peer application, letting them coexist and possibly cooperate at runtime.The API also allows to define application-level handlers for other two typical tasks of any dynamic peer-to-peer system, namely, the migration of overlay addresses across peers after new peer arrivals, and the regeneration of missing overlay addresses after peer departures.
The API of NEBLO for the C language follows.
General
typedef u_int64_t neblo_addr;Overlay addresses in NEBLO are 64 bit integers.
typedef u_int16_t neblo_port;Each NEBLO peer can use 2^16 ports.
void neblo_init ( int argc, char **argv );Initialization routine. First one to be invoked by the application.
int neblo_connect ( void );Connect to the overlay, after having set up the handlers (see below).
Messaging
#define NEBLO_MSGSIZE 65535NEBLO messages cannot be longer that this number of bytes.
#define NEBLO_MAXCP 5Max allowed degree of message redundancy (see below).
int neblo_UD_send ( unsigned int copies, neblo_port port, neblo_addr A, void *M, unsigned int len );Unidirectional communication: send message M of len bytes towards overlay address A at the specified port. The message is sent out in multiple copies, that the system will transparently dispatch towards secondary recipients (decided by the system itself) so as to provide a degree of redundancy. Parameter copies cannot exceed NEBLO_MAXCP.
typedef u_int32_t neblo_rhandle; #define R_NONE ((neblo_rhandle)(-1)) int neblo_REQ_send ( unsigned int copies, neblo_port port, neblo_addr A, void *M, unsigned int len, void *R[], unsigned int max_len, neblo_rhandle *H ); int neblo_RES_wait ( neblo_rhandle H, int kind ); int neblo_RES_test ( neblo_rhandle H, int kind );Request-response non blocking communication, split-phase fashion: send out a request message M of len bytes and wait/test for the arrival of the corresponding response. Message M is sent towards overlay address A at the specified port in multiple copies (for redundancy). Parameter copies cannot exceed NEBLO_MAXCP.
Due to redundant requests, multiple response messages might be expected back. To this end, the routine registers a vector R of receive buffers, rather than just one single buffer. The vector must count copies elements. It is allowed to have all the elements of R point to the same application buffers; this is useful when the application is not interested in keeping all responses (for instance, because responses are assumed to be all the same). Parameter max_len specifies the maximum allowed size for each response.
A value for the handle H is also provided, that should be used later with the neblo_RES_wait() and neblo_RES_test() to wait/test for the response arrival.
In function neblo_RES_wait(), if kind is set to NEBLO_WAITANY then the routine will sleep until a response has arrived (returned value is 0). Otherwise, kind may be set to NEBLO_WAITALL and in this case the routine will sleep untill all responses have arrived (returned value is 0 again). In all cases the waiting is limited by a system-defined timeout (returned value is negative when timed out).
Function neblo_RES_test() will not sleep; it returns 1 if a response, or rather all responses, have arrived; negative if timed out; 0 otherwise.
Receiver and forwarder handlers
typedef int ( * ) ( neblo_port P, neblo_addr A, void *M, unsigned int *len ) handler;The type definition of a message handler: a (pointer to a) function taking as parameters all the relevant information concerning the arrived message (the destination port P and address A, and the message M itself). The handler can overwrite the message; this is especially useful with request-response communications: at the recipient, the request is overwritten by the response, so that the runtime system can then returns the response back to source. Also the message size can change, so as to reflect the actual response size.
The value returned by a handler is interpreted by the runtime system; in case of a negative return value, the message that raised a call to the handler will not be forwarded (this holds with forwarder handlers) or no response is given back (this holds with receiver handler of request/response messaging).
void neblo_set_receiver_handler_UD ( neblo_port P, handler rec_handler ); void neblo_set_forwarder_handler_UD ( neblo_port P, handler forw_handler );Routines invoked by the application at startup time, after neblo_init() and before neblo_connect(), to attach a handler to port P so as to manage unidirectional (UD) messages for port P.
The runtime system of each peer, once detected a message of unidirectional kind arrived at port P, evaluates whether the peer itself is recipient or not; in the former case it runs rec_handler, otherwise it runs forw_handler.
In all cases it is the runtime system that also passes the appropriate parameters (port, address, and message) to the handler.
The originator of a message is also a forwarder, if it is not also the recipient.
void neblo_set_receiver_handler_REQ ( neblo_port P, handler rec_handler ); void neblo_set_forwarder_handler_REQ ( neblo_port P, handler forw_handler_req ); void neblo_set_forwarder_handler_RES ( neblo_port P, handler forw_handler_res );Routines invoked by the application at startup time, after neblo_init() and before neblo_connect(), to attach a handler to port P so as to manage request-response messages for port P.
The runtime system of each peer, once detected a request message arrived at port P, evaluates whether the peer itself is recipient or not; in the former case it runs rec_handler, otherwise it runs forw_handler_req.
However, if the detected message is a response to a previously seen request (see neblo_REQ_send}(), neblo_RES_wait(), and neblo_RES_test() above), the runtime system runs forw_handler_res, unless the peer is the originator of the previously seen request, in what case the message emerges to the application without running any handler.
In all cases it is the runtime system that also passes the appropriate parameters (port, address, and message) to the handler.
The originator of a message is also a forwarder, if it is not also the recipient.
Migration and regeneration handlers
typedef int ( * ) ( neblo_addr A, neblo_addr B ) manager;The type definition of a manager: a special kind of handler that runs in case of regeneration of lost overlay addresses (caused by peer departures) or migration of overlay addresses from peer to peer (due to peer arrivals). As with message handlers, the managers are application-level functions that define the behaviour of the application in the two events of regeneration and migration of overlay addresses.
The relevant information in both cases is an address range [A, B], to be passed to the managers by the runtime system upon their invocation.
void neblo_set_refresh_handler ( manager refresh_man ); void neblo_set_clean_handler ( manager clean_man );Routines invoked by the application at startup time, after neblo_init() and before neblo_connect(), to register handlers for regeneration or migration of an address range of the overlay.
The regeneration handler allows to recreate missing overlay addresses. The runtime system of a given peer P, once detected a ``hole'' [A, B] in its local interval of overlay addresses, attempts to seal the address interval by rebuilding the lost application state that was associated to the missing addresses. To do so, it sends hidden requests (invisible to the application) to the peers who act as secondary recipients for the addresses within [A, B]. The runtime system of such peers will then run the manager refresh_man ( A, B ), which can make use of the auxiliary routine neblo_UD_send_h() (see below) in order to send out the backup of lost state; the routing algorithm of NEBLO will implicitly deliver those messages to peer P.
The migration handler allows to manage the migration of overlay addresses from one peer to another (adjacent) after the arrival of a new peer. As soon as a migrant address range [A, B] has changed ownership from peer P to peer Q, because of the arrival of Q in the overlay, the runtime system on peer P first runs refresh_man ( A, B ) to send out copy of the state currently associated to [A, B]; the routing algorithm of NEBLO will implicitly deliver those messages to Q. It then runs clean_man ( A, B ) to remove such state from P.
Auxiliaries
int neblo_in_range ( neblo_addr X, neblo_addr A, neblo_addr B );Probe the inclusion of a given overlay address X in the address range [A, B]. The result is boolean. Useful within regeneration handlers.
int neblo_UD_send_h ( neblo_port port, neblo_addr A, void *M, unsigned int msg_len );A variant of neblo_UD_send(), for exclusive use in regeneration handlers.
int neblo_REQ_send_h ( neblo_port port, neblo_addr A, void *M, unsigned int len, void *R[], unsigned int max_len, neblo_rhandle *H );A variant of neblo_REQ_send(), for exclusive use in regeneration handlers. The vector R<\tt> of pointers to individual response buffers should only count one entry (no multiple copies allowed).
int neblo_my_id( neblo_addr *R );Variable R takes the id. of the caller peer. Returns 0 on success, -1 otherwise.
void neblo_lock( void );Takes the internal mutex, used by NEBLO to protect potentially non-thread-safe routines (like malloc(), free(), and all the OpenSSL routines). NEBLO applications should always use this mutex to protect their own invocations of any OpenSSL routines as well as malloc() and free().
int neblo_trylock( void );Same as above but non blocking. Returns 1 if internal mutex already taken by another thread, 0 otherwise (in the latter case, the mutex is taken by caller).
void neblo_unlock( void );Releases the NEBLO internal mutex.