Operational Data, RPCs and Notifications
Operational data
Writing API-agnostic code for YANG-modeled operational data is challenging. Sysrepo, for instance, has completely different API to fetch operational data. So how can we write API-agnostic callbacks that can be used by both the Sysrepo plugin, and any other northbound client that might be written in the future?
As an additional requirement, the callbacks must be designed in a way that makes in-place XPath filtering possible. As an example, a management client might want to retrieve only a subset of a large YANG list (e.g. a BGP table), and for optimal performance it should be possible to filter out the unwanted elements locally in the managed devices instead of returning all elements and performing the filtering on the management application.
To meet all these requirements, the four callbacks below were introduced in the northbound architecture:
/*
* Operational data callback.
*
* The callback function should return the value of a specific leaf or
* inform if a typeless value (presence containers or leafs of type
* empty) exists or not.
*
* xpath
* YANG data path of the data we want to get
*
* list_entry
* pointer to list entry
*
* Returns:
* pointer to newly created yang_data structure, or NULL to indicate
* the absence of data
*/
struct yang_data *(*get_elem)(const char *xpath, void *list_entry);
/*
* Operational data callback for YANG lists.
*
* The callback function should return the next entry in the list. The
* 'list_entry' parameter will be NULL on the first invocation.
*
* list_entry
* pointer to a list entry
*
* Returns:
* pointer to the next entry in the list, or NULL to signal that the
* end of the list was reached
*/
void *(*get_next)(void *list_entry);
/*
* Operational data callback for YANG lists.
*
* The callback function should fill the 'keys' parameter based on the
* given list_entry.
*
* list_entry
* pointer to a list entry
*
* keys
* structure to be filled based on the attributes of the provided
* list entry
*
* Returns:
* NB_OK on success, NB_ERR otherwise
*/
int (*get_keys)(void *list_entry, struct yang_list_keys *keys);
/*
* Operational data callback for YANG lists.
*
* The callback function should return a list entry based on the list
* keys given as a parameter.
*
* keys
* structure containing the keys of the list entry
*
* Returns:
* a pointer to the list entry if found, or NULL if not found
*/
void *(*lookup_entry)(struct yang_list_keys *keys);
These callbacks were designed to provide maximum flexibility. Each callback does one and only one task, they are indivisible primitives that can be combined in several different ways to iterate over operational data. The extra flexibility certainly has a performance cost, but it’s the price to pay if we want to expose FRR operational data using several different management interfaces (e.g. Sysrepo+Netopeer2). In the future it might be possible to introduce optional callbacks that do things like returning multiple objects at once. They would provide enhanced performance when iterating over large lists, but their use would be limited by the northbound plugins that can be integrated with them.
The [[Plugins - Writing Your Own]] page explains how the northbound plugins can fetch operational data using the aforementioned northbound callbacks, and how in-place XPath filtering can be implemented.
Example
Now let’s move to an example to show how these callbacks are implemented in practice. The following YANG container is part of the ietf-rip module and contains operational data about RIP neighbors:
container neighbors {
description
"Neighbor information.";
list neighbor {
key "address";
description
"A RIP neighbor.";
leaf address {
type inet:ipv4-address;
description
"IP address that a RIP neighbor is using as its
source address.";
}
leaf last-update {
type yang:date-and-time;
description
"The time when the most recent RIP update was
received from this neighbor.";
}
leaf bad-packets-rcvd {
type yang:counter32;
description
"The number of RIP invalid packets received from
this neighbor which were subsequently discarded
for any reason (e.g. a version 0 packet, or an
unknown command type).";
}
leaf bad-routes-rcvd {
type yang:counter32;
description
"The number of routes received from this neighbor,
in valid RIP packets, which were ignored for any
reason (e.g. unknown address family, or invalid
metric).";
}
}
}
We know that this is operational data because the neighbors
container is within the state
container, which has the
config false;
property (which is applied recursively).
As expected, the gen_northbound_callbacks
tool also generates
skeleton callbacks for nodes that represent operational data:
{
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor",
.cbs.get_next = ripd_state_neighbors_neighbor_get_next,
.cbs.get_keys = ripd_state_neighbors_neighbor_get_keys,
.cbs.lookup_entry = ripd_state_neighbors_neighbor_lookup_entry,
},
{
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor/address",
.cbs.get_elem = ripd_state_neighbors_neighbor_address_get_elem,
},
{
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor/last-update",
.cbs.get_elem = ripd_state_neighbors_neighbor_last_update_get_elem,
},
{
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor/bad-packets-rcvd",
.cbs.get_elem = ripd_state_neighbors_neighbor_bad_packets_rcvd_get_elem,
},
{
.xpath = "/frr-ripd:ripd/state/neighbors/neighbor/bad-routes-rcvd",
.cbs.get_elem = ripd_state_neighbors_neighbor_bad_routes_rcvd_get_elem,
},
The /frr-ripd:ripd/state/neighbors/neighbor
list within the
neighbors
container has three different callbacks that need to be
implemented. Let’s start with the first one, the get_next
callback:
static void *ripd_state_neighbors_neighbor_get_next(void *list_entry)
{
struct listnode *node;
if (list_entry == NULL)
node = listhead(peer_list);
else
node = listnextnode((struct listnode *)list_entry);
return node;
}
Given a list entry, the job of this callback is to find the next element
from the list. When the list_entry
parameter is NULL, then the first
element of the list should be returned.
ripd uses the rip_peer
structure to represent RIP neighbors, and
the peer_list
global variable (linked list) is used to store all RIP
neighbors.
In order to be able to iterate over the list of RIP neighbors, the
callback returns a listnode
variable instead of a rip_peer
variable. The listnextnode
macro can then be used to find the next
element from the linked list.
Now the second callback, get_keys
:
static int ripd_state_neighbors_neighbor_get_keys(void *list_entry,
struct yang_list_keys *keys)
{
struct listnode *node = list_entry;
struct rip_peer *peer = listgetdata(node);
keys->num = 1;
(void)inet_ntop(AF_INET, &peer->addr, keys->key[0].value,
sizeof(keys->key[0].value));
return NB_OK;
}
This one is easy. First, we obtain the RIP neighbor from the
listnode
structure. Then, we fill the keys
parameter according
to the attributes of the RIP neighbor. In this case, the neighbor
YANG list has only one key: the neighbor IP address. We then use the
inet_ntop()
function to transform this binary IP address into a
string (the lingua franca of the FRR northbound).
The last callback for the neighbor
YANG list is the lookup_entry
callback:
static void *
ripd_state_neighbors_neighbor_lookup_entry(struct yang_list_keys *keys)
{
struct in_addr address;
yang_str2ipv4(keys->key[0].value, &address);
return rip_peer_lookup(&address);
}
This callback is the counterpart of the get_keys
callback: given an
array of list keys, the associated list entry should be returned. The
yang_str2ipv4()
function is used to convert the list key (an IP
address) from a string to an in_addr
structure. Then the
rip_peer_lookup()
function is used to find the list entry.
Finally, each YANG leaf inside the neighbor
list has its associated
get_elem
callback:
/*
* XPath: /frr-ripd:ripd/state/neighbors/neighbor/address
*/
static struct yang_data *
ripd_state_neighbors_neighbor_address_get_elem(const char *xpath,
void *list_entry)
{
struct rip_peer *peer = list_entry;
return yang_data_new_ipv4(xpath, &peer->addr);
}
/*
* XPath: /frr-ripd:ripd/state/neighbors/neighbor/last-update
*/
static struct yang_data *
ripd_state_neighbors_neighbor_last_update_get_elem(const char *xpath,
void *list_entry)
{
/* TODO: yang:date-and-time is tricky */
return NULL;
}
/*
* XPath: /frr-ripd:ripd/state/neighbors/neighbor/bad-packets-rcvd
*/
static struct yang_data *
ripd_state_neighbors_neighbor_bad_packets_rcvd_get_elem(const char *xpath,
void *list_entry)
{
struct rip_peer *peer = list_entry;
return yang_data_new_uint32(xpath, peer->recv_badpackets);
}
/*
* XPath: /frr-ripd:ripd/state/neighbors/neighbor/bad-routes-rcvd
*/
static struct yang_data *
ripd_state_neighbors_neighbor_bad_routes_rcvd_get_elem(const char *xpath,
void *list_entry)
{
struct rip_peer *peer = list_entry;
return yang_data_new_uint32(xpath, peer->recv_badroutes);
}
These callbacks receive the list entry as parameter and return the
corresponding data using the yang_data_new_*()
wrapper functions.
Not much to explain here.
Iterating over operational data without blocking the main pthread
One of the problems we have in FRR is that some “show” commands in the CLI can take too long, potentially long enough to the point of triggering some protocol timeouts and bringing sessions down.
To avoid this kind of problem, northbound clients are encouraged to do one of the following:
Create a separate pthread for handling requests to fetch operational data.
Iterate over YANG lists and leaf-lists asynchronously, returning a maximum number of elements per time instead of returning all elements in one shot.
In order to handle both cases correctly, the get_next
callbacks need
to use locks to prevent the YANG lists from being modified while they
are being iterated over. If that is not done, the list entry returned by
this callback can become a dangling pointer when used in another
callback.
Currently the Sysrepo plugin runs only in the main pthread. The plan in the short-term is to introduce a separate pthread only for handling operational data, and use the main pthread only for handling configuration changes, RPCs and notifications.
RPCs and Actions
The FRR northbound supports YANG RPCs and Actions through the rpc()
callback, which is documented as follows in the lib/northbound.h file:
/*
* RPC and action callback.
*
* Both 'input' and 'output' are lists of 'yang_data' structures. The
* callback should fetch all the input parameters from the 'input' list,
* and add output parameters to the 'output' list if necessary.
*
* xpath
* xpath of the YANG RPC or action
*
* input
* read-only list of input parameters
*
* output
* list of output parameters to be populated by the callback
*
* Returns:
* NB_OK on success, NB_ERR otherwise
*/
int (*rpc)(const char *xpath, const struct list *input,
struct list *output);
Note that the same callback is used for both RPCs and actions, which are
essentially the same thing. In the case of YANG actions, the xpath
parameter can be consulted to find the data node associated to the
operation.
As part of the northbound retrofitting process, it’s suggested to model
some EXEC-level commands using YANG so that their functionality is
exposed to other management interfaces other than the CLI. As an
example, if the clear bgp
command is modeled using a YANG RPC, and a
corresponding rpc
callback is written, then it should be possible to
clear BGP neighbors using NETCONF and RESTCONF with that RPC (the Sysrepo
plugin has full support for YANG RPCs and actions).
Here’s an example of a very simple RPC modeled using YANG:
rpc clear-rip-route {
description
"Clears RIP routes from the IP routing table and routes
redistributed into the RIP protocol.";
}
This RPC doesn’t have any input or output parameters. Below we can see
the implementation of the corresponding rpc
callback, whose skeleton
was automatically generated by the gen_northbound_callbacks
tool:
/*
* XPath: /frr-ripd:clear-rip-route
*/
static int clear_rip_route_rpc(const char *xpath, const struct list *input,
struct list *output)
{
struct route_node *rp;
struct rip_info *rinfo;
struct list *list;
struct listnode *listnode;
/* Clear received RIP routes */
for (rp = route_top(rip->table); rp; rp = route_next(rp)) {
list = rp->info;
if (list == NULL)
continue;
for (ALL_LIST_ELEMENTS_RO(list, listnode, rinfo)) {
if (!rip_route_rte(rinfo))
continue;
if (CHECK_FLAG(rinfo->flags, RIP_RTF_FIB))
rip_zebra_ipv4_delete(rp);
break;
}
if (rinfo) {
RIP_TIMER_OFF(rinfo->t_timeout);
RIP_TIMER_OFF(rinfo->t_garbage_collect);
listnode_delete(list, rinfo);
rip_info_free(rinfo);
}
if (list_isempty(list)) {
list_delete_and_null(&list);
rp->info = NULL;
route_unlock_node(rp);
}
}
return NB_OK;
}
If the clear-rip-route
RPC had any input parameters, they would be
available in the input
list given as a parameter to the callback.
Similarly, the output
list can be used to append output parameters
generated by the RPC, if any are defined in the YANG model.
The northbound clients (CLI and northbound plugins) have the
responsibility to create and delete the input
and output
lists.
However, in the cases where the RPC or action doesn’t have any input or
output parameters, the northbound client can pass NULL pointers to the
rpc
callback to avoid creating linked lists unnecessarily. We can
see this happening in the example below:
/*
* XPath: /frr-ripd:clear-rip-route
*/
DEFPY (clear_ip_rip,
clear_ip_rip_cmd,
"clear ip rip",
CLEAR_STR
IP_STR
"Clear IP RIP database\n")
{
return nb_cli_rpc("/frr-ripd:clear-rip-route", NULL, NULL);
}
nb_cli_rpc()
is a helper function that merely finds the appropriate
rpc
callback based on the XPath provided in the first argument, and
map the northbound error code from the rpc
callback to a vty error
code (e.g. CMD_SUCCESS
, CMD_WARNING
). The second and third
arguments provided to the function refer to the input
and output
lists. In this case, both arguments are set to NULL since the YANG RPC
in question doesn’t have any input/output parameters.
Notifications
YANG notifations are sent using the nb_notification_send()
function,
documented in the lib/northbound.h file as follows:
/*
* Send a YANG notification. This is a no-op unless the 'nb_notification_send'
* hook was registered by a northbound plugin.
*
* xpath
* xpath of the YANG notification
*
* arguments
* linked list containing the arguments that should be sent. This list is
* deleted after being used.
*
* Returns:
* NB_OK on success, NB_ERR otherwise
*/
extern int nb_notification_send(const char *xpath, struct list *arguments);
The northbound doesn’t use callbacks for notifications because notifications are generated locally and sent to the northbound clients. This way, whenever a notification needs to be sent, it’s possible to call the appropriate function directly instead of finding a callback based on the XPath of the YANG notification.
As an example, the ietf-rip module contains the following notification:
notification authentication-failure {
description
"This notification is sent when the system
receives a PDU with the wrong authentication
information.";
leaf interface-name {
type string;
description
"Describes the name of the RIP interface.";
}
}
The following convenience function was implemented in ripd to send authentication-failure YANG notifications:
/*
* XPath: /frr-ripd:authentication-failure
*/
void ripd_notif_send_auth_failure(const char *ifname)
{
const char *xpath = "/frr-ripd:authentication-failure";
struct list *arguments;
char xpath_arg[XPATH_MAXLEN];
struct yang_data *data;
arguments = yang_data_list_new();
snprintf(xpath_arg, sizeof(xpath_arg), "%s/interface-name", xpath);
data = yang_data_new_string(xpath_arg, ifname);
listnode_add(arguments, data);
nb_notification_send(xpath, arguments);
}
Now sending the authentication-failure YANG notification should be as
simple as calling the above function and provide the appropriate
interface name. The notification will be processed by all northbound
plugins that subscribed a callback to the nb_notification_send
hook.
The Sysrepo plugin, for instance, uses this hook to relay the notifications
to the sysrepod daemon, which can generate NETCONF notifications to subscribed
clients. When no northbound plugin is loaded, nb_notification_send()
doesn’t
do anything and the notifications are ignored.