Difference between revisions of "Containers"

Revision as of 22:34, 9 November 2007

Some (incomplete) ramblings about netsnmp_containers...

Introduction

Containers are a generic data interface, similar to a database. Like a database, you use an index (aka key) to access and sort the data. Containers use a compare function provided by the user to determine the sort order. The function is called with a pointer to two data items, and must return a value indicating which of the two has the lesser index value.

Types of Containers

There are several base types of containers:

sorted_singly_linked_list
unsorted_singly_linked_list
lifo (a last in, first out stack)
fifo (a first in, first out stack)
binary_array
null (for testing)

Some of these have aliases:

table_container (binary_array)
linked_list (sorted_singly_linked_list)
ssll_container (sorted_singly_linked_list)

Some types also come with a comparison routine other than the usual OID index:

string or string:binary_array (binary_array with a string comparison function)

Developer Tip:
 New types are registered via netsnmp_container_register() and netsnmp_container_register_with_compare().

Getting a Container

To get a container of a particular type,

netsnmp_container_find("type1:type2");

This will search for a container of "type1", and if not found, of "type2". It is a good idea to specify a custom name each time that you use a container. In the future, it will be possible to specify that certain tokens should map to certain container types. For example, say you are using a container while implementing the XyzWidgetMib, and you wanted to use a linked list, you could create your container like so:

container = netsnmp_container_find("XyzWidgetMib:linked_list");

This would use a linked list container, and (once implemented), allow runtime-configuration to change the container type to a binary array, should the linked list perform poorly with a large data set.

MIB indexes vs Container indexes

SNMP tables have indexes. A table index may have multiple components which, taken together, uniquely identify a row.

Now, when the two are used together, it easy to get confused. All of the MIB indexes, taken together, are the primary container index. Even if a MIB table has a dozen indexes, the container only has one.

To clarify futher, here's an example. Let's say we are creating a SNMP interface to a hotel reservation system. The (simplified) table looks like this:

guestTable

 guestEntry INDEX { building, room }
   building INTEGER
   room     INTEGER
   name     STRING

So, the MIB has two indexes, the building and room numbers. The primary MIB index is the building, and the secondary MIB index is the room. However, the primary index of the container will be the combined index OIDs (building.room).

Comparison routines

The default compare routine for containers assumes that the data record's first component is a netsnmp_index, so when using an OID as a key, you don't need to provide a comparison routine. By providing a second compare function, you can to access the data in a different order. If you wanted to provide your own compare routine, the primary container index compare function might look something like this:

 int
 _compare_room(guestTable *lh_guest, guestTable *rh_guests)
 {
   /* compare building, then room */
   if(lh_guest->building == rh_guest->building) {
       if(lh_guest->room == rh_guest->room)
          return 0;
       else {
          if(lh_guest->room < rh_guest->room)
             return -1;
          else
             return 1;
       }
   }
   else {
      if(lh_guest->building < rh_guest->building)
        return -1;
      else
        return 1;
   }
 }

So we now have a container with the guest data, and we can look up data by building and room number.

netsnmp_container *
netsnmp_container_init(void)
{
    netsnmp_container *container;

    container = netsnmp_container_find("guest room:table_container");
    if (NULL == container)
        return NULL;
    container->container_name = strdup("room container");
    container->compare = (netsnmp_container_compare*)_compare_room;

    return container;
}

Subcontainers

A container can have a sub-container. There are two types of sub-containers: secondary indexes, and subsets.

Secondary Indexes

So we now have a container with the guest data, and we can look up data by building and room number. What if our application now needs to generate a guest report, but sorted by name? We have the data, but in the wrong sort order. This is where you would use a secondary index to the container. The new compare might look like this:

int
_compare_names(guestTable *lh_guest, guestTable *rh_guests)
{
   /* compare name, then building and room */
   int rc = strcmp(lh_guest->name,rh_guest->name);
   if(rc != 0)
      return rc;

   if(lh_guest->building == rh_guest->building) {
      if(lh_guest->room == rh_guest->room)
           return 0;
      else {
         if(lh_guest->room < rh_guest->room)
            return -1;
         else
            return 1;
      }
   }
   else {
      if(lh_guest->building < rh_guest->building)
         return -1;
      else
         return 1;
   }
}

Setting up the container with two indexes would look something like this:

netsnmp_container *
netsnmp_container_init(void)
{
    netsnmp_container *container1, *container2;

    /*
     * create 2 containers.
     */
    container1 = netsnmp_container_find("guest room:table_container");
    if (NULL == container1)
        return NULL;
    container1->container_name = strdup("room container");

    container2 = netsnmp_container_find("guest name:table_container");
    if (NULL == container2) {
        CONTAINER_FREE(container1);
        return NULL;
    }
    container2->container_name = strdup("guest container");
    container2->compare = (netsnmp_container_compare*)_compare_name;

    netsnmp_container_add_index(container1, container2);

    return container1;
}

Sub-sets

Sometimes, it can be useful to access a sub-set of a data set. For example, if you have a container of all the IP addresses for some system, it might be useful to be able to work with only the IPv4 addresses, without having to iterate over all the addresses and ignore the IPv6 (or other) addresses. This can be done with a secondary index with a filter (only avaliable in more recent releases).

Container Operations

Several functions are provided for easy use of containers. These functions should be used is almost all cases, since the do the 'right' thing for special cases such as insert filters and subcontainers.

Creating a container

Adding and removing data

CONTAINER_INSERT

The container insert function doesn't need much explanation. It inserts an item into a container. Note that an insert_filter may filter items from bein inserted into one or more sub-containers.

CONTAINER_REMOVE

The container remove function doesn't need much explanation. It removes and item from a container. It does not free the item.

CONTAINER_CLEAR

CONTAINER_CLEAR(container, callback, context)

The container clear function is a special function that is optimized for clearing a container. It is effectively the same as iterating over the container and calling CONTAINER_REMOVE for each item, calling the specified callback function as you go. The context parameter is for you use, and is also passed to the callback function. Pass NULL if you don't need any context.

/* clear a container that only needs a simple free() call on each item. */
CONTAINER_CLEAR(container,
                (netsnmp_container_obj_func*)netsnmp_container_simple_free,
                 NULL);

/*
 * here is an example of a function that could be passed for a more
 * complex free
 */
static void 
_complex_free(void *data, void *context)
{
    struct mystruct *s = (struct mystruct *)data;

    if (data == NULL)
	return;
    free(s->some_ptr);
    free(s);
 }

Finding data in a container

CONTAINER_FIND

Iterating over items in a container

CONTAINER_FOR_EACH

You can define a callback function and pass it to CONTAINER_FOR_EACH, like so:

void my_callback(void *data, void* context) {
   fprintf(stdout,"callback for data %x and context %x\n", data, context);
}

int main(int argc, char **argv) {
   void              *my_context = 0xdeadbeef;
   netsnmp_container *c = netsnmp_container_find("binary_array");

   /* insert data in container */
   CONTAINER_FOR_EACH(c, my_callback, my_context);
}

The function my_callback will be called for each data item in the container.

CONTAINER_FIRST and CONTAINER_NEXT

You can use CONTAINER_FIRST and CONTAINER_NEXT to iterate over a container:

netsnmp_container *c = netsnmp_container_find("binary_array");
void              *data;

for( data = CONTAINER_FIRST(c); data; data = CONTAINER_NEXT(c, data)) {
   fprintf(stdout, "data %x\n", data);
}

The disadvantage of this is that CONTAINER_NEXT does a search to find the previous item before it can return the next item. For a large container, this can be horribly inefficient. Using a container iterator, as described below, is a much better idea.

CONTAINER_ITERATOR(x)

Some containers support iterating over the contents, like so:

netsnmp_container *c = get_my_container();
netsnmp_container_it *it;
void *data;

it = CONTAINER_ITERATOR(c);
if (NULL == it)
   return -1;
for( data = ITERATOR_FIRST(it); data ; data = ITERATOR_NEXT(it) {
   fprintf(stdout, "data %x\n", data);
}
ITERATOR_RELEASE(it);

Miscellaneous operations

CONTAINER_SIZE

This function returns the number of items currently in the container.

Releasing a container

CONTAINER_FREE

The container free function releases internal resources used by the container and frees the container itself. It does not release the memory used by any items contained within the container. You must use either the CONTAINER_REMOVE or CONTAINER_CLEAR functions to release resources used by the items in the container before calling CONTAINER_FREE.

Advanced topics

CONTAINER_SET_OPTIONS and CONTAINER_CHECK_OPTION

CONTAINER_SET_OPTIONS copies the specified flags to the container options. It DOES NOT OR the flags with existing flags, so it allows one to overwrite existing flags. Unfortunately, there doesn't seem to be a way to get the existing flags, making addition of a single flag difficult.

CONTAINER_CHECK_OPTION returns 1 if the specified flags are set, or 0 if they are not set.

CONTAINER_GET_SUBSET

Note: get subset returns allocated memory (netsnmp_void_array). User is responsible for releasing this memory (free(array->array), free(array)). DO NOT FREE ELEMENTS OF THE ARRAY, because they are the same pointers stored in the container.

CONTAINER_COMPARE(x,l,r)

tbd

Shared Container/cache

Often multiple MIB implementations will need to use the same data. Instead of having redundant copies of the data, they can use the same container. This example code is based on the ifTable/ifXTable code, and uses the cache helper as well.

/** ifXTable uses the same cache (and thus container) as ifTable */
if_ctx->cache =
    netsnmp_cache_find_by_oid(ifTable_oid, ifTable_oid_size);
if (NULL != if_ctx->cache) {
    if_ctx->container = (netsnmp_container *) if_ctx->cache->magic;

@@ Line 239: / Line 239: @@
 ==== CONTAINER_FIRST and CONTAINER_NEXT ====
+You can use <code>CONTAINER_FIRST</code> and <code>CONTAINER_NEXT</code> to iterate over a container:
+ netsnmp_container *c = netsnmp_container_find("binary_array");
+ void              *data;
+ for( data = CONTAINER_FIRST(c); data; data = CONTAINER_NEXT(c, data)) {
+    fprintf(stdout, "data %x\n", data);
+ }
+The disadvantage of this is that <code>CONTAINER_NEXT</code> does a search to find the previous item before it can return the next item. For a large container, this can be horribly inefficient. Using a container iterator, as described below, is a much better idea.
 ==== CONTAINER_ITERATOR(x) ====