ONL Tutorial

The ONL Tutorial

Tutorial >> More Plugins

TOC

Copy Packets

The ONL testbed does not recognize multicast IP packets. However, this page describes how to copy and forward a packet to multiple output ports; i.e., multicast a packet at an ingress port. The multicast plugin is installed at an ingress port where it puts copies of a packet into those VOQs corresponding to the desired outputs.

Allocate memory for the packet buffer
Fill in the buffer with the packet copy
Modify the intraport shim so that it contains the desired VOQ and the reclassify bit is 0 (to prevent packet reclassification when it is passed to the FPX).
Forward the packet

Recall that each packet has a shim prepended to it that carries along information that is used by the FPX. The shim is an intraport shim when it is attached to an IP packet while still in a port processor, and it is an interport shim when it is attached to an IP packet that is transmitted from an input port to an output port.

The multicast_handle_message Function

The variable inst->destVec is a bit vector whose eight rightmost bits indicate with a 1 whether an output port should get a copy of the incoming packet. The bits are scanned from right-to-left as i is iterated through the VOQ numbers 0 to 7. A 1 in the ith position (from the right) indicates that VOQ i should get a copy of the packet.

The destination vector value is initially 0 (indicating no output ports should get a copy), but its value can be set by sending a message to the plugin. Lines 21-29 handle the two messages that add a VOQ and remove a VOQ from the destination vector inst->destVec. For example, in order to add port 3 to the list of output ports that should receive a packet, use the RLI to send a command of 2 and argument of 3 to the plugin instance. This will result in the destination vector being OR'ed with the value of "1<<3" (or 0x8).

 1  int multicast_handle_msg(
 2    struct rp_instance *this,	// pointer to instance structure
 3    void *buf,		// buffer containing integer values
 4    				// buf[0]=instance id
 5    				// buf[1]=command type
 6    				// buf[2]=first parameter value
 7    				// buf[3]=second parameter value ...
 8    u_int8_t flags,		// ignore
 9    u_int8_t seq,		// sequence number of message
10    u_int8_t *len		// number of values in buf
11  ) {
12    struct multicast_instance *inst = (struct multicast_instance *) this;
13    u_int32_t	*vals	= (u_int32_t *) buf;
14    u_int32_t	id 	= (u_int32_t) ntohl(vals[0]);
15    u_int32_t	typ	= (u_int32_t) ntohl(vals[1]);
16    int		i;
17  
18    if (typ == 1) {			// Return destVec
19      vals[0] = (u_int32_t) htonl(inst->destVec);
20      *len = 1*sizeof(u_int32_t);
21    } else if (typ == 2) {		// add VOQ
22      i = ntohl(vals[2]);
23      if (i >= 0 && i < 8) inst->destVec |= 1 << i;
24      *len = 0*sizeof(u_int32_t);
25    } else if (typ == 3) {		// remove VOQ
26      i = ntohl(vals[2]);
27      if (i >= 0 && i < 8) inst->destVec &= ~(1 << i);
28      *len = 0*sizeof(u_int32_t);
29    }
30  
31    return 0;
32  }

The multicast_handle_packet Function

The multicast_handle_packet routine uses three new functions that operate on the shim fields of a packet buffer:

msr_pkt_shim(buffer): Return the address of the shim in a buffer.
msr_vin_make(p, 0): Construct a Virtual INterface (VIN) field for port p.
msr_shim_set_ovin(shim, vin): Set the Output VIN (OVIN) field in a shim to a vin.

The OVIN field in the intraport shim indicates to the FPX the destination output port (and therefore VOQ).

 1  void copy(char *src, char *dst, int len) {	// Copy len bytes from src to dst.
 2    while (len-- > 0) { *dst++ = *src++; }
 3  }
 4  
 5  void multicast_handle_packet(
 6  	struct rp_instance *this,	// pointer to instance structure
 7  	void *bufferList		// pointer to list of packet buffers
 8  ) {
 9    struct multicast_instance	*inst;
10    msr_bufhdr_t			*buffer, *newBuf;
11    MSR_Shim_t			*shim, *newShim;
12    int				i, first, select, len;
13  
14    inst = (struct multicast_instance *) this;
15    buffer = msr_firstBuffer(bufferList);
16    shim = msr_pkt_shim(buffer);
17  
18    if (inst->destVec == 0) {	// remove packet from input list
19      msr_removeBuffer(bufferList, buffer);     
20      PLUGIN_IPBUF_FREE_FCT(buffer);
21      return;
22    }
23  
24    len = msr_iplen(msr_pkt_iph(buffer));
25    first = 1;
26  
27    for (i = 0, select = 1; i < 8; i++, select <<= 1) {
28      if ((inst->destVec & select) == 0)		continue;
29      if (first == 1) {		// don't need to make a copy
30        first = 0;
31        msr_shim_set_ovin(shim, msr_vin_make(i, 0));	// set VOQ in shim to i
32        continue;
33      }
34      // create copy to send to output i
35      if ((newBuf = PLUGIN_IPBUF_ALLOC_FCT()) == NULL)
36					return; // terminate early if no buffers
37      msr_addBuffer(bufferList, newBuf);
38  
39      // copy original pkt including shim (sets reclassify bit to 0)
40      newShim = msr_pkt_shim(newBuf);
41      copy( (char *) shim, (char *) newShim, len + MSR_FPX_SHIM_SZ);
42      msr_shim_set_ovin(newShim, msr_vin_make(i, 0));	// set VOQ in shim to i
43    }
44  }

The multicast_handle_packet code tour:

Lines 1-3: The copy function is used to copy the entire incoming packet buffer (including the shim) to a new packet buffer before forwarding.
Line 15-16: buffer points to the incoming packet that will be multicasted. shim points to the beginning of the shim field in the buffer.
Lines 18-22: We remove the buffer from the incoming packet buffer list and free the memory if the destination vector indicates no output ports.
Lines 27-43: This is the loop where all of the multicasting is being done. It iterates through the output ports from 0 to 7 while it examines the bits of inst->destVec from right-to-left. A 1 bit indicates that the port should get a copy of the incoming packet.
Line 28: We go to the next port if the ith bit from the right is 0.
Lines 29-33: We will send the original packet buffer (except with its OVIN changed to i) to the first selected output destination. Note that this packet buffer is still on the incoming packet buffer list and will get forwarded when we return from this function.
Lines 35-42: For all selected output ports after the first one encountered, do four things:
1. Allocate a packet buffer;
2. Place the buffer on the incoming packet buffer list;
3. Copy the entire packet including the shim to this new buffer; and
4. Set the OVIN to port i.
When control returns from multicast_handle_packet, the packets on the buffer list will be forwarded. Note that there is more to Line 41 (which copies the shim and IP packet) than may be apparent. When you allocate a packet buffer (PLUGIN_IPBUF_ALLOC_FCT), the shim area is initialized so that the packet will get reclassified when it is returned to the FPX. This would not be desirable since the packet would be returned to the SPC. But since Line 41 copies the shim from the original packet (which has its reclassify bit equal to 0) into the new shim, the reclassify bit is effectively set to 0. Alternatively, we could have explicitly set the reclassify bit to 0.

Tutorial >> More Plugins	TOC