USERSPACE MAD ACCESS

Device files

  Each port of each InfiniBand device has a "umad" device attached.
  For example, a two-port HCA will have two devices, while a switch
  will have one device (for switch port 0).

Creating MAD agents

  A MAD agent can be created by filling in a struct ib_user_mad_reg_req
  and then calling the IB_USER_MAD_REGISTER_AGENT ioctl on a file
  descriptor for the appropriate device file.  If the registration
  request succeeds, a 32-bit id will be returned in the structure.
  For example:

	struct ib_user_mad_reg_req req = { /* ... */ };
	ret = ioctl(fd, IB_USER_MAD_REGISTER_AGENT, (char *) &req);
        if (!ret)
		my_agent = req.id;
	else
		perror("agent register");

  Agents can be unregistered with the IB_USER_MAD_UNREGISTER_AGENT
  ioctl.  Also, all agents registered through a file descriptor will
  be unregistered when the descriptor is closed.

Receiving MADs

  MADs are received using read().  The buffer passed to read() must be
  large enough to hold at least one struct ib_user_mad.  For example:

	struct ib_user_mad mad;
	ret = read(fd, &mad, sizeof mad);
	if (ret != sizeof mad)
		perror("read");

  In addition to the actual MAD contents, the other struct ib_user_mad
  fields will be filled in with information on the received MAD.  For
  example, the remote LID will be in mad.lid.

  If a send times out, a receive will be generated with mad.status set
  to ETIMEDOUT.  Otherwise when a MAD has been successfully received,
  mad.status will be 0.

  poll()/select() may be used to wait until a MAD can be read.

Sending MADs

  MADs are sent using write().  The agent ID for sending should be
  filled into the id field of the MAD, the destination LID should be
  filled into the lid field, and so on.  For example:

	struct ib_user_mad mad;

	/* fill in mad.data */

	mad.id  = my_agent;	/* req.id from agent registration */
	mad.lid = my_dest;	/* in network byte order... */
	/* etc. */

	ret = write(fd, &mad, sizeof mad);
	if (ret != sizeof mad)
		perror("write");

/dev files

  To create the appropriate character device files automatically with
  udev, a rule like

    KERNEL="umad*", NAME="infiniband/%k"

  can be used.  This will create a device node named

    /dev/infiniband/umad0

  for the first port, and so on.  The InfiniBand device and port
  associated with this device can be determined from the files

    /sys/class/infiniband_mad/umad0/ibdev
    /sys/class/infiniband_mad/umad0/port
