RDMA_CM(7) Librdmacm Programmer's Manual RDMA_CM(7)NAMErdma_cm - RDMA communication manager.
SYNOPSIS
#include <rdma/rdma_cma.h>
DESCRIPTION
Used to establish communication over RDMA transports.
NOTES
The RDMA CM is a communication manager used to setup reliable, con‐
nected and unreliable datagram data transfers. It provides an RDMA
transport neutral interface for establishing connections. The API is
based on sockets, but adapted for queue pair (QP) based semantics: com‐
munication must be over a specific RDMA device, and data transfers are
message based.
The RDMA CM only provides the communication management (connection set‐
up / teardown) portion of an RDMA API. It works in conjunction with
the verbs API defined by the libibverbs library. The libibverbs
library provides the interfaces needed to send and receive data.
CLIENT OPERATION
This section provides a general overview of the basic operation for the
active, or client, side of communication. A general connection flow
would be:
rdma_create_event_channel
create channel to receive events
rdma_create_id
allocate an rdma_cm_id, this is conceptually similar to a socket
rdma_resolve_addr
obtain a local RDMA device to reach the remote address
rdma_get_cm_event
wait for RDMA_CM_EVENT_ADDR_RESOLVED event
rdma_ack_cm_event
ack event
rdma_create_qp
allocate a QP for the communication
rdma_resolve_route
determine the route to the remote address
rdma_get_cm_event
wait for RDMA_CM_EVENT_ROUTE_RESOLVED event
rdma_ack_cm_event
ack event
rdma_connect
connect to the remote server
rdma_get_cm_event
wait for RDMA_CM_EVENT_ESTABLISHED event
rdma_ack_cm_event
ack event
Perform data transfers over connection
rdma_disconnect
tear-down connection
rdma_get_cm_event
wait for RDMA_CM_EVENT_DISCONNECTED event
rdma_ack_cm_event
ack event
rdma_destroy_qp
destroy the QP
rdma_destroy_id
release the rdma_cm_id
rdma_destroy_event_channel
release the event channel
An almost identical process is used to setup unreliable datagram (UD)
communication between nodes. No actual connection is formed between
QPs however, so disconnection is not needed.
Although this example shows the client initiating the disconnect,
either side of a connection may initiate the disconnect.
SERVER OPERATION
This section provides a general overview of the basic operation for the
passive, or server, side of communication. A general connection flow
would be:
rdma_create_event_channel
create channel to receive events
rdma_create_id
allocate an rdma_cm_id, this is conceptually similar to a socket
rdma_bind_addr
set the local port number to listen on
rdma_listen
begin listening for connection requests
rdma_get_cm_event
wait for RDMA_CM_EVENT_CONNECT_REQUEST event with a new
rdma_cm_id
rdma_create_qp
allocate a QP for the communication on the new rdma_cm_id
rdma_accept
accept the connection request
rdma_ack_cm_event
ack event
rdma_get_cm_event
wait for RDMA_CM_EVENT_ESTABLISHED event
rdma_ack_cm_event
ack event
Perform data transfers over connection
rdma_get_cm_event
wait for RDMA_CM_EVENT_DISCONNECTED event
rdma_ack_cm_event
ack event
rdma_disconnect
tear-down connection
rdma_destroy_qp
destroy the QP
rdma_destroy_id
release the connected rdma_cm_id
rdma_destroy_id
release the listening rdma_cm_id
rdma_destroy_event_channel
release the event channel
RETURN CODES
= 0 success
= -1 error - see errno for more details
< -1 error - see include/asm-generic/errno*.h for more details
Librdmacm functions return 0 to indicate success, and a negative return
value to indicate failure. If a function operates asynchronously, a
return value of 0 means that the operation was successfully started.
The operation could still complete in error; users should check the
status of the related event. If the return value is -1, then errno can
be examined for additional information regarding the reason for the
failure. If the return value is < -1, then additional error reasons
can be obtained by comparing the returned value with the values listed
in include/asm-generic/errno-base.h and include/asm-generic/errno.h.
SEE ALSOrdma_create_event_channel(3), rdma_get_cm_event(3), rdma_create_id(3),
rdma_resolve_addr(3), rdma_bind_addr(3), rdma_create_qp(3),
rdma_resolve_route(3), rdma_connect(3), rdma_listen(3), rdma_accept(3),
rdma_reject(3), rdma_join_multicast(3), rdma_leave_multicast(3),
rdma_notify(3), rdma_ack_cm_event(3), rdma_disconnect(3),
rdma_destroy_qp(3), rdma_destroy_id(3), rdma_destroy_event_channel(3),
rdma_get_devices(3), rdma_free_devices(3), rdma_get_peer_addr(3),
rdma_get_local_addr(3), rdma_get_dst_port(3), rdma_get_src_port(3),
rdma_set_option(3)ucmatose(1), udaddy(1), mckey(1), rping(1)librdmacm 2008-01-02 RDMA_CM(7)
