Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

SCTP Specific Socket Functions in Linux

DZone's Guide to

SCTP Specific Socket Functions in Linux

The Stream Control Transmission Protocol (SCTP) lets you multiplex a single socket connection for multiple streams and multiple hosts. This post shows the key functions you need to know to use it.

Free Resource

Modernize your application architectures with microservices and APIs with best practices from this free virtual summit series. Brought to you in partnership with CA Technologies.

Preface

In the previous posts, we have used socket related functions common for protocols other than SCTP. For example with connect() you can connect to a remote peer with either TCP or SCTP socket. recvmsg() and sendmsg() can also be used with UDP and SCTP. This is normal because SCTP has features similar to TCP (connection-oriented) and UDP (message based). However, SCTP's unique features deserve specific API. In this post we will review some SCTP only functions for sending and receiving messages, establishing associations and extracting local and peer IP addresses from the association.

The sample code for this post is in two separate branches - one-to-many_advanced and one-to_many_peeloff. For the following material you need the first branch, so let's check it out:

git checkout one-to-many_advanced

All the functions described in this post are in libsctp and you need to link your application to it. This is compiler specific and off-topic for this post, so I will not talk about it. You can see how I did this with cmake in the project's CMakeLists.txt.

Obtaining Association ID

For one-to-many style sockets, some of the functions we are about to review required association id as an input parameter. One easy way to get it is to subscribe for sctp_association_event notification and to wait for SCTP_ASSOC_CHANGE event. Then you can obtain the association id from sac_assoc_id parameter of struct sctp_assoc_change. Check the previous post for more details about notifications.

Getting Local and Remote Association Addresses

Let's start with something trivial - getting local and peer IP address(es) of an association. There are two functions for this job. sctp_getpaddrs() returns the peer IP addresses of the association and it is described in Section 9.3. Respectively sctp_getladdrs() returns the local IP addresses and you can find more information about it in Section 9.5. They both have got similar signatures:

int sctp_getpaddrs(int sd, sctp_assoc_t id, struct sockaddr **addrs);
int sctp_getladdrs(int sd, sctp_assoc_t id, struct sockaddr **addrs);

Their input parameters are:

  • sd - The socket for the association.
  • id - If the socket is one-to-many style, this parameter represents the association id. For one-to-one sockets, it is ignored.
  • addrs - Empty pointer which will be assigned to an array, containing the local/peer IP addresses. The size of the array can be obtained from the return value of the function.

On success, both functions return the number of IP addresses saved in addrs array. If the socket is not bound/connected the return value is 0. On error, -1 is returned and addrs is undefined.

One very important thing! Note that you pass to the function the address of a pointer variable. Both functions allocate memory for the array with the addresses and it is your job to free it when you no longer need the addresses. For your convenience there are dedicated functions for this - sctp_freepaddrs() (described in Section 9.4) and sctp_freeladdrs() (described in Section 9.6). They also have got similar signatures:

int sctp_freepaddrs(struct sockaddr *addrs);
int sctp_freeladdrs(struct sockaddr *addrs);

Again, don't forget these functions or you will get memory leaks!

Now let's see how to use these functions in practice. In common.h there are two functions - get_assoc_local_addresses() and get_assoc_peer_addresses(). They are identical, but we will review both of them. First is get_assoc_local_addresses():

int get_assoc_local_addresses(const sctp_assoc_t assoc_id, const int socket, char* text_buf, int text_buf_size)
{
 struct sockaddr *addrs = NULL;
 int addr_count = -1;

 memset(text_buf, 0, text_buf_size);
 text_buf_size--; //leave space for the null terminator

 if((addr_count = sctp_getpaddrs(socket, assoc_id, &addrs)) == -1) {
 printf("Error occured while calling sctp_getpaddrs()\n");
 return 1;
 }

 snprintf(text_buf, text_buf_size, "Peer IP addresses: ");

 fill_addresses(addrs, addr_count, text_buf, text_buf_size);

 sctp_freepaddrs(addrs);

 return 0;
}

On line 3 we declare a NULL pointer to a struct sockaddr. It will be passed to sctp_getpaddrs(). On lines 6 and 7 the text buffer is prepared. It is filled in with zeros and the size is decremented with 1, in order to leave space for the NULL terminator. Online 9 sctp_getpaddrs() is called. Remember that its last parameter was a pointer to a struct sockaddr pointer (struct sockaddr**) and we have to pass the address of the pointer (&addrs) we defined on line 3. On lines 14-16 the address(es) are processed and saved in the text buffer. This has nothing to do with the SCTP API, so I will skip this function. The last thing we need to to is to free the array with the addresses. We do this just after they are saved in the text buffer - on line 18 with sctp_freepaddrs().

get_assoc_peer_addresses() is identical:

int get_assoc_peer_addresses(const sctp_assoc_t assoc_id, const int socket, char* text_buf, int text_buf_size)
{
 struct sockaddr *addrs = NULL;
 int addr_count = -1;

 memset(text_buf, 0, text_buf_size);
 text_buf_size--; //leave space for the null terminator

 if((addr_count = sctp_getladdrs(socket, assoc_id, &addrs)) == -1) {
 printf("Error occured while calling sctp_getpaddrs()\n");
 return 1;
 }

 snprintf(text_buf, text_buf_size, "Local IP addresses: ");

 fill_addresses(addrs, addr_count, text_buf, text_buf_size);

 sctp_freeladdrs(addrs);

 return 0;
}

The label of the log message is changed and we call sctp_getladdrs() and sctp_freeladdrs() in order to work with the local association addresses.

Binding and Connecting

The SCTP specific API provides two new functions for binding to local address and connecting to a peer - sctp_bindx() (described in Section 9.1) and sctp_connectx() (described in Section 9.9). The main difference between them and bind()/connect() is that the sctp_ versions accept an array of IP addresses as input parameters. These functions are useful when you want to use SCTP's multi-homing feature, but you can also use them with a single IP address.

The signature of sctp_bindx() is:

int sctp_bindx(int sd, struct sockaddr *addrs, int addrcnt, int flags);

Its input parameters are:

  • sd - SCTP socket.
  • addrs - Pointer to the array of with IP addresses.
  • addrcnt - How many IP addresses are located in the array.
  • flags - Can be SCTP_BINDX_ADD_ADDR, if you want to add bind addresses and SCTP_BINDX_REM_ADDR, to remove addresses.

On success 0 is returned. If there is an error, sctp_bindx() returns -1. Another difference between sctp_bindx() and bind() is that the former can be called more than once, however, you can't remove all local IP addresses from an association.

Let's see how this function is used in server's main():

struct sockaddr_in bind_addr;
memset(&bind_addr, 0, sizeof(struct sockaddr_in));
bind_addr.sin_family = ADDR_FAMILY;
bind_addr.sin_port = htons(server_port);
bind_addr.sin_addr.s_addr = INADDR_ANY;

if(sctp_bindx(server_fd, (struct sockaddr*)&bind_addr, 1, SCTP_BINDX_ADD_ADDR) == -1) {
 perror("bind");
 return 5;
}

Pretty straightforward. We bind to only one IP address so we pass it directly to sctp_bindx(). The size of the array is 1 and we use SCTP_BINDX_ADD_ADDR flag because we want to bind addresses.

sctp_connectx() has got the following signature:

int sctp_connectx(int sd, struct sockaddr *addrs, int addrcnt, sctp_assoc_t *id);

Its parameters are:

  • sd - SCTP socket.
  • addrs - Pointer to the array of with IP addresses.
  • addrcnt - How many IP addresses are located in the array.
  • id - Pointer to a sctp_assoc_t. If it is not NULL, the id of the newly created association will be written there.

sctp_connectx() returns 0 on success and -1 on error. Here is a sample code from client.c:

printf("Connecting...\n");

sctp_assoc_t assoc_id = 0;
if(sctp_connectx(client_fd, (struct sockaddr*)&peer_addr, 1, &assoc_id) == -1) {
 perror("sctp_connectx");
 return 6;
}

printf("OK. Association id is %d\n", assoc_id);

Almost identical to connect(). The only difference is that we pass a pointer to sctp_assoc_t and on successful connect the association ID is logged on the screen.

Sending and Receiving Data

We already saw how to set and retrieve some SCTP specific parameters with ancillary control messages and sendmsg()/recvmsg(). Now we are about to see an easier way to use these parameters thanks to sctp_sendmsg() (described in Section 9.7) and sctp_recvmsg() (described in Section 9.8). Both functions wrap the socket interface from the previous post, so where necessary I will redirect to sections from there.

I want to make one clarification before we go on. According to RFC 6458, sctp_recvmsg() and sctp_sendmsg() are deprecated and you should use sctp_sendv() and sctp_recvv() instead. However for some reason, they are implemented neither in Linux nor FreeBSD. I was unable to find out why so if you know something about this please leave a comment.

Let's have a look at sctp_sendmsg()'s signature:

ssize_t sctp_sendmsg(int sd, const void *msg, size_t len, const struct sockaddr *to, socklen_t tolen,
 uint32_t ppid, uint32_t flags, uint16_t stream_no, uint32_t timetolive, uint32_t context);

The return value is the number of bytes sent or -1 in the case of an error. There are quite a lot of parameters, but most of them are already familiar:

  • sd - SCTP socket.
  • msg - The buffer with the message payload.
  • len - The length of the buffer.
  • to - Destination IP address.
  • tolen - Length of the destination IP address.
  • ppid - Payload protocol ID.
  • flags - The same as snd_flags instruct sctp_sndinfo.
  • stream_no - Output stream ID.
  • timetolive - Time to live (in milliseconds).
  • context - Context id, which is sent to the upper layer if an error occurs during the message sending.

If you need more details about some of the parameters, check my previous post and especially Ancillary data that can be passed to sendmsg(). Now let's have a look at some code:

int send_reply(int server_fd, struct sockaddr_in* dest_addr)

{
 char buf[8];
 memset(buf, 0, sizeof(buf));
 strncpy(buf, "OK", sizeof(buf)-1);

 if(sctp_sendmsg(server_fd, buf, strlen(buf), (struct sockaddr*)dest_addr, sizeof(struct sockaddr_in), 0, 0, 0, 0, 0) == -1) {
 printf("sendmsg() error\n");
 return 1;
 }

 return 0;
}

The example is self-explanatory. The string "OK" is sent over the wire, without setting any of the SCTP-specific parameters.

Let's move on to sctp_recvmsg() and its signature:

ssize_t sctp_recvmsg(int sd, void *msg, size_t len, struct sockaddr *from, socklen_t *fromlen,
 struct sctp_sndrcvinfo *sinfo, int *msg_flags);

On success, the function returns the number of bytes read. Return tvalue of -1 indicates an error. Its input parameters are:

  • sd - SCTP socket.
  • msg - Pointer to a user-provided buffer for the incoming message.
  • len - The size of the buffer.
  • from - Pointer to a user-provided buffer, which will be filled in with the source IP address of the incoming message.
  • fromlen - Size of the source IP address buffer.
  • sinfo - Pointer to a structure, which contains SCTP specific parameters. It is similar to struct sctp_rcvinfo, so I won't describe it. For more information check Section 5.3.2.
  • flags - The values are the same as the ones described in Receiving messages with recvmsg() section from SCTP Linux API: One-to-many style interface post.

You can see code with sctp_recvmsg() in the next section, where the function get_connection() is listed.

Peel Off

Peel off is a function call which extracts an association from a one-to-many style socket and allows you to work with it individually in one-to-one style. Function's name is sctp_peeloff(). It is described in Section 9.2 and its definition is:

int sctp_peeloff(int sd, sctp_assoc_t assoc_id);

On success, the function returns the new socket, which is in one-to-one style. On error, -1 is returned. The function has got the following input parameters:

  • sd - One-to-many SCTP socket.
  • assoc_id - Association ID of the target, that we want to peel off. As we already discussed, this ID can be obtained via SCTP_ASSOC_CHANGE event or sctp_connectx().

Let's see an example. In the branch one-to_many_peeloff I have written a SCTP server which waits for new connections on one-to-many style sockets extracts the newly created association and spawns a thread which handles it in one-to-one style. First, let's switch to this branch:

git checkout one-to_many_peeloff

All the modifications in this branch are in the server. It initially creates a one-to-many (SOCK_SEQPACKET) socket and waits for a connection. Then when the association is established the server peels it off to one-to-one style socket and passes it to a new thread, which handles the client. This scenario has a lot of drawbacks from a scalability point of view, but it is simple enough to see sctp_peeloff() in action.

Let's start with server's main(). The only change there is that in the main loop get_connection() is called :

while(1) {
 if(get_connection(server_fd)) {
 break;
 }
}

Now let's see what it does:

int get_connection(int server_fd)
{
 int sender_addr_size = sizeof(struct sockaddr_in);
 char payload[1024];

 int buffer_len = sizeof(payload) - 1;
 memset(&payload, 0, sizeof(payload));

 struct sockaddr_in sender_addr;
 memset(&sender_addr, 0, sender_addr_size);

 struct sctp_sndrcvinfo snd_rcv_info;
 memset(&snd_rcv_info, 0, sizeof(snd_rcv_info));

 int msg_flags = 0;

 while(1) {
 int recv_size = 0;

 if((recv_size = sctp_recvmsg(server_fd, &payload, buffer_len, (struct sockaddr*)&sender_addr, &sender_addr_size, &snd_rcv_info, &msg_flags)) == -1) {
 printf("recvmsg() error\n");
 return 1;
 }

 if(!(msg_flags & MSG_NOTIFICATION)) {
 printf("Warning! Received payload in client handling loop!!!\n");
 break;
 }

 if(!(msg_flags & MSG_EOR)) {
 printf("Notification received, but the buffer is not big enough.\n");
 continue;
 }

 int assoc_id = get_association_id((union sctp_notification*)payload, recv_size);
 if(assoc_id < 0) {
 printf("Error getting association id\n");
 break;
 }

 if(handle_client(server_fd, assoc_id) < 0) {
 printf("Error handling client\n");
 break;
 }


 }

 return 0;
}

This is nothing more than the get_message() function with modified while loop. The idea here is to use sctp_recvmsg() to get the notification about a new association, to extract the association id (with get_association_id()) and then peel off the association in handle_client(). I want to remark that if your purpose is to peel off each association, it is totally pointless to use one-to-many style sockets! Stick with one-to-one and save yourself from doing all the extra work. But this project is only for demonstration purposes, so we don't bother.

get_association_id() does some error checking and returns the association id from struct sctp_assoc_change:

int get_association_id(union sctp_notification *notif, size_t notif_len)
{
 // http://stackoverflow.com/questions/20679070/how-does-one-determine-the-size-of-an-unnamed-struct
 int notif_header_size = sizeof( ((union sctp_notification*)NULL)->sn_header );

 if(notif_header_size > notif_len) {
 printf("Error: Notification msg size is smaller than notification header size!\n");
 return -1;
 }

 if(notif->sn_header.sn_type != SCTP_ASSOC_CHANGE) {
 printf("Invalid notification passed to get_association_id()\n");
 return -2;
 }

 if(sizeof(struct sctp_assoc_change) > notif_len) {
 printf("Error notification msg size is smaller than struct sctp_assoc_change size\n");
 return -3;
 }

 struct sctp_assoc_change* n = &notif->sn_assoc_change;

 if(n->sac_state != SCTP_COMM_UP) {
 printf("Notification with invalid state passed to get_association_id()\n");
 return -4;
 }

 return n->sac_assoc_id;
}

We have done this before, so let's see what happens in handle_client():

int handle_client(int server_fd, int assoc_id)
{
 int* client_conn_fd = NULL;

 if((client_conn_fd = (int*)calloc(1, sizeof(int))) == NULL) {
 printf("Error allocating memory\n");
 return 1;
 }

 *client_conn_fd = sctp_peeloff(server_fd, assoc_id);
 if(*client_conn_fd == -1) {
 printf("Error in peeloff\n");
 free(client_conn_fd);
 return 2;
 }

 if(disable_notifications(*client_conn_fd) != 0) {
 printf("Error disabling notifications\n");
 free(client_conn_fd);
 return 3;
 }

 pthread_attr_t thread_attr;
 memset(&thread_attr, 0, sizeof thread_attr);

 if(pthread_attr_init(&thread_attr)) {
 printf("Error initialising thread attributes\n");
 free(client_conn_fd);
 return 4;
 }

 if(pthread_attr_setdetachstate(&thread_attr, PTHREAD_CREATE_DETACHED)) {
 printf("Error setting detached attribute to thread\n");
 free(client_conn_fd);
 return 5;
 }

 pthread_t new_thread;
 memset(&new_thread, 0, sizeof(new_thread));

 if(pthread_create(&new_thread, &thread_attr, &client_thread, (void*)client_conn_fd)) {
 printf("Error creating thread\n");
 free(client_conn_fd);
 return 6;
 }

 return 0;
}

On lines 5-8, we allocate memory for the new socket on the heap, because we have to pass it to the client handling thread. Then on lines 10-15 we peel off the association. We are in one-to-one style now and we don't want to receive notifications anymore. They are disabled on lines 17-21. Finally, on lines 23-45 a thread is created for the new client.

The function client_thread() handles the messages from the clients. Here is its implementation:

void* client_thread(void* client_conn_fd)
{
 //save the fd to local variable, to avoid memory leaks
 int fd = *(int*)client_conn_fd;
 free(client_conn_fd);

 while (1) {
 const int buf_size = 1024;
 char buf[buf_size];
 memset(buf, 0, buf_size);

 int recv_size = recv(fd, buf, buf_size-1, 0);

 if(recv_size == -1) {
 if(errno == ENOTCONN) {
 printf("Client closed the connection\n");
 return NULL;
 }

 perror("recv()");
 return NULL;
 }
 else if (recv_size == 0) {
 printf("Client closed the connection\n");
 return NULL;
 }

 printf("%d: %s\n",recv_size, buf);

 if(send(fd, "OK", 2, 0) == -1) {
 perror("send()");
 return NULL;
 }
 }
 return NULL;
}

Our first job is to save the socket on the stack deallocate the memory. The rest is identical to the server behavior you have seen so far.

Conclusion

In this post, we saw how to access most of the SCTP features in a simplified and more convenient manner. On the projects, I was working on so far, this interface in combination with the notifications were sufficient to get the job done. I recommend you to keep your life simple and stick with the sctp_XXX functions. Fall back to ancillary data only in case you need something really specific. Thanks for reading!

The Integration Zone is proudly sponsored by CA Technologies. Learn from expert microservices and API presentations at the Modernizing Application Architectures Virtual Summit Series.

Topics:
sctp ,linux ,socket communicaiton ,integration

Published at DZone with permission of Tsvetomir Dimitrov. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}