[OpenAFS] speed of vos dump on linux gbit ethernet

[Christof Hanke]

This is a multi-part message in MIME format.
--------------040408090307090802090503
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

Sven Oehme wrote:

> 
> i know the difference between afsd.conf and the fileserver params , but 
> i have no debian box in front of me, so i can't see what's in the script 
> :-)
> but you answered me what i asked Lars for , there are no params :-)
> 
> Lars ,
> 
> i would start with the following changes in the BosConfig file :
> 
> parm /usr/lib/openafs/fileserver -nojumbo -L -rxpck 400 -udpsize 1310720
> parm /usr/lib/openafs/volserver -nojumbo  -log -udpsize 1310720
> 
But note that there are limits on the OS-level to the socket buffer 
sizes as well.

On _linux_ these are :
receive buffer :
/proc/sys/net/core/rmem_max
send buffer :
/proc/sys/net/core/wmem_max

The value "1310720" you used above is the max size set by default on my 
SuSE-machine (and probably on yours as well).

Use the small attached C-Program to check this.
Also note that the linux kernel gives you a different buffer size 
(always (?) double) than you asked for.
Read http://www.ussg.iu.edu/hypermail/linux/kernel/0202.3/0292.html
for that. Although the thread is about 2.4.x kernels it seems to be the 
case for 2.6.x as well.

Further interesting tuning parameters seem to be :
/proc/sys/net/unix/max_dgram_qlen (Queue length of datagrams at the 
socket level)
/proc/sys/net/core/netdev_max_backlog (Queue length of packets at the 
interface level)

Just to give credits, I have most this info from

http://datatag.web.cern.ch/datatag/howto/tcp.html

Although it mainly deals with tcp, the socket operations and interface 
settings are valid for udp as well, thus you find more parameters to 
check out there.

Christof

--------------040408090307090802090503
Content-Type: text/plain;
 name="check_bufsize.c"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="check_bufsize.c"

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <netdb.h>
#include <errno.h>
#include <sys/select.h>
#include <unistd.h>
#include <string.h>



int main (int argc, char **argv)
{
  int sockfd, port = 0;
  int bufsize,desired_bufsize;
  struct sockaddr_in serv_addr;
  socklen_t len;
  if (argc < 2 )
     {
       printf (" usage : udp_buffersize [desired buf-size]\n");
       exit(-1);      
     }
  desired_bufsize = atoi(argv[1]); 
  bzero ((char *) &serv_addr, sizeof (serv_addr));
  serv_addr.sin_family = AF_INET;
  serv_addr.sin_port = htons (port);
  if ((sockfd = socket (AF_INET, SOCK_DGRAM, 0)) < 0) {
    printf ("client: can't open datagram socket\n");
    exit(EXIT_FAILURE);
  }
  len = sizeof(bufsize);
  getsockopt(sockfd, SOL_SOCKET, SO_RCVBUF, &bufsize, &len);
  printf("default udp buffer size on this system : %d\n",bufsize);
  setsockopt(sockfd, SOL_SOCKET, SO_RCVBUF, &desired_bufsize, len);
  printf("trying to set buffer size to %d\n",desired_bufsize);
  getsockopt(sockfd, SOL_SOCKET, SO_RCVBUF, &bufsize, &len);
  printf("new receive buffer size = %d\n", bufsize);
  getsockopt(sockfd, SOL_SOCKET, SO_SNDBUF, &bufsize, &len);
  printf("Send buffer size (should be default size) = %d\n", bufsize);
  while (1) {
     ;
  }
  exit(EXIT_SUCCESS);
}

--------------040408090307090802090503--