RDMA FTP client (rcftp) and server (rftpd) are a set of tools for high
performance data transfer using RDMA technology.

New Features
==
* introduce maximum buffer pool size (maxbufpoolsiz)
* introduce dedicate disk IO block size (disk_io_siz)
* client and server CPU usage statistics

Installation
==
1. If you have installed previous version, please use upgrade command,
 # rpm -Uvh rftpd-0.15-rc4.x86_64.rpm
 # rpm -Uvh rcftp-0.15-rc4.x86_64.rpm

2. Comfirm version
 # rpm -qa rcftp rftpd
 rcftp-0.15-rc4
 rftpd-0.15-rc4

Configuration
==
* You could copy the sample configration(rcftprc and rftpdrc) files to /etc directory.

* Configuration file lookup:
 1 load default value
 2 load system configuration
 if env SYSTEM_RCFTPRC or env SYSTEM_RFTPDRC exist
     system configuration = env SYSTEM_RCFTPRC or env SYSTEM_RFTPDRC
 else
     system configuration = /etc/rcftprc (client) or /etc/rftpdrc (server)
 if (system configuration exist)
     load system configuration
 3 load user configuration
 if env RCFTPRC or env RFTPDRC exist
     user configuration = env RCFTPRC or env RFTPDRC
 else
     user configuration = $HOME/.rcftprc (client) or $HOME/.rftpdrc (server)
 if (user configuration exist)
     load user configuration
     override system configuration with user configuration

* The application both client and server could be configured with several parameters as follows,

+-------------+----------+--------+--------------------------------+
|   option    |  default |  app   |   description                  |
+-------------+----------+--------+--------------------------------+
| cbufsiz     |  1MB     | rcftp  | block size of ring buffer      |
|             |          | rftpd  |                                |
+-------------+----------+--------+--------------------------------+
| maxbufpoolsiz | 500MB  | rcftp  | maximum size of ring buffer    |
|             |          | rftpd  |                                |
+-------------+----------+--------+--------------------------------+
| cbufnum     |  10      | rcftp  | number of ring buffer block    |
| (deprecated)|          | rftpd  |                                |
+-------------+----------+--------+--------------------------------+
| srvcomport  |  21      | rftpd  | server side communication      |
|             |          |        | channel listening port         |
+-------------+----------+--------+--------------------------------+
| usesendfile |  no      | rcftp  | If set 'yes', TCP data channel |
|             |          | rftpd  | uses sendfile() to transfer    |
|             |          |        | data                           |
+-------------+----------+--------+--------------------------------+
| usesplice   |  no      | rcftp  | If set 'yes', TCP data channel |
|             |          | rftpd  | uses splice() to transfer data |
+-------------+----------+--------+--------------------------------+
| devzerosiz  |  100G    | rcftp  | set /dev/zero size for         |
|             |          | rftpd  | limited data transfer          |
+-------------+----------+--------+--------------------------------+
| rmtaddrnum  |  100     | rcftp  | number of pre-allocated        |
|             |          | rftpd  | remote addr information        |
+-------------+----------+--------+--------------------------------+
| evbufnum    |  100     | rcftp  | number of pre-allocated        |
|             |          | rftpd  | send control messages buf      |
+-------------+----------+--------+--------------------------------+
| recvbufnum  |  100     | rcftp  | number of pre-allocated        |
|             |          | rftpd  | recv control messages buf      |
+-------------+----------+--------+--------------------------------+
| rcstreamnum |  1       | rcftp  | number of parallel data        |
|             |          |        | data channels                  |
+-------------+----------+--------+--------------------------------+
| readernum   |  1       | rcftp  | number of reader               |
|             |          | rftpd  |                                |
+-------------+----------+--------+--------------------------------+
| writernum   |  1       | rcftp  | number of writer               |
|             |          | rftpd  |                                |
+-------------+----------+--------+--------------------------------+
| directio    |  no      | rcftp  | on/off direct IO feature when  |
|             |          | rftpd  |  doing file io                 |
|             |          |        | CHR device like /dev/zero and  |
|             |          |        |  /devnull don't support this   |
+-------------+----------+--------+--------------------------------+
| disk_io_siz |  1M      | rcftp  | size of disk io operation      |
|             |          | rftpd  |                                |
+-------------+----------+--------+--------------------------------+
| rdma_qp_sq  |  64      | rcftp  | maximum iodepth of rdma send   |
| _depth      |          | rftpd  | queue                          |
+-------------+----------+--------+--------------------------------+
| rdma_qp_rq  |  64      | rcftp  | maximum iodepth of rdma recv   |
| _depth      |          | rftpd  | queue                          |
+-------------+----------+--------+--------------------------------+
| rdma_cq     |  32      | rcftp  | maximum iodepth of rdma        |
| _depth      |          | rftpd  | completion queue               |
+-------------+----------+--------+--------------------------------+
| wc_thread   |  4       | rcftp  | number of threads for work     |
| _num        |          | rftpd  | completion event handling      |
+-------------+----------+--------+--------------------------------+
| wc_event    |  100     | rcftp  | number of events for work      |
| _num        |          | rftpd  | completion event handling      |
+-------------+----------+--------+--------------------------------+

Usage
=====

0. Before start rcftp/rftpd, please confirm maximum locked memory
for your user account

 $ ulimit -l    # max locked memory

   RDMA pins memory blocks to avoid page swap or migration. Enlarge
   this parameter if it is not sufficient. It must be larger than
   maxbufpoolsiz.

 $ ulimit -n    # open files

   RFTP supports directory hirarchy transfer. You should confirm the
   number of files you want to transfer is less than this
   limitation. Otherwise, a 'can not open file' error will occur.
   
1. Startup server manually
 # rftpd -D

2. Check the network status
 # netstat -an | grep [port]
 tcp    0   0  0.0.0.0:port           0.0.0.0:*             LISTEN
 
3. Startup client and login to the server
 # rcftp [server addr] [port]
 ...
 Name: ftp
 Password: ftp

ftp/ftp is a kind of anonymous user/passwd pair, and the server would
not set user id or group id which makes regular user permission
startup server possible.

4. Change directories
 ftp> lcd [local working directory]
 ftp> cd [remote working directory]

5.1. Transfer data via RDMA
 ftp> rput local-file [ remote-file ]
 ftp> rget remote-file [ local-file ]
 ftp> rmput [[file1] ...] [[dir1] ...]

5.2. Transfer data via RDMA from memory to memory
 ftp> rput /dev/zero /dev/null

5.3. After you set the 'usesplice' or 'usesendfile' as 'yes', the apps will avoid data copy between user space and kernel space
 ftp> put local-file [ remote-file ]

6. Close the connection
 ftp> by
 221 Goodbye.

Issues
======

* splice has higher priority than sendfile. so if 'usesplice' and
  'usesendfile' are both set as 'yes', the application will use splice
  to send data.

* A script could be used as an input for convenient, as shown in follows.

 # rcftp -n -i < [script]
 in the script file, you could write some task as you want,

------------
open server-addr port
user username password
bin
prompt
lcd local-dir
cd remote-dir
rput local-file remote-file
bye
------------