id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc
34	Data corruption when FTP server writing data to client	mtmoore	mtmoore	"When FTPing a file to an FTP server that exposed an OrangeFS file system, I can consistently get data corruption when the ProFTPD server has a specific configuration.  If I set the send and
recv socket buffer size settings for ProFTPD (as configured in the attached config file), it will often corrupt the data.  As far as I could determine, all these settings do is modify the socket
options for the FTP sockets using SO_SNDBUF and SO_RCVBUF options set with setsockopt.

Here is my environment configuration:

FTP server: ProFTPD (I just built and installed the latest version available at proftpd.org).  I've attached the configuration file.
FTP client: The standard ""ftp"" command line utility present in RedHat Linux
FS Cluster: 4 Node OrangeFS Cluster using the attached config file.
All nodes involved were running 64bit RHEL5.5

Here is the command line used for the pvfs2-client:
/usr/sbin/pvfs2-client --logtype syslog -p /usr/sbin/pvfs2-client-core --logstamp datetime --acache-timeout=30000 --ncache-timeout=30000 --desc-size 8388608 --desc-count 5

The FTP client and server were on the same machine, so I simply FTP'd to localhost.

We initially saw the corruption when copying a 110G file, but a 118G file copied just fine.  I've been able to reliably reproduce the corruption with a 10G file and sometimes with a 1G file.  I was
never able to reproduce the corruption with a file smaller than 500M.

The corruption itself seems to manifest itself by replacing good data with Nulls.  The file size is always correct.

Another observation is that it usually takes several attempts for a file transfer to cause corruption, but once it does, it is fairly consistent.

Also, I attempted to reproduce the issue by using the ""cp"" command, ""dd"", and ""rsync"".  None of these operations would reproduce the issue.  Even using the ""curl"" command line utility to
make the FTP transfer worked.  The only cases that cause corruption are our custom FTP client and the command line ftp client.  I'm assuming their access pattern triggers some edge case in
OrangeFS.

I also put debug statements in the proftpd code to write the data to a local file immediately before writing to the OrangeFS FS.  The debug file would be correct (without corruption), while the
file written out to OrangeFS would be corrupted, so this isn't an issue of the data being corrupted in-flight to the FTP server.

Any help you can provide would be greatly appreciated.

Thanks!

--
Benjamin Severs"	defect	closed	major		Client-core	latest	fixed