Brock Palen
2014-09-11 19:17:45 UTC
I made a logstash parser for transfer logs, but ran into issues because it appears that that there are Transfer stats
Entries that do not have full information:
Eg:
What I expect:
[15298] Wed Jul 16 11:08:42 2014 :: Transfer stats: DATE=20140716150842.633689 HOST=flux-xfer1.engin.umich.edu PROG=globus-gridftp-server NL.EVNT=FTP_INFO START=20140716150841.618581 USER=brockp FILE=/tmp/brockp/linux_x64_64_sfx.exe BUFFER=87380 BLOCK=262144 NBYTES=280590752 VOLUME=/ STREAMS=4 STRIPES=1 DEST=[141.212.30.10] TYPE=RETR CODE=226
What I find and choke on:
[22304] Mon Sep 8 16:45:52 2014 :: Transfer stats: DATE=20140908204552.177237 HOSTS=1944 VOLUME=/ STREAMS=2 STRIPES=1 DEST=[141.212.30.10] TYPE=STOR CODE=226
What are the meanings of these two different formats? Is the first one, with the transfer size etc, the final summary of the transfer? Can I just ignore the others if I want to collect statistics on our gridftp servers?
Brock Palen
www.umich.edu/~brockp
CAEN Advanced Computing
XSEDE Campus Champion
***@umich.edu
(734)936-1985
Entries that do not have full information:
Eg:
What I expect:
[15298] Wed Jul 16 11:08:42 2014 :: Transfer stats: DATE=20140716150842.633689 HOST=flux-xfer1.engin.umich.edu PROG=globus-gridftp-server NL.EVNT=FTP_INFO START=20140716150841.618581 USER=brockp FILE=/tmp/brockp/linux_x64_64_sfx.exe BUFFER=87380 BLOCK=262144 NBYTES=280590752 VOLUME=/ STREAMS=4 STRIPES=1 DEST=[141.212.30.10] TYPE=RETR CODE=226
What I find and choke on:
[22304] Mon Sep 8 16:45:52 2014 :: Transfer stats: DATE=20140908204552.177237 HOSTS=1944 VOLUME=/ STREAMS=2 STRIPES=1 DEST=[141.212.30.10] TYPE=STOR CODE=226
What are the meanings of these two different formats? Is the first one, with the transfer size etc, the final summary of the transfer? Can I just ignore the others if I want to collect statistics on our gridftp servers?
Brock Palen
www.umich.edu/~brockp
CAEN Advanced Computing
XSEDE Campus Champion
***@umich.edu
(734)936-1985