优化dstat监控日志

dstat是一个用来替换 vmstat,iostat netstat,nfsstat和ifstat这些命令的工具, 是一个全能系统信息统计工具。因为实际使用起来，感觉不是很方便，因此作了一些修改，方便数据的采集和事后问题的排查。用下面的命令来运行修改后的dstat（dstat代码下载）

/home/oracle/dbafree/dstat -tydncs --output /home/oracle/dbafree/dstat.log--pidfile /home/oracle/dbafree/dstat.lck --nocolor 10 

1.支持pidfile参数，并通过pidfile来防止并发执行。如果重复次启动dstat，会出现如下错误:

pidfile ['/home/oracle/dbafree/dstat.lck'] is exist,process ['2350'] is running!Failed to create pidfile /home/oracle/dbafree/dstat.lck 0

2.自动记录dstat.log及自动收集元数据dstat_raw.log。 参数通过—output指定，dstat_raw.log自动根据output_raw.log来生成,–output必需以.log后缀结尾。

[oracle@testdb /home/oracle/dbafree]$ ls -ltrtotal 104-rwxr-xr-x 1 oracle oinstall 84182 Dec  1 19:20 dstat-rw-r--r-- 1 oracle oinstall     4 Dec  1 19:34 dstat.lck-rw-r--r-- 1 oracle oinstall  5526 Dec  1 19:36 dstat_raw.log-rw-r--r-- 1 oracle oinstall  3468 Dec  1 19:36 dstat.log

查看dstat.log，对于这个文件，在本地保存，方便直接通过日志文件排查问题。

[oracle@testdb /home/oracle/dbafree]$ tail -100f dstat.log----system---- ---system-- -dsk/total- -net/total- ----total-cpu-usage---- ----swap---  date/time   | int   csw | read  writ| recv  send|usr sys idl wai hiq siq| used  free01-12 19:48:24|2558    65k|  59M   59M|   0     0 | 13   2  83   2   0   1|  12k   30G01-12 19:48:34|1189  1425 | 288k 1007k|  18k   12k|  0   0 100   0   0   0|  12k   30G01-12 19:48:44|1098  1170 | 189k  139k|  13k 9173B|  0   0 100   0   0   0|  12k   30G01-12 19:48:54|1509  2815 | 230k  156k|  13k 9097B|  0   0 100   0   0   0|  12k   30G01-12 19:49:04|1095  1216 | 189k  125k|  12k 8522B|  0   0 100   0   0   0|  12k   30G01-12 19:49:14|1100  1173 | 221k  108k|  14k 9391B|  0   0 100   0   0   0|  12k   30G01-12 19:49:24|1100  1152 | 214k   89k|  13k 9068B|  0   0 100   0   0   0|  12k   30G01-12 19:49:34|1575  2946 | 291k  128k|  17k   11k|  0   0 100   0   0   0|  12k   30G01-12 19:49:44|1099  1143 | 189k  111k|  13k 9165B|  0   0 100   0   0   0|  12k   30G01-12 19:49:54|1097  1144 | 214k   77k|  13k 9116B|  0   0 100   0   0   0|  12k   30G01-12 19:50:04|1112  1374 | 452k  313k|  13k 8814B|  0   0 100   0   0   0|  12k   30G01-12 19:50:14|1522  2823 | 325k  379k|  13k 9017B|  0   0 100   0   0   0|  12k   30G01-12 19:50:24|1124  1197 | 480k  226k|  13k 9356B|  0   0 100   0   0   0|  12k   30G01-12 19:50:34|1185  1336 | 515k  243k|  18k   12k|  0   0 100   0   0   0|  12k   30G01-12 19:50:44|1110  1162 | 240k  100k|  14k 9596B|  0   0 100   0   0   0|  12k   30G01-12 19:50:54|1620  3022 |2128k  660k|  18k   12k|  0   0 100   0   0   0|  12k   30G01-12 19:51:04|1103  1221 | 205k  171k|  13k 9198B|  0   0 100   0   0   0|  12k   30G

查看dstat_raw.log，这个是元数据文件，可以采集到数据库中，作曲线分析和对比等使用。

[oracle@testdb /home/oracle/dbafree]$ tail -100f dstat_raw.log"Dstat 0.7.0 CSV output""Author:","Dag Wieers ",,,,"URL:","http://dag.wieers.com/home-made/dstat/""Host:","testdb",,,,"User:","oracle""Cmdline:","dstat -tydncs --output /home/oracle/dbafree/dstat.log --pidfile /home/oracle/dbafree/dstat.lck --nocolor 10",,,,"Date:","01 Dec 2013 19:48:24 CST""system","system",,"dsk/total",,"net/total",,"total cpu usage",,,,,,"swap","date/time","int","csw","read","writ","recv","send","usr","sys","idl","wai","hiq","siq","used","free"01-12 19:48:24,2557.975,64684.613,61511538.970,62029695.379,0.0,0.0,12.667,1.510,83.137,2.069,0.043,0.574,12288.0,32210173952.001-12 19:48:34,1189.300,1425.200,294912.0,1031168.0,18249.300,12292.600,0.169,0.031,99.781,0.006,0.0,0.013,12288.0,32210173952.001-12 19:48:44,1097.500,1170.200,193331.200,142028.800,13558.0,9173.300,0.119,0.006,99.862,0.006,0.0,0.006,12288.0,32210173952.001-12 19:48:54,1508.800,2815.200,235929.600,159948.800,13467.600,9097.200,0.119,0.044,99.806,0.0,0.025,0.006,12288.0,32210173952.001-12 19:49:04,1094.800,1216.200,193331.200,127795.200,12733.400,8522.0,0.181,0.025,99.787,0.0,0.0,0.006,12288.0,32210173952.001-12 19:49:14,1099.700,1173.300,226099.200,110694.400,13836.100,9390.600,0.106,0.006,99.869,0.006,0.0,0.013,12288.0,32210173952.001-12 19:49:24,1100.200,1152.100,219545.600,91136.0,13703.600,9068.0,0.113,0.019,99.862,0.0,0.0,0.006,12288.0,32210173952.001-12 19:49:34,1575.100,2945.700,298188.800,130867.200,17909.600,11672.500,0.156,0.056,99.756,0.006,0.019,0.006,12288.0,32210173952.001-12 19:49:44,1098.900,1143.400,193331.200,113868.800,13741.600,9164.700,0.106,0.0,99.887,0.006,0.0,0.0,12288.0,32210173952.001-12 19:49:54,1097.100,1143.600,219545.600,78745.600,13245.600,9116.0,0.106,0.006,99.875,0.006,0.0,0.006,12288.0,32210173952.001-12 19:50:04,1111.900,1374.400,462848.0,320102.400,12805.500,8813.700,0.338,0.100,99.556,0.006,0.0,0.0,12288.0,32210173952.001-12 19:50:14,1521.500,2823.100,332595.200,388403.200,13783.200,9016.500,0.144,0.044,99.775,0.006,0.025,0.006,12288.0,32210173952.001-12 19:50:24,1124.200,1197.0,491520.0,231116.800,13683.800,9356.0,0.119,0.013,99.856,0.006,0.0,0.006,12288.0,32210173952.001-12 19:50:34,1185.300,1336.300,527564.800,249241.600,18165.400,11826.200,0.144,0.013,99.831,0.006,0.0,0.006,12288.0,32210173952.001-12 19:50:44,1110.100,1161.700,245760.0,102604.800,14268.100,9596.0,0.106,0.006,99.881,0.0,0.0,0.006,12288.0,32210173952.001-12 19:50:54,1619.600,3021.900,2178611.200,676300.800,18337.300,12409.900,0.144,0.050,99.756,0.025,0.019,0.006,12288.0,32210173952.001-12 19:51:04,1102.700,1221.100,209715.200,175308.800,13113.600,9198.100,0.188,0.031,99.756,0.013,0.006,0.006,12288.0,32210173952.0

3. 增加了日志自动归档功能，大于100m，会自动归档，归档最多保留2个。

这样，我们把dstat的代码部署到crontab中，将dstat_raw.log采集入库，就可以很方便的实现操作系统层面的监控了。