把所有的业务日志通过scribe收集上来后，需要给项目开发人员一个查询业务日志的页面，方便他们查日志也给我们运维节约时间，我们可以用节约的时间做更多更有意义的事。写的这个页面很简单后端用的是Django，前端展示用的是基于bootstrap的Metronic模板，因为不需要权限控制也不要登录验证故没有用到数据库。用一个业务日志查询页面来介绍下，其它业务日志查询的逻辑都一样甚至渲染的模板都几乎一样，唯一不同的是渲染模板传递的参数有小差异。实现的功能很简单：可以查询每天最新的日志如最新100行；根据日期查询某天日志最后100行；根据一个或多个关键字查询日志并显示最新的200行结果；每次查询的结果都可以通过页面的下载按钮下载到本地供日后分析；页面上显示的结果按时间排序，最新的显示在最前面。1、渲染模板，每个业务日志模板都由一个基础模板和各自独有的模板构成。基础模板bash.html
<!DOCTYPE html><!--[if IE 8]> <html lang="en" class="ie8"> <![endif]--><!--[if IE 9]> <html lang="en" class="ie9"> <![endif]--><!--[if !IE]><!--> <html lang="en"> <!--<![endif]--><!-- BEGIN HEAD --><head><meta charset="utf-8" /> <title>{{privatetitle}}</title><meta content="width=device-width, initial-scale=1.0" name="viewport" /><meta content="" name="description" /><meta content="" name="author" /><!-- BEGIN GLOBAL MANDATORY STYLES --><link href="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/css/bootstrap.min.css" rel="stylesheet" type="text/css"/><link href="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/css/bootstrap-responsive.min.css" rel="stylesheet" type="text/css"/><link href="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/css/font-awesome.min.css" rel="stylesheet" type="text/css"/><link href="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/css/style-metro.css" rel="stylesheet" type="text/css"/><link href="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/css/style.css" rel="stylesheet" type="text/css"/><link href="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/css/style-responsive.css" rel="stylesheet" type="text/css"/><link href="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/css/default.css" rel="stylesheet" type="text/css" id="style_color"/><link href="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/css/uniform.default.css" rel="stylesheet" type="text/css"/><!-- END GLOBAL MANDATORY STYLES --> <!-- BEGIN PAGE LEVEL STYLES --> <link rel="stylesheet" type="text/css" href="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/css/datepicker.css" /> <link href="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/css/glyphicons.css" rel="stylesheet" /><!-- END PAGE LEVEL STYLES --><link rel="shortcut icon" href="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/image/favicon.ico" /></head><!-- END HEAD --><!-- BEGIN BODY --><body class="page-header-fixed"><!-- BEGIN HEADER --><div class="header navbar navbar-inverse navbar-fixed-top"><!-- BEGIN TOP NAVIGATION BAR --><div class="navbar-inner"><div class="container-fluid"><!-- BEGIN LOGO --><a class="brand" href="http://www.xiaomastack.com/seek/"><img src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/image/logo.png" alt="logo" /></a><!-- END LOGO --><!-- BEGIN RESPONSIVE MENU TOGGLER --><a href="javascript:;" class="btn-navbar collapsed" data-toggle="collapse" data-target=".nav-collapse"><img src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/image/menu-toggler.png" alt="" /></a> <!-- END RESPONSIVE MENU TOGGLER --><!-- BEGIN TOP NAVIGATION MENU --><ul class="nav pull-right"><!-- BEGIN USER LOGIN DROPDOWN --><li class="dropdown user"><a href="http://www.xiaomastack.com/2014/11/16/logs-terrace/" class="dropdown-toggle" data-toggle="dropdown"><img alt="" src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/image/linux.jpg" /><span class="username">SomeBody</span><i class="icon-angle-down"></i></a><ul class="dropdown-menu"><li><a href="http://www.xiaomastack.com/2014/11/16/logs-terrace/"><i class="icon-tasks"></i> My Home</a></li><li class="divider"></li><li><a href="http://www.xiaomastack.com/2014/11/16/logs-terrace/"><i class="icon-key"></i> Log Out</a></li></ul></li><!-- END USER LOGIN DROPDOWN --></ul><!-- END TOP NAVIGATION MENU --></div></div><!-- END TOP NAVIGATION BAR --></div><!-- END HEADER --><!-- BEGIN CONTAINER --><div class="page-container row-fluid"><!-- BEGIN SIDEBAR --><div class="page-sidebar nav-collapse collapse"><!-- BEGIN SIDEBAR MENU --> <ul class="page-sidebar-menu"><li><!-- BEGIN SIDEBAR TOGGLER BUTTON --><div class="sidebar-toggler hidden-phone"></div><!-- BEGIN SIDEBAR TOGGLER BUTTON --></li><li class="active "> <a href="javascript:;"><i class="icon-table"></i><span class="title">日志查询</span><span class="selected"></span><span class="arrow open"></span> </a><ul class="sub-menu"> <li class={{seek01}}> <a href="http://www.xiaomastack.com/seek/aaaaa/">A日志</a></li> <li class={{seek02}}> <a href="http://www.xiaomastack.com/seek/bbbbb/nginx-php/">B日志</a></li> <li class={{seek03}}> <a href="http://www.xiaomastack.com/seek/ccccc/">C日志</a></li> <li class={{seek04}}> <a href="http://www.xiaomastack.com/seek/ddddd/services/">D日志</a></li> <li class={{seek05}}> <a href="http://www.xiaomastack.com/seek/eeeee/nginx-php/">E日志</a></li></ul></li></ul><!-- END SIDEBAR MENU --></div><!-- END SIDEBAR --><!-- BEGIN PAGE --><div class="page-content"><!-- BEGIN SAMPLE PORTLET CONFIGURATION MODAL FORM--><div id="portlet-config" class="modal hide"><div class="modal-header"><button data-dismiss="modal" class="close" type="button"></button><h3>portlet Settings</h3></div><div class="modal-body"><p>Here will be a configuration form</p></div></div><!-- END SAMPLE PORTLET CONFIGURATION MODAL FORM--><!-- BEGIN PAGE CONTAINER--><div class="container-fluid"><!-- BEGIN PAGE HEADER--><div class="row-fluid"> <div class="span12"> {% block rightareadown %}<br><h3>请选择项目</h3>{% endblock %}</div></div><!-- END PAGE CONTENT--></div><!-- END PAGE CONTAINER--></div><!-- END PAGE --> </div><!-- END CONTAINER --><!-- BEGIN FOOTER --><div class="footer"><div class="footer-inner">Copyright (c) 2014 日志查询平台. All rights reserved</div><div class="footer-tools"><span class="go-top"><i class="icon-angle-up"></i></span></div></div><!-- END FOOTER --><!-- BEGIN JAVASCRIPTS(Load javascripts at bottom, this will reduce page load time) --><!-- BEGIN CORE PLUGINS --><script src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/js/jquery-1.10.1.min.js" type="text/javascript"></script><script src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/js/jquery-migrate-1.2.1.min.js" type="text/javascript"></script><!-- IMPORTANT! Load jquery-ui-1.10.1.custom.min.js before bootstrap.min.js to fix bootstrap tooltip conflict with jquery ui tooltip --><script src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/js/jquery-ui-1.10.1.custom.min.js" type="text/javascript"></script> <script src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/js/bootstrap.min.js" type="text/javascript"></script><!--[if lt IE 9]><script src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/js/excanvas.min.js"></script><script src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/js/respond.min.js"></script> <![endif]--> <script src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/js/jquery.slimscroll.min.js" type="text/javascript"></script><script src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/js/jquery.blockui.min.js" type="text/javascript"></script> <script src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/js/jquery.cookie.min.js" type="text/javascript"></script><script src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/js/jquery.uniform.min.js" type="text/javascript" ></script><!-- END CORE PLUGINS --> <!-- BEGIN PAGE LEVEL PLUGINS --> <script type="text/javascript" src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/js/bootstrap-datepicker.js"></script> <script type="text/javascript" src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/js/date.js"></script> <!-- END PAGE LEVEL PLUGINS --><!-- BEGIN PAGE LEVEL SCRIPTS --><script src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/js/app.js"></script> <script src="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{STATIC_URL}}/js/form-components.js"></script><script>jQuery(document).ready(function() {// initiate layout and plugins App.init(); FormComponents.init();});</script><!-- END PAGE LEVEL SCRIPTS --><script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-37564768-1']); _gaq.push(['_setDomainName', 'keenthemes.com']); _gaq.push(['_setAllowLinker', true]); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://' : 'http://') + 'stats.g.doubleclick.net/dc.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })();</script></body><!-- END BODY --></html>
每个项目还有自己独有的模板且继承bash.html,独立模板用来重写{% block rightareadown %}{% endblock %}块，独立的模板几乎一样就举一例。
{% extends "base.html" %}{% block rightareadown %}<ul class="breadcrumb"> <li> <i class="icon-home"></i> <a href="http://www.xiaomastack.com/seek/">日志查询</a> <i class="icon-angle-right"></i></li><li><a href="http://www.xiaomastack.com/seek/aaaaa/">A日志</a></li></ul><div class="row-fluid"> <div class="span12"> <!-- BEGIN PORTLET--> <div class="portlet box grey"> <div class="portlet-title"> <div class="caption"> <i class="icon-reorder"></i>填入必要的查询条件 </div> <div class="tools"> <a href="javascript:;" class="collapse"></a> <a href="javascript:;" class="remove"></a> </div> </div> <div class="portlet-body form"> <!-- BEGIN FORM--> <form action="/seek/aaaaa/" class="form-horizontal" enctype="multipart/form-data" method="post"> <div class="control-group"> <label class="control-label">选择查询日期(默认当天)</label> <div class="controls"> <div class="input-append date date-picker" data-date-format="mm/dd/yyyy" data-date-viewmode="years"> <input class="m-wrap m-ctrl-medium date-picker" readonly size="16" type="text" value="" name="date"/><span class="add-on"><i class="icon-calendar"></i></span> </div> </div> </div> <div class="control-group"> <label class="control-label">选择要查询的日志</label> <div class="controls"> <select class="span6 m-wrap" multiple="multiple" data-placeholder="Choose a Category" tabindex="1" name="logdir" > {% for dir in dirs %} <option>{{ dir }}</option> {% endfor %} </select> </div> </div> <div class="control-group"> <label class="control-label">Keys</label> <div class="controls"> <input type="text" class="span6 m-wrap" name="keys" placeholder="输入关键字，多个关键字用分号分隔(不输入关键字则获取当天最新的100行记录)"> </div> </div> <div class="form-actions"> <button type="submit" class="btn blue">Submit</button> </div> {% csrf_token %} </form> <!-- END FORM--> </div> </div> <!-- END PORTLET--></div><div class="row-fluid"> <div class="portlet box grey"> <div class="portlet-title"> <div class="caption"> <i class="icon-reorder"></i>查询结果 </div> <div class="tools"> <a href="javascript:;" class="collapse"></a> </div> </div> <div class="portlet-body"> <div class="control-group"> <a href="http://www.xiaomastack.com/2014/11/16/logs-terrace/{{durl}}" class="glyphicons no-js download_alt"><i></i>下载</a> </div> <div class="scroller" data-height="550px" data-always-visible="1"> {{ value }} {% for mes in message %} {{ mes }}<br><br> {% endfor %} </div> </div> </div></div>{% endblock %}
2、视图函数，接收前端页面的POST请求并根据不同的请求条件，使用日志查询函数searchKey, noKeys, readLine查询日志，然后将结果渲染后返回给前端页面。
#coding=utf-8from django.shortcuts import render_to_responsefrom django.template import RequestContextfrom os import listdir, pathfrom search import searchKey, noKeys, readLineimport timedef seekindex(request): avg = {'privatetitle': '日志查询', 'STATIC_URL': '/static'} return render_to_response('base.html', avg)def seekvenus(request): message = '' durl = '#' value = '' logPath = r"/var/scribe_data/aaaaa" logList = listdir(logPath) dirs = [] for dir in logList: dirs.append(dir.replace("1_", '')) dirs.sort() if request.method == 'POST': try: date = request.POST['date'] except: pass try: logdir = request.POST['logdir'] except: logdir = 'nodir' try: keys = request.POST['keys'] except: pass if not date: date = time.strftime('%m/%d/%Y',time.localtime(time.time())) if date and logdir and keys: #fname temp = date.split('/') dd = temp[1] mm = temp[0] yy = temp[2] fname = '/var/scribe_data/aaaaa/'+'1_'+logdir+'/'+'1_'+logdir+'-'+yy+'-'+mm+'-'+dd+'_00000' #keys keys = keys.split(';') #seek if not path.exists(fname): value = "当天的日志文件不存在" else: dfile = searchKey(fname, keys) durl = r'/static/tmp/' + dfile.split('/')[-1] message = readLine(dfile, False) elif logdir == 'nodir': value = '请选择日志' else: temp = date.split('/') dd = temp[1] mm = temp[0] yy = temp[2] fname = '/var/scribe_data/aaaaa/'+'1_'+logdir+'/'+'1_'+logdir+'-'+yy+'-'+mm+'-'+dd+'_00000' if not path.exists(fname): value = "当天的日志文件不存在" else: dfile = noKeys(fname) durl = r'/static/tmp/' + dfile.split('/')[-1] message = readLine(dfile, True) avg = {'privatetitle': 'aaaaa日志查询', 'STATIC_URL': '/static', 'message':message, 'seek01':"active" ,'dirs':dirs, 'durl':durl, 'value':value} return render_to_response('aaaaa/seek.html', avg, context_instance=RequestContext(request))
3、日志查询函数。每次先将一定条件的查询结果保存到一个新的文件，然后再从查询结果中读取最新的100行显示，这个新的文件可以通过页面的下载链接上下载到本地。
#!/usr/bin/pythonfrom re import search, subimport loggingimport commands def searchKey(fname, keys, user='Unknown'): logging.basicConfig(filename = '/var/www/logseek/log/seek.log', level = logging.INFO, filemode = 'a', format = '%(asctime)s - %(levelname)s: %(message)s') tmpdir = r'/var/www/logseek/static/tmp/' resultFile = tmpdir + fname.split(r'/')[-1]+'-'+keys[0]+'-'+keys[-1]+'.log' file = open(fname) refile = open(resultFile, 'w') while True: lines = file.readlines(500000) if not lines: break for line in lines: for key in keys: row = search(key, str(line)) if not row: row = False break if row: refile.write(line) file.close() refile.close() keyinfo = '' for key in keys: keyinfo = keyinfo + ':' + key logging.info(user + ':' + fname + keyinfo) return resultFiledef noKeys(fname, user='Unknown'): logging.basicConfig(filename = '/var/www/logseek/log/seek.log', level = logging.INFO, filemode = 'a', format = '%(asctime)s - %(levelname)s: %(message)s') tmpdir = r'/var/www/logseek/static/tmp/' resultFile = tmpdir + fname.split(r'/')[-1]+'-'+'new100.log' comand = 'tail -n 100 '+fname+' > '+ resultFile try: stat, proStr = commands.getstatusoutput(comand) except: pass logging.info(user + ':' + fname + ' tail 100 lines') return resultFiledef readLine(fname, flag): if flag: comand = 'tail -n 100 '+fname else: comand = 'tail -n 200 '+fname try: stat, proStr = commands.getstatusoutput(comand) except: pass relist = proStr.split('\n') relist.reverse() return relist
4、最后看到的查询日志页面一个非常简单实用的日志查询页面，如果以后还有更多需求再慢慢堆代码。文章出处：http://www.xiaomastack.com/2014/11/16/logs-terrace/
就会犯错误，就会有无数次让自己跌倒的机会出现，
