运维大部分时间都在重复性的做着相同的工作,当你有一天终于不堪忍受,想提高自己的效率时,一批批自动化的工具就涌入了你的眼前。fabric是第一个出现在我视野里的工具,第一次看到就被这货的高效和简单折服,这也正是python所倡导的简单优雅…
一:安装
1 在python的世界里安装软件是件简单事情。
pip install fabric
二:使用
Fabric 作用:基于ssh远程到多台服务器执行命令;例如批量上传配置文件,批量重启服务等。
1 先直接上个简单的demo,再讲解下:
a 文件保存为:fabfile.py#!/usr/local/python2.7/bin/python2.7# -*- coding:utf8 -*-import sys from fabric.api import *reload(sys)sys.setdefaultencoding('utf-8')#ssh 尝试连接的次数env.connection_attempts = 3 #不再读取本机的known_hosts,防止出现key变了,发生异常env.disable_known_hosts = True#网络超时时间env.timeout = 60#跳过有问题的ipenv.skip_bad_hosts = True#任务并行env.parallel = Trueenv.roledefs = { 'host1':['root@221.180.146.95:2222'],'host2':['root@221.180.146.97:2222']}env.passwords = {'root@221.180.146.95:2222':'password', 'root@221.180.146.97:2222':'password', } @roles('host1')def ip_display(): run('ifconfig')@roles('host2')def uptime(): run('uptime')b 到fabfile.py同级目录下,执行:[root@printserver systemadministrator]# fab ip_display[root@221.180.146.95:2222] Executing task 'ip_display'[root@221.180.146.95:2222] run: ifconfig[root@221.180.146.95:2222] out: eth0 Link encap:Ethernet HWaddr 90:B1:1C:1A:57:31 [root@221.180.146.95:2222] out: inet addr:221.180.146.95 Bcast:221.180.146.255 Mask:255.255.255.0[root@221.180.146.95:2222] out: inet6 addr: fe80::92b1:1cff:fe1a:5731/64 Scope:Link[root@221.180.146.95:2222] out: UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1[root@221.180.146.95:2222] out: RX packets:911689505 errors:0 dropped:2961131 overruns:0 frame:0[root@221.180.146.95:2222] out: TX packets:859773232 errors:0 dropped:0 overruns:0 carrier:0[root@221.180.146.95:2222] out: collisions:0 txqueuelen:1000 [root@221.180.146.95:2222] out: RX bytes:879927007186 (819.4 GiB) TX bytes:1020348108176 (950.2 GiB)[root@221.180.146.95:2222] out: Interrupt:82 Memory:da000000-da012800 [root@221.180.146.95:2222] out: [root@221.180.146.95:2222] out: lo Link encap:Local Loopback [root@221.180.146.95:2222] out: inet addr:127.0.0.1 Mask:255.0.0.0[root@221.180.146.95:2222] out: inet6 addr: ::1/128 Scope:Host[root@221.180.146.95:2222] out: UP LOOPBACK RUNNING MTU:16436 Metric:1[root@221.180.146.95:2222] out: RX packets:9844 errors:0 dropped:0 overruns:0 frame:0[root@221.180.146.95:2222] out: TX packets:9844 errors:0 dropped:0 overruns:0 carrier:0[root@221.180.146.95:2222] out: collisions:0 txqueuelen:0 [root@221.180.146.95:2222] out: RX bytes:73990935 (70.5 MiB) TX bytes:73990935 (70.5 MiB)[root@221.180.146.95:2222] out: [root@221.180.146.95:2222] out: Done.[root@printserver systemadministrator]# fab uptime[root@221.180.146.97:2222] Executing task 'uptime'[root@221.180.146.97:2222] run: uptime[root@221.180.146.97:2222] out: 22:23:36 up 373 days, 23:55, 1 user, load average: 0.05, 0.03, 0.00[root@221.180.146.97:2222] out: Done.c 说明:在fabric中每一个任务就是一个python的函数,直接到同级目录使用fab调用执行。d fabric中常用api函数的参考local # execute a local commandrun # execute a remote command on all specific hosts, user-level permissions)sudo # sudo a command on the remote server)put # copy over a local file to a remote destination)get # download a file from the remote server)prompt # prompt user with text and return the input (like raw_input))reboot # reboot the remote system, disconnect, and wait for wait seconds)
2 我的需求是:有多台nginx缓存服务器,需要批量的去清除缓存
使用Fabric的具体流程如下:
(1)将需要批量删除的uri写入到apk_list.txt,然后批量上传到所有远端nginx缓存服务器
(2)批量执行远端服务器的删除缓存的脚本
1 删除nginx cache的脚本/usr/local/bin/nginx_purge_cache.sh#!/bin/bash#Date: 2013-06-27#Auther: budong########################################################说明:# 1.本脚本用于清除nginx缓存文件# 2.查看你的nginx是根据什么作为key来hash的,我的设置是 proxy_cache_key $uri$is_args$args;# 因此nginx会根据$uri$is_args$args作为key进行hash,因此可以模拟nginx对一个key进行再# hash找到相应的文件路径,删除(具体可随意找个缓存文件 more 一下看看)# 3.缓存设置 proxy_cache_path /data/mumayi/cache levels=1:2 keys_zone=cache_one:6000m inactive=15d max_size=200g;# 根据相应的配置,请做相应修改测试# 4.uri格式请按照同级目录下apk_list.txt中填写#####################################################while read -r linedo md5uri=`echo -n $line | md5sum | awk '{ print $1 }'` filepath=`echo "$md5uri" | awk '{print "/data/mumayi/cache/"substr($0,length($0),1)"/"substr($0,length($0)-2,2)"/"$0}'` rm -rf $filepath #echo $filepathdone < /usr/local/bin/apk_list.txt2 需要删除的cache文件的列表(只加了一个/test/test.apk做测试)/usr/local/bin/apk_list.txt/test/test.apk3 fabric脚本,雪藏已久,现在贡献出来fabfile.py#!/usr/local/python2.7/bin/python2.7# -*- coding:utf8 -*-import sysfrom fabric.api import *reload(sys)sys.setdefaultencoding('utf-8')#ssh 尝试连接的次数env.connection_attempts = 3 #不再读取本机的known_hosts,防止出现key变了,发生异常env.disable_known_hosts = True#网络超时时间env.timeout = 60#跳过有问题的ipenv.skip_bad_hosts = True#任务并行env.parallel = Trueenv.roledefs = { 'host1':['root@221.180.146.95:2222'],'host2':['root@221.180.146.97:2222']}env.passwords = {'root@221.180.146.95:2222':'password', 'root@221.180.146.97:2222':'password', } @roles('host1')def upload_apk_list(): put('/usr/local/bin/apk_list.txt','/usr/local/bin/apk_list.txt')@roles('host1')def rm_apk_cache(): run('/bin/sh /usr/local/bin/nginx_purge_cache.sh') run('/usr/local/nginx/sbin/nginx -s stop') run('/usr/local/nginx/sbin/nginx')4 批量删除缓存时执行:[root@printserver systemadministrator]# fab upload_apk_list[root@221.180.146.95:2222] Executing task 'upload_apk_list'[root@221.180.146.95:2222] put: /usr/local/bin/apk_list.txt -> /usr/local/bin/apk_list.txtDone.[root@printserver systemadministrator]# fab rm_apk_cache[root@221.180.146.95:2222] Executing task 'rm_apk_cache'[root@221.180.146.95:2222] run: /bin/sh /usr/local/bin/nginx_purge_cache.sh[root@221.180.146.95:2222] run: /usr/local/nginx/sbin/nginx -s stop[root@221.180.146.95:2222] run: /usr/local/nginx/sbin/nginxDone.[root@printserver systemadministrator]# 5 当然上面实际只删除了一台服务器的缓存,可以在host1中继续增加主机或者定义新的role都行。
三:是时候结束了
自言自语:
业务上有需求,偶尔需要删除所有的nginx缓存服务器上的违法文件。如果是手动的一台一台登录上去、删除,真的有些抗不住,这个任务用fabric配合自己写的删除nginx缓存的脚本比较完美。这篇文章,本想写的好些,按说包含的内容也不少,无奈还是力不从心,总觉的那个地方不对劲,就当做是自己的回忆吧,对以前的工作做个总结。
参考资料:
What is Fabric:http://www.pythonforbeginners.com/systems-programming/how-to-use-fabric-in-python/
使用Fabric自动化你的任务:http://www.cnblogs.com/holbrook/archive/2012/03/05/2380398.html
Python fabric实现远程操作和部署: http://wklken.me/posts/2013/03/25/python-tool-fabric.html
配置 fabric 穿越跳板机: http://mozillazg.com/2013/12/python-fabric-gateway.html
官网:http://docs.fabfile.org/en/1.5/index.html