高并发、大流量网卡调优

周五晚上LVS集群的一台LB由于CPU单核耗尽挂掉了,最后查到原因是网卡调优没有生效,今天查了一下网卡调优的资料,感谢同事higkoo给我讲解了一下这方面的东西,下面是关于我们公司网卡调优的相关知识,欢迎大家共同探讨,让我们的机器跑的更high。

1、Broadcom的网卡建议关闭GRO功能

ethtool -K eth0 gro offethtool -K eth1 gro offethtool -K eth2 gro offethtool -K eth3 gro off

2、关闭irqbalance服务并手动分配网卡中断

service irqbalance stopchkconfig irqbalance off# 查看网卡中断号grep ethx /proc/interrupts# 分配到每颗颗CPU核上cat /proc/irq/{84,85,86,87,88,89,90,91,92,93}/smp_affinityecho 1 > /proc/irq/84/smp_affinityecho 2 > /proc/irq/85/smp_affinityecho 4 > /proc/irq/86/smp_affinityecho 8 > /proc/irq/87/smp_affinityecho 10 > /proc/irq/88/smp_affinityecho 20 > /proc/irq/89/smp_affinityecho 40 > /proc/irq/90/smp_affinityecho 80 > /proc/irq/91/smp_affinityecho 100 > /proc/irq/92/smp_affinityecho 200 > /proc/irq/93/smp_affinityPS:smp_affinity的值可以用下面脚本算哈,此脚本来自:#!/bin/bash#echo “统计cpu的16进制”[ $# -ne 1 ] && echo ‘$1 is Cpu core number’ && exit 1CCN=$1echo “Print eth0 affinity”for((i=0; i<${CCN}; i++))doecho ==============================echo “Cpu Core $i is affinity”((affinity=(1<<i)))echo “obase=16;${affinity}” | bcdone使用方法:sh 脚本名字 空格 cpu核数

3、开启网卡的RPS功能(Linux内核2.6.38或以上版本支持)

# Enable RPS (Receive Packet Steering)rfc=4096cc=$(grep -c processor /proc/cpuinfo)rsfe=$(echo $cc*$rfc | bc)sysctl -w net.core.rps_sock_flow_entries=$rsfefor fileRps in $(ls /sys/class/net/eth*/queues/rx-*/rps_cpus)doecho fff > $fileRpsdonefor fileRfc in $(ls /sys/class/net/eth*/queues/rx-*/rps_flow_cnt)doecho $rfc > $fileRfcdonetail /sys/class/net/eth*/queues/rx-*/{rps_cpus,rps_flow_cnt}

献上一个完整的脚本:

vi /opt/sbin/change_irq.sh#!/bin/bashethtool -K eth0 gro offethtool -K eth1 gro offethtool -K eth2 gro offethtool -K eth3 gro offservice irqbalance stopchkconfig irqbalance offcat /proc/irq/{84,85,86,87,88,89,90,91,92,93}/smp_affinityecho 1 > /proc/irq/84/smp_affinityecho 2 > /proc/irq/85/smp_affinityecho 4 > /proc/irq/86/smp_affinityecho 8 > /proc/irq/87/smp_affinityecho 10 > /proc/irq/88/smp_affinityecho 20 > /proc/irq/89/smp_affinityecho 40 > /proc/irq/90/smp_affinityecho 80 > /proc/irq/91/smp_affinityecho 100 > /proc/irq/92/smp_affinityecho 200 > /proc/irq/93/smp_affinity# Enable RPS (Receive Packet Steering)rfc=4096cc=$(grep -c processor /proc/cpuinfo)rsfe=$(echo $cc*$rfc | bc)sysctl -w net.core.rps_sock_flow_entries=$rsfefor fileRps in $(ls /sys/class/net/eth*/queues/rx-*/rps_cpus)doecho fff > $fileRpsdonefor fileRfc in $(ls /sys/class/net/eth*/queues/rx-*/rps_flow_cnt)doecho $rfc > $fileRfcdonetail /sys/class/net/eth*/queues/rx-*/{rps_cpus,rps_flow_cnt}chmod +x /opt/sbin/change_irq.shecho “/opt/sbin/change_irq.sh” >> /etc/rc.localPS:记得修改网卡中断号,别直接拿来用哦

本文出自 “屌丝运维男” 博客,谢绝转载!

,那段雨骤风狂。人生之旅本就是风雨兼程,是要说曾经拥有,

高并发、大流量网卡调优

相关文章:

你感兴趣的文章:

标签云: