Redis集群方案,Codis安装测试

1,关于豌豆荚开源的Codis

Codis是豌豆荚使用Go和C语言开发、以代理的方式实现的一个Redis分布式集群解决方案,且完全兼容Twemproxy。Twemproxy对于上一层的应用来说, 连接Codis Proxy(Redis代理服务)和连接原生的Redis服务器没有明显的区别,上一层应用能够像使用单机的 Redis一样对待。Codis底层会处理请求的转发、不停机的数据迁移等工作, 所有底层的一切处理, 对于客户端来说是透明的。总之,可以简单的认为后台连接的是一个内存无限大的Redis服务。Codis遵循MIT开源协议发布,更多关于Codis的信息请登录其在GitHub的主页查看。

Codis 是一个分布式 Redis 解决方案, 对于上层的应用来说, 连接到 Codis Proxy 和连接原生的 Redis Server 没有明显的区别 (不支持的命令列表), 上层应用可以像使用单机的 Redis 一样使用, Codis 底层会处理请求的转发, 不停机的数据迁移等工作, 所有后边的一切事情, 对于前面的客户端来说是透明的, 可以简单的认为后边连接的是一个内存无限大的 Redis 服务.

Codis 由四部分组成:

Codis Proxy (codis-proxy) Codis Manager (codis-config) Codis Redis (codis-server) ZooKeeper

架构图:

项目地址: https://github.com/wandoulabs/codis 中文说明文档: https://github.com/wandoulabs/codis/blob/master/doc/tutorial_zh.md Redis 开源集群方案: Redis 3.0 rc4已经发布了,也是一种方案可以研究下,线上系统不建议使用。

2,安装Codis&配置

参考文档: 首先按照golang,,下载地址:https://golang.org/dl/,最新的1.4.2版本。 如果被墙使用golang中国下载。

cd /usr/loca/wget https://storage-amd64.tar.gztar -zxvf go1-amd64.tar.gz

设置环境变量,vi /etc/profile

#set go & codis environmentGOROOT=/usr/local/goPATH=$PATH:$GOROOT/binGOPATH=/usr/local/codis

下载codis

cd /usr/localgit clone https://github.com/wandoulabs/codis.git#将codis的pkg包拷贝到$GOPATH目录。mkdir -p /usr/local/codis/src/github.com/wandoulabs/codiscp /usr/local/codis/pkg/ /usr/local/codis/cmd/ /usr/local/codis/src/github.com/wandoulabs/codis -R

安装codis,需要下载依赖包。比较慢,依赖

#安装编译工具yum groupinstall “Development Tools”#执行编译测试脚本,编译go和reids。sh /usr/local/codis/bootstrap.sh go build -o bin/codis-proxy ./cmd/proxygo build -o bin/codis-config ./cmd/cconfigmake -j4 -C extern/redis-2.8.13/make[1]: Entering directory `/usr/local/codis/extern/redis-2.8.13’cd src && make allmake[2]: Entering directory `/usr/local/codis/extern/redis-2.8.13/src’rm -rf redis-server redis-sentinel redis-cli redis-benchmark redis-check-dump redis-check-aof *.o *.gcda *.gcno *.gcov redis.info lcov-html(cd ../deps && make distclean)make[3]: Entering directory `/usr/local/codis/extern/redis-2.8.13/deps'(cd hiredis && make clean) > /dev/null || true(cd linenoise && make clean) > /dev/null || true(cd lua && make clean) > /dev/null || true(cd jemalloc && [ -f Makefile ] && make distclean) > /dev/null || true(rm -f .make-*)make[3]: Leaving directory `/usr/local/codis/extern/redis-2.8.13/deps'(rm -f .make-*)echo STD=-std=c99 -pedantic >> .make-settingsecho WARN=-Wall >> .make-settingsecho OPT=-O2 >> .make-settingsecho MALLOC=jemalloc >> .make-settingsecho CFLAGS= >> .make-settingsecho LDFLAGS= >> .make-settingsecho REDIS_CFLAGS= >> .make-settingsecho REDIS_LDFLAGS= >> .make-settingsecho PREV_FINAL_CFLAGS=-std=c99 -pedantic -Wall -O2 -g -ggdb -I../deps/hiredis -I../deps/linenoise -I../deps/lua/src -DUSE_JEMALLOC -I../deps/jemalloc/include >> .make-settingsecho PREV_FINAL_LDFLAGS= -g -ggdb -rdynamic >> .make-settings(cd ../deps && make hiredis linenoise lua jemalloc)make[3]: Entering directory `/usr/local/codis/extern/redis-2.8.13/deps'(cd hiredis && make clean) > /dev/null || true(cd linenoise && make clean) > /dev/null || true(cd lua && make clean) > /dev/null || true(cd jemalloc && [ -f Makefile ] && make distclean) > /dev/null || true(rm -f .make-*)(echo “” > .make-cflags)(echo “” > .make-ldflags)MAKE hirediscd hiredis && make staticMAKE linenoisecd linenoise && makemake[4]: Entering directory `/usr/local/codis/extern/redis-2.8.13/deps/linenoise’cc -Wall -Os -g -c linenoise.cMAKE luacd lua/src && make all CFLAGS=”-O2 -Wall -DLUA_ANSI ” MYLDFLAGS=””make[4]: Entering directory `/usr/local/codis/extern/redis-2.8.13/deps/hiredis’cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb net.cmake[4]: Entering directory `/usr/local/codis/extern/redis-2.8.13/deps/lua/src’cc -O2 -Wall -DLUA_ANSI -c -o lapi.o lapi.cMAKE jemalloccd jemalloc && ./configure –with-jemalloc-prefix=je_ –enable-cc-silence CFLAGS=”-std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops ” LDFLAGS=””cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb hiredis.cmake[4]: Leaving directory `/usr/local/codis/extern/redis-2.8.13/deps/linenoise’cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb sds.ccc -O2 -Wall -DLUA_ANSI -c -o lcode.o lcode.ccc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb async.ccc -O2 -Wall -DLUA_ANSI -c -o ldebug.o ldebug.ccc -O2 -Wall -DLUA_ANSI -c -o ldo.o ldo.ccc -O2 -Wall -DLUA_ANSI -c -o ldump.o ldump.car rcs libhiredis.a net.o hiredis.o sds.o async.omake[4]: Leaving directory `/usr/local/codis/extern/redis-2.8.13/deps/hiredis’cc -O2 -Wall -DLUA_ANSI -c -o lfunc.o lfunc.ccc -O2 -Wall -DLUA_ANSI -c -o lgc.o lgc.ccc -O2 -Wall -DLUA_ANSI -c -o llex.o llex.ccc -O2 -Wall -DLUA_ANSI -c -o lmem.o lmem.ccc -O2 -Wall -DLUA_ANSI -c -o lobject.o lobject.cchecking for xsltproc… /usr/bin/xsltprocchecking for gcc… gcccc -O2 -Wall -DLUA_ANSI -c -o lopcodes.o lopcodes.ccc -O2 -Wall -DLUA_ANSI -c -o lparser.o lparser.ccc -O2 -Wall -DLUA_ANSI -c -o lstate.o lstate.ccc -O2 -Wall -DLUA_ANSI -c -o lstring.o lstring.cchecking whether the C compiler works… cc -O2 -Wall -DLUA_ANSI -c -o ltable.o ltable.ccc -O2 -Wall -DLUA_ANSI -c -o ltm.o ltm.ccc -O2 -Wall -DLUA_ANSI -c -o lundump.o lundump.cyeschecking for C compiler default output file name… a.outchecking for suffix of executables… cc -O2 -Wall -DLUA_ANSI -c -o lvm.o lvm.ccc -O2 -Wall -DLUA_ANSI -c -o lzio.o lzio.cchecking whether we are cross compiling… cc -O2 -Wall -DLUA_ANSI -c -o strbuf.o strbuf.ccc -O2 -Wall -DLUA_ANSI -c -o lauxlib.o lauxlib.cnochecking for suffix of object files… cc -O2 -Wall -DLUA_ANSI -c -o lbaselib.o lbaselib.cochecking whether we are using the GNU C compiler… cc -O2 -Wall -DLUA_ANSI -c -o ldblib.o ldblib.cyeschecking whether gcc accepts -g… cc -O2 -Wall -DLUA_ANSI -c -o liolib.o liolib.ccc -O2 -Wall -DLUA_ANSI -c -o lmathlib.o lmathlib.cyeschecking for gcc option to accept ISO C89… cc -O2 -Wall -DLUA_ANSI -c -o loslib.o loslib.cnone neededchecking how to run the C preprocessor… cc -O2 -Wall -DLUA_ANSI -c -o ltablib.o ltablib.ccc -O2 -Wall -DLUA_ANSI -c -o lstrlib.o lstrlib.ccc -O2 -Wall -DLUA_ANSI -c -o loadlib.o loadlib.ccc -O2 -Wall -DLUA_ANSI -c -o linit.o linit.cgcc -Ecc -O2 -Wall -DLUA_ANSI -c -o lua_cjson.o lua_cjson.ccc -O2 -Wall -DLUA_ANSI -c -o lua_struct.o lua_struct.cchecking for grep that handles long lines and -e… /usr/bin/grepchecking for egrep… /usr/bin/grep -Echecking for ANSI C header files… cc -O2 -Wall -DLUA_ANSI -c -o lua_cmsgpack.o lua_cmsgpack.ccc -O2 -Wall -DLUA_ANSI -c -o lua.o lua.clua_cmsgpack.c: In function ‘table_is_an_array’:lua_cmsgpack.c:370:21: warning: variable ‘max’ set but not used [-Wunused-but-set-variable]long count = 0, max = 0, idx = 0;^cc -O2 -Wall -DLUA_ANSI -c -o luac.o luac.ccc -O2 -Wall -DLUA_ANSI -c -o print.o print.car rcu liblua.a lapi.o lcode.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o strbuf.o lauxlib.o lbaselib.o ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o lua_cjson.o lua_struct.o lua_cmsgpack.o # DLL needs all object filesranlib liblua.acc -o lua lua.o liblua.a -lm cc -o luac luac.o print.o liblua.a -lm yesmake[4]: Leaving directory `/usr/local/codis/extern/redis-2.8.13/deps/lua/src’checking for sys/types.h… yeschecking for sys/stat.h… yeschecking for stdlib.h… yeschecking for string.h… yeschecking for memory.h… yeschecking for strings.h… yeschecking for inttypes.h… yeschecking for stdint.h… yeschecking for unistd.h… yeschecking whether byte ordering is bigendian… nochecking size of void *… 8checking size of int… 4checking size of long… 8checking size of intmax_t… 8checking build system type… x86_64-unknown-linux-gnuchecking host system type… x86_64-unknown-linux-gnuchecking whether pause instruction is compilable… yeschecking whether SSE2 intrinsics is compilable… yeschecking for ar… archecking whether __attribute__ syntax is compilable… yeschecking whether compiler supports -fvisibility=hidden… yeschecking whether compiler supports -Werror… yeschecking whether tls_model attribute is compilable… nochecking for a BSD-compatible install… /usr/bin/install -cchecking for ranlib… ranlibchecking for ld… /usr/bin/ldchecking for autoconf… /usr/bin/autoconfchecking for memalign… yeschecking for valloc… yeschecking configured backtracing method… N/Achecking for sbrk… yeschecking whether utrace(2) is compilable… nochecking whether valgrind is compilable… nochecking STATIC_PAGE_SHIFT… 12checking pthread.h usability… yeschecking pthread.h presence… yeschecking for pthread.h… yeschecking for pthread_create in -lpthread… yeschecking for _malloc_thread_cleanup… nochecking for _pthread_mutex_init_calloc_cb… nochecking for TLS… yeschecking whether a program using ffsl is compilable… yeschecking whether atomic(9) is compilable… nochecking whether Darwin OSAtomic*() is compilable… nochecking whether to force 32-bit __sync_{add,sub}_and_fetch()… nochecking whether to force 64-bit __sync_{add,sub}_and_fetch()… nochecking whether Darwin OSSpin*() is compilable… nochecking for stdbool.h that conforms to C99… yeschecking for _Bool… yesconfigure: creating ./config.statusconfig.status: creating Makefileconfig.status: creating doc/html.xslconfig.status: creating doc/manpages.xslconfig.status: creating doc/jemalloc.xmlconfig.status: creating include/jemalloc/jemalloc_macros.hconfig.status: creating include/jemalloc/jemalloc_protos.hconfig.status: creating include/jemalloc/internal/jemalloc_internal.hconfig.status: creating test/test.shconfig.status: creating test/include/test/jemalloc_test.hconfig.status: creating config.stampconfig.status: creating bin/jemalloc.shconfig.status: creating include/jemalloc/jemalloc_defs.hconfig.status: creating include/jemalloc/internal/jemalloc_internal_defs.hconfig.status: creating test/include/test/jemalloc_test_defs.hconfig.status: executing include/jemalloc/internal/private_namespace.h commandsconfig.status: executing include/jemalloc/internal/private_unnamespace.h commandsconfig.status: executing include/jemalloc/internal/public_symbols.txt commandsconfig.status: executing include/jemalloc/internal/public_namespace.h commandsconfig.status: executing include/jemalloc/internal/public_unnamespace.h commandsconfig.status: executing include/jemalloc/internal/size_classes.h commandsconfig.status: executing include/jemalloc/jemalloc_protos_jet.h commandsconfig.status: executing include/jemalloc/jemalloc_rename.h commandsconfig.status: executing include/jemalloc/jemalloc_mangle.h commandsconfig.status: executing include/jemalloc/jemalloc_mangle_jet.h commandsconfig.status: executing include/jemalloc/jemalloc.h commands===============================================================================jemalloc version : 3.6.0-0-g46c0af68bd248b04df75e4f92d5fb804c3d75340library revision : 1CC: gccCPPFLAGS: -D_GNU_SOURCE -D_REENTRANTCFLAGS: -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -fvisibility=hiddenLDFLAGS: EXTRA_LDFLAGS: LIBS: -lpthreadRPATH_EXTRA: XSLTPROC: /usr/bin/xsltprocXSLROOT: PREFIX: /usr/localBINDIR: /usr/local/binINCLUDEDIR: /usr/local/includeLIBDIR: /usr/local/libDATADIR: /usr/local/shareMANDIR: /usr/local/share/mansrcroot: abs_srcroot: /usr/local/codis/extern/redis-2.8.13/deps/jemalloc/objroot: abs_objroot: /usr/local/codis/extern/redis-2.8.13/deps/jemalloc/JEMALLOC_PREFIX : je_JEMALLOC_PRIVATE_NAMESPACE: je_install_suffix: autogen: 0experimental: 1cc-silence: 1debug: 0code-coverage: 0stats: 1prof: 0prof-libunwind: 0prof-libgcc: 0prof-gcc: 0tcache: 1fill: 1utrace: 0valgrind: 0xmalloc: 0mremap: 0munmap: 0dss: 0lazy_lock: 0tls: 1===============================================================================cd jemalloc && make CFLAGS=”-std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops ” LDFLAGS=”” lib/libjemalloc.amake[4]: Entering directory `/usr/local/codis/extern/redis-2.8.13/deps/jemalloc’gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/jemalloc.o src/jemalloc.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/arena.o src/arena.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/atomic.o src/atomic.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/base.o src/base.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/bitmap.o src/bitmap.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk.o src/chunk.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk_dss.o src/chunk_dss.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk_mmap.o src/chunk_mmap.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/ckh.o src/ckh.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/ctl.o src/ctl.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/extent.o src/extent.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/hash.o src/hash.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/huge.o src/huge.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/mb.o src/mb.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/mutex.o src/mutex.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/prof.o src/prof.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/quarantine.o src/quarantine.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/rtree.o src/rtree.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/stats.o src/stats.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/tcache.o src/tcache.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/util.o src/util.cgcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/tsd.o src/tsd.car crus lib/libjemalloc.a src/jemalloc.o src/arena.o src/atomic.o src/base.o src/bitmap.o src/chunk.o src/chunk_dss.o src/chunk_mmap.o src/ckh.o src/ctl.o src/extent.o src/hash.o src/huge.o src/mb.o src/mutex.o src/prof.o src/quarantine.o src/rtree.o src/stats.o src/tcache.o src/util.o src/tsd.omake[4]: Leaving directory `/usr/local/codis/extern/redis-2.8.13/deps/jemalloc’make[3]: Leaving directory Hint: To run ‘make test’ is a good idea ;)make[2]: Leaving directory `/usr/local/codis/extern/redis-2.8.13/src’make[1]: Leaving directory `/usr/local/codis/extern/redis-2.8.13’go test ./pkg/… ./cmd/… -race?_/usr/local/codis/pkg/env[no test files]ok_/usr/local/codis/pkg/models 2.253sok_/usr/local/codis/pkg/proxy/cachepool 0.009s?_/usr/local/codis/pkg/proxy/group[no test files]ok_/usr/local/codis/pkg/proxy/parser0.018s?_/usr/local/codis/pkg/proxy/redisconn [no test files]?_/usr/local/codis/pkg/proxy/redispool [no test files]ok_/usr/local/codis/pkg/proxy/router17.146s?_/usr/local/codis/pkg/proxy/router/topology[no test files]ok_/usr/local/codis/pkg/utils0.012sok_/usr/local/codis/cmd/cconfig 0.019s?_/usr/local/codis/cmd/proxy[no test files]想要成功,就一定要和成功的人在一起,不然反之

Redis集群方案,Codis安装测试

相关文章:

你感兴趣的文章:

标签云: