mac 下hadoop安装并运行例子

1 安装

#brew install hadoop

安装的是2.6.0,,目录为/usr/local/Cellar/hadoop,如果想安装其他版本,则下载tar包解压即可。地址: 2 配置 将hadoop可执行路径bin和sbin都配置到环境变量中

export export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/homeexport /sbin/

接下来配置伪分布式: (1) 配置core-site.xml

hadoop cat core-site.xml>>hdfs://localhost:9000>

(2)hdfs的节点备份设置为1 配置hdfs-site.xml

hadoop cat hdfs-site.xml> >1>

(3)配置mapred-site.xml

hadoop cat mapred-site.xml> >hdfs://localhost:9000/>

(4)将hadoop-env.sh里jdk路径配置好即可。 然后执行:格式化namenode

hadoop namenode -format hadoop start-all.shThis script is Deprecated. Instead use start-dfs.sh and start-yarn.shthe classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectivelynative-hadoop library for your platform… using builtin-java classes where applicableStarting namenodes on [localhost]localhost: namenode running as process 7082. Stop it first.localhost: datanode running as process 7163. Stop it first.Starting secondary namenodes []: secondarynamenode running as process 7266. Stop it first.the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectivelynative-hadoop library for your platform… using builtin-java classes where applicablestarting yarn daemonsresourcemanager running as process 7377. Stop it first.localhost: nodemanager running as process 7466. Stop it first. hadoop jps7163 DataNode9619 Jps7466 NodeManager7377 ResourceManager7082 NameNode7266 SecondaryNameNode hadoop

可以看到五个节点都启动了。 3 运行例子wordcount 先创建目录input 以及文件并上传到hdfs

libexec ls inputfile1.txt file2.txt libexec hadoop fs -put input/file*.txt /input libexec hadoop fs -ls /inputfound is deprecated, mapredhdfsoverride properties of core, mapredhdfsrespectively15/03/29 22:46:39 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicableFound 2 itemsr:02 /input/file1.txtr:02 /input/file2.txt libexec hadoop jar share/hadoop/mapreduce/hadoopjar wordcount /input /output运行如果不出错,结果输出到output目录如下: libexec hadoop fs -ls /outputfound is deprecated, mapredhdfsoverride properties of core, mapredhdfsrespectively15/03/29 22:56:30 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicableFound 2 itemsr:14 /output/_SUCCESSr:libexec hadoop fs /found is deprecated, mapredhdfsoverride properties of core, mapredhdfsrespectively15/03/29 22:56:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicableai 2am 3am2 1are 2i 2man 1man2 1shenya 2who 2you 1you2 1 libexec

其实,每个人都是幸福的。只是,你的幸福,常常在别人眼里。

mac 下hadoop安装并运行例子

相关文章:

你感兴趣的文章:

标签云: