Alex 的 Hadoop 菜鸟教程: 第18课 用Http的方式访问HDFS

$ sudo yum install hadoop-httpfs配置

编辑/etc/hadoop/conf/core-site.xml

<property> <name>hadoop.proxyuser.httpfs.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.httpfs.groups</name> <value>*</value> </property> 这边是定义可以使用httpfs的用户组和host，写*就是不限制

配置好之后重启hadoop启动HttpFs

$ sudo service hadoop-httpfs start使用HttpFs

打开浏览器访问:14000/webhdfs/v1?op=LISTSTATUS&user.name=httpfs 可以看到

{"FileStatuses": {"FileStatus": [{"pathSuffix": "hbase","type": "DIRECTORY","length": 0,"owner": "hbase","group": "hadoop","permission": "755","accessTime": 0,"modificationTime": 1423446940595,"blockSize": 0,"replication": 0},{"pathSuffix": "tmp","type": "DIRECTORY","length": 0,"owner": "hdfs","group": "hadoop","permission": "1777","accessTime": 0,"modificationTime": 1423122488037,"blockSize": 0,"replication": 0},{"pathSuffix": "user","type": "DIRECTORY","length": 0,"owner": "hdfs","group": "hadoop","permission": "755","accessTime": 0,"modificationTime": 1423529997937,"blockSize": 0,"replication": 0},{"pathSuffix": "var","type": "DIRECTORY","length": 0,"owner": "hdfs","group": "hadoop","permission": "755","accessTime": 0,"modificationTime": 1422945036465,"blockSize": 0,"replication": 0}]}}这个&user.name=httpfs 表示用默认用户 httpfs 访问，默认用户是没有密码的。

webhdfs/v1 这是HttpFs的根目录

访问:14000/webhdfs/v1/user?op=LISTSTATUS&user.name=httpfs 可以看到

{"FileStatuses": {"FileStatus": [{"pathSuffix": "cloudera","type": "DIRECTORY","length": 0,"owner": "root","group": "hadoop","permission": "755","accessTime": 0,"modificationTime": 1423472508868,"blockSize": 0,"replication": 0},{"pathSuffix": "hdfs","type": "DIRECTORY","length": 0,"owner": "hdfs","group": "hadoop","permission": "700","accessTime": 0,"modificationTime": 1422947019504,"blockSize": 0,"replication": 0},{"pathSuffix": "history","type": "DIRECTORY","length": 0,"owner": "mapred","group": "hadoop","permission": "1777","accessTime": 0,"modificationTime": 1422945692887,"blockSize": 0,"replication": 0},{"pathSuffix": "hive","type": "DIRECTORY","length": 0,"owner": "hive","group": "hadoop","permission": "755","accessTime": 0,"modificationTime": 1423123187569,"blockSize": 0,"replication": 0},{"pathSuffix": "hive_people","type": "DIRECTORY","length": 0,"owner": "root","group": "hadoop","permission": "755","accessTime": 0,"modificationTime": 1423216966453,"blockSize": 0,"replication": 0},{"pathSuffix": "hive_people2","type": "DIRECTORY","length": 0,"owner": "root","group": "hadoop","permission": "755","accessTime": 0,"modificationTime": 1423222237254,"blockSize": 0,"replication": 0},{"pathSuffix": "impala","type": "DIRECTORY","length": 0,"owner": "root","group": "hadoop","permission": "755","accessTime": 0,"modificationTime": 1423475272189,"blockSize": 0,"replication": 0},{"pathSuffix": "root","type": "DIRECTORY","length": 0,"owner": "root","group": "hadoop","permission": "700","accessTime": 0,"modificationTime": 1423221719835,"blockSize": 0,"replication": 0},{"pathSuffix": "spark","type": "DIRECTORY","length": 0,"owner": "spark","group": "spark","permission": "755","accessTime": 0,"modificationTime": 1423530243396,"blockSize": 0,"replication": 0},{"pathSuffix": "sqoop","type": "DIRECTORY","length": 0,"owner": "hdfs","group": "hadoop","permission": "755","accessTime": 0,"modificationTime": 1423127462911,"blockSize": 0,"replication": 0},{"pathSuffix": "test_hive","type": "DIRECTORY","length": 0,"owner": "root","group": "hadoop","permission": "755","accessTime": 0,"modificationTime": 1423215687891,"blockSize": 0,"replication": 0}]}}

很奇怪的是HttpFs的文档很少，更具体的命令要去 WebHDFS的文档里面看WebHDFS REST API

支持的命令

世界上那些最容易的事情中，拖延时间最不费力。