使用MySQL作为Hive的Metastore
使用MySQL作为Hive的Metastore
前提是成功安装了HIVE和MYSQL
在hive-site.xml中添加如下内容,指定METASTORE的地址以及连接方式
<property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://10.20.151.10:3306/hive?characterEncoding=UTF-8</value> <description>JDBC connect string for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> <description>Driver class name for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>hive_user</value> <description>username to use against metastore database</description> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>123</value> <description>password to use against metastore database</description> </property>
然后登陆到HIVE客户端,创建一个表试试
[gpadmin1@hadoop5 hive-0.6.0]$ bin/hive Hive history file=/tmp/gpadmin1/hive_job_log_gpadmin1_201106081130_1156785421.txt hive> show tables; FAILED: Error in metadata: javax.jdo.JDOFatalDataStoreException: Unknown database 'hive' NestedThrowables: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'hive' FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
报错了,提示很明显,识别不到名称为hive的database,难道要自己创建?试试
[Intranet root@www.linuxidc.com /var/lib/mysql] #mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or g. Your MySQL connection id is 41 Server version: 5.5.12 MySQL Community Server (GPL) Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | test | +--------------------+ 4 rows in set (0.00 sec) mysql> create database hive; Query OK, 1 row affected (0.00 sec) mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | hive | | mysql | | performance_schema | | test | +--------------------+ 5 rows in set (0.00 sec)
再登陆到HIVE里看看
[gpadmin1@hadoop5 hive-0.6.0]$ bin/hive Hive history file=/tmp/gpadmin1/hive_job_log_gpadmin1_201106081130_544334815.txt hive> show table; FAILED: Parse Error: line 0:-1 mismatched input '<EOF>' expecting EXTENDED in show statement hive> show tables; OK Time taken: 5.173 seconds hive> CREATE TABLE u_tmp1 (id1 INT, > id2 int > ) > ROW FORMAT DELIMITED > FIELDS TERMINATED BY ','; OK Time taken: 0.266 seconds hive> show tables; OK u_tmp1 Time taken: 0.197 seconds hive>
注:
修改数据库的binlog
Java代码
set global binlog_format=’MIXED’;
READ-COMMITTED需要把bin-log以mixed方式来记录
否则进入hive,会如下错误
FAILED: Error in metadata: javax.jdo.JDOException: Couldnt obtain a new sequence (unique id) : Binary logging not possible. Message: Transaction level ‘READ-COMMITTED’ in InnoDB is not safe for binlog mode ‘STATEMENT’