Hive安装详细步骤

一、下载hive

下载hive——地址：http://mirror.bit.edu.cn/apache/hive/

二、安装mysql

执行以下几个命令安装8.0版本mysql

//1、下载MySQLyum源（8.0版本的）
wget http://dev.mysql.com/get/mysql80-community-release-el7-3.noarch.rpm
（5.7版本的）
wget http://dev.mysql.com/get/mysql57-community-release-el7-10.noarch.rpm
//2、添加MySQLyum源
sudo yum localinstall mysql80-community-release-el7-3.noarch.rpm
(5.7版本）
sudo yum localinstall mysql57-community-release-el7-10.noarch.rpm
//3、MySQL的GPG升级了，需要更新，如果是新安装的MySQL，执行以下脚本即可：
rpm --import https://repo.mysql.com/RPM-GPG-KEY-mysql-2022
//4、安装MySQL客户端
yum -y install mysql-community-server

 开启mysql服务

# 1、开启Mysql服务
sudo service mysqld start
# 2、查看mysql是否开启
sudo servicee mysqld status
# 重启mysql服务
sudo service mysqld restart
# 3、查看初始密码
sudo grep 'temporary password' /var/log/mysqld.log
# 4、进入mysql客户端,输入查询到的初始化密码
mysql -uroot -p

mysql密码设置

# 1、首先要修改密码，高版本的mysql默认必须修改密码才能正常使用，所以如果不修改密码不能做任何事
alter user 'root'@'localhost' identified by '新密码'
注意：因为我们还没有改密码的复杂度，所以这里的密码必须足够复杂，后面会改简单的
# 2、查看mysql初始化密码的策略
SHOW VARIABLES LIKE 'validate_password%'; 
mysql> SHOW VARIABLES LIKE 'validate_password%'; 
+--------------------------------------+-------+
| Variable_name                        | Value |
+--------------------------------------+-------+
| validate_password.check_user_name    | ON    |
| validate_password.dictionary_file    |       |
| validate_password.length             | 6     |
| validate_password.mixed_case_count   | 1     |
| validate_password.number_count       | 1     |
| validate_password.policy             | LOW   |
| validate_password.special_char_count | 1     |
+--------------------------------------+-------+
7 rows in set (0.00 sec)
# 3、修改密码验证强度（重启后就失效）
set global validate_password.policy=LOW; 
# 4、修改密码允许最短长度，不能小于4
set global validate_password.length=6;
# 5、修改简单密码
alter user 'root'@'localhost' identified by '000000';

 设置mysql远程登陆

8.0版本以前

# 1、修改权限
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456' WITH GRANT OPTION;
# %表示所有远程机器
# root表示远程登陆后使用root用户
# *.*表示所有表
# 刷新权限
flush privileges;

8.0版本的因为有新的安全要求，不能自己给自己授权，所以要新建一个用户，通过那个用户来授权

# 1、查看mysql用户
use mysql;
select user, host from user;
mysql> select user, host from user;
+------------------+-----------+
| user             | host      |
+------------------+-----------+
| mysql.infoschema | localhost |
| mysql.session    | localhost |
| mysql.sys        | localhost |
| root             | localhost |
+------------------+-----------+
4 rows in set (0.00 sec)
# 2、新建用户
create user 'hadoop100'@'%' identified by '000000';
# hadoop100表示用户名
# %表示任意ip
# 000000表示该用户的密码，记得如果重启需要从新修改密码验证策略
# 3、为用户授权
grant all on *.* to 'hadoop100'@'%';
# 4、刷新权限
flush privileges;

mysql开机自启动(在linux中执行)

systemctl enable mysqld

三、解压与配置

①进入到存放hive的目录下,输入以下命令解压到指定目录下

tar -zxvf apache-hive-3.1.2-bin.tar.gz -C /opt/modules/

②配置文件重命名

将hive-defaultxml.template重命名为hive-site.xml

将hive-env.sh.template重命名为hive-env.sh

将hive-log4j.properties.template重命名为hive-log4j.properties

③修改配置

1、修改hive-env.sh配置

添加JAVA_HOME和HADOOP_HOME,exportHIVE_CONF_DIR（即hive的conf目录地址）

2、修改hive-site.xml配置

<!--判断mysql下是否有这个数据库，没有的话创建-->
<property>
	<name>javax.jdo.option.ConnectionURL</name>
	<value>jdbc:mysql://hadoop100:3306/metastore?createDatabaseIfNotExist=true</value>
	<description>JDBC connect string for a JDBC metastore</description>
</property>
<!--使用mysql driver驱动，默认是hive内置数据库derby驱动-->
<property>
	<name>javax.jdo.option.ConnectionDriverName</name>
	<value>com.mysql.jdbc.Driver</value>
	<description>Driver class name for a JDBC metastore</description>
</property>
<!--mysql账号-->
<property>
	<name>javax.jdo.option.ConnectionUserName</name>
	<value>root</value>
	<description>username to use against metastore database</description>
</property>
<!--mysql密码-->
<property>
	<name>javax.jdo.option.ConnectionPassword</name>
	<value>000000</value>
	<description>password to use against metastore database</description>
</property>
<!--显示查询出的数据的字段名称-->
<property>
  <name>hive.cli.print.header</name>
  <value>true</value>
  <description>Whether to print the names of the columns in query output.</description>
</property>
<!--在hive中显示当前所在数据库名称-->
<property>
  <name>hive.cli.print.current.db</name>
  <value>true</value>
  <description>Whether to include the current database in the Hive prompt.</description>
</property>
<!--使得查询单列数据不会执行mapreduce-->
<property>
  <name>hive.fetch.task.conversion</name>
  <value>more</value>
  <description>
    Some select queries can be converted to single FETCH task minimizing latency.
    Currently the query should be single sourced not having any subquery and should not have
    any aggregations or distincts (which incurs RS), lateral views and joins.
    1. minimal : SELECT STAR, FILTER on partition columns, LIMIT only
    2. more    : SELECT, FILTER, LIMIT only (TABLESAMPLE, virtual columns)
  </description>
</property>

3、修改hive-log4j.properties配置

设置log路径用以存放hive的log日志文件
 hive.log.dir=/opt/modules/hive-3.1.2/logs

4、拷贝数据库驱动包到hive的lib目录中

cp mysql-connector-java-5.1.27/mysql-connector-java-5.1.27-bin.jar /opt/modules/hive-3.1.2/lib/

四、打开hive

1、启动Hive，在hive目录下

bin/hive

2、修改HDFS系统中关于Hive的一些目录权限

/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6/bin/hadoop fs -chmod 777 /tmp/ 
/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6/bin/hadoop fs -chmod 777 /user/hive/warehouse

五、配置HiveServer2（注：1.几版本的hive不用配）

<property>
  <name>hive.server2.long.polling.timeout</name>
  <value>5000</value>
  <description>Time in milliseconds that HiveServer2 will wait, before responding to asynchronous calls that use long polling</description>
</property>
<property>
  <name>hive.server2.thrift.bind.host</name>
  <value>hadoop100</value>
  <description>Bind host on which to run the HiveServer2 Thrift interface.
  Can be overridden by setting $HIVE_SERVER2_THRIFT_BIND_HOST</description>
</property>
<property>
  <name>hive.server2.enable.doAs</name>
  <value>false</value>
  <description>
   Setting this property to true will have HiveServer2 execute
    Hive operations as the user making the calls to it.
  </description>
</property>

1、检查端口：
 $ sudo netstat -antp | grep 10000

2、启动服务：
$ bin/hive --service hiveserver2

3、连接服务：
$ bin/beeline
 beeline> !connect jdbc:hive2://hadoop100:10000 

六、安装hive出现的一些错误

1、Exception in thread "main" java.lang.RuntimeException: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

解决方案：在hive 配置文件hive-site.xml 中找到${system:java.io.tmpdir}，

并把所有的${system:java.io.tmpdir}都替换成具体目录，如/opt/modules/hive-0.13.1-cdh5.3.6/tmp即可

2、使用hive语句时出现：

Failed with exception java.io.IOException:java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:user.name%7D

解决方案：

①进入hive-site.xml配置文件中

将

<property>
    <name>hive.exec.local.scratchdir</name>
    <value>/opt/modules/hive-0.13.1-cdh5.3.6/tmp/${System:user.name}</value>
    <description>Local scratch space for Hive jobs</description>
</property>

配置改为如下配置即可，其它地方可以不用改

<property>
    <name>hive.exec.local.scratchdir</name>
    <value>/opt/modules/hive-0.13.1-cdh5.3.6/tmp/${user.name}</value>
    <description>Local scratch space for Hive jobs</description>
</property>

3、在Hive中无法执行任何命令java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.Session

hive的数据库MySQL在安装的时候没有初始化

# 在MySQL中
# 删除mysql中的元数据库
drop databases metastore;
# 新建一个元数据库
create database metastore;

# 在命令行中,重新初始化
schematool -dbType mysql -initSchema