005 python连接hive

环境安装

1. hive
   hive 安装包配置
2. python配置hive环境

   pip install sasl
   pip install thrift
   pip install thrift-sasl
   pip install PyHive
   

   注：安装sasl的时候可能会报错，导致安装不上，这个时候去sasl下载地址下载配置版本的sasl

   # 安装适配版本的whl
   pip install sasl-0.2.1-cp37-cp37m-win_amd64.whl
   

python 连接hive测试

1. 连接代码

   from pyhive import hive
   
   def main():
       conn = hive.Connection(
           host='192.168.1.18',
           port=10000,
           username='hive',
           database='default',
           auth='NOSASL'
       )
   
       cursor = conn.cursor()
       cursor.execute('show databases')
       res = cursor.fetchall()
       for x in res:
       	print(x)
       conn.close()
       
   if __name__ == "__main__":
       main()
   

问题解决

1. thrift.transport.TTransport.TTransportException: Could not start SASL: b’Error in sasl_client_start (-4) SASL(-4): no mechanism available: Unable to find a callback: 2’

   conn = hive.Connection(
   	host='192.168.1.18',
   	port=10000,
   	username='hive',
   	database='default',
   	auth='NOSASL'  # 添加该项，则不会报上面的错误
   )
   

2. 若连接不报错，也不返回，则需要修改hive-site.xml文件-

   # 添加
   <property>
     <name>hive.server2.authentication</name>
     <value>NOSASL</value>
     <description>
       Expects one of [nosasl, none, ldap, kerberos, pam, custom].
       Client authentication types.
         NONE: no authentication check
         LDAP: LDAP/AD based authentication
         KERBEROS: Kerberos/GSSAPI authentication
         CUSTOM: Custom authentication provider
                 (Use with property hive.server2.custom.authentication.class)
         PAM: Pluggable authentication module
         NOSASL:  Raw transport
     </description>
   </property>
   

   重启hiveserver2。

参考

1. pycharm使用pyhive连接hive
2. Python连接Hive(基于PyHive)