HiveConf
java class for current Hive configuration options
Metastore Conf
All the metadata for Hive tables and partitions are stored in Hive Metastore.
there are 3 different ways to setup metastore server using different Hive configurations:
An embedded metastore is mainly used for unit tests. Only one process can connect to the metastore at a time, so it is not really a practical solution but works well for unit tests.Derby is the default database for the embedded metastore.
In local metastore setup, each Hive Client will open a connection to the datastore and make SQL queries against it. The following config will set up a metastore in a MySQL server. Make sure that the server is accessible from the machines where Hive queries are executed since this is a local store. Also make sure the jdbc client library is in the classpath of Hive Client.
In remote metastore setup, all Hive Clients will make a connection to a metastore server which in turn queries the datastore (MySQL in this example) for metadata. Metastore server and client communicate using Thrift Protocol.
Starting with Hive 0.5.0, you can start a Thrift server by executing the following command:
#hive --service metastore
In my situation, I configure the remote metastore mode
/path/to/hive-0.12.0-bin/conf/hive-site.xml
<property>
<name>hive.metastore.uris</name>
<value>thrift://192.168.122.1:9083</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://192.168.122.1/hivedb?createDatabaseIfNotExist=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
<description>username to use against metastore database</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive123</value>
<description>password to use against metastore database</description>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
<description>location of default database for the warehouse</description>
</property>
#hive --service metastore #hive --service metastore -p <port_num>
You should create user 'hive' and grant privileges to him in mysql datatbase;
If you are using MySQL as the datastore for metadata, put MySQL client libraries in HIVE_HOME/lib before starting Hive Client or HiveMetastore Server.
when run command #hive --service metastore
there are some errors :
MetaException(message:Version information not found in metastore
Solution:
#schematool -help
#schematool -dbType mysql -initSchema
Could not create ServerSocket on address 0.0.0.0/0.0.0.0:9083
Solution:
#netstat -tulpn |grep 9083
#kill -s 9 5776 //metastore daemon is running,so kill it with its pid 5776
when start hive CLI
hive>show databases;
eroor: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
Solution:
#hive -hiveconf hive.root.logger=DEBUG,console //check detail logs
---------------------
Others:
<property>
<name>hive.server2.thrift.port</name>
<value>10000</value>
<description>Port number of HiveServer2 Thrift interface.
Can be overridden by setting $HIVE_SERVER2_THRIFT_PORT</description>
</property>
<property>
<name>hive.server2.thrift.bind.host</name>
<value>192.168.122.1</value>
<description>Bind host on which to run the HiveServer2 Thrift interface.
Can be overridden by setting $HIVE_SERVER2_THRIFT_BIND_HOST</description>
</property>
Note:Starting in Hive release 0.11.0, HCatalog is installed and configured with Hive. The HCatalog server is the same as the Hive metastore.
references:
https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/cdh5ig_hive_schema_tool.html
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_18_4.html?scroll=topic_18_4_1_unique_1__title_508_unique_1
相关推荐
hive metastore hive metastore是hive的元数据管理服务,实际应用中很多第三方框架需要访问metastore服务,如spark,impala等。同样hive metastore也提供了java接口。 使用 import org.apache.hadoop.hive.conf....
hive 开发UDF 使用maven工程 引发jar包缺失 hive 开发UDF 使用maven工程 引发jar包缺失
hive-metastore-3.1.2.jar
在hive java api的时候 : 需要导入hive运行所需的jar包,导入jar包有两种方式可实现 1.使用maven依赖直接修改配置文件,以下为配置内容:(不推荐,会下载额外很多无用包,而且时间很长在1小时左右) 2.挑选必须...
bde2020_hive-metastore-postgresql.tar
hive-metastore-2.3.2.jar 修改了配置文件,适合oracle的jar包,
资源来自pypi官网。 资源全名:hive_metastore_client-1.0.0-py3-none-any.whl
独立的Hive Metastore 该项目可在本地运行Hive Metastore,或将其嵌入到容器中。 尽管是公开的,但该项目目前尚未记录,并且可能仍承载某些基础结构特定配置。 该项目的灵感来自和多亏了他
背景:由于Hive需要开启权限管理,安装网上教程,开启权限配置,重启集群后。 使用root用户登录,进入Hive命令行界面。 执行 create database test; 发现报错: Authorization failed:No privilege 'Create' found ...
hive3.1.2编译后的驱动包
hive:/default/foobar hive -e "SELECT * FROM foobar" > $[OUTPUT]协议 !hive:/default/foobar <- !hive:/default/sample_07 [hive] CREATE TABLE IF NOT EXISTS ${env:TABLEOUT} LIKE ${env:TABLEIN}; INSERT ...
hadoop-hue-hive-cookbook TODO:在此处输入食谱说明。 支持的平台 TODO:列出您支持的平台。 属性 钥匙 类型 描述 ... " recipe[hadoop-hue-hive::default] " ] } 许可证和作者 作者:: YOUR_NAME ()
Hive 事务设计
"jdbc:hive://localhost:10000/default". Currently, the only dbname supported is "default". 2.从hive导出数据到本地时注意指定的目录/home/bigdata/syg。如果指定的目录存在,hive会先删除该目录下的所有文件。...
自述文件概括Django 开放式计算工具版本:0.1 Hive 是一个试图帮助其他需要计算能力的项目的项目。 它是一个网络平台 ( ),它试图让任何人都可以使用高性能集群。 本项目使用docker运行计算程序。 这允许使安装过程...
hive相关jar包,连接大数据hive所用jar包,最新3.1.1版本
详细调研hivemetastore管理机制.zip
## Hive metastore 使用达梦数据库存储元数据 - 使用说明 1. **在达梦数据库中创建metastore将使用的用户和表空间** 2. **配置hive-site.xml** 3. **元数据初始化** ### 常见问题
This book is your go-to resource for using Hive: authors Scott Shaw, Ankur Gupta, David Kjerrumgaard, and Andreas Francois Vermeulen take you through learning HiveQL, the SQL-like language specific ...