The hive shell runs successfully! There are no errors or warnings with hava.
But when running application.sh, the startup fails
/usr/local/spark/bin/spark-submit \
--class cn.spark.sql.Demo \
--num-executors 3 \
--driver-memory 512m \
--executor-memory 512m \
- -executor-cores 3 \
--files /usr/local/hive/conf/hive-site.xml \
--driver-class-path /usr/local/hive/lib/mysql- connector-java.jar \
/usr/local/java/sql/sparkstudyjava.jar \
and error message:
Exception in thread "main" java.lang.IllegalArgumentException: Error while
instantiating'org.apache.spark.sql.hive.HiveSessionState':
...
Caused by: java.lang.IllegalArgumentException : Error while instantiating
'org.apache.spark.sql.hive.HiveExternalCatalog':
...
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Unable to
instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
...
Caused by: java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop .hiv e.ql.metadata.SessionHiveMetaStoreClient
...
Caused by: MetaException(message:Hive Schema version 1.2.0 does not match
metastore's schema version 2.1.0 Metastore is not upgraded or corrupt )
...
I tried many ways to solve this error, but the error still occurs.
How to fix it?
Execute The following command & see if the output is different from /usr/local/hive.
$which hive
If both are the same hive directory, please add the following properties in hive-site.xml.
hive.metastore.schema.verification
false
Environment: spark2.11 hive2.2 hadoop2.8.2
< /p>
The hive shell runs successfully! There are no errors or warnings with hava.
But when running application.sh, the startup fails
/usr/local/spark/bin/spark-submit \
--class cn.spark.sql.Demo \
--num-executors 3 \
--driver-memory 512m \
--executor-memory 512m \
- -executor-cores 3 \
--files /usr/local/hive/conf/hive-site.xml \
--driver-class-path /usr/local/hive/lib/mysql- connector-java.jar \
/usr/local/java/sql/sparkstudyjava.jar \
and error message:
Exception in thread "main" java.lang.IllegalArgumentException: Error while
instantiating'org.apache.spark.sql.hive.HiveSessionState':
...
Caused by: java.lang.IllegalArgumentException : Error while instantiating
'org.apache.spark.sql.hive.HiveExternalCatalog':
...
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Unable to
instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
...
Caused by: java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop .hive.ql. metadata.SessionHiveMetaStoreClient
...
Caused by: MetaException(message:Hive Schema version 1.2.0 does not match
metastore's schema version 2.1.0 Metastore is not upgraded or corrupt)
...
I tried many ways to solve this error, but the error still occurs.
How to fix it?
Maybe hive refers to another version of the configuration unit (different configuration).
Execute the following command & see if the output is the same as /usr/local /hive is different.
$which hive
If both are the same hive directory, please add it in hive-site.xml The following properties.
hive.metastore.schema.verification
false