論壇首頁 綜合技術論壇

[sqoop1 error] java.lang.ClassNotFoundException org.apache.hadoop.hive.conf.Hiv

瀏覽 112 次
精華帖 (0) :: 良好帖 (0) :: 新手帖 (0) :: 隱藏帖 (0)
作者 正文
   發表時間:2019-06-03  

?

?

問題:

[zkkafka@yanfabu2-37 ~]$ sqoop import \
> --connect jdbc:mysql://10.156.50.36:3306/mqh \
> --username root \
> --password root \
> --table device \
> --fields-terminated-by '\t' \
> --delete-target-dir \
> --num-mappers 1 \
> --hive-import \
> --hive-database test \
> --hive-table hive_bbs_product_snappy;
Warning: /home/zkkafka/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/zkkafka/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/05/29 12:00:27 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
19/05/29 12:00:27 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/05/29 12:00:27 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/05/29 12:00:27 INFO tool.CodeGenTool: Beginning code generation
Wed May 29 12:00:27 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
19/05/29 12:00:27 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `device` AS t LIMIT 1
19/05/29 12:00:27 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `device` AS t LIMIT 1
19/05/29 12:00:27 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/zkkafka/hadoop
注: /tmp/sqoop-zkkafka/compile/0edae8bed32dad40d1fe8ca424e30387/device.java使用或覆蓋了已過時的 API。
注: 有關詳細信息, 請使用 -Xlint:deprecation 重新編譯。
19/05/29 12:00:31 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-zkkafka/compile/0edae8bed32dad40d1fe8ca424e30387/device.jar
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/zkkafka/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/zkkafka/hbase/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
19/05/29 12:00:32 INFO tool.ImportTool: Destination directory device deleted.
19/05/29 12:00:32 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/05/29 12:00:32 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/05/29 12:00:32 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/05/29 12:00:32 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/05/29 12:00:32 INFO mapreduce.ImportJobBase: Beginning import of device
19/05/29 12:00:32 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/05/29 12:00:32 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
Wed May 29 12:00:39 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
19/05/29 12:00:39 INFO db.DBInputFormat: Using read commited transaction isolation
19/05/29 12:00:40 INFO mapreduce.JobSubmitter: number of splits:1
19/05/29 12:00:40 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1558676658010_0008
19/05/29 12:00:41 INFO impl.YarnClientImpl: Submitted application application_1558676658010_0008
19/05/29 12:00:41 INFO mapreduce.Job: The url to track the job: http://master1:8088/proxy/application_1558676658010_0008/
19/05/29 12:00:41 INFO mapreduce.Job: Running job: job_1558676658010_0008
19/05/29 12:00:49 INFO mapreduce.Job: Job job_1558676658010_0008 running in uber mode : false
19/05/29 12:00:49 INFO mapreduce.Job:  map 0% reduce 0%
19/05/29 12:00:57 INFO mapreduce.Job:  map 100% reduce 0%
19/05/29 12:00:59 INFO mapreduce.Job: Job job_1558676658010_0008 completed successfully
19/05/29 12:01:00 INFO mapreduce.Job: Counters: 30
	File System Counters
		FILE: Number of bytes read=0
		FILE: Number of bytes written=128603
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=87
		HDFS: Number of bytes written=156
		HDFS: Number of read operations=4
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=2
	Job Counters 
		Launched map tasks=1
		Other local map tasks=1
		Total time spent by all maps in occupied slots (ms)=5465
		Total time spent by all reduces in occupied slots (ms)=0
		Total time spent by all map tasks (ms)=5465
		Total vcore-milliseconds taken by all map tasks=5465
		Total megabyte-milliseconds taken by all map tasks=5596160
	Map-Reduce Framework
		Map input records=6
		Map output records=6
		Input split bytes=87
		Spilled Records=0
		Failed Shuffles=0
		Merged Map outputs=0
		GC time elapsed (ms)=103
		CPU time spent (ms)=1380
		Physical memory (bytes) snapshot=168935424
		Virtual memory (bytes) snapshot=2104311808
		Total committed heap usage (bytes)=93323264
	File Input Format Counters 
		Bytes Read=0
	File Output Format Counters 
		Bytes Written=156
19/05/29 12:01:00 INFO mapreduce.ImportJobBase: Transferred 156 bytes in 27.7324 seconds (5.6252 bytes/sec)
19/05/29 12:01:00 INFO mapreduce.ImportJobBase: Retrieved 6 records.
19/05/29 12:01:00 INFO mapreduce.ImportJobBase: Publishing Hive/Hcat import job data to Listeners for table device
Wed May 29 12:01:00 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
19/05/29 12:01:00 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `device` AS t LIMIT 1
19/05/29 12:01:00 WARN hive.TableDefWriter: Column register_time had to be cast to a less precise type in Hive
19/05/29 12:01:00 INFO hive.HiveImport: Loading uploaded data into Hive
19/05/29 12:01:00 ERROR hive.HiveConfig: Could not load org.apache.hadoop.hive.conf.HiveConf. Make sure HIVE_CONF_DIR is set correctly.
19/05/29 12:01:00 ERROR tool.ImportTool: Import failed: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
	at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:50)
	at org.apache.sqoop.hive.HiveImport.getHiveArgs(HiveImport.java:392)
	at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:379)
	at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:337)
	at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:241)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:537)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:44)
	... 12 more

?

解決方法:
cp /home/zkkafka/hive/lib/hive-exec-**.jar  /home/zkkafka/sqoop/lib/

?

?

?

?

?

?

?

?

?

?

?

?

?

?

?

捐助開發者?

在興趣的驅動下,寫一個免費的東西,有欣喜,也還有汗水,希望你喜歡我的作品,同時也能支持一下。 當然,有錢捧個錢場(支持支付寶和微信 以及扣扣群),沒錢捧個人場,謝謝各位。

?

個人主頁http://knight-black-bob.iteye.com/



?
?
?謝謝您的贊助,我會做的更好!

論壇首頁 綜合技術版

跳轉論壇:
Global site tag (gtag.js) - Google Analytics 全天上海快3计划网 铁力市| 凤凰县| 阿克陶县| 汾阳市| 长宁县| 舞钢市| 三原县| 彭山县| 神木县| 崇信县| 于田县| 措勤县| 肥城市| 施甸县| 九寨沟县| 仪陇县| 全南县| 商都县| 内黄县| 昭苏县| 安溪县| 闸北区| 盐池县| 连平县| 突泉县| 平南县| 乌拉特前旗| 吉隆县| 曲阜市| 腾冲县| 安西县| 古浪县| 博白县| 姚安县| 儋州市| 怀来县| 安阳县| 阿坝|