大约3亿条记录
代码语言：javascript复制SQL> SELECT count(*) FROM INFO;
COUNT(*)
----------
294239674
SQL>导入Hive
代码语言：javascript复制[root@node1 sqoop-1.4.7]# bin/sqoop import --connect jdbc:oracle:thin:@node1:1521:ORA --username test --password test --table info --hive-import --target-dir temp_table --hive-table info --null-string '\N' --null-non-string '\N' --fields-terminated-by ' 01' --hive-drop-import-delims -m 1 --hive-overwrite --delete-target-dir
Warning: /opt/sqoop-1.4.7/bin/../../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
18/05/25 15:03:27 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
18/05/25 15:03:27 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/05/25 15:03:27 INFO oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop is disabled.
18/05/25 15:03:27 INFO manager.SqlManager: Using default fetchSize of 1000
18/05/25 15:03:27 INFO tool.CodeGenTool: Beginning code generation
18/05/25 15:03:40 INFO manager.OracleManager: Time zone has been set to GMT
18/05/25 15:03:40 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM info t WHERE 1=0
18/05/25 15:03:40 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/hadoop-2.7.5
Note: /tmp/sqoop-root/compile/cc373056b9d4974d8e7a97947a06ac4b/info.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
18/05/25 15:03:42 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/cc373056b9d4974d8e7a97947a06ac4b/info.jar
18/05/25 15:03:43 INFO tool.ImportTool: Destination directory temp_table deleted.
18/05/25 15:03:43 INFO manager.OracleManager: Time zone has been set to GMT
18/05/25 15:03:43 INFO manager.OracleManager: Time zone has been set to GMT
18/05/25 15:03:43 INFO mapreduce.ImportJobBase: Beginning import of info
18/05/25 15:03:43 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
18/05/25 15:03:43 INFO manager.OracleManager: Time zone has been set to GMT
18/05/25 15:03:43 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
18/05/25 15:03:44 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
18/05/25 15:03:46 INFO db.DBInputFormat: Using read commited transaction isolation
18/05/25 15:03:46 INFO mapreduce.JobSubmitter: number of splits:1
18/05/25 15:03:46 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1526097883376_0063
18/05/25 15:03:46 INFO impl.YarnClientImpl: Submitted application application_1526097883376_0063
18/05/25 15:03:46 INFO mapreduce.Job: The url to track the job: http://bigdata03-test:8088/proxy/application_1526097883376_0063/
18/05/25 15:03:46 INFO mapreduce.Job: Running job: job_1526097883376_0063
18/05/25 15:03:56 INFO mapreduce.Job: Job job_1526097883376_0063 running in uber mode : false
18/05/25 15:03:56 INFO mapreduce.Job: map 0% reduce 0%
18/05/25 15:32:01 INFO mapreduce.Job: Task Id : attempt_1526097883376_0063_m_000000_0, Status : FAILED
Error: java.io.IOException: SQLException in nextKeyValue
at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:556)
at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.sql.SQLRecoverableException: No more data to read from socket
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1157)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:350)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
at oracle.jdbc.driver.T4CPreparedStatement.fetch(T4CPreparedStatement.java:1121)
at oracle.jdbc.driver.OracleResultSetImpl.close_or_fetch_from_next(OracleResultSetImpl.java:384)
at oracle.jdbc.driver.OracleResultSetImpl.next(OracleResultSetImpl.java:288)
at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:237)
... 12 more
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
18/05/25 15:32:10 INFO mapreduce.Job: Task Id : attempt_1526097883376_0063_m_000000_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: java.sql.SQLException: Listener refused the connection with the following error:
ORA-12528, TNS:listener: all appropriate instances are blocking new connections
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:170)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:161)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.RuntimeException: java.sql.SQLException: Listener refused the connection with the following error:
ORA-12528, TNS:listener: all appropriate instances are blocking new connections
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:223)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:168)
... 10 more
Caused by: java.sql.SQLException: Listener refused the connection with the following error:
ORA-12528, TNS:listener: all appropriate instances are blocking new connections
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:489)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:553)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:254)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:528)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:216)
... 11 more
Caused by: oracle.net.ns.NetException: Listener refused the connection with the following error:
ORA-12528, TNS:listener: all appropriate instances are blocking new connections
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:399)
at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1140)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:340)
... 19 more
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
18/05/25 15:32:29 INFO mapreduce.Job: Task Id : attempt_1526097883376_0063_m_000000_2, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: java.sql.SQLException: Listener refused the connection with the following error:
ORA-12528, TNS:listener: all appropriate instances are blocking new connections
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:170)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:161)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.RuntimeException: java.sql.SQLException: Listener refused the connection with the following error:
ORA-12528, TNS:listener: all appropriate instances are blocking new connections
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:223)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:168)
... 10 more
Caused by: java.sql.SQLException: Listener refused the connection with the following error:
ORA-12528, TNS:listener: all appropriate instances are blocking new connections
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:489)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:553)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:254)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:528)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:216)
... 11 more
Caused by: oracle.net.ns.NetException: Listener refused the connection with the following error:
ORA-12528, TNS:listener: all appropriate instances are blocking new connections
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:399)
at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1140)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:340)
... 19 more
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
18/05/25 15:32:34 INFO mapreduce.Job: map 100% reduce 0%
18/05/25 15:32:35 INFO mapreduce.Job: Job job_1526097883376_0063 failed with state FAILED due to: Task failed task_1526097883376_0063_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0
18/05/25 15:32:36 INFO mapreduce.Job: Counters: 8
Job Counters
Failed map tasks=4
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=1710308
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=1710308
Total vcore-milliseconds taken by all map tasks=1710308
Total megabyte-milliseconds taken by all map tasks=1751355392
18/05/25 15:32:36 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
18/05/25 15:32:36 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 1,732.3058 seconds (0 bytes/sec)
18/05/25 15:32:36 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
18/05/25 15:32:36 INFO mapreduce.ImportJobBase: Retrieved 0 records.
18/05/25 15:32:36 ERROR tool.ImportTool: Import failed: Import job failed!
[root@node1 sqoop-1.4.7]#
