When importing data from Oracle using Sqoop, it fails with the following error: Error: java.lang.RuntimeException: java.lang.RuntimeException: java.sql.SQLException: ORA-09817: Write to audit file failed. Linux-x86_64 Error: 28: No space left on device Additional information: 12 ORA-02002: error while writing to audit trail Cause: This issue occurs when there is no sufficient space in the /var/log/audit directory on the Oracle server. Solution: To resolve the issue, clear the space on the /var/log/audit directory of the Oracle server. ... View more

For now, it is not possible to encrypt the data during its transmission from Sqoop. This is a known limitation, Jira SQOOP-917 is already in place for the feature. ... View more

If the mysqldump is for different version other than Hive 2.1.1000 in HDP 2.6. Then do the following: 1. Stop Hive services from Ambari. 2. Create new database under MySQL as say hive2: mysql> create database hive2; Query OK,1 row affected (0.00 sec) mysql> grant all privileges on hive2.* to 'hive'@'%' identified by'hive'; Query OK,0 rows affected (0.00 sec) 3. Restore database as: mysql -u hive -phive hive2 < dumpfilename.sql 4. Update database connection string for mysql under Ambari -> Hive configs. 5. Save configurations and try restarting. Since there is different in VERSION, service startup would fail. 6. Run Hive schematool command to upgrade the schema as below: hive@ssnode260 bin]$ /usr/hdp/2.6.0.3-8/hive2/bin/schematool -upgradeSchema -dbType mysql 7. Restart Hive services from Ambari. If the Hive metadata version is same as Hive 2.1.1000 in HDP 2.6, then follow steps 1 through 5. ... View more

SYMPTOM When running Hive queries from Resource Manager log the following error is displayed: 2017-02-07 15:08:32,140 ERROR impl.MetricsSinkAdapter (MetricsSinkAdapter.java:publishMetricsFromQueue(148)) - Got sink exception, retry in 4600ms org.apache.hadoop.metrics2.MetricsException: Failed to putMetrics at org.apache.hadoop.metrics2.sink.timeline.HadoopTimelineMetricsSink.putMetrics(HadoopTimelineMetricsSink.java:216) at org.apache.hadoop.metrics2.impl.MetricsSinkAdapter.consume(MetricsSinkAdapter.java:186) at org.apache.hadoop.metrics2.impl.MetricsSinkAdapter.consume(MetricsSinkAdapter.java:43) at org.apache.hadoop.metrics2.impl.SinkQueue.consumeAll(SinkQueue.java:87) at org.apache.hadoop.metrics2.impl.MetricsSinkAdapter.publishMetricsFromQueue(MetricsSinkAdapter.java:134) at org.apache.hadoop.metrics2.impl.MetricsSinkAdapter$1.run(MetricsSinkAdapter.java:88) Caused by: java.net.UnknownHostException: http at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at java.net.Socket.<init>(Socket.java:425) at java.net.Socket.<init>(Socket.java:280) ROOT CAUSE This issue occurs when reverse lookup returns incorrect hostname for the IP address. RESOLUTION To resolve this issue, fix the DNS issue under /etc/resolv.conf. ... View more

The table definition "LINES TERMINATED BY" only supports newline '\n' right now. This is a known issue and Jira Hive 11996 has already been raised for the issue: To handle the newline characters within the data, you can use the Omniture Data format which uses a EscapedLineReader which gets around Omniture's pesky escaped tabs and newlines. Please note that the data files need to include '\' characters before the newline character within the data and run the below command in sequence and required jars are attached along with data file: add jar /tmp/omnituredata-1.0.2-SNAPSHOT-jar-with-dependencies.jar; add jar /tmp/omnituredata-1.0.2-SNAPSHOT-javadoc.jar; add jar /tmp/omnituredata-1.0.2-SNAPSHOT-sources.jar; add jar /tmp/omnituredata-1.0.2-SNAPSHOT.jar; (Note: jars are available on the HDFS /tmp folder). CREATE TABLE test8(id string,desc string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS INPUTFORMAT 'org.rassee.omniture.hadoop.mapred.OmnitureDataFileInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat' LOCATION '/apps/hive/warehouse/test8'; Sample file under HDFS location '/apps/hive/warehouse/test8' is as: [hive@sindhu root]$ hdfs dfs -cat /apps/hive/warehouse/test8/file.txt id desc 1 Hi\ I am a member and would like to open savings accts for both my kids aged 12 and 16.\ Is that possible and what documents do I need to bring?\ Also do I need to make an appt first?\ Thx! Also, the inputformat as TEXT does not understand the escaped newline characters. 4 rows selected (0.165 seconds) 0: jdbc:hive2://sindhu:2181/> CREATE TABLE test9(id string,desc string) 0: jdbc:hive2://sindhu:2181/> ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' 0: jdbc:hive2://sindhu:2181/> STORED AS textfile LOCATION '/apps/hive/warehouse/test9'; No rows affected (0.209 seconds) 0: jdbc:hive2://sindhu:2181/> select * from test9; +---------------------------------------------------------------------------------------+-------------------+--+ | test9.id | test9.desc | +---------------------------------------------------------------------------------------+-------------------+--+ | id | desc | | 1 | Hi\ | | I am a member and would like to open savings accts for both my kids aged 12 and 16.\ | NULL | | Is that possible and what documents do I need to bring?\ | NULL | | Also do I need to make an appt first?\ | NULL | | Thx! | NULL | | 2 | hi jihidp\ | | uiunoo! | NULL | | 3 | hi who are you\ | | talking with | NULL | +--------------------------+ ... View more