@ChineduLB , Did you mean that you got the same error while trying to export the sample data you provided earlier? Have you tried to update your driver in case it might be old? Cheers Eric ... View more

@ChineduLB , I have tested in my lab, I can successfully export the data you provided into Teradata table. Below is the output from Teradata table after Sqoop export: accno |acc_co_no|acc_appsys_id|acc_sub_no|acc_pd_ctgy_cd|aml_acc_id|aml_pty_id|accno1 |arng_id|acc_nm |acc_stat_cd_deriv|acc_dom_cntry|acc_opn_dt|acc_cls_dt|cc_no|bnkctr_nm|brn_no|acc_tin |accdm_actv_in|accpr_actv_in|acctr_actv_in|btch_id |lst_row_updt_ts |acc_st_cd|jle_in|prmy_pty_in|acc_pty_role_cd|partitioncolumn|sa_data_date|sa_data_yyyymm|sa_last_updated_timestamp |sa_last_updated_by| ----------------|---------|-------------|----------|--------------|----------|----------|---------------------|-------|---------------------|-----------------|-------------|----------|----------|-----|---------|------|---------|-------------|-------------|-------------|--------|-------------------|---------|------|-----------|---------------|---------------|------------|--------------|-----------------------------|------------------| 4400665121475865|317 |555 | |CCA |1634538008|677509401 |000004400665121475865| |Davidio Lynca Flavour|Open |US |2016-04-23| | | |8BEB00|140361381|1 |1 |1 |20190424|2019-05-08 15:49:11|OT |1 |0 |AUT |1 |2019-06-13 |20190613 |2019-06-13 06:50:32.038000000|qrstasluu | I think you might want to narrow down the issue by reducing the number of columns or number of records in your data to see which ones caused the issue. Cheers Eric ... View more

@ChineduLB Can you please share below info: - the DDL of the table in Teradata - some sample data from the file in HDFS It looks to me some issues in the data, I suggest you also narrow down the issue by slowly reducing the number of columns from the file and see which one is actually causing the issue and which rows. Cheers Eric ... View more

Hi Choon Kiat, Thanks for the confirmation. I have checked further, "Unable to obtain Principal Name for authentication" can happen when the JCE jars are not up to date on the client machine and not able to use the encryption key provided by Kerberos KDC. Can you please download the latest/appropriate JCE jars and place them in directory $JAVA_HOME/jre/lib/security. The files are: US_export_policy.jar local_policy.jar These files are dependent on the Java version. For Java8, it can be found here: https://www.oracle.com/java/technologies/javase-jce8-downloads.html Please give it a try and see how it goes. Regards Eric ... View more

Hi Gabor, Can you share the krb5.conf file on the HMS host for review? Cheers Eric ... View more

Hi Your JDBC connection string: jdbc:hive2://{host}:{port}/{database};AuthMech=1;KrbRealm=Domain;KrbHostFQDN={server};KrbServiceName=hive;KrbAuthType=2;LogLevel=6;LogPath=c:\ProgramData\MIT\Kerberos5\log.log Is "Domain" the real string or a placeholder? From your krb5, your Realm should be "AIU.XXXXXX", so I want to confirm. Cheers Eric ... View more

@TanChoonKiat Your JDBC Connection string is wrong, assuming that you are using Cloudera Hive JDBC Driver: jdbc:hive2://xx.xx.xxx.xx:10001/default;AuthMech=1;principal=hive/domain@domain;KrbHostFQDN=10.11.121.21;KrbServiceName=hive;KrbAuthType=2;LogLevel=6;LogPath=c:\ProgramData\MIT\Kerberos5\log.log 1. No need for "principal" 2. Can you please check what's the full principals from your KDC server? KrbHostFQDN should be normally the domain name, not IP. 3. you are missing KrbRealm, which should be the part after "@" from your principal Please try to correct those and try again. Full JDBC doc can be found here: https://docs.cloudera.com/documentation/other/connectors/hive-jdbc/latest/Cloudera-JDBC-Driver-for-Apache-Hive-Install-Guide.pdf Cheers Eric ... View more

@TanChoonKiat Can you please share your JDBC connection string? Cheers Eric ... View more

@aleesmin , Couple of questions: 1. I am not familiar with golden gate , but are those inserts and updates run inside transactions? 2. have you checked if the duplicate data in the same resulted text file in HDFS or on different files? 3. I see that you used "--direct", have you tried non-direct mode? Cheers Eric ... View more

Please use the query ID hive_20200308155555_9a444e04-9523-4443-b312-64a94e5a5138 and search in HS2 log to find out the exact reason for the failure. Looks like beeline did not capture the full error from server side. Cheers Eric ... View more

Hi @Dombai_Gabor, Can you check the keytab file under /var/run/cloudera-scm-agent/process/*HIVEMETASTORE* directories are valid and you can kinit using those keytab files without issues? Also, does the error happen on start up or after running for a while? Thanks Eric ... View more

@MrCarlo , Have you tried to create a VIEW on top of the table in Oracle and then import the VIEW instead? It is supported in Sqoop as mentioned below: http://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_selecting_the_data_to_import Cheers Eric ... View more

@RobinRo , BDR will pick random hosts to run jobs, have you checked whether the fail always from the same host? This is the first thing I will check. The stacktrace pointed out that the error happened when BDR job tried to get file info from HDFS and timed out, so if you confirm the issue is from the same host, then we should check the connectivity between this host and NN. Cheers Eric ... View more

@chhaya_vishwaka, If "mysql -h tdicsadn01.td.com -u rman -p rman2019" still not working, please share the output of below query from MySQL database: SELECT * FROM mysql.user WHERE User = 'rman'\G Need to make sure that there are no other entries might interfere with the permissions. Cheers Eric ... View more

Is table-b's value fixed? You mentioned 1 or 2. Are they pre-defined? If you know the values up-front, then you don't need to do cross join, just add conditions based on the fixed number of values, like below: SELECT * FROM table-a WHERE TABLE-a.date_col <= from_unixtime(unix_timestamp()-(1*24*60*60))) OR TABLE-a.date_col <= from_unixtime(unix_timestamp()-(2*24*60*60))) OR TABLE-a.date_col <= from_unixtime(unix_timestamp()-(3*24*60*60))) ... Not sure if it still meet your requirement. Cheers ... View more

@sridarvs Based on https://www.techonthenet.com/oracle/errors/ora06502.php, it looks like a data conversion issue. Have you tried to just import into HDFS without Hive table and see if it makes any difference? Can you also share the DDL of table TXN in Oracle? Cheers Eric ... View more

@Ellyl , Please also share the exact version of either HDP or CDH in use, this can help to determine the code that we need to check and see why we hit NPE. Cheers Eric ... View more

Does it have to be Cross JOIN? You are generating tens or hundreds of million rows, it is very intensive. Is it true that for every row in table-b, you have to join across in table-a? Cheers Eric ... View more

@ElistonCole, If you have access to O'Reilly, below Hive book is useful. It was published in 2018, fairly new compare with others: https://learning.oreilly.com/library/view/apache-hive-essentials/9781788995092/ Regarding Spark, below book published this year is good: https://learning.oreilly.com/library/view/learning-spark-2nd/9781492050032/ Hope that helps. Cheers Eric ... View more

Glad it works now, thanks for sharing. Cheers ... View more

@dewi If HS2 was unable to connect to HMS, then you should check HMS log as well. What error is reported over there? Cheers Eric ... View more

@Kureikana , Have you tried "hdfs dfs -get" command? If not working, what error do you get? Cheers ... View more

@mirelagrio , Sorry, I misunderstood your question. So for the audit log entry that appear in the web UI, can you confirm that it was logged using the policy that you have audit log OFF? Could it be coming from other policies? Cheers Eric ... View more

@mirelagrio, I don't think this feature will stop logging audit info in Solr. I believe that audit will still get logged and Ranger provides an ability to easily exclude them in the UI for users to see real user's entries. So changing this will not save you disk space. Cheers Eric ... View more