Cloudera Community
nuspengcheng
user rank
Contributor

Re: Sqoop - Can't parse input data

by Contributor in Support Questions
‎04-12-2017 05:39 AM
‎04-12-2017 05:39 AM
In my case, both the source database and target database are MySQL, so do you have any idea if the --ctrl-chars extra argument can also be applied to MySQL during sqoop import or sqoop export ? ... View more

Re: Sqoop - Can't parse input data

by Contributor in Support Questions
‎04-12-2017 02:57 AM
‎04-12-2017 02:57 AM
@Sunil Kumar Bolisetty, thanks for your reply. But it seems the '--ctrl-chars' extra arguments is only applicable for Netezza , not applicable for MySQL, right ? ... View more

Sqoop - Can't parse input data

by Contributor in Support Questions
‎04-11-2017 04:58 AM
‎04-11-2017 04:58 AM
Hi All, I am using sqoop 1.4.6 to transfer data between hive and MySQL. The sqoop-import job was succeeded, but the sqoop-export job was failed with the error "Can't parse input data". The main reason I think is that there are some special characters(i.e. ^@) in the HDFS plain file which was imported by sqoop-import as shown in below. The sqoop import command I used is : sqoop import --connect jdbc:mysql://******:3306/smart_voice_data --table sqoop_import_test --username ****** --password ****** --verbose --hive-database hive_smart_voice_data --hive-table hive_sqoop_import_test --lines-terminated-by "\n" --hive-drop-import-delims --split-by id --null-non-string NULL --null-string NULL --fields-terminated-by "|" --delete-target-dir --hive-overwrite The sqoop export command I used is : sqoop export --connect "jdbc:mysql://******:3306/smart_voice_data?useUnicode=true&characterEncoding=gbk" --username ****** --password ****** --table sqoop_hi_filog_goi_vidaa4_normalized --export-dir hdfs://cdh5node1:8020/user/hive/warehouse/hive_smart_voice_data.db/hive_sqoop_import_test/ --input-null-string "\\\\N" --input-null-non-string "\\\\N" --fields-terminated-by '|' --input-lines-terminated-by '\n' Questions: 1 、Is there any way to escape/filter the special character ^@ (\000) when using sqoop to import data into HDFS? 2、 Is there any way to skip the special character ^@(\000) when using sqoop to export data to mysql , or is there any solution to make sqoop-export can parse the special character ^@ ? 3、Is there any way to skip the error and continue the remaining job when doing sqoop-export ? Any help is really appreciated , thanks a lot ! Pengcheng ... View more
Labels:

Re: Atlas application.log keeps increasing

by Contributor in Support Questions
‎01-18-2017 01:38 AM
‎01-18-2017 01:38 AM
@hardik desai @anaik, thanks for your reply. Just like what anaik said, I can see multiple log files, such as application.log.2017-01-17 ... application.log.2017-01.xx within the atlas log folder, which means Atlas enables DailyRollingFileAppender log rotate mechanism by default. Now that there is no ERROR and WARN appearing in application.log, that Audit: UNKNOWN/10.18.218.21 performed request GET so frequently is also a normal case for atlas, right ? I just wonder why atlas needs to perform the GET request so frequently , and is there any place to config the frequence of sending GET request ? ... View more

Atlas application.log keeps increasing

by Contributor in Support Questions
‎01-17-2017 09:27 AM
‎01-17-2017 09:27 AM
Hi All, I added Atlas component on HDP2.5 , the Altlas status shown in Ambari is normal without any alerts as shown in bleow : However, when I watched Atlas log file application.log, in which the below INFO message are printed endlessly , and the size of application.log are also kept increasing. 2017-01-17 17:10:24,253 INFO - [qtp699780352-19 - 3d52c102-c14b-42de-a41a-9cc8d60880d0:] ~ Audit: UNKNOWN/10.18.218.21 performed request GET http://hdp25node21:21000/api/atlas/admin/status (10.18.218.21) at time 2017-01-17T09:10Z (AuditFilter:91) 2017-01-17 17:10:24,253 INFO - [qtp699780352-19 - 3d52c102-c14b-42de-a41a-9cc8d60880d0:] ~ Audit: UNKNOWN/10.18.218.21-10.18.218.21 performed request GET http://hdp25node21:21000/api/atlas/admin/status (10.18.218.21) at time 2017-01-17T09:10Z (AUDIT:104) 2017-01-17 17:10:25,617 INFO - [qtp699780352-17 - 1a0b0647-cd66-41ba-9ff5-1d2e54536b14:] ~ Audit: UNKNOWN/10.18.218.21 performed request GET http://hdp25node21:21000/api/atlas/admin/status (10.18.218.21) at time 2017-01-17T09:10Z (AuditFilter:91) 2017-01-17 17:10:25,617 INFO - [qtp699780352-17 - 1a0b0647-cd66-41ba-9ff5-1d2e54536b14:] ~ Audit: UNKNOWN/10.18.218.21-10.18.218.21 performed request GET http://hdp25node21:21000/api/atlas/admin/status (10.18.218.21) at time 2017-01-17T09:10Z (AUDIT:104) 2017-01-17 17:10:25,686 INFO - [qtp699780352-18 - 7d0371fe-31fc-4e8d-b0e5-fa58df6e6318:] ~ Audit: UNKNOWN/10.18.218.21 performed request GET http://hdp25node21:21000/api/atlas/admin/status (10.18.218.21) at time 2017-01-17T09:10Z (AuditFilter:91) 2017-01-17 17:10:25,686 INFO - [qtp699780352-18 - 7d0371fe-31fc-4e8d-b0e5-fa58df6e6318:] ~ Audit: UNKNOWN/10.18.218.21-10.18.218.21 performed request GET http://hdp25node21:21000/api/atlas/admin/status (10.18.218.21) at time 2017-01-17T09:10Z (AUDIT:104) 2017-01-17 17:10:28,861 INFO - [qtp699780352-12 - 69a31ee6-6c3a-4e3a-8095-6b1a7685d7eb:] ~ Audit: UNKNOWN/10.18.218.21 performed request GET http://hdp25node21:21000/api/atlas/admin/status (10.18.218.21) at time 2017-01-17T09:10Z (AuditFilter:91) 2017-01-17 17:10:28,861 INFO - [qtp699780352-12 - 69a31ee6-6c3a-4e3a-8095-6b1a7685d7eb:] ~ Audit: UNKNOWN/10.18.218.21-10.18.218.21 performed request GET http://hdp25node21:21000/api/atlas/admin/status (10.18.218.21) at time 2017-01-17T09:10Z (AUDIT:104) 2017-01-17 17:10:31,176 INFO - [qtp699780352-19 - 3917084f-90dd-46d0-bd64-a1fd6a9b4633:] ~ Audit: UNKNOWN/10.18.218.21 performed request GET http://hdp25node21:21000/api/atlas/admin/status (10.18.218.21) at time 2017-01-17T09:10Z (AuditFilter:91) 2017-01-17 17:10:31,176 INFO - [qtp699780352-19 - 3917084f-90dd-46d0-bd64-a1fd6a9b4633:] ~ Audit: UNKNOWN/10.18.218.21-10.18.218.21 performed request GET http://hdp25node21:21000/api/atlas/admin/status (10.18.218.21) at time 2017-01-17T09:10Z (AUDIT:104) Is this case above normal since I only saw "INFO" instead of "WARN" , "ERROR" in application.log file ? Or is anything going wrong ? Because from my understanding , the log file are not supposed to be kept increasing so quickly la ? application.log is attached as application.zip , any response or help is really appreciated ! application.zip ... View more

Re: oozie job fails without any errors

by Contributor in Support Questions
‎01-08-2017 09:43 AM
‎01-08-2017 09:43 AM
@simran kaur, I just met the same problem, did you solve this ? If yes , could you give me some hints, many many thanks ! ... View more

Re: NameNode Last Checkpoint script alert definiti...

by Contributor in Support Questions
‎01-07-2017 04:53 AM
‎01-07-2017 04:53 AM
@Yukti Agrawal , I have met just the same problem, after performing the above steps ,then after sometime these alerts are coming again and again , did you solve this problem ? I set dfs.namenode.checkpoint.period to 1 hour , but it seems it did not work, since I checked the fsimage file which is not generated automatically per hour , thanks ... View more

Re: Need help please. I have used Ambari and HDP 2...

by Contributor in Support Questions
‎01-05-2017 02:21 AM
‎01-05-2017 02:21 AM
Hi Prakash, I met just the same problem, did you solve it ? ... View more

Does zeppelin support multiple hive queries in one...

by Contributor in Support Questions
‎12-27-2016 11:20 AM
‎12-27-2016 11:20 AM
Hi All, I just wonder if zeppelin is able to run multiple hive queries in just one paragraph ? If yes , how to do ? if no, how to understand the below description in https://zeppelin.apache.org/docs/0.6.2/interpreter/hive.html "You can also run multiple queries up to 10 by default. Changing these settings is not implemented yet." Any response is welcome, thanks ! ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎04-12-2017 05:39 AM
Community Statistics
Member Since ‎12-27-2016 04:53 AM
Last Visited ‎04-12-2017 05:39 AM
Posts 15
Kudos received 1