that output comes after the reduce function not map ... View more

@iamfromsky Did You get any resolution for this? . I am facing the same scenario but no help so far in solving this. Jobs are running form certain tool is not able to connect to HMS and fails with the below error. ERROR org.apache.thrift.transport.TSaslTransport: [pool-5-thread-207]: SASL negotiation failure javax.security.sasl.SaslException: DIGEST-MD5: IO error acquiring password [Caused by org.apache.hadoop.security.token.SecretManager$InvalidToken: token expired or does not exist: HIVE_DELEGATION_TOKEN ... View more

Hi,   after set the parameter "-max_cached_file_handles=0 " as your workaround shows me, I got another issue, it's agent heartbeat timeout. the ticket URL as below: <a href="https://community.cloudera.com/t5/Support-Questions/Cloudera-Manager-agent-bad-healthy/m-p/283865#M210854" target="_blank">https://community.cloudera.com/t5/Support-Questions/Cloudera-Manager-agent-bad-healthy/m-p/283865#M210854</a>    My CDH env has been online more than half year, agent heartbeat timeout has never been happened, but after comparing the date of setting  the impala parameter and agent  heartbeat issue date , it seems there are connection, but I am not sure .       what I mean is the agent heartbeat timeout issue happened after I set the impala parameter "-max_cached_file_handles=0 ".   is that impossible ? ... View more

@Sona, Sorry I missed your question in May. For (1), please refer to my previous update. For (2), yes all paths that store Hive databases/tables should be managed by Hive/Sentry, so those paths should be configured under Sentry Synchronization Path Prefixes setting and need to be owned by "hive:hive". The idea of Sentry is to have everything managed by "hive" so that no one can do direct modifications without going through Hive/Sentry. Cheers Eric ... View more

Hi iamfromsky, To confirm, are you able to change your LB's setting to check less frequent to see if the number of error/warning will also reduce? However, based on my experience, the error produced by LB's tcp check normally is SASL related, not like "java.net.SocketException: Connection reset". It would be good to modify LB setting to confirm though. Cheers Eric ... View more

any soulution for this issue ?   CDH 6.2.0 hue so many little bugs, why ?   sqoop action in editor or oozie could not execute correctly.    even i can't sumit sqoop job in oozie . pls have  a detail like below:   [06/Jul/2019 19:51:44 +0800] resource DEBUG GET /admin/configuration Got response in 16ms: {"oozie.email.smtp.auth":"false","oozie.service.ELService.functions.coord-job-submit-data":"\n coord:dataIn=org.apache.oozie.coord.CoordELFunctions#ph1_coord_dataIn_echo,\n coord:dataOut=org.apache.oozie.coord.CoordELFunctions#ph1_coord_dataOut_echo,\n coord:nominalTime=org.apache.oozie.coord.CoordELFunctions#ph1_coord_nominalTime_echo_wrap,\n coord:actualTime=org.apache.oozie.coord.CoordELFunctions#ph1_coord_actualTime_echo_wrap,\n coord:dateOffset=org.apache.oozie.coord.CoordELFunctions#ph1_coord_dateOffset_echo,\n coord:dateTzOffset=org.apache.oozie.coord.CoordELFunctions#ph1_coord_dateTzOffset_echo,\n coord:formatTime=org.apache.oozie.coord.CoordELFunctions#ph1_coord_formatTime_echo,\n coord:epochTime=org.apache.oozie.coord.CoordELFunctions#ph1_coord_epochTime_echo,\n coord:actionId=org.apache.oozie.coord.CoordELFunctions#ph1_coord_actionId_echo,\n coord:name=org.apache.oozie.coord.CoordELFunctions#ph1_coord_name_echo,\n coord:conf=org.apache.oozie.coord.CoordELFunctions#coord_conf,\n coord:user=org.apache.oozie.coord.CoordELFunctions#coord_user,\n coord:databaseIn=org.apache.oozie.coord.HCatELFunctions#ph1_coord_databaseIn_echo,\n coord:databaseOut=org.apache.oozie.coord.HCatELFunctions#ph1_coord_databaseOut_echo,\n coord:tableIn=org.apache.oozie.coord.HCatELFunctions#ph1_coord_tableIn_echo,\n coord:tableOut=org.apache.oozie.coord.HCatELFunctions#ph1_coord_tableOut_echo,\n coord:dataInPartitionFilter=org.apache.oozie.coord.HCatELFunctions#ph1_coord_dataInPartitionFilter_echo,\n coord:dataInPartitionMin=org.apache.oozie.coord.HCatELFunctions#ph1_coord_dataInPartitionMin_echo,\n coord:dataInPartitionMax=org.apache.oozie.coord.HCatELFunctions#ph1_coord_dataInPartitionMax_echo,\n coord:dataInPartitions=org.apache.oozie.coord.HCatELFunctions#ph1_coor... [06/Jul/2019 19:51:44 +0800] exceptions_renderable ERROR Potential detail: 'statement' [06/Jul/2019 19:51:44 +0800] exceptions_renderable ERROR Potential trace: [('/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/apps/oozie/src/oozie/views/editor2.py', 413, '_submit_workflow_helper', 'job_id = _submit_workflow(request.user, request.fs, request.jt, workflow, mapping)'), ('/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/apps/oozie/src/oozie/views/editor2.py', 453, '_submit_workflow', 'job_id = submission.run()'), ('/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/desktop/libs/liboozie/src/liboozie/submission2.py', 58, 'decorate', 'deployment_dir = self.deploy()'), ('/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/desktop/libs/liboozie/src/liboozie/submission2.py', 370, 'deploy', "action.data['type'] in ('sqoop', 'sqoop-document') and action.data['properties']['statement'] in '--hive-import'):")] [06/Jul/2019 19:51:44 +0800] middleware INFO Processing exception: Workflow 提交失败: Traceback (most recent call last): File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11-py2.7.egg/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11-py2.7.egg/django/utils/decorators.py", line 185, in inner return func(*args, **kwargs) File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/apps/oozie/src/oozie/decorators.py", line 115, in decorate return view_func(request, *args, **kwargs) File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/apps/oozie/src/oozie/decorators.py", line 77, in decorate return view_func(request, *args, **kwargs) File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/apps/oozie/src/oozie/views/editor2.py", line 369, in submit_workflow return _submit_workflow_helper(request, workflow, submit_action=reverse('oozie:editor_submit_workflow', kwargs={'doc_id': workflow.id})) File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/apps/oozie/src/oozie/views/editor2.py", line 415, in _submit_workflow_helper raise PopupException(_('Workflow submission failed'), detail=smart_str(e), error_code=200)   if i use sqoop in hue editor:    ... 83 more 22:23:44.775 [4430f9a6-d62d-47db-add0-b8c79715be8f main] WARN hive.metastore - Failed to connect to the MetaStore Server... 22:23:44.776 [4430f9a6-d62d-47db-add0-b8c79715be8f main] INFO hive.metastore - Waiting 1 seconds before next connection attempt. 22:23:45.776 [4430f9a6-d62d-47db-add0-b8c79715be8f main] INFO hive.metastore - Trying to connect to metastore with URI thrift://oyoshbddnprd2.ahotels.tech:9083 22:23:45.779 [4430f9a6-d62d-47db-add0-b8c79715be8f main] ERROR org.apache.thrift.transport.TSaslTransport - SASL negotiation failure javax.security.sasl.SaslException: GSS initiate failed could you give me some advises?     it's really impact too much, cause many bi user and etl user can't use this function./ ... View more

The HBase shell currently only prints out ASCII printable range of characters, and not unicode, to make it easier to pass around values. In practice, HBase keys are often not designed to be readable and are binary forms (such as encoded integers of hashed values, etc.).   That said, the HBase shell is a programmable JRuby console, so you can use HBase Java APIs within it to get a desired output if you are going to be relying on HBase shell for your scripting work.   Here's a simple example:   hbase(main):013:0> config = org.apache.hadoop.hbase.HBaseConfiguration.create => #<Java::OrgApacheHadoopConf::Configuration:0x4a864d4d> hbase(main):014:0> table = org.apache.hadoop.hbase.client.HTable.new(config, 't') => #<Java::OrgApacheHadoopHbaseClient::HTable:0x5e85c21b> hbase(main):015:0> scanner = table.getScanner(Scan.new()) => #<Java::OrgApacheHadoopHbaseClient::ClientScanner:0x5aa76ad2> hbase(main):030:0> scanner.each do |row| hbase(main):031:1* key = String.from_java_bytes(row.getRow()) hbase(main):032:1> puts "'#{key}'" hbase(main):033:1> end '我' ... View more

CDH5.10.2 should have the fix for that specific issue.   ... View more

how exactly spark streaming can handle it, can you give as exampl ? thanks ... View more

thanks.   my pain point is when i finished a spark , java, hive, sqoop job, i tested it on Linux System , mostly it's normal.   then i deploy the job in oozie by Hue, it always throw some errors like Class not found . i know it because some jars missed, then i am going to search which jars is missed, if found , i add this jar file.   so i mean i don't know which jar files is needed while i deploy these job in oozie, i need the try and try to found missed jars.  this is my pain point.   these jobs executed on Linux System model is normal  i think it's because the job load all the classpath jar files, so it's no this kind issue. ... View more