david_morin
Explorer

In fact the problem is related to th ...

by Explorer in Support Questions
‎02-17-2019 06:11 PM
‎02-17-2019 06:11 PM
In fact the problem is related to the java Orc api when parallelism is activated (multi-thread) I use Flink and when I set a parallelism > 1 on the Sink that generates Orc files, I face to this issue: data are unreadable. I've seen some tickets about this issue like this one: https://jira.apache.org/jira/browse/ORC-361 At the moment I use a parallelism of 1 but I have to fix this issue in order to scale my ingest pipeline. All help is welcome. Thx ... View more

Orc files in hdf: NullPointerException (RunLengthI...

by Explorer in Support Questions
‎02-11-2019 02:34 PM
‎02-11-2019 02:34 PM
Hello, I face to one error when I try to read my Orc files from Hive (external table) or Pig or with hive --orcfiledump .. These files are generated with Flink using the Orc Java API with Vectorize column. If I create these files locally (/tmp/...), push them to hdfs, then I can read the content of these files from Pig or with the use of External table in Hive. If I change the path and use hdfs directly, then I face to this error : Failure while running task:java.lang.NullPointerException at org.apache.hadoop.hive.ql.io.orc.RunLengthIntegerReaderV2.readValues(RunLengthIntegerReaderV2.java:54) at org.apache.hadoop.hive.ql.io.orc.RunLengthIntegerReaderV2.next(RunLengthIntegerReaderV2.java:302) at org.apache.hadoop.hive.ql.io.orc.TreeReaderFactory$TimestampTreeReader.next(TreeReaderFactory.java:1105) at org.apache.hadoop.hive.ql.io.orc.TreeReaderFactory$StructTreeReader.next(TreeReaderFactory.java:2079) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.next(RecordReaderImpl.java:1082) at org.apache.hadoop.hive.ql.io.orc.OrcNewInputFormat$OrcRecordReader.nextKeyValue(OrcNewInputFormat.java:108) And the same if I get these files locally. ... View more
Labels:

Re: HDP 2.6.4 - Problem during application of Kerb...

by Explorer in Support Questions
‎03-21-2018 11:06 PM
‎03-21-2018 11:06 PM
One HCC thread related to this kind of error: https://community.hortonworks.com/questions/84101/failed-to-start-hive-server2-illegal-character-in.html ... View more

Re: HDP 2.6.4 - Problem during application of Kerb...

by Explorer in Support Questions
‎03-21-2018 11:02 PM
‎03-21-2018 11:02 PM
I pursue in the resolution of my problem and my understanding.. When I activate HDFS Audit in Ranger Configs and restart a Namenode to apply config, I obtain these errors in the log file ' /var/lib/ambari-agent/data/output-XX.txt': call returned (3, '') 2018-03-21 23:44:37,745 - Will retry 7 time(s), caught exception: (u"Execution of 'curl --location-trusted -k --negotiate -u : -b /var/lib/ambari-agent/tmp/cookies/7768 3cd0-2605-4b42-a750-509ecc713ba9 -c /var/lib/ambari-agent/tmp/cookies/77683cd0-2605-4b42-a750-509ecc713ba9 -w '%{http_code}' '{{policymgr_mgr_url}}/login.jsp' --connect -timeout 10 --max-time 12 -o /dev/null 1>/tmp/tmp21P7Ww 2>/tmp/tmpfrrHEy' returned 3. curl: (3) [globbing] nested braces not supported at pos 2\n",). Sleeping for 8 sec(s) ... View more

Re: HDP 2.6.4 - Problem during application of Kerb...

by Explorer in Support Questions
‎03-21-2018 04:19 PM
‎03-21-2018 04:19 PM
Hello, I'm still trying to deploy a "kerberized" cluster HDP with Ambari blueprint. With the Ranger service but with all plugins desactivated, it's ok. A soon as I try to activate plugins, the cluster never starts (Namenodes are still stopped) : "ranger-hdfs-plugin-enabled" : "Yes"... A JVM related to the Namenode is started but that's all. This is the command to format Namenode: org.apache.hadoop.hdfs.server.namenode.NameNode -format -nonInteractive Then, there is no log file related to the namenode in /var/log/hadoop/hdfs/ If someone has some advices or an example of blueprint that works with Kerberos and Ranger (+ plugins) I will be eternally grateful 🙂 ... View more

Re: HDP 2.6.4 - Problem during application of Kerb...

by Explorer in Support Questions
‎03-20-2018 02:30 PM
‎03-20-2018 02:30 PM
I think the problem is related to missing steps during post install but I'm not sure because the error is not really obvious. Example for Hdfs: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_security/content/hdfs_plugin_kerberos.html Questions about that : - If these steps are required, does I have to create this 'rangerlookup' user manually and its keytab (cluster is kerberized) ? - Is it blueprint compliant ? ... View more

Re: HDP 2.6.4 - Problem during application of Kerb...

by Explorer in Support Questions
‎03-12-2018 12:24 AM
‎03-12-2018 12:24 AM
Despite the copy of these ranger-* files before application of Kerberos, I still obtain the same error. ... View more

Re: HDP 2.6.4 - Problem during application of Kerb...

by Explorer in Support Questions
‎03-09-2018 11:25 PM
‎03-09-2018 11:25 PM
Thanks for your reply It seems ok. I spawn a new cluster from scratch and give a feedback. ... View more

HDP 2.6.4 - Problem during application of Kerberos

by Explorer in Support Questions
‎03-09-2018 10:11 PM
‎03-09-2018 10:11 PM
Hello, I face to a problem related to Ranger during application of Kerberos with Ambari onto my HDP cluster (2.6.4) FATAL conf.Configuration (Configuration.java:loadResource(2712)) - error parsing conf file:/.../hadoop/2.6.4.0-91/0/xasecure-audit.xml java.io.FileNotFoundException: /.../hadoop/2.6.4.0-91/0/xasecure-audit.xml (No such file or directory) I've found a temporary work-around by desactivate audit in Ranger configuration. How can I resolve this problem in order to reactivate audit. Thanks in advance, Regards, David ... View more
Contact Me
Online
Offline
Last Visited
‎02-17-2019 06:11 PM
Public Statistics
Date Registered ‎03-09-2018 10:04 PM
Last Visited ‎02-17-2019 06:11 PM
Total Messages Posted 12
Total Kudos Received 1