Re: when inserting data from hive parquet table wi...

takefumi_ohide · ‎11-07-2016

In case of Hive on Tez, decreasing tez.grouping.max-size might help you. I faced almost same problem before and I decreased tez.grouping.max-size from 1GB to 256MB, then that problem almost(not perfectly) solved.

takefumi_ohide · ‎08-18-2016

I confirmed that it's not necessary to increase maxSsf, only maxBufSize should be increased for this solution. Thanks @Hajime for your comment! And I'll edit my first answer.

takefumi_ohide · ‎08-18-2016

To tell the truth, I don't know what is SSF and what increasing SSF means... There's no need to increase SSF possibly. I'll try later, sorry.

takefumi_ohide · ‎08-17-2016

I found a solution... but it's just a quick hack, not an appropriate fix I think. modify hue/desktop/core/ext-py/sasl-0.1.1/sasl/saslwrapper.cpp to increase maxBufSize. build it with "python setup.py build" and get a new _saslwrapper.so replace hue/build/env/lib/python2.6/site-packages/sasl-0.1.1-py2.6-linux-x86_64.egg/_saslwrapper.so with it restart Hue. Detail of step 1 and 2 below. Step 1. In hue/desktop/core/ext-py/sasl-0.1.1/sasl/saslwrapper.cpp you can find a line "ClientImpl() : conn(0), cbIndex(0), maxBufSize(65535), minSsf(0), maxSsf(65535), externalSsf(0), secret(0) {}" around line 35, so modified maxBufSize to larger value. Step 2. In hue/desktop/core/ext-py/sasl-0.1.1, execute "python setup.py build", so you've got _saslwrapper.so in hue/desktop/core/ext-py/sasl-0.1.1/uild/lib.linux-x86_64-2.6/_saslwrapper.so

takefumi_ohide · ‎08-16-2016

Hue 3.10.0 + HiveServer2 with hive.server2.thrift.sasl.qop=auth-conf sometimes(or always on certain conditions) reports a SASL error shown below: Error in sasl_decode (-1) SASL(-1): generic failure: Unable to find a callback: 32775 (code THRIFTTRANSPORT): TTransportException('Error in sasl_decode (-1) SASL(-1): generic failure: Unable to find a callback: 32775',) I can see this message when executing, for example, "select * from db.table limit 12;", but cannot see when executing "select * from db.table limit 11;"... These queries are same only except for "limit" clause. I searched with that error message, so I found this post: https://groups.google.com/a/cloudera.org/forum/#!topic/hue-user/0LfySJBERsc A poster said "Seems hue doesn't play nice with hive.server2.thrift.sasl.qop set to auth-conf. Works OK with only auth though." in this post. Is that also true on Hue 3.10.0? Is it impossible to use sasl.qop=auth-conf on Hue 3.10.0?

takefumi_ohide · ‎08-03-2016

Thank you very much! It went well!

takefumi_ohide · ‎08-03-2016

I found that Ambari creates Hive View on its restart automatically. But I want to block that. How do I block Ambari from creating Hive View on its restart?

takefumi_ohide · ‎06-23-2016

@Predrag Minovic Thank you very much!!! I've succeeded in adding a JournalNode.

takefumi_ohide · ‎06-23-2016

Is using Ambari REST API the only way for it? I found some articles using Ambari REST API for this problem. https://community.hortonworks.com/articles/14612/process-for-moving-journal-nodes-from-one-host-to.html https://gist.github.com/uprush/441861460f1d4c93b13f http://zdatainc.com/2015/09/add-journalnode-to-ambari-managed-hadoop-cluster/

takefumi_ohide · ‎06-23-2016

A host that JournalNode is assigned went down by hardware trouble. So I want to move that JournalNode to another host. However, I can't find the way how to move or add JournalNode by Ambari. Is it impossible to move or add JournalNode by Ambari?

takefumi_ohide · ‎06-09-2016

@Sagar Shimpi Thank you for your reply. I understood.

takefumi_ohide · ‎06-09-2016

@vperiasamy Thank you for your answer! I hope I could configure it on Ambari WebUI...

takefumi_ohide · ‎06-08-2016

I found it may be possible to set a session timeout of Ranger Admin WebUI by modifying <session-timeout> parameter in web.xml (usually placed in /usr/hdp/${version}/ranger-admin/ews/webapps/WEB-INF), however, this parameter cannot be set via Ambari WebUI. Is it the best way to modify web.xml on Ranger Admin host? If not, what is the best way for it?

takefumi_ohide · ‎06-07-2016

@Benjamin Leonhardi Thank you very much for your answer! I understood well!

takefumi_ohide · ‎06-07-2016

I modified(tampered) a local policy file written by Ranger HDFS Plugin to test against illegal or malicious operation, but authorization rules are not changed. For example: 1. user "ohide" cannot read /user/ohide 2. admit user "ohide" to read /user/ohide by Ranger 3. confirm user "ohide" can read /user/ohide 4. delete an entry added by step 2 from a local policy cache file in NameNode host (where Ranger HDFS Plugin running) 5. try to read /user/ohide by user "ohide" and succeeded. This behavior is appropriate I think, but I do not know and want to know how to avoid not to read tampered policy cache file. Does anyone know the answer of my question?

Online	Offline
Last Visited	‎02-20-2018 01:15 AM

Date Registered	‎02-02-2016 10:02 AM
Last Visited	‎02-20-2018 01:15 AM
Total Messages Posted	17
Total Kudos Received	3

Re: when inserting data from hive parquet table wi...

Re: Hue 3.10.0 + HiveServer2 with hive.server2.th...

Re: when inserting data from hive parquet table wi...

Re: Hue 3.10.0 + HiveServer2 with hive.server2.th...

Re: Hue 3.10.0 + HiveServer2 with hive.server2.th...

Re: Hue 3.10.0 + HiveServer2 with hive.server2.th...

Hue 3.10.0 + HiveServer2 with hive.server2.thrift...

Re: How do I block Ambari from creating Hive View ...

How do I block Ambari from creating Hive View on i...

Re: Is it impossible to move or add JournalNode by...

Re: Is it impossible to move or add JournalNode by...

Is it impossible to move or add JournalNode by Amb...

Re: What is the best way to modify session timeout...

Re: What is the best way to modify session timeout...

What is the best way to modify session timeout of ...

Re: How does Ranger HDFS plugin avoid reading tamp...

How does Ranger HDFS plugin avoid reading tampered...