Member since
09-12-2017
5
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
7175 | 03-05-2018 01:04 AM |
05-02-2018
03:44 AM
1 Kudo
@jirapong this is a known issue which we've recently seen in CDS 2.3 On Spark 2.3 the nativeLoader (SnappyNativeLoader’s) parentClassLoader is now an ExecutorClassLoader , whereas the parentClassLoader was a Launcher$ExtClassLoader prior to Spark 2.3. This created incompatibility with the snappy version (snappy-java.1.0.4.1) packaged with CDH. We are currently working on a solution in a future release, but there are two workarounds: 1) Use a later version of the Snappy library, which works with the above-mentioned class loader change, for example, snappy-java-1.1.4. Place the new snappy-java library on a local file system (for example /var/snappy). Then run your spark application with the user classpath options as shown below: spark2-shell --jars /var/snappy/snappy-java-1.1.4.jar --conf spark.userClassspathFirst=true --conf spark.executor.extraClassPath="./snappy-java-1.1.4.jar" 2) Instead of using Snappy, you can set the compression by changing the codec to LZ4 or UNCOMPRESSED (which you've already tested).
... View more
03-05-2018
01:04 AM
Yes. I've end-up using impersonate user following this article http://gethue.com/ldap-or-pam-pass-through-authentication-with-hive-or-impala/ and plus <property> <name>hadoop.proxyuser.hue_hive.groups</name> <value>*</value> </property> on core-site.xml by this answer http://community.cloudera.com/t5/Web-UI-Hue-Beeswax/Failed-to-validate-proxy-privilege-of-hue-hive-for-administrator/m-p/49742
... View more
09-20-2017
03:12 AM
Thank you very much. You're right. And this turn out to have me start whole cluster from zero again.
... View more