About quadoss

quadoss · ‎06-13-2016

@Mandeep Cheema Yes, it will work with HDP 2.4.x.

quadoss · ‎06-07-2016

@Tim Dobeck You can look at this http://gethue.com/hue-3-7-with-sentry-app-and-new-search-widgets-are-out/ which has the information of how to perform the install along with the tarball of 3.7

quadoss · ‎06-07-2016

@Sagar Shimpi By default the shell actions are not allowed to run as another user as sudo is blocked. If you want a yarn application to run as someone other than yarn (i.e. the submitter), then you need to enable the linux container executor so that the containers are started up by the submitting user. Also note the below setting information which also needs to be changed as well to achieve this. With yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users=false (default), it runs as yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user (default is 'nobody') With yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users=true, it runs as the user submitting the workflow. Stating that there are issues around this also where it does not work as expected because of the issues https://issues.apache.org/jira/browse/YARN-2424 https://issues.apache.org/jira/browse/YARN-3462 The current suggestion that I can make is to add line to change the ownership of the file which was created using shell.

quadoss · ‎05-31-2016

@sankar rao You may want to visit these links as they are documented about the use of different Authorizations. Hive Authorization Storage Based Authorization in the Metastore Server

quadoss · ‎05-12-2016

@Sushil Saxena It seems like there are likely to be multiple versions of hive jars. When I state multiple version of hive jars meaning there different hdp release hive jars in the oozie share lib location. Was this a upgraded cluster from 2.1.x/2.2.x to 2.3.4 release?. One thing that I can suggest here is to perform the following provided you have not copied over thirdparty jars like oracle jar/ mysql jar file to the oozie share lib location. If you have then you would need to copy over all the 3rd party jars again to hdfs. From the oozie server host, using oozie user: 1. hdfs dfs -rm -r /user/oozie/share/lib -> to remove the lib folder completely 2. cd to /usr/hdp/current/oozie-client/bin 3. run "./oozie-setup.sh sharelib create -fs <get the fs.defaultFS from core-site.xml>" 4. Restart Oozie service This should help to address the issue.

quadoss · ‎05-12-2016

@simran kaur, it seems like this is a duplicate of the post https://community.hortonworks.com/questions/30437/error-in-oozie-class-orgapacheoozieactionhadoopsqo.html#answer-33008 where you were able to get past the issue. If you do, please update the solution here and close this thread.

quadoss · ‎05-12-2016

@simran kaur, There are couple of ways to address this. One as @rbiswas mentioned and the other one is to create a lib folder where the workflow is receding in HDFS and place the mysql connector jar file there. For example, if your workflow is located in hdfs at /user/simran/sqoopwf/workflow.xml. Take the path "/user/simran/sqoopwf/" and create a lib folder here hdfs dfs -mkdir -p /user/simran/sqoopwf/lib and then place the mysql connector jar to this location hdfs dfs -put <mysql-connector-java-version.jar> /user/simran/sqoopwf/lib/. Then kick off the oozie job which should work.

quadoss · ‎05-11-2016

@Chris McGuire I'm not sure whether this would work on saveAsTable command since I have very limited to no knowledge on spark. I'm hoping that this property should work for the spark streaming job as well.

quadoss · ‎05-11-2016

@Mamta Chawla, Hive does not do data validation based on the fields. Its the users responsibility to check the data and see if it matches with the table being created. May be this link can help you out with what you are looking for: https://community.hortonworks.com/articles/1283/hive-script-to-validate-tables-compare-one-with-an.html

quadoss · ‎05-11-2016

@Chris McGuire, @Eric Walk, Yes, that could well be the reason. There are properties for hive to merge spark file. The property is called hive.merge.sparkfiles by default this is false. You may want to enable it and also look at this wiki for hive-spark configuration: https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started

Online	Offline
Last Visited	‎07-03-2024 11:44 AM

Member Since	‎03-07-2019 04:31 PM
Last Visited	‎07-03-2024 11:44 AM
Posts	316
Kudos received	17

Cloudera Community

Re: Hue - default database for user

Re: How to configure hive so beeline does not ask ...

Re: Table not found in Hive Action

Re: Merge multiple directories into one table in H...

Re: Files created after running oozie shell action...

Re: Install hue in HDP 2.4

Re: Can you install Hue 3.7.1 onto HDP 2.4.0?

Re: Files created after running oozie shell action...

Re: we are using hdp 2.1.5 and I have create user ...

Re: Hive Metastore - Method not found

Re: oozie task with sqoop : java.lang.ClassNotFoun...

Re: error in oozie Class org.apache.oozie.action....

Re: Hive compactions on External table

Re: Data Type Validation in Hive

Re: Hive compactions on External table