Hi    What i have seen is tht the share option only gives you read or read+modify permisson. There is nothing as such execute? If i give read+modify other users will be ale to run the oozie workflow. I have seen it does not happen. As the permission on the underlying hdfs folder for the workflow is only for my user and it does not get modified. drwxrwx--- - kuaksha hue 0 2020-10-28 10:42 /user/hue/oozie/workspaces/hue-oozie-1520605312.96   Please elaborate and help.   Regards Akshay ... View more

thanks. its worked nicely ... View more

That particular package only comes from the MySQL Community repository.  Not from the OS repos or from the SCL repos. ... View more

this workflow is working? Just workflow and it will send to email? ... View more

Hi. How can you find your workflow from list of all workflows? They are nameless... ... View more

Ye even  I cannot run shell jobs thru HUE. CAn anyone help.? ... View more

Looks like you already have another thread opened: http://community.cloudera.com/t5/Batch-SQL-Apache-Hive/Hive-Safety-Valve-configuration-is-not-applied-HiveConf-of-name/td-p/64037 Will follow up there. ... View more

Hi @cconner   I've connected to the Hue database on Mysql.  I see all the the tables prefixed with oozie_ however I do not see any meaninful data in these tables.  Can you explain where in this DB schema the workflow definitions are stored?   Thanks   Shak ... View more

Partitioner is not invoked when used in oozie mapreduce action (Creating workflow using HUE). But works as expected when running using hadoop jar commad in CLI,   I have implemented secondary sort in mapreduce and trying to execute it using Oozie (From Hue).   Though I have set the partitioner class in the properties, the partitioner is not being executed. So, I'm not getting output as expected.   The same code runs fine when run using hadoop command. And here is my workflow.xml   <workflow-app name="MyTriplets" xmlns="uri:oozie:workflow:0.5"> <start to="mapreduce-598d"/> <kill name="Kill"> <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <action name="mapreduce-598d"> <map-reduce> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <configuration> <property> <name>mapred.output.dir</name> <value>/test_1109_3</value> </property> <property> <name>mapred.input.dir</name> <value>/apps/hive/warehouse/7360_0609_rx/day=06-09-2017/hour=13/quarter=2/,/apps/hive/warehouse/7360_0609_tx/day=06-09-2017/hour=13/quarter=2/,/apps/hive/warehouse/7360_0509_util/day=05-09-2017/hour=16/quarter=1/</value> </property> <property> <name>mapred.input.format.class</name> <value>org.apache.hadoop.hive.ql.io.RCFileInputFormat</value> </property> <property> <name>mapred.mapper.class</name> <value>PonRankMapper</value> </property> <property> <name>mapred.reducer.class</name> <value>PonRankReducer</value> </property> <property> <name>mapred.output.value.comparator.class</name> <value>PonRankGroupingComparator</value> </property> <property> <name>mapred.mapoutput.key.class</name> <value>PonRankPair</value> </property> <property> <name>mapred.mapoutput.value.class</name> <value>org.apache.hadoop.io.Text</value> </property> <property> <name>mapred.reduce.output.key.class</name> <value>org.apache.hadoop.io.NullWritable</value> </property> <property> <name>mapred.reduce.output.value.class</name> <value>org.apache.hadoop.io.Text</value> </property> <property> <name>mapred.reduce.tasks</name> <value>1</value> </property> <property> <name>mapred.partitioner.class</name> <value>PonRankPartitioner</value> </property> <property> <name>mapred.mapper.new-api</name> <value>False</value> </property> </configuration> </map-reduce> <ok to="End"/> <error to="Kill"/> </action> <end name="End"/> When running using hadoop jar command, I set the partitioner class using JobConf.setPartitionerClass API.   Not sure why my partitioner is not executed when running using Oozie. Inspite of adding   <property> <name>mapred.partitioner.class</name> <value>PonRankPartitioner</value> </property> ... View more

Hi,@MC,My Hue still can not work, can help me? Thank you!         ... View more

Hue sends queries to HiveServer2, so you would want to implement it in the CM configuration for HS2, which is this safety valve in the Hive service: HiveServer2 Advanced Configuration Snippet (Safety Valve) for hive-site.xml ... View more

First, try to index only couple of records. If that doesn't work then recheck the data values and data types used. If it works then index the data in small batches ... I faced a similar issue if more than 200 records are indexed. ... View more

If you use CM you don't need to use the Hue safety valve. Make sure the Trift Server is added and started Make sure in the Hue configuration that the HBase service and the HBase thrift service are checked Restart Hue and that's it! Romain ... View more

I decided to do a quick check on the attempt processing one of oozie's hive actions:   org.apache.hadoop.mapred.YarnChild 10.0.0.9 33024 attempt_1431965085423_0015_m_000000_0 2   ...and collected a quick stack trace which shows the hive client does indeed run inproc with the JVM running the attempt:   "main" prio=10 tid=0x00007f4dbc027800 nid=0xac2c in Object.wait() [0x00007f4dc5c2b000]    java.lang.Thread.State: WAITING (on object monitor)     at java.lang.Object.wait(Native Method)     - waiting on <0x00000000fe43fa48> (a org.apache.hadoop.ipc.Client$Call)     at java.lang.Object.wait(Object.java:503)     at org.apache.hadoop.ipc.Client.call(Client.java:1454)     - locked <0x00000000fe43fa48> (a org.apache.hadoop.ipc.Client$Call)     at org.apache.hadoop.ipc.Client.call(Client.java:1399)     at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)     at com.sun.proxy.$Proxy32.getJobReport(Unknown Source)     at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport(MRClientProtocolPBClientImpl.java:133)     at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)     at java.lang.reflect.Method.invoke(Method.java:606)     at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:323)     - locked <0x00000000ff3bafd0> (a org.apache.hadoop.mapred.ClientServiceDelegate)     at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:422)     at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:575)     at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:183)     at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:603)     at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:601)     at java.security.AccessController.doPrivileged(Native Method)     at javax.security.auth.Subject.doAs(Subject.java:415)     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)     at org.apache.hadoop.mapred.JobClient.getJobUsingCluster(JobClient.java:601)     at org.apache.hadoop.mapred.JobClient.getJobInner(JobClient.java:611)     at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:636)     at org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:288)     at org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:547)     at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:435)     at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)     at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)     at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)     at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1604)     at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1364)     at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1177)     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004)     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994)     at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:247)     at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)     at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)     at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:345)     at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:443)     at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:459)     at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739)     at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)     at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:323)     at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:284)     at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)     at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66)     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)     at java.lang.reflect.Method.invoke(Method.java:606)     at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)     at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)     at java.security.AccessController.doPrivileged(Native Method)     at javax.security.auth.Subject.doAs(Subject.java:415)     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)   Thanks ... View more

Great. It works !!!   Many thanks for your suggestion. Alessio   ... View more

  I ended up going with option 2 which solved the immediate problem.  Thanks. ... View more

Great! And for information, look at the Hue configuration section and look at Sentry, you should just need to check it and it will configure Hue automatically (no need to use any safety valve or put the Sentry service on the same host as Hue) Romain ... View more

Hi,   to read data in avro format from Hive you have to use an Avro SerDe. Maybe a good starting point will be http://www.michael-noll.com/blog/2013/07/04/using-avro-in-mapreduce-jobs-with-hadoop-pig-hive/   But this is not related to this topic since the solr sink will put data into Solr. I'd suggest to use just a HDFS sink to put your data on HDFS and create an (external or not) Hive table afterwards. You do not need Solr and/or Morphlines for this.   best, Gerd ... View more

Hey   In Step 4 are we editing the file /opt/cloudera/parcels/CDH-4.5.0-1.cdh4.5.0.p0.30/etc/oozie/conf.dist/oozie-site.xml manually?   We are using CM.   I see changes in CM but not in the  oozie-site.xml  or on Oozie admin UI .   Getting error    [JA020: Could not load credentials of type [hbase] with name [hbase_credentials]]; perhaps it was not defined in oozie-site.xml?]   Thanks   ... View more

Glad to hear it!! ... View more

That's right, you'll have to put the following in your Hue Service->Configuration->Hue Server->Advanced-> Hue Server Configuration Safety Valve for hue_safety_valve_server.ini:     [hadoop] [[mapred_clusters]] [[[default]]] jobtracker_host=cdh45-2.qa.test.com thrift_port=9290 jobtracker_port=8021 submit_to=true hadoop_mapred_home={{HADOOP_MR1_HOME}} hadoop_bin={{HADOOP_BIN}} hadoop_conf_dir={{HADOOP_CONF_DIR}} security_enabled=true logical_name=logicaljt [[[jtha]]] jobtracker_host=cdh45-1.qa.test.com thrift_port=9290 jobtracker_port=8021 submit_to=true hadoop_mapred_home={{HADOOP_MR1_HOME}} hadoop_bin={{HADOOP_BIN}} hadoop_conf_dir={{HADOOP_CONF_DIR}} security_enabled=true logical_name=logicaljt     Leaving off "secuirty_enabled=true" if you're not using kerberos. ... View more

Hi Chris, thanks, restarting the MGMT services solved the test issues. bye...Gerd... ... View more

Ok great. I restarted all MapReduce and Hbase daemons in addition to the HDFS daemons and it is working properly now. Thank you!   Previously I had only restarted HDFS. ... View more

Thanks for quick&clear reply ... View more

No, I stoped all jobes before changing configuration in MapReduce service, and restarted all cluster. I also checked folder mapred/local/taskTracker/hdfs/jobcache and i am sure it was empty.   Thank you for the link, but I found nothing about jobcache folder.    Also, after the job failes or completed job folder are deleted from jobcache folder whith all attemp_xxxx folders.     ... View more

Hi Chris,   I tried your suggestions... and various alternatives: [zk: localhost:2181(CONNECTED) 8] ls / [configs, solr,n1:2181, zookeeper, solr.xml, solr,n1.hdc:2181, solr] [zk: localhost:2181(CONNECTED) 9] ls /solr [clusterstate.json, aliases.json, solr.xml, live_nodes, overseer, collections, overseer_elect] [zk: localhost:2181(CONNECTED) 10] ls /solr/collections [] [zk: localhost:2181(CONNECTED) 11] ls /solr,n1:2181 [solr] [zk: localhost:2181(CONNECTED) 12] ls /solr,n1:2181/solr [configs] [zk: localhost:2181(CONNECTED) 13] ls /solr,n1:2181/solr/configs [collection4] [zk: localhost:2181(CONNECTED) 14] ls /solr,n1.hdc:2181/solr [configs] [zk: localhost:2181(CONNECTED) 15] ls /solr,n1.hdc:2181/solr/configs [collection2, collection1] [zk: localhost:2181(CONNECTED) 16] Btw, my nodes are all in the domain hdc.  ... View more

Thanks Chris.. I understood the process when i spent more time on it. I have tried the same and it did work.. Probably i might have asked question little too early.. 🙂 Thanks for your resonse... ... View more