Hi harsh, The issue related to hive action works when single workflow is running, but when we try to run multiple workflows at same time it shows error with following INFO: 2016-01-25 02:13:20,396 INFO [Thread-68] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:1 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:0 RackLocal:0 2016-01-25 02:13:20,400 INFO [Thread-68] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://xxxx:8020 /user/cdhadmin/.staging/job_1453585092595_0349 2016-01-25 02:13:20,403 INFO [Thread-68] org.apache.hadoop.ipc.Server: Stopping server on 59772 2016-01-25 02:13:20,404 INFO [IPC Server listener on 59772] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 59772 2016-01-25 02:13:20,404 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted 2016-01-25 02:13:20,405 INFO [Ping Checker] org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: TaskAttemptFinishingMonitor thread interrupted 2016-01-25 02:13:20,404 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder ... View more

docu56048_OneFS-7.2-CLI-Administration-Guide.pdf   Configure HDFS authentication properties on the Hadoop client If you want clients running Hadoop 2.2 and later to connect to an access zone through Kerberos, you must make some modifications to the core-site.xml and hdfs- site.xml files on the Hadoop clients. Before you begin Kerberos must be set as the HDFS authentication method and a Kerberos authentication provider must be configured on the cluster. Procedure 1. Go to the $HADOOP_CONF directory on your Hadoop client. 2. Open the core-site.xml file in a text editor. 3. Set the value of the hadoop.security.token.service.use_ip property to false as shown in the following example: <property> <name>hadoop.security.token.service.use_ip</name> <value>false</value> </property> 4. Save and close the core-site.xml file. 5. Open the hdfs-site.xml file in a text editor. 6. Set the value of the dfs.namenode.kerberos.principal.pattern property to the Kerberos realm as shown in the following example: <property> <name>dfs.namenode.kerberos.principal.pattern</name> <value>hdfs/*@storage.company.com</value> </property> 7. Save and close the hdfs-site.xml file. ... View more

so, i've started to play with this and met interesting thing. When I try to proceed data with lzma i read in two times more data then i'm actually have on the HDFS. For example, hadoop client (hadoop fs -du) shows some numbers like 100GB. then i run MR (like select count(1) ) over this data and check MR counters and find "HDFS bytes read" two times more (like 200GB). In case of gzip and bzip2 codecs hadoop client file size and MR counters are the similar ... View more

Thanks Harsh. ... View more

Hi,   Thanks   One more issue with SLA events monitoring is setting the nominal time in workflow, Our workflow kicks based on data in events. If the data is available then it will start. so how to set the nominal time of workflow to current time when the job kicks in if data is available.   Please help!!!!       ... View more