@Koosha Tahmasebipour you can leverage DistributedCache to upload your third party jars to HDFS https://hadoop.apache.org/docs/r2.7.2/api/org/apache/hadoop/filecache/DistributedCache.html for MapReduce ... View more

@Giuseppe Maldarizzi once you confirm it works, please accept the answer to close the thread. ... View more

@jigar.patel I stand corrected, you do not need Ambari to set DEBUG on RM, here's the command vagrant@u1401:~$ yarn daemonlog -getlevel u1402.ambari.apache.org:8088 org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl Connecting to http://u1402.ambari.apache.org:8088/logLevel?log=org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl Submitted Log Name: org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl Log Class: org.apache.commons.logging.impl.Log4JLogger Effective Level: INFO vagrant@u1401:~$ yarn daemonlog -setlevel u1402.ambari.apache.org:8088 org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl DEBUG Connecting to http://u1402.ambari.apache.org:8088/logLevel?log=org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl&level=DEBUG Submitted Log Name: org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl Log Class: org.apache.commons.logging.impl.Log4JLogger Submitted Level: DEBUG Setting Level to DEBUG ... Effective Level: DEBUG ... View more

@Giuseppe Maldarizzi looks like you're correct, forceFromStart is removed from Storm in favor of EarliestTime/LatestTime, it is a problem with documentation, I will speak to docs team to remove that and replace with correct info. The associated JIRAs are https://issues.apache.org/jira/browse/STORM-563 and STORM-650 ... View more

I will follow up with engineering on this but for now, can you try the following kafka.api.OffsetRequest.LatestTime() that should only grab latest messages ... View more

not unless you install your own version of HBase on top of HDP. You will have to do that manually, deploy bits to each node, configure hbase-site.xml, etc, if you have some configuration/automation management software, it's not such a big task, doing it manually will be involved. ... View more

you could try daemonlog command. In that case, Ambari version is irrelevant. I don't know your HDP version but I checked whether this command is available in earlier Hadoop and it's available at least in 2.4.x. http://hadoop.apache.org/docs/r2.7.3/hadoop-yarn/hadoop-yarn-site/YarnCommands.html Get/Set the log level for a Log identified by a qualified class name in the daemon. Example: $ bin/yarn daemonlog -setlevel 127.0.0.1:8088 org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl DEBUG ... View more