Hi All, I am running the hdf sandbox and trying to do a sample - couple of challenges 1. how to see the logs - when I ssh into the docker container by ssh root@172.17.0.1 - I don't see any /usr/hdf or /var/log/registry directories 2. while adding a avro schema - gets error ' Given schema is invalid' - attached screenshot Thanks, Avijeet ... View more

Hi All, I am looking at the HDF3.0 sandbox - couple of observations 1. It included HDFS, YARN, MR2, HBASE etc. - I thought these are not part of HDF so can HDF be all services except HIVE, Spark 2. In Ambari I see a lot of services with the briefcase with a + symbol - what exactly this symbol means - I started a few services still it shows the same symbol instead of green tick Thanks, Avijeet ... View more

Hi All, I tried HIVE insert/update/delete operations, I see that it launches a tez job and bit slow. I have a couple of questions - 1. when we use hive streaming processors (in nifi, streamsets) does it insert record by record? I believe no. just to confirm. 2. it seems bit risky to design an acid hive table, is there an approach where it can be used (such as IOT ) safely without having locking/concurrency issues etc. Thanks, Avijeet ... View more

Hi All, While enabling ambari hive view (unsecured hadoop cluster) - as hive user - getting error Service 'hiveserver' check failed: Cannot connect to hive I have disabled 'Run as end user instead of Hive user' on HIVE Config and proxy user hive.users and hosts are set to * pls suggest if there is some way to fix it. Service 'hiveserver' check failed: java.lang.Exception: Cannot connect to hive java.lang.Exception: Cannot connect to hive at org.apache.ambari.view.hive2.actor.message.job.ExecutionFailed.<init>(ExecutionFailed.java:28) at org.apache.ambari.view.hive2.actor.JdbcConnector.notifyConnectFailure(JdbcConnector.java:385) at org.apache.ambari.view.hive2.actor.JdbcConnector.connect(JdbcConnector.java:422) at org.apache.ambari.view.hive2.actor.JdbcConnector.handleNonLifecycleMessage(JdbcConnector.java:179) at org.apache.ambari.view.hive2.actor.JdbcConnector.handleMessage(JdbcConnector.java:171) at org.apache.ambari.view.hive2.actor.HiveActor.onReceive(HiveActor.java:38) at akka.actor.UntypedActor$anonfun$receive$1.applyOrElse(UntypedActor.scala:167) at akka.actor.Actor$class.aroundReceive(Actor.scala:467) at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) at akka.actor.ActorCell.invoke(ActorCell.scala:487) Thanks, Avijeet ... View more

HI All, while trying the twitter example on storm - getting this error I am using 1.0.1 on hdp2.5 I compiled the classes with 1.0.1 for storm-core and storm-hdfs also provided exclusion for hadoop-hdfs and hadoop-client in pom.xml 2017-04-17 09:52:33.863 STDERR [INFO] WARNING: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2017-04-17 09:52:34.408 o.a.s.h.b.HdfsBolt [INFO] Preparing HDFS Bolt... 2017-04-17 09:52:34.586 o.a.s.util [ERROR] Async loop died! java.lang.RuntimeException: Error preparing HdfsBolt: No FileSystem for scheme: hdfs at org.apache.storm.hdfs.bolt.AbstractHdfsBolt.prepare(AbstractHdfsBolt.java:123) ~[stormjar.jar:?] at org.apache.storm.daemon.executor$fn__6573$fn__6586.invoke(executor.clj:798) ~[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37] at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:482) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37] at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101] pls let me know if any other information required. Thanks, Avijeet ... View more

Hi, I tried to run the Wordcount example storm-starter-0.0.1-storm-0.9.0.1.jar on storm in hdp2.5 The supervisor keeps going down and doesn't run giving below errro java.io.FileNotFoundException: File '/datadrive/hadoop/storm/supervisor/stormdist/WordCount-2-1491988512/stormconf.ser' does not exist at org.apache.storm.shade.org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:292) I killed the topology still, same error persisits. pls suggest. Thanks, Avijeet ... View more

Hi All, what would be the side effects of doing batch ingestion through nifi. Lets say large files copies, if we ingest through nifi, how would it behave? same with large db copies. Thanks, Avijeet ... View more

Hi All, by default hive-atlas integration is enabled when we set up HDP2.5. And this integration needs kafka, hbase etc. as a side effect Is there a way to enable / disable this integration easily. Thanks, Avijeet ... View more

Hi All, With all the DevOps tools available (Docker, swram, kubernetes, Ansible, Puppet, Chef etc.) around configuration management, cluster management, container clustering etc. I was wondering where does Amabri fits in, how is it developed? What about Ambari blue prints? Thanks, Avijeet ... View more