Member since
07-25-2016
12
Posts
3
Kudos Received
0
Solutions
08-15-2016
08:02 AM
Thanks a lot @Timothy Spann ! I will check it up 🙂
... View more
08-15-2016
08:02 AM
I see, @vkumar . Thanks!
... View more
08-13-2016
05:40 AM
2 Kudos
Hi!
I have gotten the HDP 2.5 Sandbox. I am trying to use Zeppelin but I see that a lot of interpreters that were "included" in HDP 2.4 are not available anymore. How does one add a hive interpreter to Zeppelin?
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Zeppelin
08-10-2016
03:30 PM
@Pierre Villard , I understand now. It is like you say a fairly simple situation. I have sent the adming an email requesting a permanent Queue to be defined. Do you recommend the Exchange to be set as "Direct" as well? or it is the same if it is defined as "topic"?
... View more
08-10-2016
01:26 PM
This is still cracking my nuts 😛 ...
I have been in contact with the admin on the Publisher side. He has told me I can create my own Queue when I bind to the Exchange (like I do in my Scala script / connector). I can use any binding key or just "#" to filter events. If I have more than one process / instance I can use the same Queue name (that's what I do in NiFi when running simultaneously my Scala script and NiFi doesn't give errors, and runs tasks but without Reading a single byte), or I can leave an empty Queue and RabbitMQ automatically creates the name for me...
Still the Exchange is of the "topic" kind and non-durable. I just don't know how to create the Queue and bind it to the Exchange within NiFi, it looks like it should be easy but I just don't see how...
... View more
08-09-2016
12:54 PM
@Pierre Villard, my Scala script is just consuming from the remote stream (remote publisher). I am sure from the configurations for the Rabbit Connector I was given that the queue is not durable. I will contact the admins on the remote server / Publisher to see if it is possible to change the configs to a durable queue. Thanks a lot for Your help!
... View more
08-09-2016
12:51 PM
Yes, @ozhurakousky ! I think you are right, the queue is set to auto-deletion ... I will be contacting the admins on the Publisher side 🙂 thank you!
... View more
08-09-2016
12:23 PM
@ozhurakousky, I am not able to run the Python script against the server (due to their security set-up). The exchange and queue are however defined: I ran a script on Scala that invoques the Rabbit Connector I have, where the exchange is declared together with the queue. Weirdly enough, once the scrip was running I started the ConsumeAMQP processor in NiFi: As you can see, there are no error Messages, the processor is able to Connect With the given queue once I am running the Scala script. Still there is no read / Write Activity: I have also Attached a "PutHDFS" procesor Connected to the ConsumeAMQP processor but it makes no difference...
... View more
08-09-2016
11:24 AM
Hi @ozhurakousky, thanks for Your reply!
I don't have access to the Publisher side however, in my scala Connector I am able to declare the "exchange", "queue" and "binding" and then connect to the messaging stream so I think it is safe to assume that they are set up. I wonder on how to pass the same kind of declarations as in my Scala Connector for "exchange", "queue" and "binding" but in this case within NiFi.
... View more
08-09-2016
11:07 AM
1 Kudo
Hortonworks
post:
Hi! I wonder
about the status of the ConsumeAMQP processor for NiFi since I am interested in
consuming a live RabbitMQ stream. This
stream I am currently trying to consume is a live stream generated by a remote
server. I am able to for example use a connector written as a Scala script to
consume the stream. In the connector I configure some parameters: factory.setUsername factory.setPassword factory.setVirtualHost factory.setHost factory.setPort Following the channel, queue and
binding declarations: channel.exchangeDeclare channel.queueDeclare channel.queueBind What I want to do is to be able to do is to set up NiFi in my virtual HDP 2.4 environment to consume the data stream and push it into HDFS, for example. NiFi's processor seems to be easy to configure. Here is a screenshot of the ConsumeAMQP processor's configuration: Once I introduce the parameters needed and start the processor I get the following errors: I am a little confused when it comes to how to configure the Queue correctly, since I am not able to introduce the channel nor bind it to the Queue. The issue is not a blocked firewall since I am using the same terminal I use to Connect to the Rabbit stream via an Scala script. I have been looking around for some tutorial or documentation page for hortonworks and RabbitMQ wihtout success. Any help / tips / pointers and comments could be of great help!
... View more