arturo_amador
Explorer

Re: How to add Hive to Zeppelin in HDP 2.5

by Explorer in Support Questions
‎08-15-2016 08:02 AM
‎08-15-2016 08:02 AM
Thanks a lot @Timothy Spann ! I will check it up 🙂 ... View more

Re: How to add Hive to Zeppelin in HDP 2.5

by Explorer in Support Questions
‎08-15-2016 08:02 AM
‎08-15-2016 08:02 AM
I see, @vkumar . Thanks! ... View more

How to add Hive to Zeppelin in HDP 2.5

by Explorer in Support Questions
‎08-13-2016 05:40 AM
2 Kudos
‎08-13-2016 05:40 AM
2 Kudos
Hi! I have gotten the HDP 2.5 Sandbox. I am trying to use Zeppelin but I see that a lot of interpreters that were "included" in HDP 2.4 are not available anymore. How does one add a hive interpreter to Zeppelin? ... View more
Labels:

Re: NiFi's ConsumeAMQP RabbitMQ to HDFS

by Explorer in Support Questions
‎08-10-2016 03:38 PM
‎08-10-2016 03:38 PM
Thanks a lot for the great help, @Pierre Villard ! ... View more

Re: NiFi's ConsumeAMQP RabbitMQ to HDFS

by Explorer in Support Questions
‎08-10-2016 03:30 PM
‎08-10-2016 03:30 PM
@Pierre Villard , I understand now. It is like you say a fairly simple situation. I have sent the adming an email requesting a permanent Queue to be defined. Do you recommend the Exchange to be set as "Direct" as well? or it is the same if it is defined as "topic"? ... View more

Re: NiFi's ConsumeAMQP RabbitMQ to HDFS

by Explorer in Support Questions
‎08-10-2016 01:26 PM
‎08-10-2016 01:26 PM
This is still cracking my nuts 😛 ... I have been in contact with the admin on the Publisher side. He has told me I can create my own Queue when I bind to the Exchange (like I do in my Scala script / connector). I can use any binding key or just "#" to filter events. If I have more than one process / instance I can use the same Queue name (that's what I do in NiFi when running simultaneously my Scala script and NiFi doesn't give errors, and runs tasks but without Reading a single byte), or I can leave an empty Queue and RabbitMQ automatically creates the name for me... Still the Exchange is of the "topic" kind and non-durable. I just don't know how to create the Queue and bind it to the Exchange within NiFi, it looks like it should be easy but I just don't see how... ... View more

Re: NiFi's ConsumeAMQP RabbitMQ to HDFS

by Explorer in Support Questions
‎08-09-2016 12:54 PM
‎08-09-2016 12:54 PM
@Pierre Villard, my Scala script is just consuming from the remote stream (remote publisher). I am sure from the configurations for the Rabbit Connector I was given that the queue is not durable. I will contact the admins on the remote server / Publisher to see if it is possible to change the configs to a durable queue. Thanks a lot for Your help! ... View more

Re: NiFi's ConsumeAMQP RabbitMQ to HDFS

by Explorer in Support Questions
‎08-09-2016 12:51 PM
‎08-09-2016 12:51 PM
Yes, @ozhurakousky ! I think you are right, the queue is set to auto-deletion ... I will be contacting the admins on the Publisher side 🙂 thank you! ... View more

Re: NiFi's ConsumeAMQP RabbitMQ to HDFS

by Explorer in Support Questions
‎08-09-2016 12:23 PM
‎08-09-2016 12:23 PM
@ozhurakousky, I am not able to run the Python script against the server (due to their security set-up). The exchange and queue are however defined: I ran a script on Scala that invoques the Rabbit Connector I have, where the exchange is declared together with the queue. Weirdly enough, once the scrip was running I started the ConsumeAMQP processor in NiFi: As you can see, there are no error Messages, the processor is able to Connect With the given queue once I am running the Scala script. Still there is no read / Write Activity: I have also Attached a "PutHDFS" procesor Connected to the ConsumeAMQP processor but it makes no difference... ... View more

Re: NiFi's ConsumeAMQP RabbitMQ to HDFS

by Explorer in Support Questions
‎08-09-2016 11:30 AM
‎08-09-2016 11:30 AM
Trying it, @ozhurakousky ... ... View more

Re: NiFi's ConsumeAMQP RabbitMQ to HDFS

by Explorer in Support Questions
‎08-09-2016 11:24 AM
‎08-09-2016 11:24 AM
Hi @ozhurakousky, thanks for Your reply! I don't have access to the Publisher side however, in my scala Connector I am able to declare the "exchange", "queue" and "binding" and then connect to the messaging stream so I think it is safe to assume that they are set up. I wonder on how to pass the same kind of declarations as in my Scala Connector for "exchange", "queue" and "binding" but in this case within NiFi. ... View more

NiFi's ConsumeAMQP RabbitMQ to HDFS

by Explorer in Support Questions
‎08-09-2016 11:07 AM
1 Kudo
‎08-09-2016 11:07 AM
1 Kudo
Hortonworks post: Hi! I wonder about the status of the ConsumeAMQP processor for NiFi since I am interested in consuming a live RabbitMQ stream. This stream I am currently trying to consume is a live stream generated by a remote server. I am able to for example use a connector written as a Scala script to consume the stream. In the connector I configure some parameters: factory.setUsername factory.setPassword factory.setVirtualHost factory.setHost factory.setPort Following the channel, queue and binding declarations: channel.exchangeDeclare channel.queueDeclare channel.queueBind What I want to do is to be able to do is to set up NiFi in my virtual HDP 2.4 environment to consume the data stream and push it into HDFS, for example. NiFi's processor seems to be easy to configure. Here is a screenshot of the ConsumeAMQP processor's configuration: Once I introduce the parameters needed and start the processor I get the following errors: I am a little confused when it comes to how to configure the Queue correctly, since I am not able to introduce the channel nor bind it to the Queue. The issue is not a blocked firewall since I am using the same terminal I use to Connect to the Rabbit stream via an Scala script. I have been looking around for some tutorial or documentation page for hortonworks and RabbitMQ wihtout success. Any help / tips / pointers and comments could be of great help! ... View more
Contact Me
Online
Offline
Last Visited
‎08-16-2016 12:46 PM
Public Statistics
Date Registered ‎07-25-2016 12:43 PM
Last Visited ‎08-16-2016 12:46 PM
Total Messages Posted 12
Total Kudos Received 3