Member since
09-18-2015
191
Posts
81
Kudos Received
40
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2045 | 08-04-2017 08:40 AM | |
5421 | 05-02-2017 01:18 PM | |
1109 | 04-24-2017 08:35 AM | |
1116 | 04-24-2017 08:21 AM | |
1333 | 06-01-2016 08:54 AM |
05-03-2016
05:55 PM
Hi @Kevin Stear
I'm afraid my answer is somewhat good news/bad news. The bad news is, there's not really anything out of the box that I'm aware of. The good news is that there's some new functionality on the near term Ambari roadmap that should make this a LOT easier. Take a look at https://issues.apache.org/jira/browse/AMBARI-15139 See if that scratches the itch you're looking for... you could do something similar on the cheap before that functionality officially lands, or maybe it's enough to let you know that you should have a far easier time in the not too distant future. Hope that helps.
... View more
05-03-2016
03:03 PM
1 Kudo
Hi @mpandit at the moment, I believe if you're just looking to encrypt some of the information, you'd need to use something like the SplitContent processor, to split out the elements you want to encrypt, and then EncryptContent to encrypt those elements separately from the rest of the data in each flowfile. This of course means you'll potentially need to re-assemble the data when it reaches its target location.
... View more
05-03-2016
02:40 PM
Ahh now that's clearer, unfortunately I don't believe that NiFi is able to do that currently.
... View more
05-03-2016
02:05 PM
2 Kudos
Hi @Amar ch So the putElasicsearch processor as you have identified, is designed to write individual flowfiles, or indeed batches of flowfiles. Those batches are controlled via the "Batch Size" property. I guess it really depends what you mean by bulk insert, I don't see any limiations on the "Batch Size" so it should be possible to increase that until you get the size insert you require. For more information on the properties, please take a look at: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.elasticsearch.PutElasticsearch/index.html From reviewing the JIRA associated with the processor, it does look as if the putElasticsearch projcessor does make use of the bulk load api. https://issues.apache.org/jira/browse/NIFI-1275
... View more
05-03-2016
12:56 PM
@Sayan Khasnabis Thanks for that! I'll notify the correct teams to update the information. Thanks again and let us know if you find any other issues like this.
... View more
05-03-2016
12:46 PM
1 Kudo
@ASHISH Rastogi You have a "/" (without the quotes) at the end of your proxy configuration, straight after the port number, if you remove that the wget command should work too!
... View more
05-03-2016
12:24 PM
Hi @ASHISH Rastogi given you're using CentOS I'd suggest either MySQL or Postgres, both are supported options and available from the CentOS repos. Installation is via the normal yum methods.
... View more
04-29-2016
01:43 PM
@ASHISH Rastogi you would need to start the installation with the following steps: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_installing_manually_book/content/ch_getting_ready_chapter.html If you would like someone to walk you through it step by step above and beyond the documentation, I'd suggest that you look into Hortonworks support services and possibly even Hortonworks consulting services http://hortonworks.com/services/support/
... View more
04-29-2016
01:16 PM
@ASHISH Rastogi ok so 11GB of RAM with about 5.9GB of RAM free. That should be enough to deploy HDP, you just need to follow the instructions mentioned in the posts above!
... View more
04-29-2016
12:46 PM
1 Kudo
@ASHISH Rastogi Honestly, you'll really struggle with that little RAM. Even the Hortonworks Sandbox suggests a bare mimimum of 8GB of RAM for a single node HDP environment, and while you might manage to trim that down a bit by reducing things to the bare minimum, it won't be a plesant experience. I would recommend you perhaps investigate the HDP Sandbox instead and just run it on VMware or Virtualbox http://hortonworks.com/products/sandbox/ If you're absolutely dead set on your origional approach, then I think you'll have to go without Ambari and instead do everything manually, the complete step by step for this is shown here: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_installing_manually_book/content/ch_getting_ready_chapter.html But honestly that's quite a complex procedure, and accident prone. Best of luck!
... View more