The solution you are looking for is:  ReplaceText:   https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.12.0/org.apache.nifi.processors.standard.ReplaceText/   You can find loads of examples here in the forum with this search:   https://community.cloudera.com/t5/forums/searchpage/tab/message?advanced=false&allow_punctuation=false&q=replaceText     If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.   Thanks, Steven ... View more

@aniket5003 NiFi can be added to ambari using one of the HDF Management packs.  Depending on your relationship with Cloudera, you may need to use your account to get after the NiFi 1.12.1 management pack.  I do know other versions are out on the open internet (1.9 and below), but newest versions will require a cloudera username and password to access repos and artifacts.   Once you have a management pack added to ambari,  you should be able to install nifi and other HDF components in an HDP cluster.   Additionally you can get 1.12.1 from nifi.apache.org and install outside of ambari interface if you need something quick.   If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.   Thanks, Steven ... View more

@Elf  IMO anything is possible with ambari.  That said, out of the box, maybe it would not appear to be possible without some advanced ambari admin skills.      I took a look at the link you provided and that is an example of how to spin up a single machine with many of the services you may already have in your ambari cluster.     To install griffin in an ambari cluster you would need to pick a node, install griffin, missing requirements (services/components not in your cluster), and thoughtfully modify the configuration to use the existing services from the ambari cluster.  For example,  feed griffin you configuration locations for hadoop, hdfs, hive, etc and NOT use the specific directions to install those parts based on sample documentation.   If you do decide to go down this path, please update here with your progress or create new Questions with specific errors you may have.   ... View more

@ujay Of course.   The link referenced xml is a template file.  Click through and get the raw xml code and save to a file.  From here you import the template:   Then in the upper navigation grab the template icon and drag to the nifi canvas:       It should automatically choose the last template uploaded:   Once the template is on the canvas click through into the process group created:   You will need to do some work in Controller Services so check out the notes in the Red Box.  The flow is an example of how to generate many flow files and detect duplicates.  Be sure to do some research on the processor (google search) to understand how others have resolved working with the processor as your begin to integrate this into your own flow.    This community is also a great research tool too:     ... View more

@praneet Adding the value to the processor (+) is a suitable method.  You just need to make sure you get the right string in that field.  It's blocked out, but appears not just the actual token but prepended with "Bearer".  Try just the token string.   One thing I like to do for any API, before I start to work on invokeHttp configuration, is to use Postman to help me identify all of the required settings to test connection to an API.  Once this is complete, I can definitively ensure that nifi invokeHttp is sending the same request.     If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.     Thanks, Steven ... View more

@Manoj90 In addition o @PVVK point, you need to be careful with routing relationships back on the originating processor.  During development I like to use a output port that I call End of Line or EOL1, EOL2, EOL3 as I need more in larger flows.   This is to evaluate if something goes to fail, retry, etc.  Later once I am certain the flow is working as I need, I either auto terminate these routes, or I route them out of my processor group to an Event Notification system.     It looks like this:   Using an output port to hold un needed routes during testing     If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.     Thanks, Steven ... View more

@Elf Can you sure the method and failure details for install with ambari?   There is always a way to work around those issues even if its some manual adjustments in ambari or the agent during install.     ... View more

@mansu You need to make sure the file and locations have the correct permissions for the nifi user.  For example in linux:   chown -R nifi:nifi /path/to/files     If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.     Thanks, Steven ... View more

@gnavarro54 This error is suggesting that your yarn node is out of memory.   You need to inspect what services are running on your node may be causing it to not have enough memory to start yarn.  If there are to o many services for your node spec (amount of ram) then all of the services may not be able to start.   If possible add more ram, or considering removing some services from that node to another node.     If you respond back with more details about your environment, myself or other community members can respond with deeper details.     If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.     Thanks, Steven ... View more

@vikrant_kumar24    Based on what you describe you are either exceeding the capabilities of the outbound network connection from nifi, or the receiving end API.  I suspect the latter but its hard to tell without seeing the number of nodes and processor scheduling settings. You can try to slow down the execution of invokeHttp to confirm.  If it is set to 0 sec it can create a lot of connections to the endpoint.  Try 5,10,15 seconds and re-evaluate.    Also If you have more than 1 one nifi node, it can create a great deal of concurrent connections.        If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.     Thanks, Steven ... View more

@surajnag This is a great question.  Here are some of my ideas which I have used in development and production nifi flows.   During development route all flow exceptions to an output port.  I call these EOL, or End of Line and increment them with #s, like EOL1, EOL2, EOL3.  I use these to hold failure, retry, original, no-retry, etc type outbound connections during testing.  As I move to production, some of these may be auto terminated, and some may remain. In production, I route ALL major points of failure which are not auto terminated to an output port called ERROR.  In my flows I sometimes have multiple ERROR output ports for difference purposes.   Depending on the use case, the error is sent to some type of event notification.  For Example:  send an email, or post to a slack channel.  In other cases these are routed to a process group that is in a stopped/disabled state.  Based on the error, I may make some change in the flow, and then enable/start the flow to create a method to REPLAY that flowfile. In reference to successful end of flow, I may just auto terminate, as I want to assume the flow is always on, and everything has succeeded if it is not routed as above in #1 or #2.  In other cases the bottom or end of the flow is also routed to some type of Event Notification.  However, be careful here, as you could create tons of notifications depending on your flow. I hope some of these ideas are helpful.     If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.     Thanks, Steven ... View more

@wenfeng  Check that the user hue is able to write to the target directory.  You may need to create a hue user directory with proper permissions if the hue user cannot write to the hive user directory.     If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.     Thanks, Steven ... View more

@alex15   I suspect the issue you have is that nifi use is not able to execute the script. Make sure the user ownership of the file is correct, also confirm read/write permissions.  In unix/linux these are chown and chmod.        If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.     Thanks, Steven ... View more

1. Check on the user permission of the jar file you added to your class path. Make sure jar file is in right place. 2. When you add driver manually also add this to sqoop command  --driver =  org . postgresql . Driver  ... View more

@lukolas The example you provide sees to be REGEX not expression language.  You would need to test some kind of expression language in that Topic Name(s) property.   Another suggestion would be to have a file, or for example generateflowfile, which contains a list of topics.  Then split/extract that list into attributes, then send that attribute to the topic name.     Having a ton of topics going to single processors can become a bottleneck which creates tons of downstream flowfiles from each topic.  So be careful with tuning and concurrency in reference to number and topics and messages per topic.   If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.     Thanks, Steven @ DFHZ ... View more

@CSRabbit  Installing the PostgreSQL JDBC Driver Download the PostgreSQL JDBC driver from http://jdbc.postgresql.org/download.html and copy it to the/var/lib/sqoop/ directory. For example: $ curl -L 'http://jdbc.postgresql.org/download/postgresql-9.2-1002.jdbc4.jar' -o postgresql-9.2-1002.jdbc4.jar $ sudo cp postgresql-9.2-1002.jdbc4.jar /var/lib/sqoop/  Snippet from (reference) for cloudera 5.9x but should be similar for other versions, ambari/hdp, or native sqoop.  Just make sure you have the right download url and path for sqoop.      If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.     Thanks, Steven @ DFHZ ... View more

I suspect you have not completed a step, or missing something.  The cacerts works for me in all cases if the cert is publicly trusted (standard public cert from public CA) which it should be.    You should share info on the configurations you tried and what if any errors you got from that.  The bare minimum settings you need for that are keystore (file location), password, key type (jks), and TLS version.   Assuming you copied your java cacert file to all nodes as /nifi/ssl/cacerts the controller service properties should look like:       If cacerts doesnt work, then you must create keystores and/or trust stores with the public cert.   Use the openssl command to get the cert.  That command looks like:   openssl s_client -connect https://secure.domain.com   You can also get it from the browser when you visit the elk interface; for example cluster health, or indexes.   Double click cert lock icon in the browser then use the browser's interface to see/view/download public certificate.  You need the .cer or .crt file.   Then you use the cert to create the keystore with keytool commands.  An example is:   keytool -import -trustcacerts -alias ambari -file cert.cer -keystore keystore.jks   Once you have created a keystore/truststore file you need to copy it to all nifi nodes, ensure the correct ownership, and make sure all the details are correct in the SSL Context Service.  Lastly you may need to modify the TLS type until testing works.   Here is working example of getting the cert and using it with keytool from a recent use case:   echo -n|openssl s_client -connect https://secure.domain.com | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > publiccert.crt keytool -import -file publiccert.crt -alias astra -keystore keyStore.jks -storepass password -noprompt   keytool -import -file publiccert.crt -alias astra -keystore trustStore.jks -storepass password -noprompt   mkdir -p /etc/nifi/ssl/ cp *.jks /etc/nifi/ssl chown -R nifi:nifi /etc/nifi/ssl/     ... View more

@HarshR  You need to configure your SSLContextService with a keystore/truststore built with the cert you get from the Elasticsearch Cluster.   You can also try cacerts that is included with your Java.  This is usually easier to do.  More details here for cacerts:   https://community.cloudera.com/t5/Support-Questions/Connecting-to-DataSift-HTTPS-API-using-NiFi-GetHTTP/td-p/102276     If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.     Thanks, Steven @ DFHZ ... View more

@Francesco_Fa   If you do not want to use embedded sqlite, the options available for this are: Mysql Postgres Oracle https://docs.gethue.com/administrator/administration/database/       If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.     Thanks, Steven @ DFHZ ... View more