Member since
02-16-2016
176
Posts
197
Kudos Received
17
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3193 | 11-18-2016 08:48 PM | |
5683 | 08-23-2016 04:13 PM | |
1553 | 03-26-2016 12:01 PM | |
1454 | 03-15-2016 12:12 AM | |
15575 | 03-14-2016 10:54 PM |
04-25-2016
02:58 PM
@james.jones @Zaheer N I tried that and it works really well. Definitely Groovy !
... View more
04-10-2016
03:05 AM
@Suresh Babu Shanmugam I find using free form query to be more effective, because you can control exact interval/range for which you want to bring data.
... View more
04-10-2016
03:00 AM
@Ian HellströmYou don't need to specify exact SQL query using NiFi also. Query can be generated using NiFi expressions which can be driven from any input file/table. NiFi is not the best tool for performing any ETL kind of operations but to ingest data to HDFS, it is effective.
... View more
04-08-2016
03:51 AM
5 Kudos
With HDF 1.1.2.1 release, HDF supports accessing Kerberos enabled kafka topics. For a stanalone NiFi node, following instructions can be used. 1. new jass file should be created with following entries. Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="./conf/nifi.keytab"
useTicketCache=false
principal="nifi@EXAMPLE.COM”;
};
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useTicketCache=false
renewTicket=true
serviceName="kafka"
useKeyTab=true
keyTab="./conf/nifi.keytab"
principal="nifi@EXAMPLE.COM";
};
*** Both the KafkaClient and Client configs should use the same principal and key tab *** 2. bootstrap.conf file should be changed to include following line. java.arg.15=-Djava.security.auth.login.config=/<path>/zookeeper-jaas.conf Restart NiFi node after changes to bootstrap.conf. 3. GetKafka/PutKafka processors should be modified to include a new property security.protocol and value for this property should be set to PLAINTEXTSASL kafka.png That's it ! Now NiFi can read from Kerberos enabled kafka topics. Thank You @rgarcia for helping me with these configurations.
... View more
Labels:
04-08-2016
02:31 AM
1 Kudo
@Artem Ervits My database is Oracle, so I used sqlplus to execute a script. Where script has following content. execute spname(param)
exit I used ExecuteProcess processor to call sqlplus Note: This approach assumes that client tools for database are installed on NiFi Server.
... View more
04-07-2016
06:27 PM
2 Kudos
Thank You @Artem Ervits for your suggestion. I was able to call my stored proc using sqlplus and ExecuteProcess processor. Amazing community !
... View more
04-06-2016
12:02 PM
4 Kudos
Can I execute stored procedure on database using ExecuteSQL proessor ? Is there any sample ?
... View more
Labels:
- Labels:
-
Apache NiFi
03-31-2016
04:39 AM
4 Kudos
@Randy Gelhausen @Geoffrey Shelton Okot @Vadim In my opinion, Nifi has an advantage over Sqoop because development time is significantly reduced. If you are not planning on bringing TB of data and your incremental data volume is not huge, you can use NiFi to bring data. Bringing data in parallel using Sqoop is a double edge sword. Data gets loaded faster, but it puts load on source data store which may not be acceptable. In an enterprise environment, there are limits on number of connections open to database and parallelism in NiFi can be obtained working on independent datasets or a partition of target data. To me, NiFi shortcomings are more on operational side with not robust options for notification in case of failure.
... View more
03-29-2016
06:30 PM
@Andrew Grande To be clear, does HDF 1.1.2.1 also support Kerberized Hbase ?
... View more
03-26-2016
12:01 PM
2 Kudos
@auto gun Look into Host Groups to manage hosts with different configurations. https://developer.ibm.com/hadoop/blog/2015/11/10/override-component-configurations-with-ambari-configuration-groups/ You can create one host group with 12 disks and 2nd host group with 16 disks. Once your groups are correctly added, any missing data on initial nodes in 12/,13/,14/,15 will be redistributed to new nodes by HDFS. At that point you can free your space.
... View more