Member since
07-09-2018
19
Posts
4
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
5719 | 08-21-2018 01:19 PM | |
974 | 08-20-2018 01:39 AM | |
1242 | 07-12-2018 08:26 PM | |
1055 | 07-09-2018 07:07 PM |
08-24-2018
06:57 PM
@Ron Labau if you are using QueryDatabaseTable processor then it is regular sql function on db. if you are using convertRecord processor then we are doing in NiFi.
... View more
08-21-2018
04:13 PM
I actually think the pull is not my problem. its inserts to redshift, which are slow. think i need to put this data in files, and do copy commands to redshift. guess i need to move to saving this data in files, and then copy/load. Or i guess i can manually load tables, and then set an initial max value column, to have the nifi job start from there, and continue updating. These large tables are challenging.
... View more
08-20-2018
01:39 AM
2 Kudos
Think I found that is was just the initial memory allocation only being at the default 512mb. upped it to 25gb and seems to be running fine.
... View more
07-12-2018
08:26 PM
Well, I took an image of the server just after setup, and i tried again on a new ubuntu server, and it worked! Think nifi is running fine, about to check it out. I removed installing the ambari-metrics. not sure what the difference is, bcz i didn't change much other than that. Oh, one part did complain about disk space, when i went back and tried to reinstall again, so I bumped up from 8 to 32gb. Seems zookeeper and nifi installed ok.
... View more
07-09-2018
07:07 PM
1 Kudo
ah, i think i figured it out, used the hostname -f command to get the host, and that did the trick, used that as the hostname. on my way now!
... View more