@mclark Thank you Matt. We are trying to setup a standalone server. lets say if we cant afford RAID 10 disks. What are the alternate better options. Can we go with RAID 1 for all.?? and in future if we decided to go for RAID 10 can we easily change the config files and migrate.? ... View more

Hi @mclark I just read this article by you . It was very helpful. https://community.hortonworks.com/articles/7882/hdfnifi-best-practices-for-setting-up-a-high-perfo.html In your section about Hardware , i am trying to understand about this. using RAID 1 array we will be able to point different repositories to different hard disk drives.?? if so do we need to specify the size of each disk as the one we are going to use for content repo needs to be in TBs whereas flow file repo and prov repo doesn't have to be like that... anything else apart from repos needs to be on seperate drives.? in your break down here each / is point to a different drive.?? (1 hardware RAID 1 array) RAID 1 array (This could also be a RAID 10) logical volumes: -/ -/boot -/home -/var -/var/log/nifi-logs <-- point all your NiFi logs (logback.xml) here -/opt <-- install NiFi here under a sub-directory -/database-repo <-- point NiFi database repository here -/flowfile-repo <-- point NiFi flowfile repository here ... View more

I am also interested to know if Nifi is processer heavy or memory heavy tool. ... View more

@Matt Burgess , @jfrazee I wrote this Shift transformation in NiFi JOLT , which is flattening above JSON..just wanted to make sure of this would be the correct approach..?? Only thing is if we come across a new Tag in input then we will have to write transformations to it. { "glossary": { "title": "Glossary Title", "GlossDiv": { "title*": "GlossDivTitles", "GlossList": { "GlossEntry": { "ID": "GlossEntryID", "SortAs": "GlossEntrySortAs", "GlossTerm": "GlossEntryGT", "Acronym": "GlossEntryAcronym", "Abbrev": "GlossEntryAbbr", "GlossDef": { "*" : "GlossDef&" }, "GlossSee": "GlossEntryGSee" } } } } } ... View more

Hi @Matt Burgess @jfrazee Where can i find help for JOLT JSON transformations..i am trying to flatten this example json with multi levels. { "glossary": { "title": "example glossary", "GlossDiv": { "title": "S", "GlossList": { "GlossEntry": { "ID": "SGML", "SortAs": "SGML", "GlossTerm": "Standard Generalized Markup Language", "Acronym": "SGML", "Abbrev": "ISO 8879:1986", "GlossDef": { "para": "A meta-markup language, used to create markup languages such as DocBook.", "GlossSeeAlso": ["GML", "XML"] }, "GlossSee": "markup" } } } } } ... View more

🙂 thanks. its there. they should have removed it from the other place i was looking at. ... View more

@Matt Burgess i tried to do that , but it is not filling in the Input or Specs. ... View more

@jfrazee when I tried to look at these examples , its not showing any JSON or specs. All I get is empty screen like this.. when I tried this link below http://jolt-demo.appspot.com/#bucketToPrefixSoup ... View more