Member since
01-03-2017
27
Posts
10
Kudos Received
3
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2920 | 11-21-2018 06:00 PM | |
5177 | 11-21-2018 05:09 PM | |
1268 | 11-21-2018 03:19 PM |
12-10-2018
05:23 PM
@Zhao Chaofeng Looks like you are missing appropriate permissions for hbase to access the hdfs directory.
... View more
12-10-2018
04:45 PM
2 Kudos
@Gowthaman V check out this gist which has an example flow to properly use the distributed cache - https://gist.github.com/ijokarumawak/8ba9a2a1b224603f877e960a942a6f2b Also this might help you - https://stackoverflow.com/questions/44590296/how-does-one-setup-a-distributed-map-cache-for-nifi
... View more
11-21-2018
06:00 PM
Have you tried ssh'ing to the Host machine ip and port 4200 (seems like you are able to ssh from host machine to sandbox using 127.0.0.1:4200 ) ?
... View more
11-21-2018
05:09 PM
1 Kudo
@Julez Nyzl
JSON Jolt is mainly used to modify the structure of the JSON document. The NiFi JSON processor also supports expression language.
For you specific case, I would recommend using a combination of both Json Path processor and Jolt processor
Step 1 : extract the "value" part from the document e.g "test" into a flow file variable using the json path processor
Step 2 : use jolt processor to transform the key "id" into "number" and for the vaule part use the nifi expression language to manipulate the flow file variable as you would need it. Jolt spec to transform "id" into "number" [
{
"operation": "shift",
"spec": {
"id": "number"
}
}
]
here is a link for How to use NiFi EL in jolt processor by @Yolanda M. Davis - https://community.hortonworks.com/articles/105965/expression-language-with-jolt-in-apache-nifi.html
... View more
11-21-2018
03:19 PM
1 Kudo
HDP & HDF are hardened and flexible platforms with several options to configure security https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/security.html . Industry certifications such as PCI, HIPAA are based on how HDP/HDF is implemented by the customer/project. Here is a link to a whitepaper - https://community.hortonworks.com/storage/attachments/1601-hortonworks-pci-compliance-wp.pdf Also this HCC Q&A talk about it more - https://community.hortonworks.com/questions/11973/do-we-need-any-customization-in-hortonworks-to-imp.html
... View more
11-21-2018
03:05 PM
Can you try writing it out to hdfs location where you have write permissions ?
... View more
07-19-2018
04:37 AM
2 Kudos
A quick example NiFi flow to demonstrate sending logs to syslog from NiFi. Steps: 1. Wait for new compressed files (gz) to land in folder (/tmp/nifi/syslog/src/*.gz) with log lines. 2. Decompress the file 3. Split the file into individual lines 4. Send data to syslog NiFi Template : Download the following NiFi template and import the flow into your NiFi instance. Instructions to import a NiFi template is available here - Importing NiFi templates send-to-syslog.xml Send to Syslog Template Walkthrough : The Flow is broken down into 3 sections Section 1 : Test Data Generation Section 2 : Processing (decompress files and send to local syslog) Key Highlights ExtractText Processor : We are using the extract text processor to get the content of the flowfile into the flowfile attribute "msg". We are grabbing the entire content of the flowfile into "msg" by using the regular expression (.*). You may get only specific fields by providing appropriate regular expression here. PutSyslog processor : We are using the flowfile attribute ${msg} in the syslog message body. We have also prefixed string "NifiLogger" to the syslog message (optional). Section 3 : Validation (receive syslog messages)
... View more
Labels:
04-20-2018
06:51 AM
You can use the BINARY data type in Hive. Store the photo/image as binary in the hive table. You may retrieve it back from the query results and display it in your frontend application. https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-MiscTypes
... View more