Member since
07-30-2019
3133
Posts
1565
Kudos Received
909
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
156 | 01-09-2025 11:14 AM | |
888 | 01-03-2025 05:59 AM | |
439 | 12-13-2024 10:58 AM | |
497 | 12-05-2024 06:38 AM | |
392 | 11-22-2024 05:50 AM |
11-17-2016
07:37 PM
1 Kudo
@Saikrishna Tarapareddy Are you running your NiFi on Windows or Linux based OS? I do not have a Windows based server available to look at this, but try changing the configurations for path to the keystore and truststore in your nifi.properties file to the fully qualified windows path rather then .\conf\nifidev.truststore.jks Matt
... View more
11-17-2016
06:53 PM
1 Kudo
@Hellmar Becker You may want to try merging your CSV files before sending them to the ConvertCSVtoAvro processor. Try configuring your MergeContent as follows: Feed merged output from this processor to the ConvertCSVtoAvro, then that output to ConvertAvroToORC processor, and finally off to hive. Thanks, Matt
... View more
11-17-2016
06:30 PM
@Hellmar Becker What format are your original messages in before you convert them to Avro?
... View more
11-17-2016
05:29 PM
1 Kudo
@Saikrishna Tarapareddy It seems to not like your users.xml file. Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authorizer': FactoryBean threw exception on object creation; nested exception is org.apache.nifi.authorization.exception.AuthorizerCreationException: javax.xml.bind.UnmarshalException - with linked exception: [org.xml.sax.SAXParseException; systemId: file:/F:/Tools/HDF-2.0.1.0/nifi/./conf/users.xml; lineNumber: 1; columnNumber: 1; Premature end of file. The users.xml file is created using the values configured in the authorizers.xml file when NiFi is started for the first time after being configured for SSL. Subsequent restarts of NiFi will not result in a new users.xml file being created. This File should not be manually created or edited. Since this is the first time trying to get HTTPS working, try deleting the users.xml and authorizations.xml files and restart again. Do you see the same error still? If so, can you share the configuration from your authorizers.xml file? Thanks, Matt
... View more
11-16-2016
04:03 PM
1 Kudo
Templates when created are not inherently made available to all users. They are tightly coupled to the authorization policies of the process group they are created in. Since these templates are coupled in this way to a particular process group, that process group cannot be deleted until those associated templates are removed. If you access the "templates" UI, you will see a "Process Group Id" column. You should see the process group ID for your "Templates" process group associated with two of your templates. NiFi does not give you the ability to re-associated templates with another existing process group. The idea here is to allow users who are restricted via access policies to some specific process group(s) in NiFi to upload there templates which may contain sensitive configurations and prevent other users from being able to load/use them. Of course a user can adjust the access policies once the template has been uploaded; however, the template remains associated with the Process group in which it was uploaded. Thanks, Matt
... View more
11-15-2016
08:25 PM
Side note: You can install a NiFi 1.x or HDF 2.x and use the flow.xml.gz file from an older 0.5.x+ or HDF 1.1+ version of NiFi. The newer version of NiFi will convert it to the new format.
... View more
11-15-2016
08:22 PM
1 Kudo
@sagar gaikwad Difference between templating the entire NiFi canvas and copying the actual flow.xml.* files. When you generate a template it is saved as and xml and contains all the configuration from the canvas with the EXCEPTION of sensitive property values (passwords). Sensitive processor properties are encrypted using the sensitive props key configured in the nifi.properties file. So by removing these values when a template is created the templates to be loaded into any other NiFi successfully. If you capture the flow.xml.* files directly from NiFi, you will also need to save/reserve the value set for the sensitive props key in order to use them on another system. The above is true no matter which version of NiFi you are using. NiFi 0.x /HDF 1.x versions: With these versions of NiFi you have a NiFi Cluster Manager (NCM) and NiFi nodes. The Nodes all have identical flow.xml.gz files and an identical list of templates inside the /templates directory. The NCM has a flow.tar file. This file is not the same as what is on the nodes. The flow.tar consists of numerous files important to the NCM only. (NCM specific controller services, primary node designator, etc..) The flow.tar also includes a flow.xml file. It is from this flow.xml file on the NCM that the flow.xml.gz and templates inside the /templates dir on the nodes are derived from. You can build a new cluster if you have the flow.tar file as it contains everything new nodes need plus what ta new NCM needs. The flow.tar cannot be used on a standalone install of NiFi nor can it be used by the nodes directly themselves. It is intended for an NCM only. The flow.xml.gz file from any of the nodes can be used to stand-up a standalone NiFi instance. You may choose to keep a backup of both. NiFi 1.x /HDF 2.x versions: These versions of NiFi have eliminated the need for a NCM (zero-master clustering). As a result every instance of Nifi is a node and they all have identical flow.xml.gz files. These flow.xml.gz files do not contain the exact same content as the nodes from 0.x versions of NiFi. The templates have all been moved into the flow.xml.gz file and there is no longer a need for a separate /templates dir. With this version you can keep a backup or flow.xml.gz from any node. The flow.xml.gz file can be used to buidl another cluster or a standalone NiFi instance. Thanks, Matt
... View more
11-15-2016
02:09 PM
@Iyappan Gopalakrishnan Adding new users to a secured NiFI 0.x / HDF 1.x version of NiFi can be done directly via the NIFi UI. I am assuming you have already secured your NiFi and manually added the initial admin to the authorized-users.xml file. Adding new users is pretty easy. Simply have the new user attempt to access the https web address for your nifi 0.7 install. After authenticating successfully (Done via client certs, ldap, etc), they will see a screen telling them they are not authorized for access and giving them the opportunity to request access. Once they request access, the "Admin" user who does have access to the UI will need need to go into the users UI by clicking on the below Icon which can be found in the upper right corner of the NiFi UI: From within that UI you will see all existing users and any pending authorization users. Simply click on the pencil icon to the right of any pending users to authorize them for 1 or more of the available user roles (Administrator, Dataflow Manger, Read Only, or Provenance). The administration-guide included with your install can give you more details on each of the roles and what privileges each grants to the users. Thanks, Matt
... View more
11-15-2016
01:53 PM
@Rajkumar Singh @Iyappan Gopalakrishnan The link here references an administration-guide for NIFI 1.x baseline. The NIFI 0.x baseline is very different. I suggest reading the administration-guide that was bundled with your download version for more accurate information on adding and setting up new users. Thanks, Matt
... View more
11-15-2016
01:45 PM
@Karthik Narayanan The link you reference is good for troubleshooting adding new nodes in NiFi 0.x and HDF 1.x versions of NiFi which had an NCM. NiFi 1.x and HDF 2.x version are very different now and do not have an NCM.
... View more