Member since
01-27-2023
229
Posts
73
Kudos Received
45
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
708 | 02-23-2024 01:14 AM | |
910 | 01-26-2024 01:31 AM | |
619 | 11-22-2023 12:28 AM | |
1425 | 11-22-2023 12:10 AM | |
1617 | 11-06-2023 12:44 AM |
03-29-2023
02:14 PM
What do you mean with it is working fine on local server but there is a problem when you access it remotely? How do you access it remotely? Are you using the Apache NiFi version or the Clouder version? Now, as I do not have all the info yet, I would go ahead and ask a dumb question. If you are trying to access it remotely, I assume that you are using a different device (pc, laptop, vm, etc.), so therefore, are you sure that you are using the same keyboard layout and you are typing the correct password?
... View more
03-29-2023
08:04 AM
1 Kudo
Hi @swanifi, I would start by uploading the full error stack and not a photo of a partial error, taken with your phone. Afterwards, I would tell you that the mentioned processor, based on my knowledge, is not an official processor, belonging to Apache NiFi or Cloudera NiFi stack. This means that in most cases, you will have to discuss this topic directly with the developer of that custom made processor. In addition to this, I would double check the MarkLogic DB for any restrictions in term of connectivity (restrict the number of inserts via API to a number of X items)
... View more
03-29-2023
06:07 AM
To be honest I do not really understand your question. If you split your original file into smaller chunks, you will have X flowfiles instead of a single flowfile. Those flowfiles will then go sequentially in your HTTP response Processor, based on your load balance strategy.
... View more
03-29-2023
05:46 AM
hi @saquibsk, I assume that you are using the single user provider policy and not the integration with the LDAP/Kerberos and so on. Most likely, when you deleted and re-extracted your NiFi Folder you have over-written your authorizations.xml, authorizers.xml, login-identity-provider.xml and users.xml files. From my point of view, your problem is related to the login-identity-providers.xml, as this is the file where you have your password for your user. You could try and get a back-up of this file, extract the password from there and paste it into your current file. Otherwise, you can generate a new password for your user. Or (not sure if it will work) you can see what encryption NiFi uses for hashing the password, and generate your old password using that hash and paste it within your login-identity-provider.xml file. PS: make sure that you stop NiFi before modifying the file 🙂
... View more
03-28-2023
09:39 AM
Well, there you have it 🙂 Your problem is not related directly to NiFi and it is caused by the executed SQL statement. I am not very experienced with SQL Servers but you could try selecting all the fields from the table and see if you still encounter an error message. If the error still persists, you could use the convert function on the where clause --> CONVERT(datetime,your_value,25) or select convert(varchar, your_value, 25)
... View more
03-28-2023
08:04 AM
I see that you are executing a different select in your database than the one executed in NiFi. Try with that exact SQL statement and see if it works. In what you are executing now, I see no WHERE condition set on that column date. In NiFi you are executing: "SELECT dt_change FROM CompanyXYZ.dbo.test WHERE dt_change > '2023-03-28 09:55:38.783' " As for the convert function, you receive no error message, it works just fine. But as stated above, you need to select the correct format for your use case. All the formats are present in that link.
... View more
03-28-2023
08:03 AM
I see that you are executing a different select in your database than the one executed in NiFi. Try with that exact SQL statement and see if it works. In what you are executing now, I see no WHERE condition set on that column date. In NiFi you are executing: "SELECT dt_change FROM CompanyXYZ.dbo.test WHERE dt_change > '2023-03-28 09:55:38.783' " As for the convert function, you receive no error message, it works just fine. But as stated above, you need to select the correct format for your use case. All the formats are present in that link.
... View more
03-28-2023
07:52 AM
hi @kumsath, If you want to split your output from ExecuteSQL, you could try replacing ConvertAvroToJson with a SplitRecord in which you will configure the Record Reader as AVRO and the Record Writer as JSON, whereas the Records per Split can be set by you to any value. In this way, your ExecuteSQL results are always split into smaller chunks, based on your desired output. Of course, you can use the NiFi Expression Language and calculate the Records per Split dynamically and so on.
... View more
03-28-2023
07:43 AM
As I can see, the problem is not strictly related to NiFi, but to the query you are executing. If you try and execute this query directly on your Database, does it work? You first need to identify where the issue occurs, when you query the database or when you make the conversion. If the problem occurs when fetching the data, I think you will need to use the convert function from sql to achieve your goal. Something like: CONVERT(VARCHAR,your_date_Value,103), where your_date_value is your date and 103 is the date format, which can be found here: https://www.mssqltips.com/sqlservertip/1145/date-and-time-conversions-using-sql-server/ This is just an example of that function, you will need to apply it based on your use case. You could also try fetching another column instead of dt_change to further debug your flow.
... View more
03-28-2023
06:58 AM
Hi @Daniel_AB, As far as I know, when executing QueryDatabaseTable, the output you have is in AVRO Format. In order to fully use the AVRO capabilities and the data types, I would recommend to set the property "Use Avro Logical Types" to true and test how your data look like afterwards. Do you have it on false by any chance?
... View more