Thanks Andre, I did manage to get the token using access/token api. however when I provided the token in postman as Bearer authentication I still get the 403 Forbidden response. Here is my request and response info as captured by Fiddler, let me know if you see anything wrong: PUT https://[server name]:9443/nifi-api/processors/385fcdc0-0180-1000-0000-000030a768e3/run-status HTTP/1.1 Content-Type: application/json Accept: application/json, text/javascript, */*; q=0.01 Keep-Alive: timeout=100, max=50000 Authorization: Bearer [access token] User-Agent: PostmanRuntime/7.29.0 Postman-Token: 5900c41a-f704-43f3-a2e4-a425eeb22569 Host: [host name]:9443 Accept-Encoding: gzip, deflate, br Connection: keep-alive Content-Length: 215 { "revision": { "clientId": "8F3BD748-DBCC-4703-8743-1D98A24B95C2", "version": 1.16, "lastModifier": "user.name" }, "state": "RUNNING", "disconnectedNodeAcknowledged": true }   Response:   HTTP/1.1 403 Forbidden X-Frame-Options: SAMEORIGIN Content-Security-Policy: frame-ancestors 'self' X-XSS-Protection: 1; mode=block X-Content-Type-Options: nosniff Strict-Transport-Security: max-age=31540000 Content-Length: 0 Server: Jetty(9.4.45.v20220203)     ... View more

I never worked with Hive DB, but if it only takes avro format and you have csv you can use processor ConvertRecrod to convert a record from one format to another. In this processor you create CSV Reader service and the writer is Avro ... View more

There is a lot of articles\videos around learning Regular Expression. The one I provided using what is called positive lookbehind (?<=), which means: find the Id which contain alphanumeric characters with hyphen (-) that is proceeded (lookbehind) with the text "requestUid" ... View more

there are a lot of processors that can help you with that for example : PUTSQL, PutDatabaseRecord, ExecuteSQL ... View more

have you tried Selected distinct * order by Date, ERROR_MSG ... View more

It should not matter if the format of the text inside is like csv. ... View more

you can do that using ExtractText Processor to extract the line that you need with word "bind" into an attribute using regular expression "( bind.+$) ", then you ReplaceText processor to basically located the same line using the same regular expression and the the replacement  value is the attribute from the ExtractText processor (assume its called XYZ) like this ${XYZ:replace(" ","")} ... View more

Not sure if this is the best way but try using QueryRecord and create one dynamic property to funnel the data into where your query will look like this: Select distinct * from FLOWFILE ... View more

Thank you so much,   That did the trick. You seem to be a json jolt guru. Im trying to understand it myself but its not that straight forward. Can you just elaborate this syntax for me which remove null values on the base level: "*": { "*": { "@1": "&2" } } Why there is two asterisks in front of it? ... View more

Hi,   Best way I can think of store that attribute name in another attribute. You can also use execute script processor and read the attribute name you are looking for as in  :   var myAttr = flowFile.getAttribute('filename') and if the value is not  null then you can route the flow File to the desired relationship. For more information on how to use the execute script processor: https://community.cloudera.com/t5/Community-Articles/ExecuteScript-Cookbook-part-1/ta-p/248922 ... View more

can you share the jolt script so I can take a look? ... View more

Hi,   Have you tried using JsonJolt processor. Its like json transformation from one format to another. Im not Json jolt expert myself but I found a link that might help you in what you are trying to do: https://www.titanwolf.org/Network/q/1600720e-fcfb-4711-bce6-4cca97006d16/y   ... View more

Matt,   Thanks for your reply. Basically I have two local nifi's: one is that 1.11 and another is 1.14 both of them are setup almost identical in the nifi.properties, however 1.11 is working when passing the azure workspace key and Id and the other 1.14 throwing the error. the log file doesnt have much info besides the message I posted above but here is the stack trace from the log file if that helps in anyway: java.lang.RuntimeException: HTTP/1.1 403 Forbidden at org.apache.nifi.reporting.azure.loganalytics.AbstractAzureLogAnalyticsReportingTask.postRequest(AbstractAzureLogAnalyticsReportingTask.java:164) at org.apache.nifi.reporting.azure.loganalytics.AbstractAzureLogAnalyticsReportingTask.sendToLogAnalytics(AbstractAzureLogAnalyticsReportingTask.java:147) at org.apache.nifi.reporting.azure.loganalytics.AzureLogAnalyticsReportingTask.sendMetrics(AzureLogAnalyticsReportingTask.java:137) at org.apache.nifi.reporting.azure.loganalytics.AzureLogAnalyticsReportingTask.onTrigger(AzureLogAnalyticsReportingTask.java:107) at org.apache.nifi.controller.tasks.ReportingTaskWrapper.run(ReportingTaskWrapper.java:44) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)   Im hoping someone can try the same on their end to verify if they are getting the same results ... View more

You really dont need a screenshot because you are not changing much properties:   1-  Create ListFile Processor & set the "Input Directory" to whatever directory you want to track. 2- Create a FetchFile Processor and connect the ListFile to it via the "success" relationship. under the processor properties keep the "File to Fetch" property set to "${absolute.path}/${filename}" since the path and the file name will be set in those attributes using the ListFile and that is it.   After that the content of the file will be passed via the success relation and you can do whatever you want with it just as if you are using GetFile except the ListFile will keep state of the latest file timestamp it grabbed and basically use that to grab any new files added to the folder and update the state to new timestamp and so. ... View more

I think I know what is the problem. Since you are not doing any fragmentation -like in using split processor- try setting the Support Fragmented Transaction to False in the PUTSQL. Also remove the original \ failure connection from the Convert processor. you can mute those relationship in the convert setting table by checking the box. ... View more

Hi,   You seem to be having the correct processors and sequence. Can you share screenshot with the configuration of the put SQL. Have you configured the PUTSQL with the correct DB connection? Also are you seeing data (FlowFiles) flowing into it from the above processors. It could be that you are not getting any data to begin with or your configuration in the putSQL is incorrect. By the way why are you connecting failuer,original, sql relationship from the ConvertJsonTOSQL to the PUT SQL? you will get an error this way, you need to just connect the sql releationship. Another thing , the SQL property in the PUTSQL needs to be empty so that it process the sql received from the convert processor, if you have anything in there it will not take the sql insert from the convert processor ... View more

Take  a look at the Nifi ListFile & Fetch File processors. They both work together. The ListFile will read files metadata based on the last read file modified date and will keep state of that so that only newly added files will be read. The fetch file will take the filename parameter from the ListFile processor and fetch the contents. Hope that helps ... View more

Can some one help verify this please. Im thinking that there is bug in the 1.14 version around this, because I have tried the same ReportingTask on version 1.11.4 against the same Azure Log Analytics workspace ID and Key and its working, so that tells me its not something with Azure. Both versions have the same settings when it comes to nifi.web.https and nifi.remote.input properties except for nifi.remote.input.socket.port where its set for different ports.  Thank you.   ... View more

Its Solved after upgrading to 1.14.0 ... View more

Its Solved in the latest version 1.14.0. I'm not sure if they addressed this issue in later releases of 1.13 version ... View more