Can Oozie invoke and RestFul service for notification ? Is there an example which can be used ... ... View more

We have a use case to use "InvokeHttp" for restful service, getting all the details from the DB (URL/username/password), need to pass basic authentication (username and password) via attributes retrieved from DB. Is it possible to pass authentication using attributes ? ... View more

Is there a way to clean up flow files and keep only the attributes after event processing is completed. Need to remove Flow files for security reasons. Any suggestions if we can delete or any other way of handling the above use case. ... View more

Hello, Apache Nifi doesn't show data Provenance and show status as "Searching provenance events", checked logs it shows the following error. Any suggestions to resolve the same. 2017-03-15 08:40:06,603 ERROR [Provenance Repository Rollover Thread-2] o.a.n.p.PersistentProvenanceRepository Failed to merge journals. Will try again. journalsToMerge: [./provenance_repository/journals/2642.journal.0, ./provenance_repository/journals/2642.journal.1, ./provenance_repository/journals/2642.journal.2, ./provenance_repository/journals/2642.journal.3, ./provenance_repository/journals/2642.journal.4, ./provenance_repository/journals/2642.journal.5, ./provenance_repository/journals/2642.journal.6, ./provenance_repository/journals/2642.journal.7, ./provenance_repository/journals/2642.journal.8, ./provenance_repository/journals/2642.journal.9, ./provenance_repository/journals/2642.journal.10, ./provenance_repository/journals/2642.journal.11, ./provenance_repository/journals/2642.journal.12, ./provenance_repository/journals/2642.journal.13, ./provenance_repository/journals/2642.journal.14, ./provenance_repository/journals/2642.journal.15], storageDir: ./provenance_repository, cause: java.lang.RuntimeException: java.io.FileNotFoundException: _1g.fdt 2017-03-15 08:40:06,607 ERROR [Provenance Repository Rollover Thread-2] o.a.n.p.PersistentProvenanceRepository java.lang.RuntimeException: java.io.FileNotFoundException: _1g.fdt at org.apache.lucene.index.TieredMergePolicy$SegmentByteSizeDescending.compare(TieredMergePolicy.java:258) ~[na:na] at org.apache.lucene.index.TieredMergePolicy$SegmentByteSizeDescending.compare(TieredMergePolicy.java:238) ~[na:na] at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) ~[na:1.8.0_101] at java.util.TimSort.sort(TimSort.java:234) ~[na:1.8.0_101] at java.util.Arrays.sort(Arrays.java:1512) ~[na:1.8.0_101] at java.util.ArrayList.sort(ArrayList.java:1454) ~[na:1.8.0_101] at java.util.Collections.sort(Collections.java:175) ~[na:1.8.0_101] at org.apache.lucene.index.TieredMergePolicy.findMerges(TieredMergePolicy.java:292) ~[na:na] at org.apache.lucene.index.IndexWriter.updatePendingMerges(IndexWriter.java:2020) ~[na:na] at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1984) ~[na:na] at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3029) ~[na:na] at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3134) ~[na:na] at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3101) ~[na:na] at org.apache.nifi.provenance.lucene.SimpleIndexManager.returnIndexWriter(SimpleIndexManager.java:162) ~[na:na] at org.apache.nifi.provenance.PersistentProvenanceRepository.mergeJournals(PersistentProvenanceRepository.java:1864) ~[na:na] at org.apache.nifi.provenance.PersistentProvenanceRepository$8.run(PersistentProvenanceRepository.java:1332) ~[na:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_101] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_101] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_101] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_101] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_101] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_101] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101] Caused by: java.io.FileNotFoundException: _1g.fdt at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:255) ~[na:na] at org.apache.lucene.index.SegmentCommitInfo.sizeInBytes(SegmentCommitInfo.java:219) ~[na:na] at org.apache.lucene.index.MergePolicy.size(MergePolicy.java:478) ~[na:na] at org.apache.lucene.index.TieredMergePolicy$SegmentByteSizeDescending.compare(TieredMergePolicy.java:248) ~[na:na] ... 22 common frames omitted Thanks, Nagesh ... View more

Hello, We are trying to read data from Oracle tables, "Date" based data types are converted into "Timestamp" Data types. e.g: Table is Oracle. desc hr.employees; Name Null? Type ----------------------------------------- -------- ---------------------------- EMPLOYEE_ID NOT NULL NUMBER(6) FIRST_NAME VARCHAR2(20) LAST_NAME NOT NULL VARCHAR2(25) EMAIL NOT NULL VARCHAR2(25) PHONE_NUMBER VARCHAR2(20) HIRE_DATE NOT NULL DATE JOB_ID NOT NULL VARCHAR2(10) SALARY NUMBER(8,2) COMMISSION_PCT NUMBER(2,2) MANAGER_ID NUMBER(6) DEPARTMENT_ID NUMBER(4) SSN VARCHAR2(55) and schema read in the DataFrame in Scala |-- EMPLOYEE_ID: decimal(6,0) (nullable = false) |-- FIRST_NAME: string (nullable = true) |-- LAST_NAME: string (nullable = false) |-- EMAIL: string (nullable = false) |-- PHONE_NUMBER: string (nullable = true) |-- HIRE_DATE: timestamp (nullable = false) (Incorrect data type read here) |-- JOB_ID: string (nullable = false) |-- SALARY: decimal(8,2) (nullable = true) |-- COMMISSION_PCT: decimal(2,2) (nullable = true) |-- MANAGER_ID: decimal(6,0) (nullable = true) |-- DEPARTMENT_ID: decimal(4,0) (nullable = true) |-- SSN: string (nullable = true) Hire_Date is read incorrectly as TimeStamp, is there a way to correct. Data is being read from Oracle on the fly and the application does not have an upfront knowledge of datatypes and can't convert it after being read. Thanks in advance. Nagesh ... View more