Member since
02-28-2018
15
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2699 | 05-04-2018 07:40 AM |
05-04-2018
07:40 AM
Hello, It wasn't that the case. Anyway, I tried to use UpdateAttribute processor to change the filename to .CSV and it works in this way. Thank you for your time ! Kind regards, Stefan
... View more
05-02-2018
10:04 AM
We receive the .DAT files directly from the client which for the normal process that we have in place, ETL using ODI, we are open it in excel. But I wanted to know if there is a way to convert it or read it because we need to store the data of it in a table.
... View more
04-29-2018
03:52 PM
Dear Hortonworks, How can we read a .DAT file in Nifi or transform it in CSV or JSON ? Should I create a custom processor in Java or is a possibility by reading it directly in Nifi ? Thank you ! Kind regards, Stefan
... View more
Labels:
- Labels:
-
Apache NiFi
04-27-2018
09:11 AM
Hello all, I have a weird issue with this processor, PutSQL, precisely, I am trying to execute a SQL insert script but doesn't work, meaning, is only stay in running mode without doing nothing. In the logs I do not have any errors or warning messages. The connections to the DB are correct, verified with our DBA, even more, the DBA said that there are not any sessions open. I am saying that this is weird because before is was working with the same workflow. Also, there was no other modifications or applied patches at DB level which can stop this processor to insert data. Moreover, ExecuteSQL is working, I can extract data from the tables. Also the schema/user has all the privileges needed for this task and the tables are well created, some don't have even primary key in order to avoid some other issues which maybe Nifi didn't show them. Also, the ReplaceText provide a perfect script, I copy it and execute it via Toad and SQL Dev and the line is inserted. I am using locally Nifi 1.4.0 but also I tried with Nifi 1.5.0 and is the same behavior. I tried also from other colleague computer and is the same. Any advise would be great. Thank you ! Kind regards, Stefan
... View more
Labels:
- Labels:
-
Apache NiFi
04-03-2018
08:52 AM
1 Kudo
Hello everyone ! I tried with Embedded schema, like Shu mentioned, and now everything is working perfectly. I feel so bad that it was so easy and simple, just few processors...anyway, thank you for your time and help, I really appreciate it ! Kind regards, Stefan
... View more
03-30-2018
02:44 PM
Hello @Shu, I didn't added here but of course I tried also with upper letters and still the same issue with ConvertRecord. With the EvaluateJsonPath, same issue again. I tried many other things but nothing worked. I was thinking that it will be really easy to output a file from a simple select query from a table but it seems that is not like that..
... View more
03-30-2018
12:12 PM
Json input: [ {
"INV_IDN" : "247048764",
"INV_NUMBER" : "181120060",
"USR_MDF" : "15/03/2018 08:34:00 by LDL",
"INVISSDAT" : "15/03/2018",
"INVCLICOD" : "FUNDQ",
"INVCLINAM" : "FUNDQUEST",
"INVCLI_REGCOUNTRY" : "FR",
"INVOICECLASS" : "CUS - assujetti in EU outside Lx",
"CORCLICOD" : "BNPAMFR",
"OFOCHINMB" : "20173748",
"BSNCTGDSC" : "Fund Data Management",
"BSNCTGCOD" : "LIS",
"BSNCTGDSPCOD" : "FDM",
"INVTYP" : "Credit Note",
"INVSTU" : "Validated",
"INVORGCOD" : "LU",
"CRYCOD" : "EUR",
"AMNWTHVAT" : "-7,543.23",
"AMNWTHVATINEUR" : "-7,543.23",
"MEDIAFEESINEUR" : "0",
"KCFEESINEUR" : "-7543.23"
} ] Json schema: {
"type": "record",
"name": "DnpReport",
"fields" : [
{"name": "inv_idn", "type": ["null", "string"]},
{"name": "inv_number", "type": ["null", "string"]},
{"name": "usr_mdf", "type": ["null", "string"]},
{"name": "invIssDat", "type": ["null", "string"]},
{"name": "invCliCod", "type": ["null", "string"]},
{"name": "invCliNam", "type": ["null", "string"]},
{"name": "invCli_RegCountry", "type": ["null", "string"]},
{"name": "invoiceClass", "type": ["null", "string"]},
{"name": "corCliCod", "type": ["null", "string"]},
{"name": "ofoChiNmb", "type": ["null", "string"]},
{"name": "bsnCtgDsc", "type": ["null", "string"]},
{"name": "bsnCtgCod", "type": ["null", "string"]},
{"name": "bsnCtgDspCod", "type": ["null", "string"]},
{"name": "invTyp", "type": ["null", "string"]},
{"name": "invStu", "type": ["null", "string"]},
{"name": "invOrgCod", "type": ["null", "string"]},
{"name": "cryCod", "type": ["null", "string"]},
{"name": "amnWthVat", "type": ["null", "string"]},
{"name": "amnWthVatInEur", "type": ["null", "string"]},
{"name": "mediaFeesInEur", "type": ["null", "string"]},
{"name": "kcFeesInEur", "type": ["null", "string"]}
]
}
... View more
03-30-2018
10:18 AM
Hello @Shu, @Rahul Soni, Do you have any other ideas ? Thank you ! Kind regards, Stefan
... View more
03-30-2018
08:10 AM
Hello @Rahul Soni, Since beggining I tried with null like default values but still I receive that error. Also I validated the JSON so is OK, here is my JSON: {
"type": "record",
"name": "DnpReport",
"fields" : [
{"name": "inv_idn", "type": ["null", "string"]},
{"name": "inv_number", "type": ["null", "string"]},
{"name": "usr_mdf", "type": ["null", "string"]},
{"name": "invIssDat", "type": ["null", "string"]},
{"name": "invCliCod", "type": ["null", "string"]},
{"name": "invCliNam", "type": ["null", "string"]},
{"name": "invCli_RegCountry", "type": ["null", "string"]},
{"name": "invoiceClass", "type": ["null", "string"]},
{"name": "corCliCod", "type": ["null", "string"]},
{"name": "ofoChiNmb", "type": ["null", "string"]},
{"name": "bsnCtgDsc", "type": ["null", "string"]},
{"name": "bsnCtgCod", "type": ["null", "string"]},
{"name": "bsnCtgDspCod", "type": ["null", "string"]},
{"name": "invTyp", "type": ["null", "string"]},
{"name": "invStu", "type": ["null", "string"]},
{"name": "invOrgCod", "type": ["null", "string"]},
{"name": "cryCod", "type": ["null", "string"]},
{"name": "amnWthVat", "type": ["null", "string"]},
{"name": "amnWthVatInEur", "type": ["null", "string"]},
{"name": "mediaFeesInEur", "type": ["null", "string"]},
{"name": "kcFeesInEur", "type": ["null", "string"]}
]
} Kind regards, Stefan
... View more
03-29-2018
04:11 PM
Hello @Shu, Unfortunately, both flows didn't work. First one, with ReplaceText having the properties you mentioned still outputted empty values even though the attributes are the same. For the second one with ConvertRecord, I receive this error: Failed to process StandardFlowFileRecord[...] will route to failure: java.lang.ArrayIndexOutOfBoundsException: -40
java.lang.ArrayIndexOutOfBoundsException: -40 I even try to convert the Avro to JSON and then to pass to ConvertRecord but still the same error... Any help will be welcomed ! Thank you ! Kind regards, Stefan
... View more