Member since
05-16-2016
12
Posts
0
Kudos Received
0
Solutions
04-13-2020
07:06 AM
PFB the full configuration details of PutDatabaseRecord and CSVReader. PutDatabaseRecord SS Part 1: PutDatabaseRecord SS Part 1 PutDatabaseRecord SS Part 2: PutDatabaseRecord SS Part 2: CSVReader ss part 1: CSVReader SS part 1 CSVReader ss part 2: CSVReader ss part 2 CSVReader ss part 3: CSVReader ss part 3 Please let me know if anything else is needed, also if I need to tweak on settings or add/remove processors.
... View more
04-13-2020
06:33 AM
Hi @stevenmatison , its not the same issue, this error showed up when I am trying to load a Parquet file into a particular PostgreSQL table whereas the other post is related to an error when trying to load one csv file into a particular PostgreSQL table. I'll share the SS of the whole configuration of PutDatabaseRecord and CSVReader in the other post. Here is the SS of the whole configuration of PutDatabaseRecord and CSVReader. PutDatabaseRecord SS Part 1: PutDatabaseRecord SS Part 2: ParquetReader part 1: ParquetReader Part 1 ParquetReader Part 2: ParquetReader Part 2 Note: If you see a discrepancy between the table name I wrote in prior queries and in the SS, please ignore, I did that to simply mask.
... View more
04-12-2020
12:52 PM
Hey that's a mistake in my typing as I had to mask col and table name. That's not an issue. Can you find anything else? It would be a good help in solving this then.
... View more
04-11-2020
04:34 AM
I am trying to load a csv file with headers with exactly same spelling and caps as in the PostgreSQL DB table. Error: putdatabaserecord.error Batch entry 0 INSERT INTO analytics_dev.op_working_time_nifi (T_CODE, V_NO, C_SIZE, M_TYPE, T_IND, F_EMPTY, IV, O_NAME, S_DATE, S_TIME, Q_NAME, C_HR, W_START_TIME, OP_W_MINS, T_MOVES) VALUES ('AB-2','153443','22','VS','NOT T','F','M42','206014500','4/5/2019','Night S','QB03','26','2019-04-05T22:03:36','3.683344363','3') was aborted: ERROR: column "t_code" of relation "working_time_nifi" does not exist Position: 49 Call getNextException to see other errors in the batch. Schema of the table: CREATE TABLE analytics_dev.working_time_nifi ( "T_CODE" text, "V_NO" text, "C_SIZE" bigint, "M_TYPE" text, "T_IND" text, "F_EMPTY" text, "IV" text, "O_NAME" text, "S_DATE" date, "S_TIME" text, "Q_NAME" text, "C_HR" float8, "W_START_TIME" timestamp, "OP_W_MINS" float8, "T_MOVES" bigint ); Any help as to why this is happening?
... View more
Labels:
- Labels:
-
Apache NiFi
04-11-2020
03:23 AM
This is the schema at target: CREATE TABLE dev.ctrmovement_test_ab2 ( "T_CODE" text, "C_NO" text, "L_NO" text, "INT_C_NO" bigint, "C_CATEGORY" text, "C_WEIGHT" float8, "V_SERVICE" text, "V_NAME" text, "V_NO" text, "V_LOCATION" text, "INT_VIA_NO" bigint, "B_NO" text, "FPD" text, "C_SIZE" bigint, "C_HEIGHT" bigint, "C_TYPE" text, "F_EMPTY" text, "I_CD" text, "C_ENTRY_MODE" text, "C_EXIT_MODE" text, "IV" text, "FROM_LOCATION" text, "TO_LOCATION" text, "E_NAME" text, "Q_NAME" text, "Q_COMPLETION_TIME" timestamp, "Y_COMPLETION_TIME" timestamp, "CAL_C_TIME" timestamp, "O_NAME" text, "O_FULL_NAME" text, "M_TYPE" text, "POL" text, "POD" text, "COA" text, "S_LINE" text, "PLANNED_LOCATION" text, "T_FLG" text, "ENTRY_DATE" timestamp, "EXIT_DATE" timestamp ); It is exactly the same in Parquet file, as the data was fetched using AWS DMS from the same database, same table and created a parquet file out of it. Any help as to why this should happen?
... View more
04-10-2020
05:17 AM
I am trying to write data to PostgreSQL table using PutDatabaseRecord processor reading it from AWS S3. But getting below error:
putdatabaserecord.error: INT96 not yet implemented.
Screenshot of flow is here:
SS of settings of PutDatabaseRecord is as follows:
Note: Checked S3 connectivity, listing and fetch both are happening correctly. Also checked DB connection, that too works fine with a test of a small csv file load into a table.
How to successfully do it?
... View more
Labels:
- Labels:
-
Apache NiFi
05-16-2016
12:09 AM
Can you please help me in this? http://community.cloudera.com/t5/Advanced-Analytics-Apache-Spark/Pros-and-Cons-of-fetching-data-using-SparkSQL-vs-Hive-in-Spark/m-p/40855#U40855
... View more