Member since
08-16-2016
2
Posts
0
Kudos Received
0
Solutions
03-31-2020
09:18 AM
You pointed a right direction. I added some codes to update the modification time of the files in HDFS, and the "Refresh" SQL worked now.
... View more
04-01-2017
09:25 PM
So I was able to solve the issue but not using a Serde. What I did was load the fixedwidth file into a single string and then used substring to parse through the line as follows: CREATE TABLE FIXEDWIDTH_TEMP ( whole_line STRING ) LOCATION '/data/shared_workspace/test/fixedwidth_temp'; CREATE EXTERNAL TABLE FIXEDWIDTH_STG ( ZIP_CODE_3043 STRING, ZIP4_CODE_3044 STRING, PREFIX_TITLE_8093 STRING, STATE_3038 STRING, PHONE_3032 STRING, ... .. 1800 lines more ... ) LOCATION '/data/shared_workspace/test/fixedwidth_stg'; INSERT INTO FIXEDWIDTH_STG SELECT SUBSTRING(acxiom_line,1,5), SUBSTRING(acxiom_line,6,4), SUBSTRING(acxiom_line,10,1), SUBSTRING(acxiom_line,11,3), SUBSTRING(acxiom_line,14,2), SUBSTRING(acxiom_line,16,10), SUBSTRING(acxiom_line,26,12), . . .1800 lines more FROM FIXEDWIDTH_TEMP;
... View more