Member since
09-21-2018
11
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
15258 | 10-18-2018 01:52 AM |
05-08-2019
02:53 PM
I have a valid JSON file [[{"eventEndTimestamp":"2019-05-07T03:48:01Z","eventStartTimestamp":"2019-05-07T03:48:01Z","sourceSystem":"X1","transactionCode":"asdf","transactionSuccessIndicator":"Y"}],[{"eventEndTimestamp":"2019-05-07T03:48:04Z","eventStartTimestamp":"2019-05-07T03:48:04Z","sourceSystem":"X2","transactionCode":"qwerty","transactionSuccessIndicator":"Y"}]] But when I print it in spark console, I'm getting the dataframe but with null values. The dataframe columns are inferred from this file correctly with null values. The code is working fine with other JSON files but not this one. Output: Batch: 0 ------------------------------------------- +-----------------+-------------------+----------------+--------------------------------+---------------------------+ |eventEndTimestamp|eventStartTimestamp|sourceSystem|transactionCode|transactionSuccessIndicator| +-----------------+-------------------+----------------+--------------------------------+---------------------------+ | null| null| null| null| null| +-----------------+-------------------+----------------+--------------------------------+---------------------------+
... View more
Labels:
- Labels:
-
Apache Spark
10-18-2018
02:32 AM
bash-4.1$ hdfs dfs -ls /etc/hive/data/weather -rw-r--r-- 1 cloudera supergroup 220 2018-10-17 13:03 /etc/hive/data/weather
... View more
10-18-2018
01:57 AM
CREATE TABLE weather (wban INT, date STRING, precip INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LOCATION '/etc/hive/data/weather'; Getting error while creating the table, FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:hdfs://quickstart.cloudera:8020/etc/hive/data/weather is not a directory or unable to create one)
... View more
Labels:
- Labels:
-
Apache Hive
10-18-2018
01:52 AM
1 Kudo
Thank you @Jonathan Sneep I resolved the issue. It is due to typing the query in text editor and copied it to hive cli, the single quote was malformed.
... View more
10-17-2018
07:28 AM
I have an existing sqoop job 'InsertOrder'. Now I need to edit it. I
couldn't find a way to edit, but getting only way to delete that job and
re-create it. Is it possible to edit the existing job
... View more
Labels:
- Labels:
-
Apache Sqoop
10-17-2018
04:32 AM
CREATE TABLE IF NOT EXISTS employee (eid int, name String, salary String) STORED AS TEXTFILE; LOAD DATA LOCAL INPATH '/user/hdadmin/sample.txt' OVERWRITE INTO TABLE employee; Sample.txt 1201^AGopal^A45000 1202^AManisha^A45000 1203^AMasthanvali^A40000 1204^AKiran^A40000 1205^AKranthi^A30000 Both queries were executed successfully, but the table have only NULL values, Output: hive> select * from employee; OK NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL Time taken: 0.148 seconds, Fetched: 5 row(s) Please help me to fix this
... View more
Labels:
- Labels:
-
Apache Hive
10-17-2018
04:32 AM
CREATE TABLE IF NOT EXISTS employee (eid int, name String, salary String) ROW FORMAT DELIMITED FIELDS TERMINATED BY ‘\t’ LINES TERMINATED BY ‘\n’ STORED AS TEXTFILE; I'm getting the below error: MismatchedTokenException(26!=298) at org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617) at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115) at org.apache.hadoop.hive.ql.parse.HiveParser.tableRowFormatFieldIdentifier(HiveParser.java:31777) at org.apache.hadoop.hive.ql.parse.HiveParser.rowFormatDelimited(HiveParser.java:30730) at org.apache.hadoop.hive.ql.parse.HiveParser.tableRowFormat(HiveParser.java:31006) at org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:4677) at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:2138) at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1392) at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1030) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:199) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:417) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1026) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1091) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:962) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:952) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:800) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) FAILED: ParseException line 3:44 mismatched input 't' expecting StringLiteral near 'BY' in table row format's field separator Even iIf I use '\073' , I'm getting the error: MismatchedTokenException(287!=298) at org.antlr.runtime.BaseRecognizer.recoverFromMismatchedToken(BaseRecognizer.java:617) at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115) at org.apache.hadoop.hive.ql.parse.HiveParser.tableRowFormatFieldIdentifier(HiveParser.java:31777) at org.apache.hadoop.hive.ql.parse.HiveParser.rowFormatDelimited(HiveParser.java:30730) at org.apache.hadoop.hive.ql.parse.HiveParser.tableRowFormat(HiveParser.java:31006) at org.apache.hadoop.hive.ql.parse.HiveParser.createTableStatement(HiveParser.java:4677) at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:2138) at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:1392) at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1030) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:199) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:417) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1026) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1091) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:962) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:952) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:800) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) FAILED: ParseException line 3:44 mismatched input '073' expecting StringLiteral near 'BY' in table row format's field separator Please let me know how to fix this.
... View more
Labels:
- Labels:
-
Apache Hive
09-21-2018
02:53 PM
I have a text file with values separated by ','. And I have to save the file as a text file in hive with delimiter '|'. How to apply this in query?
... View more
Labels:
- Labels:
-
Apache Hive