Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

JSON is null in DataFrame

JSON is null in DataFrame

Explorer

I have a valid JSON file


[[{"eventEndTimestamp":"2019-05-07T03:48:01Z","eventStartTimestamp":"2019-05-07T03:48:01Z","sourceSystem":"X1","transactionCode":"asdf","transactionSuccessIndicator":"Y"}],[{"eventEndTimestamp":"2019-05-07T03:48:04Z","eventStartTimestamp":"2019-05-07T03:48:04Z","sourceSystem":"X2","transactionCode":"qwerty","transactionSuccessIndicator":"Y"}]]


But when I print it in spark console, I'm getting the dataframe but with null values. The dataframe columns are inferred from this file correctly with null values. The code is working fine with other JSON files but not this one.


Output:

Batch: 0

-------------------------------------------

+-----------------+-------------------+----------------+--------------------------------+---------------------------+

|eventEndTimestamp|eventStartTimestamp|sourceSystem|transactionCode|transactionSuccessIndicator|

+-----------------+-------------------+----------------+--------------------------------+---------------------------+

| null| null| null| null| null|

+-----------------+-------------------+----------------+--------------------------------+---------------------------+


1 REPLY 1
Highlighted

Re: JSON is null in DataFrame

The above was originally posted in the Community Help track. On Sat May 11 03:19 UTC 2019, the HCC moderation staff moved it to the Data Processing track. The Community Help track is intended for questions about using the HCC site itself.

Bill Brooks, Community Manager
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Don't have an account?
Coming from Hortonworks? Activate your account here