Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to turn JSONL format to Avro

How to turn JSONL format to Avro

New Contributor

Hi

I have data being streamed through a 3rd party API which is supplying the data in the JSONL (http://jsonlines.org) format.

The work flow is simple and I can split the large lump of data into single flows

The data looks like this

{"event":"pageview: Eligibility","properties":{"time":1518652801,"distinct_id":"16196c2a998b0-0f8d4646ccc0a9-646e0a3b-49a10-16196c2a999ac","$browser":"Chrome","$browser_version":56,"$city":"Barnet","$current_url":"https://online.xxx.com/get-started/nhs/eligibility?_ga=2.231391839.1747202949.1518652791-112770749.1518652791&_gac=1.248861941.1518652791.Cj0KCQiA2Y_UBRCGARIsALglqQ2M_xmGi5PlSN63sr7EAn3uU4IW1w1PasQY6RqeVqDPpCdgzLNpXyYaAkRxEALw_wcB","$device":"Android","$initial_referrer":"https://www.xxx.nhs.uk/?gclid=Cj0KCQiA2Y_UBRCGARIsALglqQ2M_xmGi5PlSN63sr7EAn3uU4IW1w1PasQY6RqeVqDPpCdgzLNpXyYaAkRxEALw_wcB","$initial_referring_domain":"www.xxx.nhs.uk","$lib_version":"2.17.0","$os":"Android","$referrer":"https://www.xxx.nhs.uk/?gclid=Cj0KCQiA2Y_UBRCGARIsALglqQ2M_xmGi5PlSN63sr7EAn3uU4IW1w1PasQY6RqeVqDPpCdgzLNpXyYaAkRxEALw_wcB","$referring_domain":"www.xxx.nhs.uk","$region":"Barnet","$screen_height":732,"$screen_width":412,"category":"xx at xxxx","mp_country_code":"GB","mp_lib":"web"}}

So not really "proper" JSON but looking for a way to transform ultimately into AVRO

Don't have an account?
Coming from Hortonworks? Activate your account here