Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

I have to convert csv file to nested json can you help me out with flow file configuration .I am new to NIFI

avatar

output file:

{
"name" : "name",
"class" : "a",
"grade" : "A",
"address" : "state"
"url" :{

"first url":"url",

"second url":"url"

}

}

1 ACCEPTED SOLUTION

avatar

@Abhiram-4455,

Assuming you have the following input where you can have multiple records per CSV input:

Name,class,grade,address,first_url,second_url
John,10th,A,New York,https://www.facebook.com,/brnds-coupons/offers
sam,9th,A,Dallas,www.google.com,www.amazon.com

You can use the following processors to do your transformation:

SAMSAL_0-1695823148778.png

Where each processor is configured as the following:

- ConvertRecord: is to convert from CSV format to JSON

SAMSAL_1-1695823271838.png

CSVReader: (use default settings)

SAMSAL_2-1695823395667.png

Note: you can adjust the "Value Separator" in case the CSV delimiter is different from ",".

JSONRecorSertWriter: (use default settings)

SAMSAL_3-1695823517965.png

- JoltTransformationJSON:

SAMSAL_4-1695823609582.png

where "Jolt Specification" is set to the following:

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": "[&1].&",
        "first_url": "[&1].url.&",
        "second_url": "[&1].url.&"
      }
    }
  }
]

 

The output shouldlook like the following:

[
	{
		"Name": "John",
		"class": "10th",
		"grade": "A",
		"address": "New York",
		"url": {
			"first_url": "https://www.facebook.com",
			"second_url": "/brnds-coupons/offers"
		}
	},
	{
		"Name": "sam",
		"class": "9th",
		"grade": "A",
		"address": "Dallas",
		"url": {
			"first_url": "www.google.com",
			"second_url": "www.amazon.com"
		}
	}
]

 

if you need to process each record separately you can use SplitJson processor:

SAMSAL_5-1695824348064.png

Use the "split" relationship to get the split records.

 

Hope that helps.

 

 

 

 

 

 

 

 

 

 

View solution in original post

7 REPLIES 7

avatar
Community Manager

@Abhiram-4455 Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @MattWho @cotopaul  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Contributor

@Abhiram-4455 

What's the input look like?

avatar

Name, class, grade, address, first_url, second_url

John, 10th,A,New York,uhttps://www.facebook.com,/brnds-coupons/offers

avatar

Hi @Abhiram-4455 ,

The easiest way to do such transformation is to use ConvertRecord to convert the format from CSV format to JSON  format, then you can use JoltTransformation processor to do the proper transformation to the final format that you need. To see an example of this please refer to the following post: https://community.cloudera.com/t5/Support-Questions/want-to-convert-csv-to-nested-json-using-nifi/td...

If that helps please accept solution.

Thanks

avatar

I have tried this example but getting errors like: unable to pares incoming data. Lot of errors... So can you help to get step by step process for which processor need to use and and configuration 

avatar

@Abhiram-4455,

Assuming you have the following input where you can have multiple records per CSV input:

Name,class,grade,address,first_url,second_url
John,10th,A,New York,https://www.facebook.com,/brnds-coupons/offers
sam,9th,A,Dallas,www.google.com,www.amazon.com

You can use the following processors to do your transformation:

SAMSAL_0-1695823148778.png

Where each processor is configured as the following:

- ConvertRecord: is to convert from CSV format to JSON

SAMSAL_1-1695823271838.png

CSVReader: (use default settings)

SAMSAL_2-1695823395667.png

Note: you can adjust the "Value Separator" in case the CSV delimiter is different from ",".

JSONRecorSertWriter: (use default settings)

SAMSAL_3-1695823517965.png

- JoltTransformationJSON:

SAMSAL_4-1695823609582.png

where "Jolt Specification" is set to the following:

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": "[&1].&",
        "first_url": "[&1].url.&",
        "second_url": "[&1].url.&"
      }
    }
  }
]

 

The output shouldlook like the following:

[
	{
		"Name": "John",
		"class": "10th",
		"grade": "A",
		"address": "New York",
		"url": {
			"first_url": "https://www.facebook.com",
			"second_url": "/brnds-coupons/offers"
		}
	},
	{
		"Name": "sam",
		"class": "9th",
		"grade": "A",
		"address": "Dallas",
		"url": {
			"first_url": "www.google.com",
			"second_url": "www.amazon.com"
		}
	}
]

 

if you need to process each record separately you can use SplitJson processor:

SAMSAL_5-1695824348064.png

Use the "split" relationship to get the split records.

 

Hope that helps.

 

 

 

 

 

 

 

 

 

 

avatar

Thank you so much SAMSAL it's working