Support Questions
Find answers, ask questions, and share your expertise

kafka json slow

Contributor

Hello Team,

 

we are having some performance issue when sending data to Kafka using JSON as format. Performs is very slow.

 

Performance is faster when using DSV parser.

 

How can we fine tune JSON formatter?

 

Regards,

Roshan

2 ACCEPTED SOLUTIONS

Accepted Solutions

Contributor

Hello @roshanbi  

 

JSON is one of the best data exchange format  which is lightweight and compact. C/DSV can be used when large dataset, with bandwidth limits are to be sent, but DSV cannot be used where the data is complex and unstructured; there comes in JSON.

 

So, as you said, you are are seeing a better performance with DSV, that is because it does not contain attribute-value pair as in JSON (which makes it structured). To fine tune you application, you can use 

Compression.

SME || Kafka | Schema Registry | SMM | SRM

View solution in original post

Community Manager

@roshanbi, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

View solution in original post

3 REPLIES 3

Cloudera Employee

@roshanbi Can you please confirm the way you are sending the JSON data to the Kafka topic? Are you using kafka-console-producer? Please provide producer.properties file parameters used. 

Contributor

Hello @roshanbi  

 

JSON is one of the best data exchange format  which is lightweight and compact. C/DSV can be used when large dataset, with bandwidth limits are to be sent, but DSV cannot be used where the data is complex and unstructured; there comes in JSON.

 

So, as you said, you are are seeing a better performance with DSV, that is because it does not contain attribute-value pair as in JSON (which makes it structured). To fine tune you application, you can use 

Compression.

SME || Kafka | Schema Registry | SMM | SRM

View solution in original post

Community Manager

@roshanbi, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

View solution in original post