Skip to main content
roshanbi
roshanbi
Gold ⭐️
October 20, 2021
Solved

kafka json slow

Hello Team,

 

we are having some performance issue when sending data to Kafka using JSON as format. Performs is very slow.

 

Performance is faster when using DSV parser.

 

How can we fine tune JSON formatter?

 

Regards,

Roshan

Nandinin
NandininAnswer
Gold ⭐️
October 25, 2021

Hello @roshanbi  

 

JSON is one of the best data exchange format  which is lightweight and compact. C/DSV can be used when large dataset, with bandwidth limits are to be sent, but DSV cannot be used where the data is complex and unstructured; there comes in JSON.

 

So, as you said, you are are seeing a better performance with DSV, that is because it does not contain attribute-value pair as in JSON (which makes it structured). To fine tune you application, you can use 

Compression.

SME || Kafka | Schema Registry | SMM | SRM
View in thread

3 replies

arandive
Silver ⭐️⭐️
arandive
Silver ⭐️⭐️
October 21, 2021

@roshanbi Can you please confirm the way you are sending the JSON data to the Kafka topic? Are you using kafka-console-producer? Please provide producer.properties file parameters used. 

Translate
Nandinin
Gold ⭐️
NandininAnswer
Gold ⭐️
October 25, 2021

Hello @roshanbi  

 

JSON is one of the best data exchange format  which is lightweight and compact. C/DSV can be used when large dataset, with bandwidth limits are to be sent, but DSV cannot be used where the data is complex and unstructured; there comes in JSON.

 

So, as you said, you are are seeing a better performance with DSV, that is because it does not contain attribute-value pair as in JSON (which makes it structured). To fine tune you application, you can use 

Compression.

Translate
SME || Kafka | Schema Registry | SMM | SRM
    VidyaSargur
    Community Manager
    VidyaSargur
    Community Manager
    October 29, 2021

    @roshanbi, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 

    Translate
    Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
    Terms & ConditionsAccessibility statement