Hi everyone,
We are facing some difficulties:
Application description: Spark subscribes to Kafka topic, flattens out Json content and writes it using JDBC driver to Teradata Table. The Teradata Table is queried by a Talend Job that transfers the data to a DWH 3N model.
Problem: There are multiple deadlock problems in the Spark App. Is there any best practice or technical solution to tackle this problem?
Software components:
- Apache Spark 2.2.0
- Teradata 16.20
Kafka Topic has 5 partitions.
Any comment will be appreciated.
