Hi, Are you still getting the same error even though after increasing the Overhead memory? Could you please share the Error messages after increasing the Overhead memory / Executor/ Driver memory?   Thanks AK ... View more

Hi, yes . Count is the first action. But the next step is union of both hivedf and filedf and finding count. That step aslo takes same time (25s +12 mins)12.25 mins. Could you please give some suggestions. Thanks ... View more

Hi albani, Thanks for sharing the links, i found these threads earlier. These static one(defined for 3 elements). I would expect more dynamic. Eg: Today i may receive 3 elements, tomorrow may be 10 elements. Code should handle it dynamically. ... View more

Good to know @Mani! I'd kindly ask you, if you found the answer helpful and did answer your question completely, please accept this as an answer. This will encourage other HCC users to keep doing a good job, hence they will find the answer faster by just searching for the "best answer". Thanks! 🙂 ... View more