Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark and llap jobs process

Solved Go to solution
Highlighted

Spark and llap jobs process

I'm running spark jobs in server-1 and if i can run llap daemons on the same server is there any impact on running spark jobs.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Spark and llap jobs process

@kishore sanchina LLAP is a long-running service so it will preempt memory for the llap queue. The best practice is to dedicate nodes to LLAP workloads.

You can utilize the LLAPContext in Spark which will stream data from HDFS to the spark executor but this is more of a Hive process and not Spark which can incorporate some masking and filtering security features but you may see a 3x-4x performance degradation.

View solution in original post

1 REPLY 1
Highlighted

Re: Spark and llap jobs process

@kishore sanchina LLAP is a long-running service so it will preempt memory for the llap queue. The best practice is to dedicate nodes to LLAP workloads.

You can utilize the LLAPContext in Spark which will stream data from HDFS to the spark executor but this is more of a Hive process and not Spark which can incorporate some masking and filtering security features but you may see a 3x-4x performance degradation.

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here