Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Want to Implement Gatling Load Test Using Cloudera Distribution

Want to Implement Gatling Load Test Using Cloudera Distribution


I want to Implement Gatling Load Test( Gatling is scala + DSL Using Cloudera Distribution..


Gatling has advantage of 

  • Low latency execution
  • Direct Spark SQL execution
  • Configuration at one place
  • Auditing and Logging
  • Complete statement history and metrics

Livy Server

Livy is an open source REST interface for interacting with Apache Spark from anywhere. It supports executing snippets of code or programs in a Spark context that runs locally or in Apache Hadoop YARN.

Livy offers three modes to run Spark jobs:

  • Using programmatic API
  • Running interactive statements through REST API
  • Submitting batch applications with REST API

Livy provides the following features:

  • Interactive Scala, Python, and R shells
  • Batch submissions in Scala, Java, Python
  • Multiple users can share the same server (impersonation support)
  • Can be used for submitting jobs from anywhere with REST
  • Does not require any code change to your programs
  • Support Spark1/ Spark2, Scala 2.10/2.11 within one build.

Livy provides the following advantages:

  • Programmatically upload jar file and run job. Add additional applications that will connect to same cluster and upload jar with next job. If you use spark-submit, you must upload manually JAR file to cluster and run command. Everything must be prepared before run
  • Use Spark in interactive mode, hard to do with spark-submit or Thrift Server at scale.
  • Security. Reduce exposure of the cluster to the outside world.
  • Stability. Spark is a complex framework and there many factors which can affect its long term performance and stability. Decoupling Spark context and application allows to handle Spark issues gracefully, without full downtime of the application.

Gatling Server

Gatling is a highly capable load testing tool. It is designed for ease of use, maintainability and high performance. Gatling server provides the following benefits.

  • Powerful scripting using Scala
  • Akka + Netty
  • Run multiple scenarios in one simulation
  • Scenarios = code + DSL
  • Graphical reports with clear & concise graphs

Is there a use case already .As of now my thoughts are to implement below,is this possible in clodera ?any suggestions or thoughts?