Sunday, July 28, 2013

Getting Started with Gatling - alternative to JMeter

In some projects with high traffic load, tests are often overlooked due to time constraints or lack of simple tools to integrate to the project. And this is wrong because only load tests can properly validate an application or a system before deployment, both in quality of service as resource consumption.

Although Apache JMeter is a reference, its aging interface, and complexity of implementation, does not make it the ideal sexy tool that all developpers want to have. However an alternative combining ease of use, performance, and reliability exist.

Through practice, this article proposes a complete 'getting started' with Gatling.

What is Gatling?

A bit of culture that can affect our health, the Gatling project takes its name and logo of the first effective machine gun combines reliability, firepower and ease supply:
The Principle designed and developed by Richard Gatling in 1861, provides the means to efficiently parallelize the necessary mechanical operations (loading, percussion, extraction, ejection) and let cool rooms and better guns, so the high rates of fire reached incommensurate with the weapons one barrel. (dixit Wikipedia ).
That said, you can have fun doing some similarities between this gun of the 19th century and the Gatling project:
  • reliability: developed in Scala and running on the JVM;
  • firepower / parallelization of operations: Asynchronous HTTP Client based on Netty and the actor model with Akka;
  • ease supply: a domain specific language (DSL) clear and concise.
In addition Gatling use Highcharts to generate its graphs, which brings the sexy part to the project. Of course everything is "Open Source" under Apache License v2 , and available on GitHub if you want to take a look or even contribute.

Installation / Integration

Gatling is provided directly through an archive all-in-one available here. The version used in our case will be the 2.0-M2, before downloading and using it, make sure you have completed the prerequisites.

The structure of the archive once unpacked:
Then it provides a command line interface (CLI) to run a simulation:
By default, Gatling provides two simulations available in the 'user-files' directory. We can then execute one of them to make sure everything works properly:
Using a CLI is very convenient for some quick tests, but it has its limitations when working on a real project.
We prefer then use one of the following integration options:

Write your first scenario

We will write a first scenario that involves the consultation of one or more products on our flagship site: The Bees Shop.

The scenario is as follows:
  • a user visits the site and arrives on the home page;
  • the user is viewing the list of available products;
  • the user accesses 5 retail products on average.
For this rather simple scenario the command line interface is used.

A good practice is to separate Gatling simulations and data sets of scenarios. We will create three files:
  • the dataset 'products.csv' which contain the list of products;
  • the file 'ConsultProductsScenario.scala' which will be implementing our scenario;
  • the file 'BeesShopSimulation.scala' that will run our script.
This gives the following tree after files creation:
The file 'products.csv' contains two products:
  • the Long Island Iced Tea which is a cocktail made ​​of tequila, gin, vodka, rum and orange liqueur;
  • the Sex On the Beach which is a cocktail made ​​of vodka, schnapps fishing, orange juice and lemon juice.
In short, giving concrete:
productId,productName
1,Long Island Iced tea
2,Sex On The Beach
Even if you're not a proved "Scalafiste" the most difficult with Gatling, in my opinion, come not from Scala knwonledge but rather learning it's DSL. That's why in this article I will try to provide maximum variety and concrete examples, to enhance the official project wiki.

The scenario 'ConsultProductsScenario.scala' therefore be written as follows:
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import bootstrap._
 
object ConsultProductsScenario {
 
  val products = csv("products.csv").random
 
  val scn = scenario("View 5 random products")
    .exec(
      http("Home page")
        .get("/")
        .check(status.is(200)))
    .exec(
      http("View the list of products")
        .get("/product")
        .check(status.is(200)))
    .repeat(5) {
      feed(products)
      .exec(
        http("View a random product")
          .get("/product/${productId}")
          .check(status.is(200)))
    }
}
Some explanations anyway :
  • the var "products" is a feeder initialized with a random stategy;
  • the method "feed()" our data source in the current user session;
  • using EL "/product/${productId}" provides direct access to properties stored in the user session. In our case this is an ID of a product.
It is also possible to use directly a JDBC as a data source which for me is better in terms of consistency and maintenance for our scenarios. For example if a product is deleted by mistake then it could be possible to have one request of two in failure due to a 404 which completely skews the results.

So we will write our file 'BeesShopSimulation.scala', which will be for example a test of scalability:
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import scala.concurrent.duration._
 
class BeesShopSimulation extends Simulation {
 
  val httpConf = httpConfig.baseURL("http://localhost:8085/bees-shop")
 
  setUp(
    ConsultProductsScenario.scn
      .inject(rampRate(10 usersPerSec) to(100 usersPerSec) during(5 minutes))
      .protocolConfig(httpConf)
  )
}
You will notice that it extends the Simulation class that will act as a runner at the start of the test. Then we set the HTTP configuration to use:
val httpConf = httpConfig.baseURL("http://localhost:8085/bees-shop")
This configuration is rather minimalist in our case, if in your project you go through a proxy or a load balancer, so I refer you directly to the wiki part of protocol configuration.

It remains for us to set up our scenario with a ramp increasing the number of simultaneous users of 10 users/s to 100 users/s in 5 minutes.
inject(rampRate(10 usersPerSec) to(100 usersPerSec) during(5 minutes))
Simulation run through the script provided by Gatling:
End of the simulation :
The results are available in HTML format, you can see them online here, and just below the summary table of the number of requests per second:
  • in green : OK queries that have passed all check() int the scenario;
  • in red : KO queries which had failed at least on one check() or or if it there's been an exception at runtime (eg SocketTimeoutException).
Caution: Do not confuse number of concurrent users and the number of requests per second, in our case a user runs multiple requests per second.

Combine multiple scenarios

Now we want to perform a simulation combining the following three scenarios:
  • a client consults 5 random products;
  • a client search a product and make post a comment on it;
  • a client adds one product of three to their shopping cart during consultation.
The first scenario 'ConsultProductsScenario.scala' is already done, see above.

The second scenario 'SearchAndCommentProductsScenario.scala' is writing as follows:
import io.gatling.core.Predef._
import io.gatling.http.Predef._
 
object SearchAndCommentProductsScenario {
 
  val products = csv("products.csv").random
 
  val scn = scenario("Search a product")
    .exec(
      http("Home page")
        .get("/")
        .check(status.is(200)))
    .feed(products)
    .exec(
      http("Search a random product by name")
        .get("/product/")
        .queryParam("name", "${productName}")
        .check(status.is(200))
        .check(regex("""href=".*/product/(\d+)"""").find.exists.saveAs("productIdFound")))
    .exec(
      http("View a product")
        .get("/product/${productIdFound}")
        .check(status.is(200)))
    .exec(
      http("Comment a product")
        .post("/product/${productIdFound}/comment")
        .param("comment", "My 2 cents!")
        .check(status.is(200)))
}
To find a product we will pass into our GET a query parameter with the name of the desired product with the queryParam() method:
queryParam("name", "${productName}")
Then we will check that the desired product exists with a regex and we will use the saveAs() method to save the ID in a session variable:
check(regex("""href=".*/product/(\d+)"""").find.exists.saveAs("productIdFound")))
We can then post a comment on the desired product with the param() method:
post("/product/${productIdFound}/comment").param("comment", "My 2 cents!")
The third scenario 'AddProductsInCartScenario.scala', this time asking some basic Scala is as follows:
import io.gatling.core.Predef._
import io.gatling.core.validation.Validation
import io.gatling.http.Predef._
import scala.concurrent.duration._
import bootstrap._
 
object AddProductsInCartScenario {
 
  val products = csv("products.csv").random.build
  val numberOfProductsRegex: (Session) => Validation[String] = """(\d+) items"""
  val numberOfProducts: String = "numberOfProductsInCart"
 
  val scn = {
    scenario("Add 3 products in cart")
      .exec(
        http("Home page")
          .get("/")
          .check(status.is(200))
          .check(regex(numberOfProductsRegex).find.transform(_.map(_.toInt)).is(0).saveAs(numberOfProducts)))
      .feed(products)
      .asLongAs(_.get[Int](numberOfProducts, Int.MaxValue) < 3) {
        exec(
          http("View a product")
            .get("/product/${productId}")
            .check(status.is(200)))
          .randomSwitch(
           70 -> pause(1 second, 5 seconds),
           30 -> exec(
             http("Add a product in cart")
                 .post("/cart/add")
                 .param("product", "${productId}")
                 .param("quantity", "1")
                 .check(status.is(200))
                 .check(regex(numberOfProductsRegex).find.transform(_.map(_.toInt)).exists.saveAs(numberOfProducts))))
    }
  }
}
We will first check that the current shopping cart of the client is empty. The transform() method is needed to convert the number of products found in Int:
check(regex(numberOfProductsRegex).find.transform(_.map(_.toInt)).is(0).saveAs(numberOfProducts)))
Then iterates with asLongAs() so that the shopping cart contains at least three products:
asLongAs(_.get[Int](numberOfProducts, Int.MaxValue) < 3)
And then a randomSwitch() is performed to simulate the fact that a user adds a product on average three times during its consultation.

Now, the goal is to perform a simulation combining these three scenarios to achieve a stress test. Therefore, we will add our two new scenarios initialization into our previous simulation file "BeesShopSimulation.scala" and configure our simulation like this:
  • a client consults 5 random products → ~ 65 users / s for 2 minutes;
  • a client adds on product of three to their shopping during consultation → nothing for 1 minute then 75 users ~ / s for 40 seconds;
  • a client search a product and post a comment on it → nothing during 1"30 then 500 users at once.
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import scala.concurrent.duration._
 
class BeesShopSimulation extends Simulation {
 
  val httpConf = httpConfig.baseURL("http://localhost:8085/bees-shop")
 
  setUp(
    ConsultProductsScenario.scn
      .inject(ramp(8000 users) over (2 minutes))
      .protocolConfig(httpConf),
    AddProductsInCartScenario.scn
      .inject(nothingFor(1 minutes), ramp(3000 users) over (40 seconds))
      .protocolConfig(httpConf),
    SearchAndCommentProductsScenario.scn
      .inject(nothingFor(100 seconds), atOnce(500 users))
      .protocolConfig(httpConf)
  )
}
As before, the simulation is executed with the CLI, and all results are available here.

Here is the graph showing the number of requests per second :
  • in orange: the number of active user session at a given time.
As you can see, we have exceeded 1,000 concurrent users during peak load, which resulted to fail 50% of the requests at that time.

Conclusion

You'll understand Gatling is a real alternative to other performance testing tools, ease of use, efficiency, and reliability, you could in a few minutes perform a scalability test, a stress test, or a test at the limits. It is a tool that is free, in constant evolution, and it would be a shame to not use it into our projects.

The entire source code of scenarios and simulations, as well as the target application is available here on my GitHub account. This is for you the opportunity to see an example of Maven integration with Gatling Gatling, and also Java with Scala in the same project. Feel free to fork the project, code your own tests, and share your discoveries or difficulties.

To conclude, we can say with conviction that Gatling will be right once again the Bees Shop ;-)

4 comments:

  1. Hello my friend,

    When I executed the script provided by Gatling with the first part of your tutorial I have this message: "16:05:23.002 [ERROR] i.g.a.ZincCompiler$ - C:\gatling-charts-highcharts-2.0.0-M3
    a\user-files\simulations\bees-shop\BeesShopSimulation.scala:18: value protocols
    is not a member of io.gatling.core.structure.ProfiledScenarioBuilder.
    possible cause: maybe a semicolon is missing before `value protocols'?"

    My code is the following:

    import io.gatling.core.Predef._
    import io.gatling.http.Predef._
    import scala.concurrent.duration._

    class BeesShopSimulation extends Simulation {

    val httpConf = httpConfig.baseURL("http://localhost:8080/gatling-bees-shop-1.0.0-SNAPSHOT")

    setUp(
    ConsultProductsScenario.scn
    .inject(rampRate(10 usersPerSec) to(100 usersPerSec) during(5 minutes))
    .protocolConfig(httpConf)
    )
    }

    Like you can see I only changed the string inside the httpConfig.baseURL, please help me. Thanks in advance.

    ReplyDelete
    Replies
    1. Hello Henry,

      In this tutorial I have used the version 2.0.0-M2 of Gatling and not the 2.0.0-M3, so if you want to make this example works, you need to use the 2.0.0-M2 one.

      If you want that your code works with 2.0.0-M3, you can see "https://github.com/excilys/gatling/wiki/Gatling-2#wiki-http-protocol" to understand what has changed.

      Your code above should be :

      val http = http.baseURL("http://localhost:8080/gatling-bees-shop-1.0.0-SNAPSHOT")

      setUp (
      ConsultProductsScenario.scn.inject(rampRate(10 usersPerSec) to(100 usersPerSec) during(5 minutes))
      ).protocols(http)

      Enjoy this great tool :)

      Delete
    2. Hello Clement,

      Thanks for your answer. I did your advice, but now I get this message:

      09:17:26.051 [ERROR] i.g.a.ZincCompiler$ - C:\gatling-charts-highcharts-2.0.0-M3
      a\user-files\simulations\bees-shop\BeesShopSimulation.scala:7: recursive value h
      ttp needs type
      09:17:26.066 [ERROR] i.g.a.ZincCompiler$ - val http = http.baseURL("http://loc
      alhost:8080/gatling-bees-shop-1.0.0-SNAPSHOT")
      09:17:26.071 [ERROR] i.g.a.ZincCompiler$ - ^
      09:17:26.432 [ERROR] i.g.a.ZincCompiler$ - one error found
      Exception in thread "main" Compilation failed
      at sbt.compiler.AnalyzingCompiler.call(AnalyzingCompiler.scala:105)
      at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:48)
      at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:41)
      at sbt.compiler.AggressiveCompile$$anonfun$3$$anonfun$compileScala$1$1.a
      pply$mcV$sp(AggressiveCompile.scala:98)
      at sbt.compiler.AggressiveCompile$$anonfun$3$$anonfun$compileScala$1$1.a
      pply(AggressiveCompile.scala:98)
      at sbt.compiler.AggressiveCompile$$anonfun$3$$anonfun$compileScala$1$1.a
      pply(AggressiveCompile.scala:98)
      at sbt.compiler.AggressiveCompile.sbt$compiler$AggressiveCompile$$timed(
      AggressiveCompile.scala:155)
      at sbt.compiler.AggressiveCompile$$anonfun$3.compileScala$1(AggressiveCo
      mpile.scala:97)
      at sbt.compiler.AggressiveCompile$$anonfun$3.apply(AggressiveCompile.sca
      la:138)
      at sbt.compiler.AggressiveCompile$$anonfun$3.apply(AggressiveCompile.sca
      la:86)
      at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:3
      0)
      at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:2
      8)
      at sbt.inc.Incremental$.cycle(Incremental.scala:73)
      at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:33)
      at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:32)
      at sbt.inc.Incremental$.manageClassfiles(Incremental.scala:41)
      at sbt.inc.Incremental$.compile(Incremental.scala:32)
      at sbt.inc.IncrementalCompile$.apply(Compile.scala:25)
      at sbt.compiler.AggressiveCompile.compile2(AggressiveCompile.scala:146)
      at sbt.compiler.AggressiveCompile.compile1(AggressiveCompile.scala:70)
      at com.typesafe.zinc.Compiler.compile(Compiler.scala:161)
      at com.typesafe.zinc.Compiler.compile(Compiler.scala:142)
      at io.gatling.app.ZincCompiler$.main(ZincCompiler.scala:111)
      at io.gatling.app.ZincCompiler.main(ZincCompiler.scala)
      Exception in thread "main" org.apache.commons.exec.ExecuteException: Process exi
      ted with an error: 1 (Exit value: 1)
      at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecut
      or.java:377)
      at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:
      160)
      at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:
      147)
      at io.gatling.app.ZincCompilerLauncher$.apply(ZincCompilerLauncher.scala
      :54)
      at io.gatling.app.SimulationClassLoader$.fromSourcesDirectory(Simulation
      ClassLoader.scala:32)
      at io.gatling.app.Gatling$$anonfun$15.apply(Gatling.scala:171)
      at io.gatling.app.Gatling$$anonfun$15.apply(Gatling.scala:171)
      at scala.Option.getOrElse(Option.scala:120)
      at io.gatling.app.Gatling.start(Gatling.scala:171)
      at io.gatling.app.Gatling$.fromMap(Gatling.scala:59)
      at io.gatling.app.Gatling$.runGatling(Gatling.scala:80)
      at io.gatling.app.Gatling$.main(Gatling.scala:54)
      at io.gatling.app.Gatling.main(Gatling.scala)

      My code is the following:

      import io.gatling.core.Predef._
      import io.gatling.http.Predef._
      import scala.concurrent.duration._

      class BeesShopSimulation extends Simulation {

      val http = http.baseURL("http://localhost:8080/gatling-bees-shop-1.0.0-SNAPSHOT")

      setUp (
      ConsultProductsScenario.scn.inject(rampRate(10 usersPerSec) to(100 usersPerSec) during(5 minutes))
      ).protocols(http)
      }


      I`ll be waiting your advice again. Thanks is advance.

      Delete
  2. This comment has been removed by the author.

    ReplyDelete