Categories
Automated Testing Test Automation

Karate: Test Automation Made Simple?

I’ve heard this test automation tool mentioned a few times lately: Karate: Test Automation Made Simple. The idea behind the tool is all one needs to write is gherkin feature files – no step or glue code is required.

I thought I’d give it try to see how it works. My example repository is on GitHub.

Background

Karate allows you to define automated tests in gherkin (Given, When, Then) which are executed directly as either Web Browser or API tests.

Examples

A web test looks like:

Feature: Web Browser Automation using Karate

Background:
   * configure driver = { type: 'chrome' }

Scenario: Demonstrate webdriver usage
  Given driver 'https://webdriverjsdemo.github.io'
  When waitFor('#elementappearschild')
  Then assert exists('#elementappearschild').exists
  Given driver 'https://webdriverjsdemo.github.io/leave'
  When click('#homelink')
  And dialog(true)
  And waitFor('#elementappearsparent')
  Then assert exists('#elementappearsparent').exists

and an API test looks like:

Feature: API test example using Karate
  Background:
    * url 'https://webdriverjsdemo.github.io/'

  Scenario: get all users should return array of one user
    Given path 'users.json'
    When method get
    Then status 200
    And match response == '#[1]'

  Scenario: get a user should include name and age
    Given path 'user.json'
    When method get
    Then status 200
    And match response.name == '#string'
    And match response.age == '#number'

These tests are fully self-contained, there’s no need to write step definitions or any other code.

Test Execution

There’s a few ways to execute tests. There’s a command line standalone executable you can run passing in details of your tests which are parsed and run. This is a straightforward way to quickly get started, although it seems things like configuring different URLs/environments is more complicated. You can also set it up as a full Java project with IDE and debugging support.

Nice 😇

  • Provided standalone executable makes writing and running your first test super quick – including a VSCode plugin to run/debug
  • In built parallel test execution support (although I couldn’t get it to work)
  • Can easily mix API and web browser automation in same script
  • Don’t need to write “code” just follow an API and write a gherkin steps

Not sure 🤔

  • I couldn’t get parallel execution of tests to work well using the command line execution argument for Chrome browser tests
  • I’m still not sure about using plain gherkin files to define implementation detail such as css selectors and API end points 🤔
  • I’m not sure the best way to get this running in CI: probably using the executable? I didn’t spend time on this.

Summary

Whilst Karate is a unique testing tool, I am not sure how well it would scale to a med-large software project considering things like configuration and reusability seem difficult to implement in plain language gherkin files. For a small project or perhaps some throwaway test scripts I think it would be a fun way to write these types of tests.

8 replies on “Karate: Test Automation Made Simple?”

how would one do reusability in Karate with no step def? When waitFor(‘#elementappearschild’)
what if locator changes in future? Need to change for each step ?

Dev of Karate here. Thanks for the review, yes parallel-execution of browser-based tests is non-trivial in any framework, which is why we have a Docker / CI option if you go through the docs in detail. Let me know if you need a link.

Regarding the concern on “reusability”. I actually think this is over-rated – and that the so-called “Page Object Model” actually does more harm than good, but of course – just my opinion 🙂 That said, there is a way to achieve re-use in Karate, explained in point #7 here: https://hackernoon.com/the-world-needs-an-alternative-to-selenium-so-we-built-one-zrk3j3nyr

I remember when I was on a gig (when you and I worked together Alister) and a dev and I decided to ignore everything we knew about cucumber/page object model abstractions etc and implement a purist cucumber project following the instructions from the cucumber team. It came out very differently than we originally thought it would. Better in some ways, worse in others. Better in the sense the framework did alot more of the heavy lifting than we’d normally let it have and it did it well, albeit in an opinionated way and as such it wasn’t beholden to bad code/patterns behind the scenes, but worse because implementation detail crept into the behavioural language and made it much less readable.
What was true for me back then with Cucumber is true for me now with Karate.

Thanks Dean – makes sense. I think the original Cucumber team regretted the decision to support those generic steps using Capybarra since it led to hard to maintain tests and eventually removed it.

I’m just a beginner with respect to BDD and end to end testing etc and still have a long way to go, but so far I am impressed by Karate. And if it does not have the functionality build in that you need, there is JavaScript (and super easy to use within a feature file) or else Java. Please have a look here: https://en.wikipedia.org/wiki/Comparison_of_API_simulation_tools
It seems like Karate is the only framework that offers an escape route when its build in features are not sufficient.

I see that at Ekko Proxy, they mentioned support for JDBC, while for Karate they did not. But here is how super easy that is:
https://github.com/intuit/karate/blob/master/karate-demo/src/test/java/demo/dogs/dogs.feature
With automatic conversion of Java Maps to json, the full power of ‘match’ becomes available. Super easy. So it is a bit unfair that for Karate JDBC is not mentioned. Even though we have to go via Java Interop, the integration is painless.

So now all other test frameworks have become redundant 🙂
I wish there was out of the box support tor tcp, though, and for executing scripts (command line), but as mentioned, Karate at least offers an escape route to get it done.

One tiny critical note: today I found out you can’t set the delimiter or quote character for reading csv files…This should be fairly simple to implement in a next version?

But overall, this is the only framework so far that makes writing tests fun to do.

Leave a Reply

Your email address will not be published. Required fields are marked *