Karate Framework should you use it or not ?

For more than a year, I’ve been using the Karate testing framework for day-to-day activities in my QA role. In this post, I would like to share my honest feedback about it.

I will keep it short and straight to the point. Here are the PROS that I see:

• If you are familiar with Cucumber and like to use it for technical feature files, then Karate will help you a lot.
• It is fast for API testing and relatively easy to set up.

Feature: API Test Example


  Scenario: Create and verify a user
    Given url 'https://jsonplaceholder.typicode.com/users'
    And request { name: 'John Doe', email: 'john@test.com' }
    When method post
    Then status 201
    And match response.name == 'John Doe'

And here are the CONS:

• It’s challenging to set up nice reporting. If you are using JUnit, the execution will always display in the IDE that the test passed. This is because, in the background, all Karate tests are executed in one big JUnit test, so the IDE can't track the actual execution status of individual scenarios.

// JUnit 5 syntax allows the IDE to see individual Scenarios
class TestRunner {
    
    @Karate.Test
    Karate testAll() {
        return Karate.run().relativeTo(getClass());
    }
}

• For UI testing, there are too many ways to interact with elements. Some use smart locators, some use retries, and sometimes even the examples in the official documentation do not work as expected.

// 1. CSS Selector
click('#submit-button')


// 2. XPath
click("//button[@name='submit']")


// 3. Karate 'Friendly' Locator (Wildcard)
click('{}Submit')


// 4. Text Content Locator
click('{^}Submit')


// 5. Right Of Locator (Relative)
click('rightOf(#password-input)')

• The cucumber BDD format that is used in karate is not the Behaviour Driven Development format that can be read and understood by any person who works on the project

In conclusion, I would say that if I were the person deciding what tools or libraries are needed for a specific project, I would never recommend Karate.