Ratchet Tests with ScalaTest

Ever found more broken things in a project than you possible could fix in one go? I’m exactly in that situation. We have a test which checks for certain dependencies in our project. The problem: We discovered that the test was broken and didn’t report all disallowed dependencies. We do know how to fix it, but if we do it will fail. And since there are many violations it will take quite some time to fix all of them. But of course we want to prevent even more violations of our dependency rules.

The solution could be a ratchet on the tests. A contraption which accepts broken tests, but which doesn’t allow tests to fail once they succeeded.

For ScalaTest you can create appropriate tests using this simple Ratchet trait

  1. package de.schauderhaft.ratchet
  2. import org.scalatest.AbstractSuite
  3. import org.scalatest.Suite
  4. import org.scalatest.TestFailedException
  5.  
  6. trait Ratchet extends AbstractSuite {
  7.     self : Suite =>
  8.  
  9.     private var tests = Set[String]()
  10.  
  11.     def ratchet(ratchetedTests : Set[String]) {
  12.         tests = ratchetedTests
  13.     }
  14.  
  15.     override abstract def withFixture(theTest : NoArgTest) {
  16.         if (tests.contains(theTest.name)) {
  17.             var failedToFail = false
  18.             try {
  19.                 super.withFixture(theTest)
  20.                 failedToFail = true
  21.             } catch {
  22.                 case ex : TestFailedException =>
  23.             }
  24.             if (failedToFail)
  25.                 fail("Remove '%s' from the ratchet it doesn't fail anymore".format(theTest.name))
  26.         } else
  27.             super.withFixture(theTest)
  28.     }
  29. }

It adds a ratchet method to your suite. You pass it a Set of test names. These are the tests that you expect to fail. If they do fail, the Ratchet will convert that failure to a success. If a test which you expect to fail succeeds, the Ratchet will make sure it does fail with a message saying you should remove it from the tests expected to fail. Tests not registered with the ratchet method behave just as normal tests do. This is how the contraption looks in action with an example test suite: (Note tests which start witch ‘expected:’ do fail

  1. class RatchetTestDemo extends FunSuite with ShouldMatchers with Ratchet {
  2.     ratchet(Set(
  3.         "a failing test with ratchet does not fail",
  4.         "expected: a succeeding test with ratchet fails"))
  5.  
  6.     test("a failing test with ratchet does not fail") {
  7.         fail
  8.     }
  9.  
  10.     test("expected: a failing test without ratchet fails") {
  11.         fail
  12.     }
  13.  
  14.     test("a succeeding test without ratchet succeeds") {
  15.     }
  16.  
  17.     test("expected: a succeeding test with ratchet fails") {
  18.         println("hallo")
  19.     }
  20. }


Share:
  • DZone
  • Digg
  • del.icio.us
  • Reddit
  • Facebook
  • Twitter