Integration Testing with Clojure and Clojurescript

2013-03-07 13:11:00 -0800

A popular framework for integration testing web apps is Selenium, which programmatically controls a browser to run tests. Selenium has its drawbacks, namely tests can be slow, brittle and non-deterministic. In this post, we’ll look at an alternative integration testing solution that uses Clojure and Clojurescript.

The ideal way to do integration testing would be to write tests that can make frontend calls and then verify state changes on the backend. We thought that Clojure (a modern dialect of Lisp that runs on the JVM) and ClojureScript (its Javascript variant) would be good candidates for the test framework since it allows us to use essentially the same language on both the frontend and backend. An added benefit is that the backend can be written in any language that runs on the JVM, and the client code can be Javascript or anything that compiles to Javascript. As an example, Versal’s stack is primarily Scala and Coffeescript.

To allow communication between the frontend and the backend, we’ll use the excellent utility cemerick/yonder. Yonder allows the server to remotely run ClojureScript through an nREPL (network REPL) connection, and includes helper functions for setting up and managing the nREPL connection. When a client first loads the page, it opens a connection to server and then waits for commands.

(repl/connect "http://localhost:9000/repl")

The test suite must create an nREPL endpoint on the server side before it loads the client. Yonder’s mechanism for doing this is open-session, which returns a session handle for the REPL it creates. open-session takes two parameters, :prepare (function that sets up the client), and :new-server (a description for the server that acts as the nREPL endpoint). For the prepare function, we’ll use the default function included with yonder that loads the client page using PhantomJS. For the server, we’ll use the default nREPL server with an additional middleware that allows ClojureScript evaluation.

(yonder/open-session
              {:prepare yonder/prepare-phantomjs
               :new-server
               {:handler (server/default-handler
                           #'cemerick.piggieback/wrap-cljs-repl)}})

With the session returned by open-session, we can now execute ClojureScript on the client remotely from our Clojure server!

(yonder/eval session (+ 1 2 3))
(yonder/eval session (into [] (js/Array :a 'b ::c "d")))

Now we’re ready to write our integration tests. For a sample test, let’s make the client perform an Ajax POST and store the submission on the server. Then we’ll verify that we received the correct submission.

Ajax request with some sample data:

(defn ^:export submit []
  (xhr [:post "/api/submit"] {:submission 42}
       (fn [_] (log "Finished submission"))))

Compojure route on the server that sets an atom to the value it receives.

(POST "/api/submit"
                 { {s :submission} :params}
                 ;; Modify server state
                 (reset! submission s))

So our final test looks like this. Note that the prepare function opens up test.html instead of the default index.html. After we have an nREPL session, we call invoke the ajax call on the client side, wait a few seconds for the request to complete, and verify the result in the atom.

(deftest browser-test
         (yonder/with-session
           [session
            (yonder/open-session
              {:prepare (partial yonder/prepare-phantomjs
                                 {:url "http://localhost:8080/test.html"})
               :new-server
               {:handler (server/default-handler
                           #'cemerick.piggieback/wrap-cljs-repl)}})]
           (yonder/eval session (cljs.test/submit))
           (Thread/sleep 4000)
           (is (= "42" @submission))))

We can run our test from the command line with lein test.

wei:shuttle wei$ lein test

lein test shuttle.core-test
2013-03-06 13:46:34.936:INFO:oejs.Server:jetty-7.6.1.v20120215

Testing shuttle.core-test
2013-03-06 13:46:35.142:INFO:oejs.AbstractConnector:Started SelectChannelConnector@0.0.0.0:8080

Ran 1 tests containing 1 assertions.
0 failures, 0 errors.

Please check out the project at versal/shuttle!

Sandbox

2012-11-30 17:35:00 -0800

Sometimes user-provided javascript needs to be run, which is scary (Evil.js anybody?). At first, we tried to sanitize third-party code to make it safe, but this got ugly quickly. It meant that we had to prepend code to every CSS selector to make sure external styles only affected the right elements, the HTML had to always be well-formed, and let’s not even talk about the JS. It also meant we couldn’t surface meaningful javascript errors when something went wrong, because it was a small piece of javascript compiled in a larger application. We looked for another solution.

JSFiddle and JSBin seemed to have similar requirements, and handled it using iframes. As frontend developers though, our gut aversion to iframes stopped us from typing the tag into our terminals in good conscience. However, they seemed to work nicely because the scripts couldn’t touch anything but their box, CSS and JS selectors just worked, and we could monitor errors easily. This talk covers more of the benefits of iframes (and it’s in 3D).

Although this isn’t a very common requirement, we built a simple library to handle this which we call Sandbox.js. It enables a few nice things like importing external JS and CSS as well as stopping code from creating dialogs (alert/confirm/prompt) unless desired. In the near future we hope to add error handling and a nice callback system so the frame can more easily communicate with the parent without nasty globals everywhere.

Happy Sandboxing!

Beyond the Reader Monad: Dependency Injection with Jellyfish

2012-11-16 10:37:00 -0800

One handy use of the reader monad is to build up a program that has a single external dependency which can be provided at the appropriate time.

In Scala a reader might look something like this:

trait Monad[A, F[_]] {
  def map[B](f: A => B): F[B]
  def flatMap[B](f: A => F[B]): F[B]
}

class Reader[E, A](g: E => A) extends Monad[A, ({type λ[α] = E => α})#λ] {
  def apply(e: E): A = g(e)
  def map[B](f: A => B): E => B = { e => f(g(e)) }
  def flatMap[B](f: A => (E => B)): E => B = { e => f(g(e))(e) }
}

object Reader {
  implicit def function1ToReader[E, A](g: E => A): Reader[E, A] = new Reader(g)
}

We can use this to compose a program from parts which each depend on the same external context:

import scala.xml.NodeSeq
import Reader.function1ToReader

val htmlHeader: String => NodeSeq =
  { title => <head><title>{title}</title></head> }

val htmlBody: String => NodeSeq =
  { title => <body><h1>{title}</h1></body> }

val html: NodeSeq => NodeSeq => NodeSeq =
  { header => body => <html>{header}{body}</html> }

val pageFromTitle: Reader[String, NodeSeq] =
  for {
    header <- htmlHeader
    body   <- htmlBody
    page   =  html(header)(body)
  } yield page // a program which, given a title, produces an HTML page

val page: NodeSeq = pageFromTitle("Reader Example")
println(page) // <html><head><title>Reader Example</title></head><body> ...

The problem

This is an easy way to do dependency injection, but it has a drawback: the dependency has to be a single thing. This makes it hard to build a program from components that have different depenencies.

Consider if we change htmlBody to take a Date instead of a String:

val htmlBody: Date => NodeSeq = date => <body><h1>{date.toString}</h1></body>

The for expression above will no longer compile, because the reader is an instance of Reader[String, NodeSeq], so both its map and flatMap methods expect to be passed a function which takes a String, not a Date:

[error]  found   : java.util.Date => scala.xml.NodeSeq
[error]  required: String => ?
[error]   body   <- htmlBody

The solution

What we really want is a way to construct a program, and keep track of all the different dependencies as we go. At the end, we’ll have a sort of arbitrary-order curried function that we can interpret recursively, providing each dependency as needed.

val simpleProgram: Program =
  program {
    val bar: Bar = read[Bar] // retrieve the `Bar` dependency
    val foo: Foo = read[Foo] // retrieve the `Foo` dependency
    Return("foo is " + foo.x + ", bar is " + bar.x)
  }

This is what Jellyfish gives us.

How it works

A Jellyfish program is represented as an instance of the Program trait, which has two implementations:

case class Return(a: Any) extends Program
case class With[A](c: Class[A], f: A => Program) extends Program

The read function, which wraps Scala’s shift function, takes a generic function of type A => Program and wraps it in a With which tracks the type of A. This can happen an arbitrary number of times, resulting in a data structure analogous to a curried function.

Ignoring some of the wrappers, this:

val bar: Bar = read[Bar] // retrieve the `Bar` dependency
val foo: Foo = read[Foo] // retrieve the `Foo` dependency
Return("foo is " + foo.x + ", bar is " + bar.x)

becomes:

bar: Bar => {
  val foo: Foo = read[Foo] // retrieve the `Foo` dependency
  Return("foo is " + foo.x + ", bar is " + bar.x)
}

which becomes:

bar: Bar => {
  foo: Foo => {
    Return("foo is " + foo.x + ", bar is " + bar.x)
  }
}

which is a curried function with two dependencies.

An interpreter is then built to unwrap each nested With, extract the function of type A => Program, provide the appropriate instance of A, and continue until the program completes with a Return.

An example

First, write a program which retrieves dependencies via the read function:

case class Foo(x: Int)
case class Bar(x: String)

object SimpleProgram {

  import com.versal.jellyfish.{program, Program, read, Return}

  // create a program with some dependencies
  val simpleProgram: Program =
   program {
      val bar: Bar = read[Bar] // retrieve the `Bar` dependency
      val foo: Foo = read[Foo] // retrieve the `Foo` dependency
      Return("foo is " + foo.x + ", bar is " + bar.x)
    }

}

Second, write an interpreter which provides the dependencies to the program:

object SimpleInterpreter {

  import com.versal.jellyfish.{classy, Program, Return, With}

  val foo = Foo(42)
  val bar = Bar("baz")

  // run a program, injecting dependencies as needed
  def run(p: Program): Any =
    p match {
      case With(c, f) if c.isA[Foo] => run(f(foo)) // inject the `Foo` dependency
      case With(c, f) if c.isA[Bar] => run(f(bar)) // inject the `Bar` dependency
      case Return(a)                => a           // all done - return the result
    }

}

Third, run the interpreter:

val result = SimpleInterpreter.run(SimpleProgram.simpleProgram)
println(result) // prints "foo is 42, bar is baz"

Use it

Jellyfish lives on GitHub at github.com/Versal/jellyfish. To use it in your own project, enable continuations and add the Jellyfish library to your build.sbt file:

libraryDependencies += "com.versal" %% "jellyfish" % "0.1-RC1"

A performance shootout for RESTful libraries

2012-11-07 17:38:00 -0800

One of our early projects at Versal was building a thin RESTful API on top of our Scala-based platform. This would allow access to backend services by rich frontend applications, and establish conventions for data and service contracts among our systems.

Initially we went with a Play 2.0 and Swagger implementation, but its configuration felt unweildy, and the bulk of its feature set was unnecessary for just a simple RESTful API. At this point we stepped back to survey the landscape.

Some of us had experience with RESTful Java libraries such as Jersey, Spring MVC, Struts, etc., while others had used Scala libraries such as Scalatra and Spray. It was apparent that we needed to cast a wide net and compare what we found, so we created the Scamper project.

Scamper pits several libraries against each other:

Fast Test

We made our best guesses to configure each project in a comparable way, and designed both non-blocking “fast” tests and blocking “slow” tests for each using both ApacheBench and JMeter.

Based on initial analysis, the fastest implementation uses raw Servlet 3.0. Play 2.0 was somewhat disappointing, but Scalatra turns out to be a nice tradeoff between performance and implementation simplicity. We decided to go with Scalatra.

We have had good input from the community on how to optimize Play 2.0, yielding significant performance improvements. We look forward to learning more about each approach!