← Back to Code School
09/27/2013

Following my last post on course challenges, I want to now take a look at an example of challenge testing. The first part of creating tests for a challenge is to define sample attempts and the error messages that should show up for each one. This is where we can define challenge attempts and matching error messages.

Say we have a challenge where we want the user to create a function named largestNum which takes two numbers as arguments and returns the greater value. One possible answer might be:

function largestNum(a,b) {
    if(a > b){
      return a;
    }else{
      return b;
    }
}

In the following examples, these chunks of code would typically be typed into something similar to Rspec as a string, since user attempts are really just strings from the client. I’m going to omit the Rspec code to save space in this post.

Correct Answers

I first define possible correct answers, so I know when my tests/attempts should always succeed. I usually try to think of a few different ways and formats to get better success coverage.

function largestNum(a,b) {
    if(a > b){
      return a;
    }else{
      return b;
    }
}

var largestNum = function (a,b) {
    if(a > b){
      return a;
    }else{
      return b;
    }
}

var largestNum = function (a,b) {
    var val = a
    if(b > a) {
        var val = b
    }
    return val
}

var largestNum = function (a,b) {
    if(b > a) {
        return b
    }
    return a
}

function largestNum(a,b) {
    if(b > a){
      return b;
    }else{
      return a;
    }
}

function largestNum(a,b) {
    return b > a ? b : a
}

Failing Answers

For the failing answers, I like to start with what the user is initially given (in this case it’s a blank editor) and work my way to a possible correct answer. I try to add a failing test every now and then that’s not quite in line with the progression, but that catches some predictable common errors. I then also define the error messages that users should see. Our own internal tests make sure that the course outputs the correct error message for each attempt.

The bold text is the error message, while the italics are an annotation for the type of test used for an attempt.

At the very beginning of these tests, we run some code to check if the resulting output of the attempt is the correct result. If the expected output comes back positive then we skip all the rest of the tests. In this case, we would run this function twice with different numbers for a and b, and as long as the correct number is returned then we’ll consider it correct.

This is pretty important, since that ternary statement example above will probably break any parse tree tests.

//blank

You forgot to create a largestNum function, you should go ahead and give that a try.

This will probably be either a simple eval and check to make sure that largestNum exists and is a function, or we could use a parse tree if we didn’t want to require a specific function name.

function largestNum(){

}

Great! You got the largestNum function correct. One problem though, this function should accept two arguments, go ahead and make that change.

In some languages we could just run an eval and look for arguments 2 for 0 type error, but JavaScript doesn’t care, it takes whatever you give it, so we’d be using a parse tree test for this one.

function largestNum(a){

}

You’re missing an argument, this function should accept two arguments instead of one.

Same as before, use a parse tree, although this test code will come before the previous one, since it will look specifically for one variable.

function largestNum(a,b) {

}

**Your function should return the greatest value of the two values passed into it. Try starting with a conditional if statement **

Parse tree here, we’ll just look for an if statement inside of the function.

function largestNum(a,b) {
    if(a){}
}

Your if statement should compare the two variables passed into the largestNum function, try using a >

Again parse tree, we’ll pull apart that if statement and look for some sort of conditional.

function largestNum(a,b) {
    if(a > b){}
}

You’re making a good comparison, but you need to actually return something from this function

weee more parse trees, looking for that return statement.

function largestNum(a,b) {
    if(a > b){}
    return a;
}

Trying to be tricky, eh? You need to return a value based on your conditional, your function doesn’t return the correct value if the arguments are switched up, so try putting the return inside of the conditional.

Parse tree, check to see if the return statement is inside of the conditional block. At this point, I’m going to push the user into putting the return statements in the conditional like the very first correct example. Any of the other correct answers should cancel the tests early and succeed if they work, so we don’t need to worry. If we tried to handle every correct answer we would end up with a hundred tests and two days spent on a single challenge, so it’s better to push in a clear and simple direction

function largestNum(a,b) {
    if(a > b){
        return a;
    }
}

It’s nice that you’re returning a value based on your conditional, but your function still isn’t returning the correct values, you may need an else statement in your conditional.

Parse tree, look for an else statement, we’re getting close to done!

function largestNum(a,b) {
    if(a > b){
        return a;
    }else{}
}

Your function isn’t returning the correct values, make sure to use another return statement in your else statement to return the opposite variable than the one you’re returning in the first part of the if statement.

function largestNum(a,b) {
    if(a > b){
        a
    }else{
        b
    }
}

function largestNum(a,b) {
    if(a > b){
        return a
    }else{
        b
    }
}

function largestNum(a,b) {
    if(a > b){
        a
    }else{
        return b
    }
}

(these 3 get the same message)

This isn’t Ruby or CoffeeScript, make sure you use the return keyword when trying to return a variable from a function in JavaScript.

I’ve added these errors to some challenges, and we use tons of CoffeeScript and Ruby here at Envy Labs and Code School, so it’s an easy mistake to make even for a seasoned dev.


That’s about it! This was a parse tree heavy challenge, as beginner courses usually are for us. I’ve found that testing frameworks, especially in JavaScript, is easier than testing raw JavaScript because you can use things like sinonjs to spy on functions, or you can just monkey patch/stub/mock all the things. If you’ve enjoyed this post, I might be convinced to blog again on tests (and possibly the parse tree stuff we do) if you request more via Twitter @codeschool! :)