Search for Programming, AngularJS, Rails, Testing ...

Advanced Testing and Debugging in AngularJS

Learn to test your AngularJS application like a Pro using the latest and greatest

AngularJS is becoming immensely popular and mainstream which means that there is a lot of AngularJS code out there that is being tested or is yet to be tested. And now that you're well on your way to test like a pro, thanks to the abundance of articles, tutorials, books and material out there on AngularJS testing & development, testing should be a mandatory process of your web development workflow.

Full-Spectrum testing with AngularJS & Karma taught us how to test certain areas of your AngularJS application, but how do we test efficiently? How do we debug a problem down the root cause? How do we skip tests, set breakpoints, and professionally mock-out our test components so that we can catch hidden bugs and unexpected scenarios? How far can and should we go with Unit & E2E testing? What else should we consider. Well lets take a deeper dive into testing in AngularJS and expand our minds by learning how to become a professional front-end tester.

Last Updated

This page was first published on September 27th 2013 and was last updated on September 28th 2013.

Table of Contents

Presentation Slides + Video

Most of the material here was featured in a meetup talk which I spoke at in Toronto, Canada. Here is a link to the slides and a recording of the video can be found below.

What to test and what not to test

Unit and E2E (protractor) testing provide a solid testing ground to cover the front-end of your application, but there are other areas that need to be tested. The key thing to remember in this case is that testing anything should be simple and thus you you should use the right tool for the job.

So if you're using unit testing to perform live tests at your website then you should probably look into using E2E tests with Protractor. Don't use Unit testing to test your server-side API response syntax code and don't use Karma to run your server-side tests (unless you use node.js).

So what tool do we use to perform testing?

Since we're using AngularJS here, which is purely front-end code, we'll stick to using Unit and E2E tests (using Protractor) like usual. However, if you're still in the dark about how to test other areas of your application, here is a generalized breakdown of what tools might be useful depending on your situation:

What to test Tools to use and why
Testing to see if each page loads properly
Use Protractor (Selenium)

A Web Driver or an E2E test can run a full integration test through a given URL on your website and can provide data on whether any JavaScript errors occurred

Does my backend API work as expected?
Server-Side testing
Lookup which server-side test framework fits your needs. Don't solely rely integration tests to see if things are working.
If I change my front-end JavaScript API code then how do I know things are working?
Unit Testing (Jasmine or Mocha)
Ideally you want to have a solid test spec for each feature (logical branch of execution) within each block of code (functions, objects, services, methods, etc...) to keep track of what goes on in that method.
How do I know things work for browser X
Integration or Unit Testing
Typically you would setup a collection of integration tests to cover various pages/views on your application. Then when a bug appears, try to isolate the broken code into it's own service/subroutine and setup a unit test or two to cover what's going on. This way if your page/view code changes then you'll still have a reference to the unit test (since unit tests are cheap) and you won't have to worry about looking out for that bug again (since the contained service/subroutine is where it will be located). This may be challenging with DOM code, but anything is possible.
I didn't read the CHANGELOG and I want to know if things will break if I up the version on my 3rd-party code
Integration or Unit Testing
Integration tests need to cover most if not all of the views on your page and, if you upgrade to a new version of framework X, then it should be easy to find out which features are not working. However, if the changes in the 3rd-party code are more internal and purely JavaScript then an integration test may not detect any broken code. If you have good amount of unit test code coverage then you should be fine, but if you have close to nothing then setup a simple test spec which tests out the simple input and outputs of the 3rd party code (method names and return values should be enough).

How do I mix all my tests together?

Since there are so many testing tools, runners and environments working independently from one another, you may be wondering how do I summarize everything together? Well a CI testing environment is your best bet. Despite a CI environment being designed to run all the time, it will test your all of your code as soon as you save and/or commit any changes. Continuous Integration is covered later on in this article.

Preparing your test environment with Karma & Grunt

Over the past year, Karma has been updated a few times and the configuration syntax has changed. Therefore, to make the latest version of Karma work with your testing code, be sure to run karma init and prepare the new options for your test code. You can also use the yearofmoo-angularjs-seed repository which I will personally maintain to include the latest versions of AngularJS, Grunt, Karma, Bower and the new AngularJS E2E tester called Protractor. Be sure to examine the README contained on that seed repo to get an idea of how to start and run unit and E2E test operations from the command line.

Click here to view the seed repository

Write tests as you go

This is so important. You have no idea. There are large drawbacks to not writing tests as you code, and you may think that writing tests in general causes your development time to increase per feature. While this may be true, it's actually a trade off since there is a high chance you'll end up wasting time to fix or check for bugs which your tests would have caught in a few microseconds. It may also seem logical to write tests afterward, but depending on when you write them they may not be as robust as they could be if written during development time.

Let's examine and break down the consequences of when testing is done for an application in the next section...

So when do you do your testing?

Don't be shy...

Never! I just refresh the page!

You're in for a world of pain. It doesn't matter how good of a developer you are, without tests you have nothing to show for yourself. Introduce a new programmer or feature and things go haywire.

After the project is done

Still not good enough. You've already wasted how much time fixing bugs that should have never existed up until this point, so how is testing going to make things better? Well OK, your application is working, but the tests will only provide a layer of coating on top of your application. Meaning that if you want to change a feature later then you'll have to change some if not all of your tests.

After I have written a large amount of code

This is good. Your tests will cover what you have done, but they will get lost in translation. You see when you're in the middle of doing something your mind remembers all the ins and outs of your code. And if you're not testing what you're working on during the moment of your tests then the tests that are written later on will only cover the boundaries of the code that you have developed. Not to mention that you may end up writing more tests to cover features that could easily be covered in a single test.

Before I start to program

This is known as Test-First development. And while this feature is nice and it does the trick, many developers often get caught up in writing too much test code before application code which may lead to nothing getting done. (Think about how many meetings have gone nowhere since there's nothing to show and everyone is too caught up with how to get things started ... This is what having only tests and no code can result in.) Programmers sometimes overlook the fact that code gets removed and redone from time to time and (in this case) this may mean that test code may get thrown in the garbage. This also means that the test code will ultimately affect the implementation of the actual feature code. Better to code a little bit first and then test as you go.

At the same time as I write code

Here we go. Not too hot and not too cold. As you write a new feature, new method or object, make a test. If it's something that changes data or makes an impact then make a test. Now you don't have to make a new test to assert each feature, but try and include as many assertions as you can into the same test. The resulting test will shape itself nicely around your new feature.

Module & Inject Breakdown

Module and Inject make testing AngularJS Unit tests really fun and easy. They are apart of the angular-mocks.js javascript file and that is to be included into your test runner. You have more than likely already seen inject() in the previous article beforehand, but what does module() do? Well the module is there so that you can dynamically adjust or create module code for each test instead of having to swap your JavaScript code somehow in between tests.

Using module() and inject() in your tests

The code below illustrates how module and test work together. Lets try and test and create our own filter directly inside of one test "it" block.

it("should create and test the functionality of a filter", function() {

  //module-level code and tests
  module(function($filterProvider) {
    $filterProvider.register('even', function() {
      return function(array) {
        var evenValues = [];
        angular.forEach(array, function(v) {
          if(v % 2 == 0) evenValues.push(v);
        });
        return evenValues;
      };
    });
  });

  //injection-level code and tests
  inject(function($filter) {
    var filterEven = $filter('even');
    expect(filterEven(values,[1,2,3,4,5])).toBe([2,4]);
  });

});

Taking a closer look at the module block, it looks very similar to the public functions available in angular.module. And yes it is! You can basically define any module-level piece of code in the module mock function and those changes will be reflected for the test that is being run.

module() can only be run before inject()

You can make as many module() and inject() function calls as you want, but as soon as an inject() function is called then module() cannot be called again. This is just how an AngularJS application functions. If you wish to provide an injection operation at the top of all your tests, but you still wish to provide module overriding somewhere later in your tests then a workaround for doing so is by having your module code return a function (which in this case acts like an inline injection).

describe('My Tests', function() {
  beforeEach(module(function() {
    //do some stuff with providers
    return function(inject1, inject2) {
      //do some stuff with injected services
    }
  }));

  it('should check another feature', function() {
    module(function() {
      //works as normal
    });
    inject(function() {
      //works as normal
    });
  });
});

Try to avoid going crazy with this stuff since it isn't very clear what is going on and other developers reading your code may have a difficult time understanding what you just did.

Do I have to inject() for each test?

No you don't. You can avoid this by running a beforeEach function which will be executed just before each test is run. Inside that beforeEach block, you can include an inject() function and, by prepending and appending each injected object with an underscore value, you can assign each member to a value without having any naming conflicts. Here's an example:

describe('testing my code', function() {
  var $compile, $rootScope;

  beforeEach(inject(function(_$compile_, _$rootScope_) {
    $compile = _$compile;
    $rootScope = _$rootScope_;
  });

  it("should say hello", function() {
    var elm = $compile('<div>hello</div>')($rootScope);
    expect(elm.text()).toBe('hello');
  });
});

Forget what I said...

As a rule of thumb, if you can avoid doing this (using global variables in your tests) then please do avoid it. Having global variables in tests is just as bad as having them in your code. Try and house as many variables, functions, objects, data and logic inside of each spec (it) or describe block as possible. This way if new tests are introduced into your code, then you don't have to end up refactoring your tests just to make new tests work.

Powerful Mocking Strategies

You want your code to be easily testable, maintainable and loosely coupled. But there are times when parts of your code is reliant on other code and some level of code dependency shows up (and there is nothing wrong with this!). As you're well aware, the AngularJS DI allows for your code to inject other pieces of code together which makes using dependencies an everyday thing. But to test out a piece of code specifically without having the need to test out all the dependencies is tricky. You're responsible to ensure that each line of code in your application works as expect so how can you reach into those deep places of your code? You can do this using mocking.

What you're doing when using a mocking strategy is simply providing a fake object or method for another piece of code (the target code) to interact with. Then, when the target code is being executed with the mock(s), the mocked-out components direct the program flow such that each area within the target code can be reached and tested. And, since AngularJS uses a DI to handle dependencies, all you have to do is override the dependencies that you wish to mock out.

//
// application code
//
app.factory('adminLocation', function($location) {
  return function(path) {
    $location.path('/admin' + path);
  };
});

//
// test code
//
it("should say hello", function() {
  var capturedPath;

  module(function($provide) {
    $provide.value('$location',{
      path : function(path) {
        capturedPath = path; 
      }
    });
  });

  inject(function(adminLocation) {
    adminLocation('/clients');
    expect(capturedPath).toBe('/admin/clients');
  });
});

Sync' and Async' Testing

There is a 99.975921% chance that there is something asynchronous going on in your application in your own code. Infact, something as simple as an API request using $http is an example of an asynchronous request. This is expected, but testing it doesn't mean that you should also test asynchronously.

Put in the time into shaping your tests to be synchronous using mocking. As explained before, mocking allows you to capture and move around the internal flow of a program without having to change anything in your application code.

The approach to making async' code testable as if it were sync' code is simple ... Just before the asynchronous code does its thing (stuff like an animation, AJAX request or a timeout, etc...), capture what's going on (using mocks) and work around it. This works so well and it speeds up your tests a lot and AngularJS uses tricks like these in their core code to reduce about one trillion CPU cycles each time the testing suite is run ($httpBackend for example is used to mock HTTP responses for routes, templates and AJAX calls).

So let's imagine we have a piece of code that downloads data from the SoundCloud JSON API. We can imagine that accessing that API is slow, but since we've spent all this time making our code read and work off of the API so nicely, why not test the live thing? Lets take a look at our test:

it("should pull in data from the SoundCloud database",
  inject(function(SoundCloudService) {

  var data = SoundCloudService.get('dubstep').then(function(data) {
    expect(data[0].name).toContain('awesome dubstep song');
  });
});

This should work fine? But no it breaks. Why? Because AngularJS protects you by avoiding external AJAX calls using $http within unit tests. OK fine ... You can easily get around this by using your own XHR wrapper, but you don't want to do this at all because it's outside of the AngularJS framework. Instead, there are two ways to get around this:

Mocking out parts of the SoundCloud service

This approach avoids using HTTP entirely. Instead it just cuts off the request up until a point and provides some fake data back to the service. As explained before, this is how mocking works and this is useful if you want to test out the internal mechanics for a block of code without using up anything extra or remote.

Using $httpBackend

The $httpBackend service is an inner service used within $http and it will automatically assert any outbound HTTP requests that are issued via the $http service. This means that as soon as we request a search from SoundCloud, in this case dubstep, then the $httpBackend service will capture and assert the full URL path to fetch that movie (in this case https://api.soundcloud.com/tracks.json?client_id=YOUR_CLIENT_ID&q=dubstep). To make this work, what we need to do is prepare the mock code to provide the raw JSON response from the SoundCloud URI.

it("should pull in data from the SoundCloud database",
  inject(function(SoundCloudService, $httpBackend) {

  $httpBackend.expect('GET',
    'https://api.soundcloud.com/tracks.json?' + 
      'client_id=YOUR_CLIENT_ID&q=dubstep')
    .respond(200,
      '[{ title : "some awesome dubstep song from 2013", id : 20 }]');

  SoundCloudService.get('dubstep').then(function(data) {
    expect(data[0].title).toContain('awesome dubstep song');
  });

  $httpBackend.flush();
});

Skipping and Filtering Tests

It may get to a point where you want to reduce the noise of your test output when testing or creating a new feature. And since we're chugging away testing and developing code at the same time then skipping and filtering tests would be a useful trick to know how to do. How can we do such a thing? Here are some nifty shortcuts that you can include in your Jasmine or Mocha test code:

Feature Jasmine & E2E Code Mocha Code
Focus on a single test change it to iit change it to it.only
Skip a test change it to xit change it to it.skip
Focus on a describe block change describe to ddescribe change describe to describe.only
Skip a describe block change describe to xdescribe change describe to describe.skip

Echoing Data to the screen

Use dump(variable) Instead of console.log(variable) when writing tests with Karma. This will provide the desired output within karma to see the state of your variable. Don't even bother using alert(). That stuff is for the web development days back when Netscape was more popular than Chrome.

Now sometimes you might want to interact with the data in your tests. Objects such as elements contain so much extra data so you may want to use something like the browser's console to examine and expand key areas of information. How do we do this? The easiest way is to include the debugger label on a single line within your code. Google Chrome (which you should be using in your karma configuration) will see that the debugger statement is there and it will set itself up so that when you start debugging your tests it will know where to start.

//This can be placed inside tests
it('should do something', function() {
  expect(element.attr('class')).toContain('active');
  expect(element.children().length).toBe(2);

  debugger //nothing else is required

  expect(element.attr('id')).toBe('stage');
});

//Or directly inside of your JavaScript code.
ngModule.factory('session', function() {
  return {
    getID : function() {
      //another place to use debugger
      debugger
      return this.id;
    }
  };
});

But how do you get into the debugger? Just open up chrome and visit localhost:9876 (this is the URL where karma lives) and click on the debug button at the top right of the screen. A (blank) new page will open and now all you have to do is open up your console and refresh the screen. Then the page will load up and freeze (it looks like chrome is hung on some line of code just like it does when a breakpoint is set). This is what we want. Just click on the console button at the bottom left of the screen and now you can use console.log(element) or any JavaScript commands to play around with the variables that are present within the application scope to where this debugger statement is placed.

Keep in mind while Chrome understands what debugger means, older and other browsers may not. So do remember to remove it once you're done with all that crazy debugging stuff.

Using Breakpoints

Breakpoints work the same way as does the debugger trick, however, instead of placing debugger in your code, a breakpoint is placed from your browser to where you want that debugger pause to show up. This is useful for times when you don't have write access to the JavaScript code.

To set a breakpoint, open up the Chrome web inspector and view the sources tab. In there, locate the JavaScript file you wish to debug and click on the line number you wish to instruct chrome to perform a debugger pause on. If the flag is set to blue then it's active so be sure to clear them away when you're done.

To see a working example of how to use debugger and breakpoints, be sure to view the embedded video in this article.

Writing Efficient Tests

There's not much more to do when making tests run efficiently in addition to:

  • Try your best to write unit tests instead of E2E tests
  • Refactor your tests to run synchronously to speed things up
  • Use mocking and fixtures to avoid relying on remote or external services to return data
  • Combine similar tests into one to reduce redundancy
  • Make use of running unit tests in parallel if a multi-core machine is in use

In addition, these helpful tricks can be used to speed up any tests specific to AngularJS:

  • If you're doing testing on DOM elements, then try and get away without attaching them at all to the body
  • Make a seperate spec and/or describe block for each feature and include only the modules you need to run your tests
  • Inject only the services you need to use in your tests and don't use any global variables

Testing older browsers

You may be using a machine that doesn't have Internet Explorer or older versions of browsers that cannot be called up with Karma. This is OK, you can still point each device or program to the URL that karma works off of and Karma will automatically feed tests to it. For instance, If I ever need to test anything using Internet Explorer 8 or 9 then this is how I do it. (Mobile phones or emulators can be used to make this happen as well.)

Connecting an iOS emulator to Karma

Instead of manually testing with your IPhone, just pop open xCode, select the iOS simulator, open up Safari and point it to localhost:9876

Chrome 28.0.1500 (Mac OS X 10.8.4): Executed 2029 of 2029 SUCCESS (7.952 secs / 7.257 secs)
INFO [Mobile Safari 6.0.0 (iOS 6.1)]: Connected on socket id mToKSq0EACwodtg5X1Al
Chrome 28.0.1500 (Mac OS X 10.8.4): Executed 2029 of 2029 SUCCESS (8.561 secs / 7.965 secs)
Mobile Safari 6.0.0 (iOS 6.1): Executed 2003 of 2003 SUCCESS (5.443 secs / 4.774 secs)
TOTAL: 4032 SUCCESS

Using IE8 & IE9

If you really want to test an older version of IE and hook that up into Karma then you need to have Windows 7 or Windows 8 with Internet Explorer 9 or higher. The developer tools allow you to emulate older versions of IE. Simply open up developer tools and click on the IE version at the top right of the window. Make sure to select Internet Explorer 8 or 9 and choose Standards Mode. Then, on the machine that has karma running on it, grab the [local] IP address and then point the Internet Explorer browser to IP-ADDRESS:9876 and Karma will pickup what's going and run the tests against it.

Chrome 28.0.1500 (Mac OS X 10.8.4): Executed 2029 of 2029 SUCCESS (7.952 secs / 7.257 secs)
INFO [Mobile Safari 6.0.0 (iOS 6.1)]: Connected on socket id mToKSq0EACwodtg5X1Al
Chrome 28.0.1500 (Mac OS X 10.8.4): Executed 2029 of 2029 SUCCESS (8.561 secs / 7.965 secs)
Mobile Safari 6.0.0 (iOS 6.1): Executed 2003 of 2003 SUCCESS (5.443 secs / 4.774 secs)
TOTAL: 4032 SUCCESS

Invest into preparing a CI environment

A Continuous Integration (CI) machine is worth the investment. Even if you're a lone developer writing an open source tool on your free time during the weekend it is still useful. The way that it works is that each commit that you is directed to your repository is automatically detected and tested against your test code. If any failed tests show up or your code fails to load then your CI server will email you and maybe even prevent the commit from merging itself into the master branch. Now you may think that you're careful and you never commit these kinds of mistakes, but it's a powerful security system to have in place.

When developing an open-source application in AngularJS

Open source development is 10000 times easier to test automatically since all you have to do is register your application on Github and configure the testing runner to work with TravisCI. All the big boy open source projects like Ruby on Rails, PHP and Ruby source code repositories as well as AngularJS all use TravisCI to do automate the testing of their test code with their source code. Travis isn't something that runs on call, but it can be configured to test your code each time a pull request against your codebase is made.

In the event that you wish to test your code with more devices and machines then you may need to setup your own CI environment. This is usually the case when developing closed-source software so please do read the next section to see how this can be done.

When developing a closed-source application in AngularJS

Closed-source applications can still be tested with a CI server, but you will need to either pay for a private server or build your CI server on a privately-hosted machine. The Jenkins CI server is a hallmark CI testing tool which can be used to test your AngularJS code. If you're using a hosting platform like Heroku then you can also pay to use one of their useful add-on testing CI testing suites. Other options include using Sauce Labs to automate testing as well.

Visit Sauce Labs Visit TravisCI Visit JenkinsCI

Feedback Please!

If anything in the article sparks your interest or you believe should be revised or expanded on then please send me an email or tweet at @yearofmoo on twitter. I hope that more of these testing articles help you out with your AngularJS workflow. Testing is just like learning how to program again, so there's always something to be learned. And when it comes to learning how test and program better and better it's just like the cleaning of a house ... It never ends :)