7 mistakes of software testing
Though most developers know the importance of testing, it seems that a lot of them still aren’t testing enough. And if they write tests, they test just test wrong.
Unit tests
If some tests are written in applications, most of them will be unit tests. It’s easy to test a utility class by just calling all utility methods, passing some values and checking if the expected result is returned.
A first mistake arises here. Most people don’t think out of the box, or not enough. You can test that 1 + 1 =2 , that 2 + 1 = 3 and that 3 + 1 = 4. But what’s the benefit of doing almost the same test 3 times? It’s better to test the boundary cases. Are the arguments of the sum( ) method primitive types or Objects? If they are Objects, what happens if you pass null values? If an exception is thrown, is that the expected one? Does it clearly tell what the problem is?
Mocks
If for example the service layer is unit tested, all other components like the DAO layer should be mocked. In many cases this is done manually. That’s a second mistake.
By mocking stuff manually your mocks are thightly coupled to the implementation. It’s much better to use mocking frameworks. They are made for it. Just trust that they will create the mocks the way you want. Some mocking frameworks are capable of doing more than others, some are easier to use than others. My favourite mocking framework is Mockito because of its power and simplicity. EasyMock is by far the best known but imo a bit too complex to use. What was the sequence again? Expect, replay, assert, or …?
So it’s not only a matter of choosing a mocking framework. It’s also a matter of choosing the right one.
Integration tests
Integration tests are tests over different integrated parts of an application while unit tests only tests a LUW (logical unit of work). The best known type of integration tests are tests of the DAO layer. In these tests, multiple things are validated: the input parameters, the use of the ORM tool, the correctness of the generated query (functional), if the DB can be accessed, the fact that the query is working and the fact that the correct ResultSet is returned.
A third mistake that is made often is that developers provide test data in a dbUnit XML dataset. This data is most of the time no representation of live data. The best way to handle this is to make functional people provide and maintain this test data. Because you don’t want to make them struggle with a specific XML format, it’s a good practice to use the XlsDataSet format. Data can be provided in an Excel sheet. The name of the tab is the table, in the first row, each column contains the name of the column and in all other rows, effective data is added. Insertion into the DB will be done tab per tab, from left to right. Deletion is done the other way around.
Functional tests
Functional test are forgotten too often because the perception is that they are difficult and costly to maintain. It’s true that you’ll have to tweak them to make them maintainable but executing all the same tests manually over and over again will cost much more. Also when functionality is tested and automated by means of continuous integration, there’s no need for stress for screens/functionality that will be broken in the next release.
Selenium is a great framework that is able to record functionality in the browser. A fourth mistake is made here. Most people will just record and replay the tests the way they are. Selenium keeps tracks of HTML components by their HTML id. If you don’t specify id’s in your HTML code, they can easily change if a page is updated. This will break the tests immediately even when functionality still works.
A fifth mistake is made by just pasting the recorded test in a JUnit test. Common functionality like login and logout, the time to wait for a response, … should be added to a higher level. For example, a login can be done before each test, afterwards a logout is done. This way, if something changes in login or logout functionality, tests won’t break.
A sixth mistake is the absence of testing logical flows. It’s perfectly possible that an insert, a search, an update, and a delete work on their own but that the complete flow from the creation, search, update until delete of the same entry won’t work. TestSuites are a great way to group multiple tests and run them in sequence all together.
BDD
Mistake nr seven: most of the tests are technical tests instead of functional tests. It’s much more important to test logical behaviour than to test if the things work technically. That can be done by means of Behavior Driven Design.
Given some things, when an event occurs, then what should be the outcome? Mockito has some great stuff to easily write tests in a BDD way.
about:software development 