Personal opinion on software development in general, development methodologies, tools and best practices in particular

Archive for the ‘Testing’ Category

7 mistakes of software testing

Though most developers know the importance of testing, it seems that a lot of them still aren’t testing enough. And if they write tests, they test just test wrong.

Unit tests

If some tests are written in applications, most of them will be unit tests. It’s easy to test a utility class by just calling all utility methods, passing some values and checking if the expected result is returned.

A first mistake arises here. Most people don’t think out of the box, or not enough. You can test that 1 + 1 =2 , that 2 + 1 = 3 and that 3 + 1 = 4. But what’s the benefit of doing almost the same test 3 times? It’s better to test the boundary cases. Are the arguments of the sum( ) method primitive types or Objects? If they are Objects, what happens if you pass null values? If an exception is thrown, is that the expected one? Does it clearly tell what the problem is?

Mocks

If for example the service layer is unit tested, all other components like the DAO layer should be mocked. In many cases this is done manually. That’s a second mistake.

By mocking stuff manually your mocks are thightly coupled to the implementation. It’s much better to use mocking frameworks. They are made for it. Just trust that they will create the mocks the way you want. Some mocking frameworks are capable of doing more than others, some are easier to use than others. My favourite mocking framework is Mockito because of its power and simplicity. EasyMock is by far the best known but imo a bit too complex to use. What was the sequence again? Expect, replay, assert, or …?

So it’s not only a matter of choosing a mocking framework. It’s also a matter of choosing the right one.

Integration tests

Integration tests are tests over different integrated parts of an application while unit tests only tests a LUW (logical unit of work). The best known type of integration tests are tests of the DAO layer. In these tests, multiple things are validated: the input parameters, the use of the ORM tool, the correctness of the generated query (functional), if the DB can be accessed, the fact that the query is working and the fact that the correct ResultSet is returned.

A third mistake that is made often is that developers provide test data in a dbUnit XML dataset. This data is most of the time no representation of live data. The best way to handle this is to make functional people provide and maintain this test data. Because you don’t want to make them struggle with a specific XML format, it’s a good practice to use the XlsDataSet format. Data can be provided in an Excel sheet. The name of the tab is the table, in the first row, each column contains the name of the column and in all other rows, effective data is added. Insertion into the DB will be done tab per tab, from left to right. Deletion is done the other way around.

Functional tests

Functional test are forgotten too often because the perception is that they are difficult and costly to maintain. It’s true that you’ll have to tweak them to make them maintainable but executing all the same tests manually over and over again will cost much more. Also when functionality is tested and automated by means of continuous integration, there’s no need for stress for screens/functionality that will be broken in the next release.

Selenium is a great framework that is able to record functionality in the browser. A fourth mistake is made here. Most people will just record and replay the tests the way they are. Selenium keeps tracks of HTML components by their HTML id. If you don’t specify id’s in your HTML code, they can easily change if a page is updated. This will break the tests immediately even when functionality still works.

A fifth mistake is made by just pasting the recorded test in a JUnit test. Common functionality like login and logout, the time to wait for a response, … should be added to a higher level. For example, a login can be done before each test, afterwards a logout is done. This way, if something changes in login or logout functionality, tests won’t break.

A sixth mistake is the absence of testing logical flows. It’s perfectly possible that an insert, a search, an update, and a delete work on their own but that the complete flow from the creation, search, update until delete of the same entry won’t work. TestSuites are a great way to group multiple tests and run them in sequence all together.

BDD

Mistake nr seven: most of the tests are technical tests instead of functional tests. It’s much more important to test logical behaviour than to test if the things work technically. That can be done by means of Behavior Driven Design.
Given some things, when an event occurs, then what should be the outcome? Mockito has some great stuff to easily write tests in a BDD way.

Category:

Quality, Software Development, Testing, Uncategorized

Focus on quality

Why do a lot of people say that testing is a waste of time? Because it takes time to write software tests? Hmm looks kinda strange don’t you think?

Software analysis, design, programming, bugfixing,… are all time consuming activities that are part of the application development process and it’s accepted that those activities take a lot of time so no one ever thinks about skipping them. So why would people skip testing? Because they like bugs? Because they hate visibility? Because they like bad quality?
Oh now I remember, because tests are obsolete and cost time and money. And we’re getting out of time or out of budget so we skip tests. Big mistake!

Tests are also part of the development process! Why? For many reasons!

Tests create visibility and a safety net. By writing tests you can pinpoint bugs as soon as possible. And you can fix them as soon as possible.
Tests reduce the amount of bugs in new and existing features. Because we see bugs by writing tests, we can fix them and reduce the amount of bugs.
Tests allow refactoring. Because there’s a safety net, you can refactor without fear. Tests will be broken if the refactoring failed.
Tests reduce the cost of change. Because you see and fix bugs as soon as possible, you don’t introduce extremely long bugfixing phases. The longer the bugfixing phase, the more money that will be spent to fix the bugs. The graph below explains everything.

Most bugs are introduced in the coding phase and this phase is the cheapest one to repair bugs in. So we can conlude that quality is what we should focus on. You can do this by writing enough tests but there are other ways to do it:

Don’t overdesign. Of course you should think about the design first but don’t program with the idea that maybe this feature could be useful. A software application is not meant to exist of several selfmade subframeworks that contain unused features. Thinking about those features and writing them is a waste of time in fact.
Design as late as possible. If you write some code, do not think about reuseablity at that moment. Extract the code at the moment you wrote the same code twice.
Refactor as much as possible. When time’s ready, refactor code to keep it clear, readable and maintainable. Code duplication is a bad practice.
Use tools to track possible bugs. Findbugs and PMD are very useful.
Make use of continuous integration. CI will tell you that something will be broken, so if CI is green, you don’t have to bother about it.
Do code reviews of each others code. Every developer has an other point of view and will detect bugs and bad coding practices that you didn’t notice because it’s your code.

So if you don’t like to focus on quality but on costs, remember that by focusing on costs, quality won’t improve. No, quality will even be worse, meaning that it will even be more expensive to guarantuee a product without major bugs and thus costs will raise by fixing a lot of bugs at the end of development! If you focus on quality from the beginning, costs will be reduced and quality will even get better.

An easy formula can illustrate this: