At the moment i am part of the team that works on the Mansystems test framework (MTF). MTF provides users with a standard set of technical keywords that are specific for testing Mendix apps. Based on these, Mendix app developers can build their own app-specific keywords and share them within their development team and user community. At the moment a i am designing the set up and tear down actions in the framework. This task has prompted me do some research. I am reading materials online from some blogs which simply do not support the idea of tear down and set up. Some blogs really give a more elaborate argument on this discussing when to use these two. From the information i have gathered first of all let me jump to the conclusion that you need to have set up and tear down but lets look at one concept, simple design.
In Agile development, we are moved to obey the use of Simple design by avoiding obfuscation and keep our code clear, simple, and clean. The values of simple design are:
- Code is appropriate for its audience
- Code communicates its intention clearly to its audience
- Each concept is expressed in code once and only once
- Code is expressed in as few classes as possible
- Code is expressed in as few methods as possible
The mission for application code is different from the mission for test code. Application code embodies business value. Understandabilty is still critically important, but maintainability is pretty important as well.Test code defines the business behavior.it is simple, declarative in nature, and written in a standard Arrange/Act/Assert format. Test code is read frequently, as developers try to understand how the business logic works. Any mental energy spent understanding how the tests work detracts from the amount of energy available to understand the important stuff — the application. This code needs to live more towards the independent understandability end of the spectrum. Hence simple design values affect the way we write out tests and application.
The big question then, when should Setup and TearDown be used?
- Resource management.
- Environment where the application resides
- Clean up logic of resources
Considering #2 ,Imagine the clean up logic is part of your test case and you understand that if an assert fires you exit the test right then and there. so if your clean up logic came after an assert that fails in the test then your fancy cleanup logic will never be executed. Those with an appreciation for this fact have been known to encapsulate their tests with try/finally blocks, just to ensure that their cleanup happens when assert fails.
It is this logic that deserves to be factored out. The cleanup logic is not part of the context for understanding the test, it is part of the wiring that makes the test run.By moving it into Setup and TearDown, you leave just that code in the test methods that really do establish the context of the test.
Examples of this kind of logic would include:
- Resource management for opening and closing files, sockets, or other resources
- Resetting housekeeping variables defined, managed, and manipulated in the tests themselves
- Stuff you darn well want to make sure happens before and especially after each test.
So how does this affect our test framework implementation on Mendix? First of all it is easy to understand why we need set up and tear down on our framework. We need it because Mendix lincesing allows only two concurrent users to login. In short you cant use the same username to login more that 2 times in the current community license. Having Set up allows us to login once during a test case and having tear down allows us to logout once so that the app under test does not run into licensing issues due to multiple logins.
From my reading so far i break testing into 3 broad categories so that i keep the context for understanding the test.
- Set up
- Tear down
I would highly consider using set up and tear down only when it matters. i believe you should not use it when forces you to jump out of the context of your test to go and read some other part of code for you to understand the test for example variables which should have stayed local. The moment you find your self jumping out to read other positions of the code for you to understand the test then most likely you used set up and tear down wrongly.