How to Test Without Access to the Test Environment

In many of our previous articles, we have expressed the importance of achieving a high standard of testing. Potentially blocking this achievement, several factors can come together to affect the quality of your testing, factors that include your test environment.

As the virtual office that forms an essential part of any testers working day, a well-built test environment will allow a developer to simulate the conditions that a system will, and could, experience once moved into operation. However, it is not unusual for a developer to be without access to their test environment.

It may sound bizarre, but there are a great many reasons as to why access to a test environment may be limited or not feasible, for example:

  • The required hardware is still in development alongside the software, and as such is not ready for testing.
  • Due to their tendency to be expensive, it may not be feasible to provide every team member with a development prototype.
  • The hardware involved is simply too large, either physically or in terms of quantity to fit into an office space.
    • Physically large – a train and/or carriage
    • Large in quantity – a train CCTV system comprising of a multitude of cameras
  • Certain industries and systems will have security restrictions in place that could form a barrier to the test environment.
  • Where working as one of many subcontractors, an agreed interface may be used until subsystems are brought together towards the end of the project
  • In addition to all the issues associated with lack of access, sometimes it can be easier to test elements of an embedded system within a non-target environment, such as the PC based testing within the development environment.

So what happens if a tester were to find themselves without ready access to this space?

Mock Environments

Looking at a dictionary definition of the word mock (noun) you should expect to see something along the lines of ‘something made as an imitation’ or ‘not real but appearing or pretending to be exactly like something’.

In the case of mocking, you are attempting to create an object or unit, that imitates the behaviour of a real object or unit. To put it simply, if your software has a dependency on other pieces of software, you can mock those interactions to establish if your code is reacting as expected to both valid and invalid inputs. This last point is important to highlight here as it shows one of the major advantages of mocking. When it comes to testing in the real world, with the actual hardware or 3rd party software, it can be difficult to create invalid conditions, but by using mocking you are able to ensure that these situations are dealt with gracefully.

In regards to the focus of this article, primarily used in unit testing, mocking can provide an interim means to exercise code within a development environment. After all, it shouldn’t matter if behaviours are being faked, code should react the same way to faked/mocked inputs as it would to those that are real. Catching bugs and issues during Unit testing also saves time later in the project as debugging issues at the system level can be more time consuming.

Simulators

Returning to the dictionary once again, a simulator is defined as ‘a piece of equipment that is designed to represent real conditions’. In the real world, pilots have simulators to mimic the experience of flying without the risk of damage from a crash. In software, most commonly in system testing, simulators will act like the various subsystems that your software will have to communicate with.

Designed to supply a realistic interface, a simulator will mimic the various interfaces in such a way that your software will not know it is talking to a simulator. As far as it is aware the software is in a live environment, and as such is receiving real messages.

For example, for a train control system (TCS), if your code were to send out a request to increase motors to x%, you would expect two things to happen. One, the simulator returns a realistic response to the request that is either positive or negative. Two, that your code responds appropriately to the simulators response or, in the worst case scenario, that it fails safely. Of course, this is an oversimplification of a very complex system, only focusing on one specific interaction between two interfaces. In reality, a simulator for a TCS would have to replicate a multitude of subsystems to ensure a realistic environment.

Mocking and simulators are cost-effective solutions to the issue of test environment access, however they will not totally replace testing on real hardware, or interfacing with real systems and users. They have a time and a place all throughout the life of a project, and even into the operational maintenance carried our post delivery, but there is still a need for testing in the actual environment. It is also worth considering the added bonus that simulators can double up as training tools to be used as part of training environment set up.

Could we be of assistance?

Looking for support with testing requirements, particularly in regards to the development of mock software and simulators? Get in contact with our team here at Zircon, and it would be our pleasure to see how we can assist you.

Email Us
Phone Us