mardi 29 janvier 2019

Which OO design patterns are relevant for data storage for big suites of test cases?

When creating a test suite with very different tests and many independent modules to test, what is a good structure for the test data?

I am working in a project where we develop automated tests of an ERP system and we are having a discussion on the team about how to structure the data that will be used for testing.

We test things like creating a new customer, new order, new products, new supplier, verify a delivery and so on. In most cases these events take place in more or less unrelated components of the system (and in some cases even external systems). Of course, there are many different kinds of customers, products (e.g., products that are combined), orders (e.g., return orders), delivery (e.g., pickup at the company, delivery to the customer)

The team leader wants one big tree based text file with one branch for customer, one for products etc. You should then, if you have a test case X that needs one customer and one product, refer from the customer to the product. Currently, we reuse these "leafs" so if you in another test case Y needs the same product as in X you use the same product.

IMO this is a recipe for a disaster. This text file is gonna be HUGE and, as a consequence, impossible to overview and people are gonna change "objects" to make them fit their special need without consideration of how it affects other people using the same object.

Therefore I am looking for design patterns (or other ideas) for how to organize test data.

Typically the code is organized into one helper class per component that contains functions frequently used in that component and then small classes that calls functions in this helper class as well as contains some a couple of "inner methods" only needed for that specific test case.

A typical test case needs 10-20 "properties" for each element it uses (if a test case concerns one customer and one product there are 10-20 strings that describes the customer and 10-20 strings that describes the product).

I think I am looking for arguments for a system with decentralized test data. Basically I think I think there should be one physical file for each test case with the properties needed in that specific test case. There will very likely be a lot of duplicate data (e.g., we will probably use the same customer with the same address details for many test cases).

A disadvantage with such a "pattern" is that if you need to change the customer number you will need to do it in hundreds of physical files (but on the other hand, if this need ever arises, I think it would be better to change the data in the database and leave these property files as is). OTOH, an advantage with such a solution is that the tests are self contained. Everything you need is in one or two directories and if you modify something there it only risks breaking one test and you know exactly which test that is a risk. No inter-dependencies that is confusing and hard to track.

Aucun commentaire:

Enregistrer un commentaire