A performance model will be full of assumptions and estimates, and yet it is necessary to make important design decisions and capacity choices based on its predictions. This means that it will be necessary to check that it is correct, known as validating the model.
The most valuable way to validate a performance model is to monitor a production environment and then to check that all of the predictions of the model are reflected by the production system within a reasonable tolerance. This allows all of the assumptions and mathematics to be checked in a single stage. The practice should be a matter of routine after the implementation of systems, and acts as an important operational performance assurance activity.
In the time before the initial implementation, however, it is only practical to check the technical aspects of the model using a set of performance tests. Thus, it is possible to test using a defined level of user load, and to make sure that the predictions of the model match the performance profile experienced to within an acceptable tolerance. If not, the reason for the differential needs to be investigated and resolved. It will usually be necessary to run a series of tests until the model’s accuracy is sufficient for the model’s use.
It is important to stress, however, that this form of validation still leaves a significant level of uncertainty about the level of demand to be placed on the system. It is worthwhile, therefore, using the model to predict the likely performance under a range of demand-profile assumptions. At least the impact of the level of unknown for the demand can then be investigated.
Sarquol Ltd 