03 January 2010

Simulation Model: Stage Four - Verification

Thinking about Pople's Quantum Chemistry Models (or in more general Simulation Models), where five stages may be distinguished as:

1) Target
2) Formulation
3) Implementation
4) Verification
5) Prediction

I got to thinking about the fourth stage:

4) Verification
The next step is to test the model against known chemical facts to determine whether the target has been achieved. If quantitative accuracy is being sought, this can be done by various statistical criteria such as the root-mean-square difference between the results of the theoretical model and experimental data. In selecting such a dataset, it is important to make it as broad as possible, while limiting it to experimental facts known to be of high quality. If the results of such a comparison do meet the target requirements, the model may be said to
be validated.

and how this applies to almost any other type of scientific endeavor.

Can stage four be skipped? In doing so how do we know stage five (Prediction) results are correct? How do we start problem solving if stage five results are incorrect? If stage five results are correct then how do we know that several wrongs did not make a right?

There are many unanswered questions and there could be even more depending on the situation.

Without stage four, I would not be confident in the results of stage five for doing predictive science. Could one start from scratch and reproduce my results without stage four? Perhaps but most likely it would be fortuitous and if several persons tried to reproduce the results then most likely my results would be found out to not be very reliable. Without each stage where is my accountability?

Interestingly enough, these five stages could be associated (or at least have direct analogies) with doing experiment too. To take it even a step farther, these five stages could be said to be the outline of the scientific method.

Science is built in steps and with each step confidence and reliability is built. Without stage four are we really doing science or are we just playing around wasting time and money?

No comments: