For interactive web apps and APIs exhaustive testing comes from a combination of internal regressions and heavy external usage. The larger the client base, the further the potential code coverage exceeds any potential internal tests. Roll out strategies ease into releases by slowly propagating changes throughout the network of clients, until stable confidence is achieved.
But for specialized system simulations the user base may be drastically smaller, as small as a solitary user. The strategies for testing are different, and often only iterative usage over time can promote confidence in the software.
A colleague out west is in the process of setting up test cases for software I wrote over the past several months. Much of the local testing I completed while developing the models and glue was based on a straight forward scenario where I could easily visualize and verify the correct system state at each processing step. To keep my development anchored I had a baseline was generated by mature external programs, and was iteratively compared to intermediate outputs.
The process of validating code is equal parts art and science. Science is leveraged to compare data products to what is known either from external sources or engineering judgement, and art is leveraged to judge what is unknown and to efficiently sample a complex nonlinear parameter space. Resilient code isn't just written, it's surface is smoothed by the coarse winds of regular usage.
The model code I recently delivered is going to be run through the paces and shaken down to mitigate any lingering bugs and identify any limitations which I haven't already documented. I offered up a rough set of runs that will further validate the models as well as a brief mention of edge cases.
The edge cases I referred to weren't what the current model was designed to handle, but the limits of what it's capable of representing. I suggested that my colleague investigate the breaking points of the model after he's satisfied with the baseline tests, so that analysts will have a basic understanding of the model's capabilities. Hyper specialized systems simulations rarely allow for broad testing beyond the transient efforts that they support. Odds are that edge case testing will have to wait until after the current version is stamped and released.