Simulation, Experimentation, and Calculation: Which is Best?

You’ve probably already thought about the answer to that question. I hope you at least believe that one of these three is essential—rather than relying solely on instinct or heuristics for all your engineering decisions. Most, however, haven’t thought about each of the three in enough detail to fully grasp the implications and limitations of each approach.

Ideally, at this point, you’re thinking about first principles. Indeed, each of these three represents a different way of applying first principles. So let’s consider how you can best use them for your first principles.
ExperimentationIt’s hard to argue with reality. And that’s what experimentation offers. If the experiment fails, it doesn’t matter if your calculations or simulations say it should work. Experimental outcomes are the ultimate judge.
The issue with experimentation from an engineering perspective is that it always "works"—even if you aren’t aware of what’s important.
You can’t choose to ignore or suppress key variables. They’re always present and always have a value, whether you’ve thought about them or not.
You might set all the key variables you think are important, but there are still others you’ve set inadvertently—because reality has already given them values. That means you might believe you’ve experimentally found a solution to your problem, but when you implement it, an issue arises. Why? Because you were unaware of a key variable—and its value during implementation differs enough from what it was during your experimentation to cause a failure.
Experimentation won’t alert you to your ignorance of key variables until it’s too late.
An example I mentioned in my book involves an engineer designing a device to control water flow for watering plants. Their experimentally developed design worked, but once the system was implemented, variations in temperature—and thus viscosity—rendered it useless. The engineer had conducted all the experiments at roughly the same temperature. Since they didn’t realize how temperature-sensitive viscosity is, they didn’t factor it into their tests—and reality had silently set that variable for them.
CalculationCalculations have the advantage of forcing you to account for all key variables. The formulae you use have been developed after considerable attention by experts who have identified the important variables at play. If the engineer in the earlier example had taken the time to read up on the theory and find the appropriate formula, they would have learned how critical viscosity is. Then, while looking up viscosity values to put in the formula, they would have seen how much viscosity changes with temperature.
Formulae also reveal opportunities for optimization. You can see which variables are raised to a higher power and thus offer more "bang for your buck." You can also work out whether variables should be increased or decreased to maximize your output—which isn’t always obvious. Sometimes, you can even deduce if an optimum point exists.
However, there aren’t always formulae available for your exact situation.
Consider again the water control device: what if it had an outlet orifice that was non-standard, and the engineer couldn’t find a discharge coefficient for it to plug into the formula?
Experimentation could be an answer—but it might be time-consuming if multiple variants had to be fabricated and tested.
SimulationObviously the newest of the three, simulation is almost like a mix of the other two.
Simulation can come very close to reality—assuming it’s a well-developed system—and it can force you to specify all variable values, forcing you to note all those at play.
However, some simulation systems “help” you by asking you to specify a material instead of individual material properties. Thus, you might find yourself back in the same situation: unaware of all the important variables, and unaware of which ones are best to adjust for optimization.
Also, simulations still rely on limiting assumptions. Often, we must simplify systems for the sake of usability. So your simulation might not be a perfect representation of what you’re actually working on.
While simulation can offer tremendous benefits when it comes to testing ideas and improving systems, it’s still not a silver bullet.
What to Do Then?It’s likely clear to you by now that you need to use all three—experimentation, calculation, and simulation—if you want the insights, speed, and confirmation needed to find the optimum solution. Beware of any engineer who suggests you should focus on one and ignore the others.