Testing Functional structure models: It is often hard to tell whether the functional model structure chosen (which almost always in published work appears to generate consistent and robust results) is the only one tested or not.
Leamer (1983) has argued that good method should require that authors report how many regressions they undertook (and the functional forms subsequently rejected) before they found the one they chose to report. Leamer is particularly concerned that authors often will do hundreds or thousands of regressions (involving an array of functional forms and manipulations of assumptions and data) before they find one that offers statistically significant results. He believes that presenting only the one that worked, instead of talking about the hundreds or thousands that didn’t work is incomplete reporting and can lead to spurious results or at least misapplied confidence in the results.
He illustrates using an example of fertilizer usage on farms that multiple functional forms can work (i.e. a linear relationship or a quadratic relationship with either increasing or decreasing returns to scale). In many cases there is not enough data (or degrees of freedom) to properly test the functional forms and select among them (what he calls the “identification problem”).
He believes the job of any researcher is “to report economically and informatively the mapping from assumptions into inferences”, identifying which forms are accepted or rejected and why. By this he hopes researchers can reduce the “whimsical character of econometric inference.”