When considering data analysis questions, I often think of this passage from “The Wizard War” by R.V. Jones, head of British scientific intelligence during World War II.
One salutary incident that I recall concerned a British mathematical physicist and an American theoretical physicist, Dr. Charles Kittel, who had been set together, side by side, to work on the problem of deducing the characteristics of German magnetic mines laid at sea, especially the sensitivity and polarity of the firing mechanism. The data from which the characteristics were to be deduced were the reports of our minesweepers as they exploded the mines, with the positions of the explosions being reported as ranges and bearings from the minesweepers. The first thing that Kittel wanted to do was to take a few trips on a minesweeper to sample the data for himself. The British theorist refused to do this, on the argument that he could only make a few trips, and therefore any experience so gained might be heavily biased, and therefore much too dangerous as a basis for generalization. So he stuck to his desk while Kittel went out minesweeping. What Kittel immediately found was that the reports from the minesweeping crews were wildly inaccurate as regards both range and bearing, and the only item of data on which one could rely was whether the mine had exploded to port or starboard. Simplifying all the later reports down to this extremely limited observation, he nevertheless succeeded in deducing the answer; but the British theorist went on accepting the data as accurate and never reached an answer.