[New? Start here: https://meditationstuff.wordpress.com/articles/]
These quotes are money and not by me:
[…Y]ou have to enter into the formalism while retaining awareness of the ontological context it supposedly represents: you have to reach the heart of the conceptual labyrinth where the reifier of abstractions is located, and then lead them out, so they can see directly again the roots in reality of their favorite constructs, and thereby also see the aspects of reality that aren’t represented in the formalism, but which are just as real as those which are.
Also keep in mind that it’s more important to make your beliefs as correct as possible then to make them as consistent as possible. Of course the ultimate truth is both correct and consistent; however, it’s perfectly possible to make your beliefs less correct by trying to make them more consistent. If you have two beliefs that do a decent job of modeling separate aspects of reality, it’s probably a good idea to keep both around, even if they seem to contradict each other. For example, both General Relativity and Quantum Mechanics do a good job modeling (parts of) reality despite being inconsistent and we want to keep both of them. Now think about what happens when a similar situation arises in a field, e.g., biology, psychology, your personal life, where evidence is messier then it is in physics.
For Bayesian methods to even apply, you have to have already defined the space of possible evidence-events and possible hypotheses and (in a decision theoretic framework) possible actions. The universe doesn’t come pre-parsed with those. Choosing the vocabulary in which to formulate evidence, hypotheses, and actions is most of the work of understanding something. Bayesianism gives you no help with that. Thus, I expect it predisposes you take someone else’s wrong vocabulary as given.
All these back and forth comments superimpose into something pretty good (hat tip a lesswrong open thread):