As we all know, the axiom of foundation (loosely speaking, that there do not exist sets with infinitely descending membership chains in their transitive closure, but usually formulated in a manner apparently custom-designed to confuse one's students) is a standard part of the machinery of ZFC. The question is: Why?
The answer, of course, depends on the role that we think ZFC should, and must, play. Here are some options:
Option 1: Foundation blocks the paradoxes, by ruling out the existence of non-well-founded sets such as the Russell set.
Worry: This is just insane. Full ZFC is, in some intuitive sense, more likely to be inconsistent for including foundation. In other words, if full ZFC (including foundation) is consistent, then ZFC-minus-foundation is consistent. (Another way to see the point: Adding foundation to Frege's Basic Law V, or any other formulation of set theory, doesn't make the resulting theory consistent - in fact, it might make contradictions easier to prove!)
Option 2: We are attempting to formulate the one true description of the unique set-theoretic universe. This universe is given to us informally by the iterative conception (as described by Boolos and others). Foundation follows on this conception.
Worry: If this is right, then one wonders why we also include the axiom of replacement (which, it seems to me, and seemed to Boolos, does not follow on the iterative conception of set). Also, one might wonder why the iterative conception is privileged in this way. There are other conceptions of set, including but not limited to the limitation-of-size conception, that don't support foundation.
Option 3: We are looking for the 'safest' reasonable looking set theory that will support the reconstruction of current mathematics.
Worry: Setting aside what 'safe' and 'reconstruction' amount to, we need some evidence that foundation plays any ineliminable role in such a reconstruction. After all, we could have a set theory that admitted various sorts of non-well-founded sets (either Aczel style or Forster style), and then just restrict our attention to the hereditary sets (those that can be built up from the empty set, loosely - see Boolos' stuff for exact details) and proceed as before. If there are no reasons for thinking we need foundation for our reconstructions, then any reasonable notion of safe will surely rank ZFC-minus-foundation as safer than ZFC.
Option 4: We are looking for the most powerful consistent set theory.
Worry: Well, this depends on what one means by 'most powerful'. Aczel's non-well-founded set theory is (if I am remembering correctly) equiconsistent with standard ZFC, but it admits far more sets. So why isn't this more powerful? At any rate, this just means we want one of either ZFC or ZFC-minus-foundation-plus-not-foundation (since either of these is more powerful than ZFC-minu-foundation), but doesn't seem to select between them.