Structuralist philosophy of science in its contemporary guise is committed to three core theses: first, that science makes progress; second, that it is structure that is ontologically fundamental; and third, that our metaphysics must be informed by science if it is to have any value. But these three theses give rise to an obvious tension, given that we as yet lack a fundamental physics theory that can inform the claims that lie at the heart of its metaphysics. To resolve the tension, one might hope that we can regard metaphysics based on merely pro tem fundamental physics to at least be making progress toward the description of the truly fundamental level. But I will argue that any such notion of progress cannot be analogous to that which science enjoys. At the root of this is the fact that structuralist metaphysics, for all its naturalistic credentials, is in fact a form of analytic metaphysics, and the categories of the latter have an all-or-nothing character that makes them intrinsically unreceptive to any meaningful notion of approximation. However, with this now in plain sight, we are better positioned to imagine what a structuralist metaphysics have should looked like all along – a metaphysics, that is, that is tolerant enough to undergo progress as well as merely suffer change.