If someone says, "I think your AI safety scheme is horribly flawed because X will go wrong," and you reply "Nah, X will be fine because of Y and Z", then good practice calls for highlighting that Y and Z are important propositions. Y and Z may need to be debated in their own right, especially if a lot of people consider Y and Z to be nonobvious or probably false. Contrast Emphemeral premises. Needs to be paired with understanding of the Multiple-Stage Fallacy so that listing load-bearing premises doesn't make the proposition look less probable - $~$\neg X$~$ will have load-bearing premises too.
Comments
Ryan Carey
This is a fair thing for more perfectionistic researchers to ask from pragmatists.
One thing that pragmatists can fairly ask of perfectionists is "flag the obvious, imperfect solution". Often, if you don't point out that some hacks are unsatisfactory, people will think you haven't noticed them at all, or will try to build new solutions from them, and besides, you could also be mistaken about their satisfactoriness.