"The power of the sword, say the minority... is in the hands of Congress."