The Supreme Court justice made his remarks in an address to the Federalist Society.