"This whole phenomenon of the diversion of organizations from their purposes and ideals does not seem very serious when the scum rise to the top in the bridge club or the offices of a small magazine publisher."