"Every man has a certain sphere of discretion which he has a right to expect shall not be infringed by his neighbours. This right flows from the very nature of man."