"The hopeless grief of those poor colored people affected me more than almost anything else."