Albert Cassell was born in Towson, near Baltimore, Maryland, the third child of Albert Truman and Charlotte Cassell. He finished his elementary and high school education in Baltimore and in 1919 received a B.A. degree in architecture from Cornell University, where he sang in churches to help pay his expenses. His studies were interrupted by service as a second lieutenant, training officers in heavy field artillery in the United States and France during World War I (1914–1918).