White graduated from Atlanta University in 1916 and went to work for the Standard Life Insurance Company. Angered by the plan of the city's Board of Education to end public schooling for blacks after sixth grade, he became active in efforts to form an Atlanta branch of the N.A.A.C.P. , which impressed James Weldon Johnson, then the organization's field secretary. In 1918, at Johnson's recommendation, White was invited to join the national organization in New York as assistant secretary. Some of White's early duties included investigating race riots and lynchings, a dangerous assignment for which he exploited his ability to pass as white. When Johnson became the first African- American executive secretary of the N.A.A.C.P. in 1920, he and White together firmly established leadership and expanded the branch system of the organization. In 1929, while Johnson was on leave, White served as acting secretary of the organization. During that time he directed the efforts to block the confirmation of John J. Parker, a segregationist judge from North Carolina, to the U.S. Supreme Court. In 1931, his appointment to succeed Johnson as executive secretary of the N.A.A.C.P. was made official.

By strengthening the legal arm of the N.A.A.C.P. , White was able to direct campaigns for federal civil rights legislation in the form of anti-lynching laws, voting rights, laws banning poll taxes and discrimination in the U.S. armed forces, and laws in favor of desegregation and the equalization of schools. In 1937 White received the Spingarn Medal for his investigations of lynchings and lobbying for the anti-lynching bill (defeated by a narrow margin in 1938). He was instrumental in the formation of the Joint Committee on National Recovery to fight discrimination in the New Deal programs, advised President Roosevelt on the executive order for Fair Employment Practices during World War II, and forged a relationship with labor unions. In 1945 and 1948, White served as a consultant to the U.S. delegations to the newly formed United Nations.

In addition to numerous articles and two syndicated newspaper columns, White wrote several books, including two novels, the Fire in the Flint (1924) and Flight (1926); works of non-fiction, Rope and Faggot (1929) and How Far the Promised Land? (1955); and an autobiography, A Man Called White (1948).

White married his first wife, Gladys Powell, in 1922. The Whites' apartment at 409 Edgecombe Avenue, to which they moved in 1929, was known as "The House of Harlem" because of the prominent and important figures who were guests there. The Whites, who divorced. resided at 409 until 1947, and their daughter, the actress Jane White, lived there until 1961. Walter White married his second wife, Poppy Cannon, in 1949, with whom he was living at 242 East 68th Street at the time of his death.