Sullivan attended Charleston's Garnet High School for blacks and received a basketball and football scholarship to West Virginia State College in 1939. A foot injury ended his athletic career and forced Sullivan to pay for college by working in a steel mill. He also worked as a part-time minister. During a visit to West Virginia, noted black minister Adam Clayton Powell convinced Sullivan to move to New York to attend the Union Theological Seminary. He also served as Powell's assistant minister at the Abysinnian Baptist Church. Sullivan took his first active role in the civil rights movement by helping to organize a march on Washington, D.C. in the early 1940s.
He moved to Philadelphia to take over the Zion Baptist Church in 1950. Under Sullivan's leadership the congregation grew from 600 to over 4,000 in just a decade. He also began organizing a civil rights movement in Philadelphia. Sullivan believed jobs were the key to improving African-American lives and asked that the city's largest companies interview young blacks. Only two companies responded positively so Sullivan organized a boycott of various businesses. The boycott was extremely effective since blacks constituted about 20 percent of Philadelphia's population. Sullivan estimated the boycott produced thousands of jobs for African Americans in a period of four years. Sullivan's work was recognized nationally and he was asked by Dr. Martin Luther King to organize boycotts in Atlanta in the early 1960s.
Sullivan realized one of the key employment problems for blacks was a lack of training for the changing job market. African Americans had been excluded from the types of training which led to better paying jobs. He formed the Opportunities Industrialization Center (OIC) to train and instill pride in African Americans. At the same time, Sullivan established the Zion Investment Association (ZIA), a company which invested in and started new businesses.
In the 1970s, Sullivan turned much of his attention to ending the system of apartheid in South Africa. Again, he looked to financial pressure to bring about change. Sullivan was the first black board member of General Motors and encouraged the company and other corporations to use their economic influence to end apartheid. In 1985, Sullivan demanded that South African blacks be given the right to vote within two years. When South African leaders failed to meet the deadline, Sullivan called for a multinational boycott. Companies from around the world pulled their businesses out and devastated the South African economy. In 1994, reform leader Nelson Mandela was elected president of South Africa in the country's first multiracial elections. Sullivan and Mandela are now working to encourage companies to return to South Africa.