He grew up playing in the family band in which he sang and danced before switching to piano at the age of 12, then to double bass when he was 14. He is quoted as saying he did not like the way people were playing the bass so he developed his own way of playing it. Despite being admired by the likes of Milt Hinton at the age of 14, he gave up in 1941 as he did not believe he could make a living. Five months later, he once again met Hinton, who persuaded him to return to music.
In 1942 he joined the Charlie Barnet band and in 1943 gained wider public attention after recording with Coleman Hawkins on his "The Man I Love". Pettiford also recorded with Earl Hines and Ben Webster around this time. After he moved to New York, he was one of the musicians (together with Dizzy Gillespie, Thelonious Monk, Kenny Clarke) who in the early 1940s jammed at Minton's Playhouse, where the music style developed that later was called bebop. He and Dizzy Gillespie led a bop group in 1943. In 1945 Pettiford went with Hawkins to California, where he appeared in The Crimson Canary, a mystery movie known for its jazz soundtrack, which also featured Josh White. He then worked with Duke Ellington from 1945 to 1948 and for Woody Herman in 1949 before working mainly as a leader in the 1950s.
As a leader he inadvertently discovered Cannonball Adderley. After one of his musicians had tricked him into letting Adderley, an unknown music teacher, onto the stand, he had Adderley solo on a demanding piece, on which Adderley performed impressively.