Webb was born in Baltimore, Maryland, to William H. and Marie Webb. The year of his birth is disputed. The Encyclopædia Britannica and Allmusic indicate 1905, and this seems to be supported by census information.
He alternated between band tours and residencies at New York City clubs through the late 1920s. In 1931, his band became the house band at the Savoy Ballroom. He became one of the best-regarded bandleaders and drummers of the new "swing" style. Drummer Buddy Rich cited Webb's powerful technique and virtuoso performances as heavily influential on his own drumming, and even referred to Webb as "the daddy of them all". Webb was unable to read music, and instead memorized the arrangements played by the band and conducted from a platform in the center. He used custom-made pedals, goose-neck cymbal holders, a 28-inch bass drum and other percussion instruments.
In November 1938, Webb's health began to decline; for a time, however, he continued to play, refusing to give up touring so that his band could remain employed during the Great Depression. He disregarded his own discomfort and fatigue, which often found him passing out from physical exhaustion after finishing sets. Finally, he had a major operation at Johns Hopkins Hospital in Baltimore in 1939. Webb died from Pott disease on June 16, 1939, in Baltimore. Reportedly his last words were, "I'm sorry, I've got to go." Webb was buried in Baltimore County, in Arbutus Memorial Park, in Arbutus, Maryland.