Gray was born to Lurdie P. and Agnes (Gray) Knoblauch in Metamora, Illinois. His father was a saloon keeper and railroad worker who died when Glen was two years of age. He had an older sister. His widowed mother married George H. DeWilde, a coal miner, and moved her family to Roanoke. Gray graduated from Roanoke High School, in 1917 where he played basketball and acquired his nickname, "Gordo".
Gray attended the American Conservatory of Music in 1921 but left during his first year to go to Peoria, Illinois, to play with George Haschert's orchestra. From 1924 to 1929, he played with several orchestras in Detroit, Michigan.
Gray served as leader of the Casa Loma Orchestra though the orchestra itself had been formed as a collective group, with no designated leader. Their mid-1930s appearances on the long-run radio comedy-variety program, the Camel Caravan (introduced with their theme, "Smoke Rings") increased their popularity. Gray chose not to conduct the band in the early years, playing in the saxophone section while violinist Mel Jenssen acted as conductor. In 1937, the band overwhelmingly voted in favor of Glen leading the orchestra, and Gray finally accepted the job. By the mid-1940s, Gray would come to own the band and the Casa Loma name. For a time, during this period, the band featured guitarist Herb Ellis, trumpeter Bobby Hackett, pianist Nick Denucci and cornetist Red Nichols.
By 1950, the Casa Loma band had ceased touring, and Gray retired to Massachusetts. The later recordings on Capitol Records (beginning with Casa Loma in Hi-Fi in 1956 and continuing through the Sounds of the Great Bands series) were done with Gray leading a group of studio musicians in Hollywood (though several of Gray's "alumni" occasionally featured.) In all, some 14 high-fidelity and stereo recordings were made for Capitol under the name of Glen Gray and the Casa Loma Orchestra before Gray's death in 1963.
Gray and his wife had one son.