He was born in Frankfurt, and studied composition under Arnold Schoenberg in Vienna.
He gained experience as a conductor in Germany and Switzerland, as well as chief conductor at the Vienna Volksoper.
Just before the 1938 Anschluss, Herbert, then in Japan to introduce modern western music, emigrated to the United States, and became an American citizen in 1944.
Herbert was director of Opera in English (San Francisco), and in 1943 was appointed the first general director of the New Orleans Opera Association.
He proceeded to found the Houston Grand Opera in 1955, where he remained as both general director and conductor until 1972, and was music director of Opera/South (which was founded by Sister M. Elise Sisson, SBS) in Jackson, Mississippi.
He founded the San Diego Opera in 1965, and became its general director and conductor from 1969.
Herbert worked with most of the great singers of his day, and gave many of them their start in the opera world.
In April 1948, during Herbert's régime in New Orleans, the young Mario Lanza made one of his very few operatic stage-appearances, as B.F. Pinkerton in Madama Butterfly, under Herbert's bâton, and Armando Agnini's stage direction. It was announced that he had accepted the role of Alfredo Germont, in the New Orleans Opera's La traviata, in May 1948, but Hollywood apparently interfered with those plans.
Herbert was a member of the Austrian Team that won the first World Championship in contract bridge in 1937, as well as European Championship in 1933 and in the middle, a bronze in 1934.
He was credited with devising the Herbert convention.