Born in Vienna on 11 October 1901 as the youngest child of Karl Scholler. He began his studies at the Veterinary College of Vienna and continued at the University of Vienna, where he obtained his teaching degree in natural sciences and geography.

For political reasons, he was sent to the concentration camp at Dachau, which he was later permitted to leave due to illness. He continued his studies by obtaining his teaching degree, and in 1941, he concluded his mineralogical studies with a PhD. In 1948, he began working at the Mineralogical - Petrographical Department of the Natural History Museum. In 1951 he became a curate of the second grade, and 3 years later he became head of the State Institute of Gemstones. After the death of his predecessor, Alfred Schiener, in 1962, Scholler became head of the Mineralogical - Petrographical Department and two years later, he was appointed as Director of the same department.

Scholler’s most significant achievement was the new classification of the items comprising the collections, which had been badly neglected due to the turbulent times. He had a particular penchant for gems and decorative stones. After a long illness, he died on 28 April 1968 in Vienna.