He was born on 19 April 1836 in Littau bei Olmütz, the son of Ignaz Czermak, who was a tax official. After finishing school in Olmütz, Tschermak began pursuing mathematical and chemical studies at the University of Vienna, and concluded these with a teaching degree. He concluded his mineralogical studies at the Universities of Heidelberg and Tübingen resulting in his obtaining his PhD. Upon his return to Vienna, he qualified as a private lecturer for mineralogy and chemistry. In 1862, Tschermak was appointed second vice curator of the Imperial Mineralogical Cabinet. In 1867, he advanced to first vice curator. In 1868, Tschermak took over at the helm of the Cabinet after the death of Director Hoernes. Only a short while prior to this, he was made Professor of Petrography at the University of Vienna. Tschermak was instrumental in undertaking a detailed inventory of the Imperial Mineralogical Cabinet which was subsequently run in an exemplary fashion by Rudolf Koechlin along a system which is maintained even today. One of his other major accomplishments on behalf of the Cabinet was the systematic expansion of the meteorite collection.