Schwertberg Castle´s was first officially mentioned in 1327. Since 1911, the castle has been in the hands of the Counts of Hoyos.
The castle was erected on a riff on the banks of the River Aist in the second half of the 13th century. In 1563, Georg Erasmus of Tschernembl was among the Province´s most important politicians and spokesmen of the Protestants. The Counts of Tschernembl had the Castle extended and augmented in splendid Renaissance style.
Over the centuries, several notable people have owned or been linked with it. Held by the Noblemen of Meggau, Starhemberg and Kueffstein, the castle was then granted to the Thürheim dynasty in 1749. Since 1911, the castle has been privately owned by the Hoyos dynasty.