Saptashrungi Devi is believed to be Mahishasur Mardini, the slayer of the demon Mahishasur, who took the form of a buffalo. Hence, at the foot of the hill, from where one starts climbing the steps, there is the head of a buffalo, made is stone, and believed to be that of the demon.
It is believed that the Devi Mahatmya, a sacred book which extols the greatness of Devi and her exploits was composed at this place by the sage Markandeya, who performed rigorous penance on a hill opposite the one on which the Devi resides, which is now named after him.