Haller was born at Aldingen in Württemberg. After schooling in Pforzheim, where he established a friendship with Philipp Melanchthon, he studied theology in Cologne. He became a teacher in Rottweil in 1510 and in Bern in 1513, where he was appointed assistant preacher at the church of St Vincent in 1515. In 1520 he became a canon and the people’s priest.
Even before his acquaintance with Huldrych Zwingli in 1521 he had begun to preach the Reformation, his sympathetic character and his eloquence making him and the painter and writer Niklaus Manuel a great force. Zwingli became his friend and adviser and they began a lively exchange of letters.
In 1526, Haller participated in the disputation of Baden, and in 1528 in the Bern disputation, which resulted in the Bernese Reformation edict on 7 February 1528 in which Bern officially decided for the Reformation.
Zwingli's 1531 death brought the Reformation in Bern to a crisis, to which the city council reacted by calling the first Bernese Synod with 200 participants. Haller was especially concerned as Zwingli's successor Heinrich Bullinger was unable to attend. However, he received strong support from Wolfgang Capito who arrived in Bern shortly before the opening of the Synod.
In 1532 Haller became the leader of the Reformed Church of Bern. He was in contact with Guillaume Farel in the western part of Switzerland and Heinrich Bullinger in Zürich and acted as a mediator between the Calvinist Reformation and Zürich.