Why do we limit ourselves to the inspired Psalms, excluding, for example, the inspired canticles found in the Scriptures outside the Psalter? The answer is that there is no clear indication in the Scriptures that such songs were intended by God for perpetual use in His Church. In the absence of such an indication, it would seem prudent to refrain from singing them in worship. For similar reasons we would also suggest that the practice of singing prose portions of Scripture in worship is wrong. The Holy Scriptures should not be subjected to uses which are foreign to their original purpose and design. The Hebrew title of the Psalter, which is simply “praises” (Tehilim), identifies the book as a series of songs to be sung in praise of God. (Songs of Zion, Michael Bushell, 4th edition, p.24,28)
The Holy Spirit is the one who organized the book of Psalms. He did not include every inspired song in the Bible within the Psalter. The fact that God did not place every inspired song within God’s hymnbook probably indicates that some inspired songs were only used on a specific occasion or for a limited period of time. Keddie writes, “It is contended that other inspired utterances such as the songs of Moses (Ex. 15:1-19; Dt. 32:1-43) and Hannah (1 Sam. 2:1-10) in the Old Testament, and of Mary (Luke 1:46-55) and Simeon (Luke 2:29-32) in the New Testament, were exceptional effusions of praise, of an inspired nature, in connection with particular (and even absolutely unique, as with the Magnificat) ‘acts of God’ and not necessarily for perpetual use in the Church’s song…the whole book of Psalms…is to be considered as the final[ised] hymnbook of the Church. It satisfies the demands of divine provision, and is the only collection of songs of praise as such which can lay claim to primary and verbal inspiration.”
(Adapted from Bryan Schwertley's Exclusive Psalmody: A Biblical Defense)
Related Objection: Why not versify any other portion of Scripture and sing it?
THE SCRIPTURAL LAW OF WORSHIP BY PASTOR WILLIAM H. VINCENT, D.D; ALLEGHENY, PA. THE PSALMS IN WORSHIP EDITED BY JOHN MCNAUGHTER HE WRITES, But some may say: Can we not versify and sing other portions of the Word, as the gospels, and thus sing the gospel? To this it may be said: (1) God's provision is to preach the gospel to the world, not to sing to it; (2) The Gospels are not lyrical, and were never intended for the praise service of the house of God; (3) There is but one book which God has labeled the book of praise either in the Old or New Testament Church, and that book is the appointed and authorized Book of Psalms. The crucial point which we emphasize in this discussion is: What has the divine appointment? By that standard every book of praise must stand or fall. That which has not the divine appointment has no standing in God's house. The book of Psalms not only has the seal of inspiration, but it has also the clear and unmistakable appointment of God. Hence it is the book of praise for the church of God in every age and in every land. As it has songs which relate not only to the past and present if the church, but to the church in the millennial age, and since the spirit of inspiration is withdrawn from the church, the Book of Psalms is designed to continue as the manual of praise in the church until the end of time.