1 PRAISE we to the Father give,
God in whom we move and live;
Children's praise He loves to hear;
Children's songs delight His ear.
2 Praise we to the First-born bring,
Christ the Prophet, Priest and King;
Glad we raise our sweetest strain
To the Lamb that once was slain.
3 Praises for the Holy Ghost
Sent from heaven at Pentecost;
'Tis through Him alone we live,
And the precious truth receive.
4 Blest our portion, thus to be
Glorying in the Trinity;
For the Gospel from above,
For the word that God is love.
What James Montgomery actually wrote was, "Glory to the Holy Ghost" .. see https://books.google.mw/books?id=0u1mqu0IOZEC&vq=glory%20to%20god%20we%20give&pg=PA452#v=onepage&q&f=false
Of course the hymn was heavily edited for the Brethren hymn books. It was only in the Taylor edition of 1951 that the wording to which Paul alludes to, first appears.