I was just listening to my Sovereign Grace CD that says "trembling joy" in it. I think that the phrase is originally Baxter or something. I really like it.
[Edit: on further thought, I can't remember why I thought it was Baxter. Maybe it was Milton. At any rate, it looks like a lot of Puritans liked to say it. It'd be kind of fun to write a master's thesis on something like "Puritan heritage and the golden age of American English." Might be a stretch . . .]