Anything for you, Merry and Philipa! Here's my theory on the true identity of Tom Bombadil, as exposited in my incarnation as Chantal of the Writers of Rohan.
I have wondered from time to time whether Tom Bombadil might not be a living personification of Eru's Secret Fire, based on the following little passage:
Eldest, that's what I am. Mark my words, my friends: Tom was here before the river and the trees; Tom remembers the first raindrop and the first acorn. He made paths before the Big People, and saw the Little People arriving. He was here before the Kings and the graves and the Barrow-wights. When the Elves passed westward, Tom was here already, before the seas were bent. He knew the dark under the stars when it was fearless -- before the Dark Lord came from Outside.
In this passage, Tom goes back in time, until he reveals that he was present in Arda even before Melkor, who was the first of the Valar to arrive. As far as I know, the only "entity" that arrived in Arda before Melkor was the Secret Fire that brought the inert planet to life. From the Ainulindale in The Silmarillion
. . . but Iluvatar called to them [the Ainur], and said: "I know the desire of your minds that what ye have seen should verily be, not only in your thoughts, but even as ye yourselves are, and yet other. Therefore I say: Ea! Let these things Be! And I will send forth into the Void the Flame Imperishable, and it shall be at the heart of the World, and the World shall Be; and those of you that will may go down into it." And suddenly the Ainur saw afar off a light, as it were a cloud with a living heart of flame; and they knew that this was no vision only, but that Iluvatar had made a new thing: Ea, the World that Is.
Thus, Tom's imperative is "to be" not "to change" or "to defy," which might help explain the paradox of his undeniable power juxtaposed with his seeming inability to be of help against Sauron. As Goldberry says, "He is." He can influence a living creature's state of being -- i.e., he can influence Old Man Willow to go back to sleep and release his prisoners -- but he cannot change their nature -- i.e., he cannot remove the evil that gnaws at the heart of Old Man Willow. He also has no sway over the weather -- it "is." This is why the Ring has no effect on him, and he can see through its magic (Frodo remains visible to him, even when the Ring is on Frodo's finger), but he cannot negate its evil.
Consider these statements about Bombadil from the Council of Elrond:
"But within those bounds nothing seems to dismay him [Bombadil]," said Erestor. "Would he not take the Ring and keep it there, for ever harmless?"
"No," said Gandalf, "not willingly. He might do so, if all the free folk of the world begged him, but he would not understand the need. And if he were given the Ring, he would soon forget it, or most likely throw it away. Such things have no hold on his mind. He would be a most unsafe guardian; and that alone is answer enough."
"But in any case," said Glorfindel, "to send the Ring to him would only postpone the day of evil. He is far away. We could not now take it back to him, unguessed, unmarked by any spy. And even if we could, soon or late the Lord of the Rings would learn of its hiding place and would bend all his power towards it. Could that power be defied by Bombadil alone? I think not. I think that in the end, if all else is conquered, Bombadil will fall, Last as he was First: and then Night will come."
"I know little of Iarwain save the name," said, Galdor; "but Glorfindel, I think, is right. Power to defy our Enemy is not in him, unless such power is in the earth itself. And yet we see that Sauron can torture and destroy the very hills . . . ."
This theory also suggests an interesting development, should Sauron have recovered the Ring and succeeded in conquering all of Middle-Earth. What would happen to Arda when Sauron and his forces finally overcome the last remaining power in Middle-Earth -- Tom Bombadil? I would postulate that the Dark Lord would suddenly find himself the master of a dead, inert planet. With the destruction of the living incarnation of the Flame Imperishable, the Secret Fire would return to Eru Iluvatar, and Sauron would be left with nothing.
Here's another interesting bit to consider along with my theory, also from the Ainulindale in The Silmarillion
To Melkor among the Ainur had been given the greatest gifts of power and knowledge, and he had a share in all the gifts of his brethren. He had gone often alone into the void places seeking the Imperishable Flame; for desire grew hot within him to bring into Being things of his own, and it seemed to him that Iluvatar took no thought for the Void, and he was impatient of its emptiness. Yet he found not the Fire, for it is with Iluvatar.
Would it not be the height of ironies if Melkor's greatest lieutenant Sauron could have succeeded in "conquering" Arda, only to have the Flame Imperishable -- his ultimate desire -- return to Iluvatar, leaving him with an empty, dead world -- yet another Void?