This conversation deals pretty directly with your questions.
No, it's not certain that the Throat of the World is the physical echo of the Snow Tower. But the Tower's alternative name, Snow-Throat, certainly reinforces the perception that the two are linked.
"The cave" could be construed liberally. Could be something in the cave, could be a metaphor for something else, could be the cave itself moves around, etc. The Chantry of Auri-El, Labyrinthian, and High Hrothgar are all good candidates for places which may have some history and association with the Snow Tower's stone.
Ultimately, I think we're missing too many chapters from this particular story to understand it. Who could've predicted the convoluted story behind Green-Sap, for example? We don't know the who, what, where, when, or why relevant to its history. Even if we could extrapolate a couple important events related to it, we'd still be in the dark. I think Brian Regan said it best: "Adolf Hitler was rejected as a young man on his application to art school... One thing led to another... And the United States dropped two atomic bombs on the sovereign nation of Japan."
It's likely that the Snow Tower's inception dates back to the Dawn Era or the very early Merethic Era, when the world still existed as a convoluted mess which defied cause and effect.
I don't think it's a big leap to guess that the two caves are one and the same. However, before ESO, MK had implicitly encouraged the idea that Green-Sap was Falinesti, and that its stone was "a fruit". Since ESO lore paints the Elden Tree as the Bosmer's Tower, with an acorn being its stone, it can be construed as one issue where MK's view of TES was contradicted and overruled by the games, and thus serves to demonstrate the inherent fallibility of relying on OOG. Thus, I don't think it's wise to put much stock into MK's comment regarding "the cave" as Snow Throat's stone.
Rampant speculation:
The Nords heavily identify with the land, which is why it would make sense for both their tower and stone to manifest as geological and/or natural things.
In Morrowind and Oblivion, the PC was apparently forced to disrupt a Stone for the greater good. If this pattern was continued in Skyrim, then perhaps Alduin was the Stone. Him or Paarthurnax (despite how much I like him, I'm forced to conclude he will likely be dead in the future due to the events of Skyrim, as the Paarthurnax quest can only end in his death). If the mountain is indeed the tower's physical echo, and the echo bears some relation with the stone, then a dragon acting as its stone would seem suitable to me. They are said to make their homes in mountain caves, after all. A mountain could be viewed as intrinsically linked to them, like a hive to a bee.