I liked that we saw the remaining part of the island disappear down a glowing whirlpool. It appeared to explain something to me that I was wondering about, because under normal circumstances an island is a protruding piece of landmass that's simply higher than the ocean level. It's not something that can simply crumble and sink like a mythological Atlantis.
Like Walter said, the island was undermined by the Gnawers (whose actual name in Japanese can mean tons of things, including "underminers"). It's not like it disintegrated without explanation. It was destroyed from the inside out.
The place had wind constantly blowing in a similar whirl-like breeze and lower gravity as well as a time-difference with the outside world.
No specific wind pattern is highlighted in the story as far as I know (even when Schierke and Molda go flying), and only the forest had lower gravity, the underground place where the dwarves lived actually had higher gravity and the rest of the island had normal gravity.
Of course there is the question of the Gnawers and what their role on the island was. Perhaps they were necessary for balance, to absorb the Barytes or something.
It's implied they are related to the island's creation, or at least that they've been there from the beginning. I've speculated about it a bit before. However I don't think they were "absorbing Barytes".
I wonder if we'll get some background on the place in a potential Gaiseric flashback. The fact that the place was called "Skellig" suggests a connection with King Gaiseric and his skull motif.
"Skellig" is a name taken from a real set of islands off the coast of Ireland. It comes from "sceilg" which means "rock" in Irish, according to the Internet. Furthermore, in Japanese, the word "skull" that is used for "Skull Knight" is pronounced "dokuro", so there is no relation at all.
This also makes me wonder if Danan intentionally cast some spell to remove the Elfhelm creatures from the world before the rest of the island would sink, even if it was some kind of "mercy kill."
That's definitely not what it looks on the page.