The Stone Plug is a Naturally spawning Structure exclusive to Hamlet DLC. It is a unique object which can exclusively be found inside the BFB's Caves; it mainly acts as an obstacle which has to be mined in order to exit these caves.
Once the player has been transported to the BFB's Nest, its only way to escape the island is to mine the Sinkhole on its surface, and enter the Caves just below. These Caves contain many rooms, and a Stone Plug will be present in the back wall of one of them, which the player will have to mine with a Pickaxe or Opulent Pickaxe in order to exit them. This will lead the player to one of the Cave Clefts on one of the four other islands, and only then, to the surface.