Any daggerboard case needs some sort of substantial crosswise bracing, most commonly some sort of thwarts at one or both ends of the case that the case is attached to. If built off to the sides, Wahoo-style, you could probably skip the twarts and attach the case(s) to the gunwale structure.
The circular board doesn't really swivel. By design, striking a submerged object just pushes the tip of the board up and back as the board rides on the rollers inside the case, and it's free to move all the way up until the board's bottom is flush with the hull bottom if needed.