it's fluff and it doesn't affect the main story.
I hope you're right, but I already see a cynical erosion of at least our perception in that regard...
The Sea God story then, I'm used to calling it an arc. My complaints are the same as most of those who don't like the side story.
This is what I'm talking about. I made the point because an "Arc" in Berserk is typically 80-100+ episodes, the exception being the Black Swordsman Arc, and this is important because that's a huge and varied amount of story that something like the Sea God is only a small part of (the beginning of the Elf Island Chapter of the Fantasia Arc) and shouldn't be compared to an Arc's worth of plotting.
More to the point, what part of it is a "side story" or what's side material in general? Is the assassination of Julius a side story? Is the entire Golden Age arc flashback a side story? By such rigid standards it's not strictly the Black Swordsman story we started with, resumed, and continue now (it sure enriches it though, huh =). So, aside from the fact that pretty much any part of Guts' adventures is by default the main story (whether he's fighting Zodd, a tree, or a Sea God), what in the relatively small Sea God portion is not of consequence? Guts' time on the ship developing relationships with his companions? Guts' continuing struggle with the beast armor and the appearance of the superior being/moonlight boy during the battle? Casca's and Guts' interactions in the above scenarios (!)? Isma's introduction to the main cast? A battle with an old god establishing this new world where Guts was pretty much going to sacrifice himself only to be saved by his reliable companions/family? Seems like a lot of things of significance throughout that small 20-some episode sampling, which also deals directly or indirectly with a Sea God, rather than a "side story." Feel free not to enjoy it (though you also said you liked it), but I wouldn't allow the dumb complaints of "most of those who don't like the
side story" to represent your opinion, let alone curate what's objectively important or not in the series.
So yeah, I hope we're not too quick to write off anything we don't immediately think deeply on as fluff, or that fluff merchandise has us judging the whole series by such standards.