The only plausible reason I can think of for why he allowed that slap to land is that, for whatever reason, he let it happen.
That reasoning doesn't make sense to me...
He even seemed to have gotten concerned or flustered after these feelings caused him to protect Casca from those falling rocks, and he left the scene with Zodd. Surely being struck, sincerely and unexpectedly, would have elicited a more significant reaction?
My reading is just slightly different: The boy's feelings made Griffith protect Casca. And at that moment, Griffith realized that Guts and Casca's connection with the boy was a variable he hadn't counted on. It was something that resulted in him losing control for a moment. We know that surprised him, because we were able to read his thoughts afterward. So he cut his losses and ran.
Anyway, Griffith did react to being struck.

We see him staring off in the distance as Rickert leaves. And afterall, we still don't know who
really asked Rakshas to personally escort Rickert out, right? It was left ambiguous, though the sequence ends with Griffith staring after him (again!).
And what properties could Rickert possibly possess that Guts' dragon slayer doesn't? (The scene where the dragon slayer phases through Griffith is from the continuation, yes, so we can take that with a grain of salt, I guess.) It can't be that Rickert's slap connected because Rickert doesn't hate Griffith, because although he may not hate him like Guts does, that slap was one of anger. Besides, the emotions of the attacker don't influence whether their attack can land on an astral being/Godhand member/apostle or not.
I think it's pretty simple, and the continuation's depiction of "phasing" has corrupted things.
Rickert's rejection was a genuine surprise to Griffith, and so was the slap. I don't think Miura crafted that moment to require a mechanical explanation for it to have happened. I see Griffith's surprise from his reaction as Rickert left the scene in the following episode. Because in a rare moment, not showing off for his cronies, not facing anyone in particular, Griffith frowned at Rickert's decision, and Miura specifically chose to show us that, and not gloss over it.
Beyond that, I do tend to relate this with Skull Knight's words back on Vritannis beach:
"To challenge and touch the Falcon, one must reside outside of the tale." What's so special about Rickert? He has credentials! Consider that Rickert is a unique loose end in a way that Guts and Casca aren't. Skull Knight made a special rescue effort just for him, back in the Golden Age. Also, Rickert is not a sacrifice. But he's connected to Griffith in a way that no one else alive (who isn't branded) is. He was witness to who he was and who he became. He can see through the utopian guise like the other humans just can't.
All of those things place Rickert into a very special position, untouched by the karmic fire that envelopes Guts, and uninfluenced by the overwhelming charm that makes everyone else swoon to his whims. I think Griffith, who was so used to everything going his way, with all outcomes playing out within his hands, truly expected Rickert to accept his offer. And the judgment he got instead
did leave a mark, for the first time.
Now... I don't think any of that has anything to do with the ending of Berserk (lol). I think the whole scene was more about Rickert finally making a decision on the conflict he'd harbored since the Hill of Swords. And as a tasty treat on top, it was a noteworthy moment for Griffith to be turned down, and to realize that despite all appearances, not everything is within his control.