There is no simple way to solve it
Which means there are ways, but most of them are too expensive for realtime rendering (and the results in difference are not worth the performance lose). To quickly describe one...
Rendering reflection works like -> you mirror your world using water plane, and clip the geometry using that plane (leaving only parts that are below) - this will work just fine ... but only when you're using plane of course.
One of the faster solutions to deal with that comes from idea - your water's pixel position can be described using displacement along water plane normal (which is right for almost every water renderer in games these days) - which allows you to calculate distance from camera to water surface on given pixel; Now, you render your reflection this way -> you mirror your world using water plane, calculate distance from the mirrored geometry to camera, and you calculate distace of "camera-to-water-surface", discarding any pixel that is in front of the water surface.
This way you remove the effect you're seeing on your images, but ... nothing is free.
- There is overhead of calculating distance of "camera-to-water-surface"
- As you're discarding pixels, you can't really use some kind of occlusion culling when rendering reflected part of scene
- Discarding pixels is more expensive then clipping (if I remember correctly from head - clipping is done after vertex shader execution, so you should also use clip plane here, clipping anything that is above the maximum height reachable from the plane after displacement)
EDIT: Also it is important to think, whether implementing whole of the above is worth the trouble of removing small artefacts for water refraction. I'd say it also depends on how often will user see water in the end in the application - if 99% of gaming time there won't be any visible water, then it's not definitely worth it ... if 99% of gaming time you're seeing water surface, well... then I'd say you should try to make water look as good as possible.