The reason to use fewer, larger, buffers, is to reduce the number of “draw calls” issued to the GPU. The reason for this is that each “draw call” has overhead for error checking and shader setup in the API.

If you are not CPU limited on processing too many “draw calls” then there's no real gain in merging multiple terrain chunks together. Similarly, if you use a fully indirect API, such as D3D12 with render lists, then the number of “draw calls” could be essentially one, even though you have many chunks to draw.

So, set a budget for how many draw calls you can afford per subsystem (totally random example: 500 for terrain, 2000 for objects and foliage, 500 for player avatars, 2000 for GUI, 2000 for VFX) and then design your buffer management to fit within that budget.