I have object A and object B, and the world matrix for object B is 6 units along the positive x axis. From what I have read if I transform object A using the inverse of object B's world matrix, it will be transformed to object B's coordinate system. Why is that? Does it mean object A's vertices are now defined relative to object B's origin(0,0,6)?

If you transform a coord using inverse of a world matrix then you get the coord local space of this world matrix.

It's like if one person see a point and another person see the same point, you got different coord based on them.

It is basically as you say. If two objects, A and B both have a point at (0, 0). You move object B across by 6 so it's point is now at (6, 0) then relative to that, A's (0,0) is actually at (-6, 0) relative to B but in 'world space' it is still (0, 0). If you invert B's transform you get -6 translation. You apply that to A and now A's (0, 0) is now at (-6, 0) so it makes sense if that's then in 'B' relative space.

It is quite confusing. As long as you understand what space you are working in and that the space is the same then you are good.

Instead of calling it the "world matrix", I like to call if the "object to world matrix" -- transform matrices are constructed to move from one space to another. Both spaces should appear in the name so the purpose of the transform is clear.

So you've got an "A->World" transform and a "B->World" transform.
Also "Inverse(B->World)" is equal to "World->B".
So "A.world * inverse(B.world)" is actually:
"A->World, World->B", or "A->World->B", or "A->B".

Likewise the "view matrix" is actually the "world to view matrix" and "projection" is "view to projection".
"WorldView" is "object to world to view" or "object to view", and "WorldViewProj" is "object to projection".

Instead of calling it the "world matrix", I like to call if the "object to world matrix" -- transform matrices are constructed to move from one space to another. Both spaces should appear in the name so the purpose of the transform is clear.

So you've got an "A->World" transform and a "B->World" transform.
Also "Inverse(B->World)" is equal to "World->B".
So "A.world * inverse(B.world)" is actually:
"A->World, World->B", or "A->World->B", or "A->B".

Likewise the "view matrix" is actually the "world to view matrix" and "projection" is "view to projection".
"WorldView" is "object to world to view" or "object to view", and "WorldViewProj" is "object to projection".

I like that idea. Certainly is easier to remember what's going on. I think I may adopt the approach.

Instead of calling it the "world matrix", I like to call if the "object to world matrix"

A good option to have it working for all space is to use "Global Space", it works for world space and model space.