Hi,

I have written an ECS but I have some questions about the update phase. (in systems)
I have read many articles, but not found references to this sort of problems.

In order to have benefits from ECS (cache friendly, for example), they are the following requirements:

• Entity must be just an ID.
• Components must be only pure data (struct with no logic).
• Systems contains the logic and update the components.
• No interactions between systems (instead systems communicate by adding “Tag” components to entities).
  

So, the logic applied in each system is fine and all works when they are no “user code”.
But, when we deal with user code (for example the user can attach C++ code to an object (like Unity, Unreal)), the problems come:

1. Since, components contain only the data, when the user modify the local position, the world position is not updated (the world position will be computed when the Transform System will process each Transform Component. So if the user asks for the world position after modifying its local position, it will get the previous world position and not the actual.
2. When an entity is removed, its children must be removed. Since the component contains only the data and not logic, the children will not be removed (it will be on the next Parent System update). So we have some “delay” (the children will still be accessible but will be removed on the next Parent System update).
3. Supposing we have the entities A, B, C. B is a child of A. In the user code (c++ code attached to the entity), the user set the parent of B has C, then remove entity A. When the Parent System will update, it will detect that A has been removed, (it can also detect that the parent of Entity A has changed) but how the system can know if the entity A has been removed after the parent change of Entity B or before?

Adding logic into components will ruins advantage of pure ECS (doing the same actions on all same components, in a cache friendly way), so IMHO it's not a solution.
Anyone have the solution? I would like to know how are you deals with this sort of problems with your ECS implementation.

Thanks!

In order to have benefits from ECS (cache friendly, for example), they are the following requirements:

• Entity must be just an ID.
• Components must be only pure data (struct with no logic).
• Systems contains the logic and update the components.
• No interactions between systems (instead systems communicate by adding “Tag” components to entities).

None of the above are necessary for you to benefit from cache ‘friendliness’, because:

1. Entities could be classes that hold pointers to components, but that has no bearing over how you choose to store the components
2. Adding functions to a component has no effect at all on its in-memory representation (exception: virtual functions)
3. As such, where the logic is defined has no effect on how cache-coherent the data is
4. …and thus any constraint on what systems should do is invalid as there is no strict requirement for systems to exist at all.

Please don't get sucked into an invalid idea of ‘purity’.

Kylotan said:
None of the above are necessary for you to benefit from cache ‘friendliness’, because: Entities could be classes that hold pointers to components, but that has no bearing over how you choose to store the components Adding functions to a component has no effect at all on its in-memory representation (exception: virtual functions) As such, where the logic is defined has no effect on how cache-coherent the data is …and thus any constraint on what systems should do is invalid as there is no strict requirement for systems to exist at all. Please don't get sucked into an invalid idea of ‘purity’.

1/ Entities can store pointer, if you want to be cache-friendly, your components must be packed, so moved in memory. So your pointers become dangling.
2/ Adding function will do immediate update, not update all component in a pool, so → cache misses.
3/ Yes, but we must process all the component in the memory order to avoid caches misses (so not in the component itself)

Pointers can be changed. Components don't necessarily need to be moved.

Adding a function does no “immediate update”. This makes no sense.

Processing components in “memory order” doesn't mean the processing can't be done by a method on the component.

Would calling your user code from pre- and post-update ‘callbacks’ solve your issues?

Geek17 said:
When an entity is removed, its children must be removed. Since the component contains only the data and not logic, the children will not be removed (it will be on the next Parent System update). So we have some “delay” (the children will still be accessible but will be removed on the next Parent System update).

Honestly I don't see that as a problem. Removing an entity is comlicated enough (especially if you have your components in a compact memory structure). that in my own ECS, I simply delay the entire erase-operation to the end of the frame. This prevents issues like when an entity destroys itself via a script, when entities get destroyed by script-callbacks while iterating over the collection, etc…

Juliean said:

Geek17 said:
When an entity is removed, its children must be removed. Since the component contains only the data and not logic, the children will not be removed (it will be on the next Parent System update). So we have some “delay” (the children will still be accessible but will be removed on the next Parent System update).

Honestly I don't see that as a problem. Removing an entity is comlicated enough (especially if you have your components in a compact memory structure). that in my own ECS, I simply delay the entire erase-operation to the end of the frame. This prevents issues like when an entity destroys itself via a script, when entities get destroyed by script-callbacks while iterating over the collection, etc…

Yes, perhaps but for the third point:
Supposing we have the entities A, B, C. B is a child of A. In the user code (c++ code attached to the entity), the user set the parent of B has C, then remove entity A. When the Parent System will update, it will detect that A has been removed, (it can also detect that the parent of Entity A has changed) but how the system can know if the entity A has been removed after the parent change of Entity B or before?

The delayed updated cause a problem, because we dont know if the child for which the parent has changed, must be removed or not!

JoeJ said:

Would calling your user code from pre- and post-update ‘callbacks’ solve your issues?

No, because the problem is just that I don't know in which order the update has been done. (I call the user code, and the user code do a remove on entity A, then a setParent on entity B….
Same for the first point, if the user do:

const auto worldPositon = transform.getWorldPosition();
transform.localPosition += 10.0f;
const auto& newWorldPosition = transform.getWorldPosition();

worldPosition will be equal to newWorldPosition, because we are in the “script”, and the transform system has not yet updated the world….

Kylotan said:

Pointers can be changed. Components don't necessarily need to be moved.

Adding a function does no “immediate update”. This makes no sense.

Processing components in “memory order” doesn't mean the processing can't be done by a method on the component.

The goal is to have an array with no empty slot.
so | ComponentA | ComponentA | ComponentA | | | …

and not: | ComponentA | | ComponentA | | | | | | | | | | ComponentA | | … (because here you have useless cache misses (you create 1 millions of component A and remove all except the last, for each system update you walk around the 999 999 empty slots….

Adding a function in the component (I talk about function that update the world transform for example, when we call getWorldTransform) will be an immediate update (not handled directly by the system).

“Processing components in “memory order” doesn't mean the processing can't be done by a method on the component.” Can i have more information? Because if you call method on component for each component in the system, this one don't solve my issue.

Geek17 said:
Yes, perhaps but for the third point: Supposing we have the entities A, B, C. B is a child of A. In the user code (c++ code attached to the entity), the user set the parent of B has C, then remove entity A. When the Parent System will update, it will detect that A has been removed, (it can also detect that the parent of Entity A has changed) but how the system can know if the entity A has been removed after the parent change of Entity B or before? The delayed updated cause a problem, because we dont know if the child for which the parent has changed, must be removed or not!

Well, you have multiple options:

1. Send a notification to the systems when entity A is deleted. Then those systems can mark all childs as deleted. A messaging system is part of almost all ECS, and without it you will find it hard to implement certain interactions. (and don't worry about performance in that case, deleting the entity is going to be hundered times slower than any kind of message-passing anyways)
2. Simply delete those childs that are still attached when the deletion step happens. Depenging on your situation it can be totally legit for scripts to “steal” the child of an entity that is about to be deleted.

A general comment about state that changes during a frame. You should rather strife to keep the world-state intact during the frame, instead of publishing every change made. Its only a property of how PCs work that updates happen linearely. In reality, everything would update concurrently. So the more you make changes apply immediately, the more you run into issues like different results based on update order. Its not uncommon (though hard) for game-engines to have two separate copies of the full world state, where the result of the current update is written on the 2nd state, so that everything stays consistent during the update.

So its absoluetly fine, if not even better to have the world-position stay consistent, while having a pass at the end of the frame where all parent-child transforms are updated.

Geek17 said:
The goal is to have an array with no empty slot. so | ComponentA | ComponentA | ComponentA | | | … and not: | ComponentA | | ComponentA | | | | | | | | | | ComponentA | | … (because here you have useless cache misses (you create 1 millions of component A and remove all except the last, for each system update you walk around the 999 999 empty slots….

I wouldn't worry so much about cache misses in ECS. They will really only be a factor once you get tens of thousands of components, and only if the work you are doing for each component is trivial. If you are doing anything beyond a few instructions per component, the memory latency incurred due to a cache miss will be covered up by the computation. If you have that many components, you are bound to have other larger bottlenecks than cache misses (e.g. graphics rendering). Large-scale (i.e. planetary) simulations will require special optimization techniques and data structures to handle updates of that many entities, cache misses will be the least concern. Cache misses can still be a problem, but usually in more special-case low-level critical code than entity/component management (e.g. physics and audio are places where it matters).

Geek17 said:
Supposing we have the entities A, B, C. B is a child of A. In the user code (c++ code attached to the entity), the user set the parent of B has C, then remove entity A. When the Parent System will update, it will detect that A has been removed, (it can also detect that the parent of Entity A has changed) but how the system can know if the entity A has been removed after the parent change of Entity B or before?

The way that this is handled in my engine is to maintain a hierarchy of components that is constructed as components are added to the engine. When a component is added, the systems are notified and can choose to add child components to the engine as well. This hierarchical relationship is tracked by a system external to the engine, I call it an “engine scope”, and there can be multiple of these. You might have one scope for each level in a game, to handle loading/unloading of components related to that level, plus a global scope that contains components common to all levels. The hierarchy itself is stored as pointers in a hash multi-map, and each component is reference counted (this is to allow the same component to be added more than once, which can happen if multiple components share children). When a component is removed from an engine scope, the scope automatically removes all of that component's children if their reference count reaches 0, no action from systems required (though they are notified of removal and can take special action).
(note: I don't differentiate between entities/components in my engine, an entity is just another component type)

Kylotan and Juliean have already given great advise. They also have helped me several times when I have questions here.

I still want to add some more …

To handle child-parent transform, I use the similar approach as Ogre3D. My gameplay has to be split into 3 phases :-

1. set all relative transform, explicitly forbid all reading back (except some rare cases)
2. boardcast all relative transformation from parent to all children recurisively (expensive)
3. gameplay can read back world transform, but explicitly forbid all transform modification in this phase.

About no-function-for-component, it is a big cult, and here is my message as a former member of that cult.

I blatantly created functions for a component and shamelessly use them e.g. :-

graphicBackgroundComponent->setColor(VeganColorMode::EMIT,Vec3(1,1,1)*MyMath::siner(timeOO->ts()));
enemyShip->damageSelf(anotherShip->getStandardDmg());

… and … if you worry too much about cache-friendly like me, I recommend assemblage approach (Reference: https://gamedev.stackexchange.com/questions/58693/grouping-entities-of-the-same-component-set-into-linear-memory).

• With a proper amount of component packing, I can squish out some extra frame per sec (I profiled).
• It needs manual tuning e.g. packing Hp and Ship.
• More memory will be waste though (for every entity that has HP component but has no Ship component, and vice versa)

I believe that assemblage technique is usually more troublesome than it worth except when the game engine is almost mature.