[C#] Fastest way to serialize data structures?

That path doesn't end well in the long term.

Any changes to the structure over time --- and there certainly will be evolution of the structure over time --- is not accounted for in your system. Version information is a must.

C# has a built-in serialization system. It includes version information. It allows ways to specify what should be included and what should be excluded. It is automatic. You can write to various formats including binary formats, xml formats, and whatever else you want. It is both fast and efficient.

Do you have any specific reason you aren't using the runtime's serialization functionality?

What if T type contains member with type string or array of bytes, or any other managed type?

That path doesn't end well in the long term.

Any changes to the structure over time --- and there certainly will be evolution of the structure over time --- is not accounted for in your system. Version information is a must.

C# has a built-in serialization system. It includes version information. It allows ways to specify what should be included and what should be excluded. It is automatic. You can write to various formats including binary formats, xml formats, and whatever else you want. It is both fast and efficient.

Do you have any specific reason you aren't using the runtime's serialization functionality?

All I'm trying to do is find a very fast and efficient way to "strip" a structure down to raw bytes. For example, to rapidly create a static vertex buffer or index buffer in the shortest amount of time possible... situations where I don't care about reflecting changes to the data in other places, as it's for one-time use.

I may be incorrect but it's my understanding that this is faster for my limited intents and purposes...it's a very small amount of code that runs in a few ticks, according to test results.

EDIT:

I haven't performed the tests to actually confirm this, but I do believe it would also be faster if I took the address of a structure and cast to byte*, then obtain a byte* to the destination buffer and copy the memory that way. For example:

[source lang="csharp"]
public unsafe byte[] GetBytes(Vector3 vec)
{
var buffer = new byte[Vector3.SizeInBytes];
byte* pSrc = (byte *)&vec

fixed (byte* baseAddr = buffer)
for (int i = 0; i < Vector3.SizeInBytes; ++i)
baseAddr = pSrc;

return buffer;
}
[/source]

Of course I could be totally wrong about all of this... but that's why I'm hear asking. To learn. :-)

But your proposed setup only works for data types where all of the data is actually in the members, and not in memory pointed to by the members. So you wouldn't be able to serialize arrays for instance, which makes it pretty much useless for serializing vertex buffers. Unless you like declaring vertex buffers like this:


struct VertexBuffer
{
Float3 v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10 ... v10236;
}


So you wouldn't be able to serialize arrays for instance, which makes it pretty much useless for serializing vertex buffers. Unless you like declaring vertex buffers like this:

[source lang="csharp"]
public static byte[] GetBytes(this T[] array) where T : struct
{
int size = Marshal.SizeOf(typeof(T));
var buffer = new byte[size * array.Length];

for (int i = 0; i < array.Length; ++i)
{
var bytes = array.GetBytes();
Array.Copy(bytes, 0, buffer, (i * size), bytes.Length);
}

return buffer;
}[/source]

Already been taken care of, and it all seems to perform blazing-fast. :-)

EDIT:

Also, the first GetBytes(...) implementation I showed is, I believe, over-complicated and wastes memory by duplicating things. I think this would actually work better with no need to allocate a new block unmanaged memory, as long as you fix the buffer:

[source lang="csharp"]
static unsafe byte[] _getBytes(T value) where T : struct {

var buffer = new byte[Marshal.SizeOf(typeof(T))];

fixed (byte* pBuffer = buffer)
Marshal.StructureToPtr(value, (IntPtr)pBuffer, true);

return buffer;
}[/source]

EDIT2:

Yes, it does indeed work. I'm sure it's faster than that first implementation I showed. Perhaps it can even be faster than GetBytes2... we'll see...

EDIT:

Ok, I setup the testing code to run 2,000,000 tests of each method and compute the average time each one takes. GetBytes2 wins, but only by less than 0.07 ticks. The changes to GetBytes1 do give it a tiny boost of speed.

I should also mention that before actual testing commences I run one interation of the test and ignore the results. This is to "pre-JIT" the code because the first time it runs it must be JIT'ed and incurs an overhead subsequent calls will not.

Well, I guess I'm already doing things as fast as they can be done. I don't see how anything can be faster than 1-2 ticks, as that's virtually the smallest slice of processor time possible. This code is currently a working part of my engine used to generate raw bytes from structures which is then bound to the GPU as vertex and index buffers, for example. I would've used .NET's Serialization had I wanted to do anything else, like sending data over a network, but in this case I don't need any type information, persistence or anything else... I just need raw, contiguous data for the GPU, and I need it as fast as humanly (well... "machinely") possible. So I guess it's not getting any better or faster than it already is.

Thanks for all the help and your time!