www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Wrote a Protobuf-like D (de)serialisation library - asking for

reply Sinisa Susnjar <sinisa.susnjar gmail.com> writes:
Hi, D newbie here,

The library is on par with Google Protobuf performance-wise and 
does not need a pre-compiler like Protobuf does, but instead uses 
the meta programming facilities of D to (de)serialise D data 
types from/to binary messages. Some samples and unit tests are 
included.

Here's the code if anyone wants to have a look: 
https://github.com/sinisa-susnjar/msgbuf

(I am sure there is a lot of room for improvement)
Jun 12
next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 6/12/21 2:59 PM, Sinisa Susnjar wrote:

 instead uses the meta programming facilities of D to (de)serialise D 
data types from/to binary messages. Yeah! :) I did the same at work.
 (I am sure there is a lot of room for improvement)
Without reading your code carefully, I can think of two optimizations that may prove to be useful: - If a struct contains all bitwise copyable members, instead of (de)serializing each member individually, the whole struct can by memcpy'ed. This may be a performance gain especially for arrays of structs: You can memcpy the whole array at once. - Instead of allocating memory for each value (e.g. for arrays with 'new'), you can maintain a function-static buffer and reuse it, allocating only as the current buffer is not large enough for new data. (No worries with multi-threading because each thread will have its own function-static buffer.) Ali
Jun 12
parent Sinisa Susnjar <sinisa.susnjar gmail.com> writes:
On Sunday, 13 June 2021 at 00:18:56 UTC, Ali Çehreli wrote:

 - If a struct contains all bitwise copyable members, instead of 
 (de)serializing each member individually, the whole struct can 
 by memcpy'ed. This may be a performance gain especially for 
 arrays of structs: You can memcpy the whole array at once.

 - Instead of allocating memory for each value (e.g. for arrays 
 with 'new'), you can maintain a function-static buffer and 
 reuse it, allocating only as the current buffer is not large 
 enough for new data. (No worries with multi-threading because 
 each thread will have its own function-static buffer.)

 Ali
Thanks for the suggestion!
Jun 14
prev sibling parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Saturday, 12 June 2021 at 21:59:13 UTC, Sinisa Susnjar wrote:
 Hi, D newbie here,

 The library is on par with Google Protobuf performance-wise and 
 does not need a pre-compiler like Protobuf does, but instead 
 uses the meta programming facilities of D to (de)serialise D 
 data types from/to binary messages. Some samples and unit tests 
 are included.

 Here's the code if anyone wants to have a look: 
 https://github.com/sinisa-susnjar/msgbuf

 (I am sure there is a lot of room for improvement)
Is it on dub?
Jun 14
parent Sinisa Susnjar <sinisa.susnjar gmail.com> writes:
On Monday, 14 June 2021 at 07:14:27 UTC, Imperatorn wrote:

 Is it on dub?
published it just now :)
Jun 14