www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Performant BOM explosion

reply Lars Johansson <lasse 11dim.se> writes:
I'm writing my first 'real' D-program.
The input are two mySQL tables: one with parent-child nodes 
(800000), one with 70000 topnodes i.e. do not have a parent.
For each of the top nodes I like to do a full BOM explosion 
(about 1.8 million tree nodes).
So far I have managed to read the input tables into two arrays.

My assumption was using an associative array for the parent-child 
table I could do the BOM explosion by itterative seaching the AA 
starting from the top nodes.
Now when I look at the AA in the documentation I doubt it is 
possible to do this with an AA.

I take one topnode find all nodes with the topnode as parent, 
then for each of those find nodes that have the child as parent 
and so on until all top nodes are processed.
But I do not find a way to do the search for children in a 
performant way.

I would be very obliged if anyone can advise a way to create the 
complete BOM tree.

p.s.
My explanation is a simplification, but an advise is a good start.
Apr 14
parent reply Rene Zwanenburg <renezwanenburg gmail.com> writes:
On Monday, 14 April 2025 at 11:45:19 UTC, Lars Johansson wrote:
 I would be very obliged if anyone can advise a way to create 
 the complete BOM tree.
I would: - Order by parent ID when selecting from the parent-child table - Use chunkBy() to group the children per-parent - Then use map() + assocArray() to turn it into an AA. Depending on how your SQL library works it might be important to copy the children to a new dynamic array here. That will give you an AA that can be used to recursively walk through the BOM with decent enough performance.
Apr 14
parent Lars Johansson <lasse 11dim.se> writes:
On Monday, 14 April 2025 at 14:30:33 UTC, Rene Zwanenburg wrote:

 That will give you an AA that can be used to recursively walk 
 through the BOM with decent enough performance.
Thank you, all new concepts for me. By the way today I have a working PHP script run in 10 parallel lines, bouncing loads of SQL between the app and DB server, this is part of a large framework. I had to rewrite this into a stand alone python script, while doing this I got the idea to move the heavy lifting from the DB server to an in memory procedure in the app server for that I need a fast program. I started by looking at FASM but that would take to long time to develop, then I looked at RUST that would probably take even longer time. I decided to give D a chance. I'm far from sure it will be faster than the PHP/MySQL procedure. But I will use your suggestions to see where it leads me. Tanks again.
Apr 14