www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Any Idea how Fast is the dlang dataframe library ?

reply bioinfornatics <bioinfornatics fedoraproject.org> writes:
Dear,

Today I seen a powerful python library written in Rust.
homepage: https://pypi.org/project/polars/
benchmark: https://pypi.org/project/polars/

I remember that they are one year or two they had a sponsored D 
project on this topic.

So the project, is it …
i) still in development ?
ii) table ?
iii) better than polars and others ?


Thanks


Best regards
Jan 06 2022
parent reply bioinfornatics <bioinfornatics fedoraproject.org> writes:
On Thursday, 6 January 2022 at 23:05:22 UTC, bioinfornatics wrote:
 Dear,

 Today I seen a powerful python library written in Rust.
 homepage: https://pypi.org/project/polars/
 benchmark: https://pypi.org/project/polars/

 I remember that they are one year or two they had a sponsored D 
 project on this topic.

 So the project, is it …
 i) still in development ?
 ii) table ?
 iii) better than polars and others ?


 Thanks


 Best regards
*stable*
Jan 06 2022
parent reply Tejas <notrealemail gmail.com> writes:
On Thursday, 6 January 2022 at 23:06:20 UTC, bioinfornatics wrote:
 On Thursday, 6 January 2022 at 23:05:22 UTC, bioinfornatics 
 wrote:
 Dear,

 Today I seen a powerful python library written in Rust.
 homepage: https://pypi.org/project/polars/
 benchmark: https://pypi.org/project/polars/

 I remember that they are one year or two they had a sponsored 
 D project on this topic.

 So the project, is it …
 i) still in development ?
 ii) table ?
 iii) better than polars and others ?


 Thanks


 Best regards
*stable*
I believe you're talking about [magpie](https://code.dlang.org/packages/magpie) The project has been declared completed, but the author moved on and nobody has touched it since then, I think Don't know about performance, most likely inferior than anything that's being developed and released _today_, but still, it's best if you benchmark it yourself
Jan 06 2022
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Friday, 7 January 2022 at 04:04:44 UTC, Tejas wrote:
 [snip]

 I believe you're talking about 
 [magpie](https://code.dlang.org/packages/magpie)

 The project has been declared completed, but the author moved 
 on and nobody has touched it since then, I think

 Don't know about performance, most likely inferior than 
 anything that's being developed and released _today_, but 
 still, it's best if you benchmark it yourself
I had provided some feedback to the author when he was working on it and it does some interesting things. I feel like it really needs someone who cares about doing analysis in D with Dataframes to take over and continue making improvements. An evaluation of the performance capabilities would help identify what is good about magpie and what would need to get improved. Ideally, some of this work would get incorporated into mir. Mir has some basic time series functionality and you can make homogenous Dataframes (although there might be some functionality that is missing compared to magpie). However, mir does not really have an easy way to handle heterogenous types in the columns. Magpie does and it is important for many practical uses of Dataframes.
Jan 07 2022
parent reply Sergey <kornburn yandex.ru> writes:
On Friday, 7 January 2022 at 15:40:26 UTC, jmh530 wrote:
 On Friday, 7 January 2022 at 04:04:44 UTC, Tejas wrote:
 [snip]

 I believe you're talking about 
 [magpie](https://code.dlang.org/packages/magpie)
Just tried to use it - and it is not seems like usable library in its current status.
Jan 20 2022
parent jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 20 January 2022 at 19:39:09 UTC, Sergey wrote:
 On Friday, 7 January 2022 at 15:40:26 UTC, jmh530 wrote:
 On Friday, 7 January 2022 at 04:04:44 UTC, Tejas wrote:
 [snip]

 I believe you're talking about 
 [magpie](https://code.dlang.org/packages/magpie)
Just tried to use it - and it is not seems like usable library in its current status.
If there are any specific problems, you should file issues. Even if the author is no longer able to work on it, it might be something that people look at if they build a new dataframe library.
Jan 20 2022