digitalmars.D.announce - update to Dlang HDF5 format bindings/wrappers
- Laeeth Isharc (41/41) Aug 19 2015 What is HDF5, and why should you use it ?
What is HDF5, and why should you use it ? http://www.hdfgroup.org/why_hdf/ (My summary): - very large data sets, very fast access requirements, and complex datasets - share data across variety of platforms - many open-source and commercial tools that understand HDF - self-describing and can specify complex data relationships and dependencies - can contain binary data in many representations - allow direct access to parts of file without first parsing whole contents - hierarchical data objects can be expressed in natural manner (contrast experience with realational database tables) - n-dimensional datasets and each element in set may be complex object - relational databases good for field matching queries but not for sequentially processing all records in database or for subsetting data based on co-ordinate style lookup - custom proprietary binary formats often not portable, not extensible and not high-performance. technical debt to maintain data management part of code I personally find it useful for storing price data for financial instruments, and also economic data. There are bespoke time series databases, but they come at a price, which is not purely a pecuniary one. Updated wrappers are here: https://github.com/Laeeth/d_hdf5 Changes since last time - some fixes to bindings and updates to later version of HDF5 API. There is more to go to make it accessible idiomatically from D, but it's usable today. A simple example of mapping D structs to HDF5 types and back again is in the examples/traits directory. Pull requests and offers to help maintain it are welcome. It's still at an alpha stage, but already useful. Laeeth.
Aug 19 2015