www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - D Beginner Trying Manual Memory Management

reply "jmh530" <john.michael.hall gmail.com> writes:
I'm new to D. I have some modest knowledge of C++, but am more 
familiar with scripting languages (Matlab, Python, R). D seems so 
much easier than C++ in a lot of ways (and I just learned about 
rdmd today, which is pretty cool). I am concerned about 
performance of D vs. C++, so I wanted to learn a little bit more 
about manual memory management, in case I might ever need it (not 
for any particular application).

The D Language book Section 6.3.4-5 covers the topic. I basically 
copied below and made some small changes.

import core.stdc.stdlib;
import std.stdio;

class Buffer {
	private void* data;
	// Constructor
	this()
         {
             data = malloc(1024);
	}
	// Destructor
         ~this()
         {
             free(data);
	}
}
unittest {
	auto b = new Buffer;
	auto b1 = b;
	destroy(b); //changed from clear in book example
	assert(b1.data is null);
	writeln("Unit Test Finished");
}

I was thinking that it might be cool to use scope(exit) to handle 
the memory management. It turns out the below unit test works.

unittest {
	auto b = new Buffer;
	scope(exit) destroy(b);
	writeln("Unittest Finished");
}

However, if you leave in the auto b1 and assert, then it fails. I 
suspect this is for the same reason that shared pointers are a 
thing in C++ (it can't handle copies of the pointer).

Alternately, you can use some other scope and something like this
unittest {
	{
		Buffer b = new Buffer;
		scope(exit) destroy(b);
	}
	destroy(b);
	writeln("Unittest Finished");
}

will fail because b has already been destroyed. I thought this 
behavior was pretty cool. If you followed this approach, then you 
wouldn't have to wait until the end of the program to delete the 
pointers. The downside would be if you need to write a lot of 
pointers and there would be a lot of nesting. (likely the 
motivation for the reference counting approach to smart pointers).

I wasn't sure how to figure out a way to combine these two 
components so that I only have to write one line. I thought one 
approach might be to put a scope(exit) within the constructor, 
but that doesn't work. Also, if I try to do it within a template 
function, then the scope(exit) is limited to the function scope, 
which isn't the same thing.

Outside of writing a unique_ptr template class (which I have 
tried to do, but couldn't get it to work) I don't really know how 
to combine them into one line. Any ideas?
Jan 12 2015
next sibling parent reply ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
On Mon, 12 Jan 2015 19:29:53 +0000
jmh530 via Digitalmars-d-learn <digitalmars-d-learn puremagic.com>
wrote:

the proper answer is too long to write (it will be more an article that
a forum answer ;-), so i'll just give you some directions:

  import std.typecons;

  {
    auto b =3D scoped!B(); // `auto` is important here!
    ...
  }

`scoped!` allocating class instance *on* *stack*, and automatically
calls destructor when object goes out of scope.

but you'd better consider using struct for such things, as struct are
stack-allocated by default (unlike classes, which are reference type
and should be allocated manually).

there is a big difference between `class` and `struct` in D, much
bigger that in C++ (where it's only about default protection,
actually).
Jan 12 2015
parent reply "jmh530" <john.michael.hall gmail.com> writes:
Thanks for the reply, I wasn't familiar with scoped. I was aware 
that structs are on the stack and classes are on the heap in D, 
but I didn't know it was possible to put a class on the stack. 
Might be interesting to see how this is implemented.

After looking up some more C++, I think what I was trying to do 
is more like make_unique than unique_ptr.

On Monday, 12 January 2015 at 19:42:14 UTC, ketmar via 
Digitalmars-d-learn wrote:

 On Mon, 12 Jan 2015 19:29:53 +0000
 jmh530 via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com>
 wrote:

 the proper answer is too long to write (it will be more an 
 article that
 a forum answer ;-), so i'll just give you some directions:

   import std.typecons;

   {
     auto b = scoped!B(); // `auto` is important here!
     ...
   }

 `scoped!` allocating class instance *on* *stack*, and 
 automatically
 calls destructor when object goes out of scope.

 but you'd better consider using struct for such things, as 
 struct are
 stack-allocated by default (unlike classes, which are reference 
 type
 and should be allocated manually).

 there is a big difference between `class` and `struct` in D, 
 much
 bigger that in C++ (where it's only about default protection,
 actually).
Jan 12 2015
parent reply ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
On Mon, 12 Jan 2015 20:14:19 +0000
jmh530 via Digitalmars-d-learn <digitalmars-d-learn puremagic.com>
wrote:


 Thanks for the reply, I wasn't familiar with scoped. I was aware=20
 that structs are on the stack and classes are on the heap in D,=20
 but I didn't know it was possible to put a class on the stack.=20
 Might be interesting to see how this is implemented.


actually, there is nothing complicated there (if you don't want to
write an universal thing like `emplace!` ;-). it builds a wrapper
struct big enough to hold class instance, copies class `.init` there
and calls class' constructor. the rest of the magic is done by the
compiler: when struct goes out of scope, compiler calls struct
destructor, which in turn calls class destructor. ah, and it forwards
all other requests with `alias this` trick.


 After looking up some more C++, I think what I was trying to do=20
 is more like make_unique than unique_ptr.


i don't remember C++ well, but nevertheless i encouraging you to take a
look at `std.typecons`. there are some handy things there, like
`Rebindable!` or `Nullable!`. and some funny things like `BlackHole!`
and `WhiteHole!`. ;-)

it even has `RefCounted!`, but it doesn't play well with classes yet
(AFAIR).
Jan 12 2015
parent reply "aldanor" <i.s.smirnov gmail.com> writes:
On Monday, 12 January 2015 at 20:30:45 UTC, ketmar via 
Digitalmars-d-learn wrote:

 it even has `RefCounted!`, but it doesn't play well with 
 classes yet
 (AFAIR).


I wonder if it's possible to somehow make a version of refcounted 
that would work with classes (even if limited/restricted in some 
certain ways), or is it just technically impossible because of 
reference semantics?
Jan 12 2015
parent reply ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
On Mon, 12 Jan 2015 21:37:27 +0000
aldanor via Digitalmars-d-learn <digitalmars-d-learn puremagic.com>
wrote:


 On Monday, 12 January 2015 at 20:30:45 UTC, ketmar via=20
 Digitalmars-d-learn wrote:

 it even has `RefCounted!`, but it doesn't play well with=20
 classes yet
 (AFAIR).


 I wonder if it's possible to somehow make a version of refcounted=20
 that would work with classes (even if limited/restricted in some=20
 certain ways), or is it just technically impossible because of=20
 reference semantics?


it's hard. especially hard when you considering inheritance (which is
not playing well with templates) and yes, ref semantics.

on the other side i found myself rarely using classes at all. i mostly
writing templates that checks if a passed "thing" has all the
neccessary methods and properties in place and just using that. with D
metaprogramming abilities (and `alias this` trick ;-) inheritance
becomes not so important. and so classes too. sometimes i'm using
structs with delegate fields to simulate some sort of "virtual methods"
'cause i tend to constantly forgetting about that `class` thingy. ;-)

OOP is overrated. at least c++-like (should i say "simula-like"?)
OOP. ;-)
Jan 12 2015
next sibling parent reply "aldanor" <i.s.smirnov gmail.com> writes:
On Monday, 12 January 2015 at 21:54:51 UTC, ketmar via 
Digitalmars-d-learn wrote:

 On Mon, 12 Jan 2015 21:37:27 +0000
 aldanor via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com>
 wrote:


 On Monday, 12 January 2015 at 20:30:45 UTC, ketmar via 
 Digitalmars-d-learn wrote:

 it even has `RefCounted!`, but it doesn't play well with 
 classes yet
 (AFAIR).


 I wonder if it's possible to somehow make a version of 
 refcounted that would work with classes (even if 
 limited/restricted in some certain ways), or is it just 
 technically impossible because of reference semantics?


 it's hard. especially hard when you considering inheritance 
 (which is
 not playing well with templates) and yes, ref semantics.

 on the other side i found myself rarely using classes at all. i 
 mostly
 writing templates that checks if a passed "thing" has all the
 neccessary methods and properties in place and just using that. 
 with D
 metaprogramming abilities (and `alias this` trick ;-) 
 inheritance
 becomes not so important. and so classes too. sometimes i'm 
 using
 structs with delegate fields to simulate some sort of "virtual 
 methods"
 'cause i tend to constantly forgetting about that `class` 
 thingy. ;-)

 OOP is overrated. at least c++-like (should i say 
 "simula-like"?)
 OOP. ;-)


I see, thanks! :) I've started liking structs more and more 
recently as well and been pondering on how to convert a 
class-based code that looks like this (only the base class has 
any data):

class Base { T m_variable; }
class Common : Base { /* tons of methods; uses m_variable */ }
class Extra : Base { /* another ton of methods; uses m_variable 
*/ }
class A : Extra, Common { ... }
class B : Common { ... }
class C : Extra { ... }

to refcounted structs with alias this but couldn't quite figure 
out how to do it (other than use mixin templates...). Even if the 
multiple alias this DIP was implemented, I don't think it would 
help much here :/
Jan 12 2015
next sibling parent reply ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
On Mon, 12 Jan 2015 22:07:13 +0000
aldanor via Digitalmars-d-learn <digitalmars-d-learn puremagic.com>
wrote:


 I see, thanks! :) I've started liking structs more and more=20
 recently as well and been pondering on how to convert a=20
 class-based code that looks like this (only the base class has=20
 any data):


it's hard to tell by brief description. but having multiple inheritance
immediately rings an alarm ring for me. something is very-very-very
wrong if you need to have a winged whale. ;-)
Jan 13 2015
parent reply "aldanor" <i.s.smirnov gmail.com> writes:
On Tuesday, 13 January 2015 at 08:33:57 UTC, ketmar via 
Digitalmars-d-learn wrote:

 On Mon, 12 Jan 2015 22:07:13 +0000
 aldanor via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com>
 wrote:


 I see, thanks! :) I've started liking structs more and more 
 recently as well and been pondering on how to convert a 
 class-based code that looks like this (only the base class has 
 any data):


 it's hard to tell by brief description. but having multiple 
 inheritance
 immediately rings an alarm ring for me. something is 
 very-very-very
 wrong if you need to have a winged whale. ;-)


A real-world example: 
http://www.hdfgroup.org/HDF5/doc/cpplus_RM/hierarchy.html

H5::File is both an H5::Location and H5::CommonFG (but not an 
H5::Object)
H5::Group is both an H5::Object (subclass of H5::Location) and 
H5::CommonFG
H5::Dataset is an H5::Object
Jan 13 2015
parent reply ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
On Tue, 13 Jan 2015 16:08:15 +0000
aldanor via Digitalmars-d-learn <digitalmars-d-learn puremagic.com>
wrote:


 On Tuesday, 13 January 2015 at 08:33:57 UTC, ketmar via=20
 Digitalmars-d-learn wrote:

 On Mon, 12 Jan 2015 22:07:13 +0000
 aldanor via Digitalmars-d-learn=20
 <digitalmars-d-learn puremagic.com>
 wrote:


 I see, thanks! :) I've started liking structs more and more=20
 recently as well and been pondering on how to convert a=20
 class-based code that looks like this (only the base class has=20
 any data):


 it's hard to tell by brief description. but having multiple=20
 inheritance
 immediately rings an alarm ring for me. something is=20
 very-very-very
 wrong if you need to have a winged whale. ;-)


 A real-world example:=20
 http://www.hdfgroup.org/HDF5/doc/cpplus_RM/hierarchy.html
=20
 H5::File is both an H5::Location and H5::CommonFG (but not an=20
 H5::Object)
 H5::Group is both an H5::Object (subclass of H5::Location) and=20
 H5::CommonFG
 H5::Dataset is an H5::Object


i see something named "CommonFG" here, which seems to good thing to
move out of hierarchy altogether.

bwah, i don't even sure that given hierarchy is good for D. C++ has no
UFCS, and it's incredibly hard to check if some entity has some
methods/properties in C++, so they have no other choice than to work
around that limitations. it may be worthful to redesign the whole thing
for D, exploiting D shiny UFCS and metaprogramming features. and,
maybe, moving some things to interfaces too.
Jan 13 2015
parent reply "Laeeth Isharc" <laeethnospam spammenot_laeeth.com> writes:
 I see, thanks! :) I've started liking structs more and more 
 recently as well and been pondering on how to convert a 
 class-based code that looks like this (only the base class 
 has any data):


 it's hard to tell by brief description. but having multiple 
 inheritance
 immediately rings an alarm ring for me. something is 
 very-very-very
 wrong if you need to have a winged whale. ;-)


 A real-world example: 
 http://www.hdfgroup.org/HDF5/doc/cpplus_RM/hierarchy.html
 
 H5::File is both an H5::Location and H5::CommonFG (but not an 
 H5::Object)
 H5::Group is both an H5::Object (subclass of H5::Location) and 
 H5::CommonFG
 H5::Dataset is an H5::Object


 i see something named "CommonFG" here, which seems to good 
 thing to
 move out of hierarchy altogether.

 bwah, i don't even sure that given hierarchy is good for D. C++ 
 has no
 UFCS, and it's incredibly hard to check if some entity has some
 methods/properties in C++, so they have no other choice than to 
 work
 around that limitations. it may be worthful to redesign the 
 whole thing
 for D, exploiting D shiny UFCS and metaprogramming features. 
 and,
 maybe, moving some things to interfaces too.


I just finished reading aldanor's blog, so I know he is slightly 
allergic to naked functions and prefers classes ;)

With Ketmar, I very much agree (predominantly as a user of HDF5 
and less so as an inexperienced D programmr writing a wrapper for 
it).  It's a pain to figure out just how to do simple things 
until you know the H5 library.  You have to create an object for 
file permissions before you even get started, then similarly for 
the data series (datasets) within, another for the dimensions of 
the array, etc etc  - that doesn't fit with the intrinsic nature 
of the domain.

There is a more general question of bindings/wrappers - preserve 
the original structure and naming so existing code can be ported, 
or write a wrapper that makes it easy for the user to accomplish 
his objectives.  It seems like for the bindings preserving the 
library structure is fine, but for the wrapper one might as well 
make things easy.

Eg here https://gist.github.com/Laeeth/9637233db41a11a9d1f4
line 146.  (sorry for duplication and messiness of code, which I 
don't claim to be perfectly written - I wanted to try something 
quickly and have not yet tidied up).

So rather than navigate the Byzantine hierarchy, one can just do 
something like this (which will take a struct of PriceBar - 
date,open,high,low,close - and put it in your desired dataset and 
file, appending or overwriting as you prefer).
	dumpDataSpaceVector!PriceBar(file,ticker,array(priceBars[ticker]),DumpMode.truncate);

which is closer to h5py in Python.  (It uses reflection to figure 
out the contents of a non-nested struct, but won't yet cope with 
arrays and nested structs inside).  And of course a full wrapper 
might be a bit more complicated, but I truly think one can do 
better than mapping the HDF5 hierarchy one for one.


Laeeth.
Jan 13 2015
next sibling parent ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
On Tue, 13 Jan 2015 17:08:37 +0000
Laeeth Isharc via Digitalmars-d-learn
<digitalmars-d-learn puremagic.com> wrote:


 I just finished reading aldanor's blog, so I know he is slightly=20
 allergic to naked functions and prefers classes ;)


that's due to absense of modules in C/C++. and namespaces aren't of big
help here too. and, of course, due to missing UFCS, that prevents nice
`obj.func()` for free functions. ;-)
Jan 13 2015
prev sibling parent reply "aldanor" <i.s.smirnov gmail.com> writes:
On Tuesday, 13 January 2015 at 17:08:38 UTC, Laeeth Isharc wrote:

 I see, thanks! :) I've started liking structs more and 
 more recently as well and been pondering on how to convert 
 a class-based code that looks like this (only the base 
 class has any data):


 it's hard to tell by brief description. but having multiple 
 inheritance
 immediately rings an alarm ring for me. something is 
 very-very-very
 wrong if you need to have a winged whale. ;-)


 A real-world example: 
 http://www.hdfgroup.org/HDF5/doc/cpplus_RM/hierarchy.html
 
 H5::File is both an H5::Location and H5::CommonFG (but not an 
 H5::Object)
 H5::Group is both an H5::Object (subclass of H5::Location) 
 and H5::CommonFG
 H5::Dataset is an H5::Object


 i see something named "CommonFG" here, which seems to good 
 thing to
 move out of hierarchy altogether.

 bwah, i don't even sure that given hierarchy is good for D. 
 C++ has no
 UFCS, and it's incredibly hard to check if some entity has some
 methods/properties in C++, so they have no other choice than 
 to work
 around that limitations. it may be worthful to redesign the 
 whole thing
 for D, exploiting D shiny UFCS and metaprogramming features. 
 and,
 maybe, moving some things to interfaces too.


 I just finished reading aldanor's blog, so I know he is 
 slightly allergic to naked functions and prefers classes ;)

 With Ketmar, I very much agree (predominantly as a user of HDF5 
 and less so as an inexperienced D programmr writing a wrapper 
 for it).  It's a pain to figure out just how to do simple 
 things until you know the H5 library.  You have to create an 
 object for file permissions before you even get started, then 
 similarly for the data series (datasets) within, another for 
 the dimensions of the array, etc etc  - that doesn't fit with 
 the intrinsic nature of the domain.

 There is a more general question of bindings/wrappers - 
 preserve the original structure and naming so existing code can 
 be ported, or write a wrapper that makes it easy for the user 
 to accomplish his objectives.  It seems like for the bindings 
 preserving the library structure is fine, but for the wrapper 
 one might as well make things easy.

 Eg here https://gist.github.com/Laeeth/9637233db41a11a9d1f4
 line 146.  (sorry for duplication and messiness of code, which 
 I don't claim to be perfectly written - I wanted to try 
 something quickly and have not yet tidied up).

 So rather than navigate the Byzantine hierarchy, one can just 
 do something like this (which will take a struct of PriceBar - 
 date,open,high,low,close - and put it in your desired dataset 
 and file, appending or overwriting as you prefer).
 	dumpDataSpaceVector!PriceBar(file,ticker,array(priceBars[ticker]),DumpMode.truncate);

 which is closer to h5py in Python.  (It uses reflection to 
 figure out the contents of a non-nested struct, but won't yet 
 cope with arrays and nested structs inside).  And of course a 
 full wrapper might be a bit more complicated, but I truly think 
 one can do better than mapping the HDF5 hierarchy one for one.


 Laeeth.


In the hierarchy example above (c++ hdf hierarchy link), by using 
UFCS to implement the shared methods (which are achieved by 
multiple inheritance in the c++ counterpart) did you mean 
something like this?

// id.d
struct ID { int id; ... }

// location.d
struct Location { ID _id; alias _id this; ... }

// file.d
public import commonfg; // ugh
struct File { Location _location; alias _location this; ... }

// group.d
public import commonfg;
struct File { Location _location; alias _location this; ... }

// commonfg.d { ... }
enum isContainer(T) = is(T: File) || is(T : Group);
auto method1(T)(T obj, args) if (isContainer!T) { ... }
auto method2(T)(T obj, args) if (isContainer!T) { ... }

I guess two of my gripes with UFCS is (a) you really have to use 
public imports in the modules where the target types are defined 
so you bring all the symbols in whether you want it or not (b) 
you lose access to private members because it's not the same 
module anymore (correct me if I'm wrong?). Plus, you need to 
decorate every single free function with a template constraint.

// another hdf-specific thing here but a good example in general 
is that some functions return you an id for an object which is 
one of the location subtypes (e.g. it could be a File or could be 
a Group depending on run-time conditions), so it kind of feels 
natural to use polymorphism and classes for that, but what would 
you do with the struct approach? The only thing that comes to 
mind is Variant, but it's quite meh to use in practice.
Jan 13 2015
next sibling parent ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
On Tue, 13 Jan 2015 18:35:15 +0000
aldanor via Digitalmars-d-learn <digitalmars-d-learn puremagic.com>
wrote:


 I guess two of my gripes with UFCS is (a) you really have to use=20
 public imports in the modules where the target types are defined=20
 so you bring all the symbols in whether you want it or not (b)=20
 you lose access to private members because it's not the same=20
 module anymore (correct me if I'm wrong?). Plus, you need to=20
 decorate every single free function with a template constraint.


you can make a package and set protection to `package` instead of
`private`, so your function will still be able to access internal
fields, but package users will not. this feature is often missed
by the people who are used to `public`/`protected`/`private` triad.
Jan 13 2015
prev sibling parent reply "Laeeth Isharc" <Laeethnospam nospam.laeeth.com> writes:
 In the hierarchy example above (c++ hdf hierarchy link), by 
 using UFCS to implement the shared methods (which are achieved 
 by multiple inheritance in the c++ counterpart) did you mean 
 something like this?

 // id.d
 struct ID { int id; ... }

 // location.d
 struct Location { ID _id; alias _id this; ... }

 // file.d
 public import commonfg; // ugh
 struct File { Location _location; alias _location this; ... }

 // group.d
 public import commonfg;
 struct File { Location _location; alias _location this; ... }

 // commonfg.d { ... }
 enum isContainer(T) = is(T: File) || is(T : Group);
 auto method1(T)(T obj, args) if (isContainer!T) { ... }
 auto method2(T)(T obj, args) if (isContainer!T) { ... }

 I guess two of my gripes with UFCS is (a) you really have to



 // another hdf-specific thing here but a good example in 
 general is that some functions return you an id for an object 
 which is one of the location subtypes (e.g. it could be a File 
 or could be a Group depending on run-time conditions), so it 
 kind of feels natural to use polymorphism and classes for that, 
 but what would you do with the struct approach? The only thing 
 that comes to mind is Variant, but it's quite meh to use in 
 practice.


Void unlink(File f){}
Void unlink(Group g){}

For simple cases maybe one can keep it simple, and despite the 
Byzantine interface what one is trying to do when using HDF5 is 
not intrinsically so complex.
Jan 14 2015
parent reply "aldanor" <i.s.smirnov gmail.com> writes:
On Wednesday, 14 January 2015 at 14:54:09 UTC, Laeeth Isharc 
wrote:

 In the hierarchy example above (c++ hdf hierarchy link), by 
 using UFCS to implement the shared methods (which are achieved 
 by multiple inheritance in the c++ counterpart) did you mean 
 something like this?

 // id.d
 struct ID { int id; ... }

 // location.d
 struct Location { ID _id; alias _id this; ... }

 // file.d
 public import commonfg; // ugh
 struct File { Location _location; alias _location this; ... }

 // group.d
 public import commonfg;
 struct File { Location _location; alias _location this; ... }

 // commonfg.d { ... }
 enum isContainer(T) = is(T: File) || is(T : Group);
 auto method1(T)(T obj, args) if (isContainer!T) { ... }
 auto method2(T)(T obj, args) if (isContainer!T) { ... }

 I guess two of my gripes with UFCS is (a) you really have to



 // another hdf-specific thing here but a good example in 
 general is that some functions return you an id for an object 
 which is one of the location subtypes (e.g. it could be a File 
 or could be a Group depending on run-time conditions), so it 
 kind of feels natural to use polymorphism and classes for 
 that, but what would you do with the struct approach? The only 
 thing that comes to mind is Variant, but it's quite meh to use 
 in practice.


 Void unlink(File f){}
 Void unlink(Group g){}

 For simple cases maybe one can keep it simple, and despite the 
 Byzantine interface what one is trying to do when using HDF5 is 
 not intrinsically so complex.


So your solution is copying and pasting the code?

But now repeat that for 200 other functions and a dozen more 
types that can be polymorphic in weirdest ways possible...
Jan 14 2015
parent reply "Laeeth Isharc" <Laeeth.nospam nospam-laeeth.com> writes:
 struct File { Location _location; alias _location this; ... }

 // group.d
 public import commonfg;
 struct File { Location _location; alias _location this; ... }

 // commonfg.d { ... }
 enum isContainer(T) = is(T: File) || is(T : Group);
 auto method1(T)(T obj, args) if (isContainer!T) { ... }
 auto method2(T)(T obj, args) if (isContainer!T) { ... }

 I guess two of my gripes with UFCS is (a) you really have to



 // another hdf-specific thing here but a good example in 
 general is that some functions return you an id for an object 
 which is one of the location subtypes (e.g. it could be a 
 File or could be a Group depending on run-time conditions), 
 so it kind of feels natural to use polymorphism and classes 
 for that, but what would you do with the struct approach? The 
 only thing that comes to mind is Variant, but it's quite meh 
 to use in practice.


 Void unlink(File f){}
 Void unlink(Group g){}

 For simple cases maybe one can keep it simple, and despite the 
 Byzantine interface what one is trying to do when using HDF5 
 is not intrinsically so complex.


 So your solution is copying and pasting the code?

 But now repeat that for 200 other functions and a dozen more 
 types that can be polymorphic in weirdest ways possible...


If you are simply have a few lines calling the API and the 
validation is different enough for file and group (I haven't 
written unlink yet) then why not (and move proper shared code out 
into helper functions).  The alternative is a long method with 
lots of conditions, which may be the best in some cases but may 
be harder to follow.

I do like the h5py and pytables approaches.  One doesn't need to 
bother too much with the implementation when using their library. 
  However, what I am doing is quite simple from a data perspective 
- a decent amount of it, but it is not an interesting problem 
from a theoretical perspective - just execution.  Now if you are 
higher octane as a user you may be able to see what I cannot.  
But on the other hand, the Pareto principle applies, and in my 
view a library should make it simple to do simple things.  One 
can't get there if the primary interface is a direct mapping of 
the HDF5 hierarchy, and I also think that is unnecessary with D.

But I very much appreciate your work as the final result is 
better for everyone that way, and you are evidently a much longer 
running user of D than me.  I never used C++ as it just seemed 
too ugly! and I suspect the difference in backgrounds is shaping 
perspectives.

What do you think the trickiest parts are with HDF5?  (You 
mention weird polymorphism).



Laeeth
Jan 14 2015
parent reply "aldanor" <i.s.smirnov gmail.com> writes:
On Wednesday, 14 January 2015 at 16:27:17 UTC, Laeeth Isharc 
wrote:

 struct File { Location _location; alias _location this; ... }

 // group.d
 public import commonfg;
 struct File { Location _location; alias _location this; ... }

 // commonfg.d { ... }
 enum isContainer(T) = is(T: File) || is(T : Group);
 auto method1(T)(T obj, args) if (isContainer!T) { ... }
 auto method2(T)(T obj, args) if (isContainer!T) { ... }

 I guess two of my gripes with UFCS is (a) you really have to



 // another hdf-specific thing here but a good example in 
 general is that some functions return you an id for an 
 object which is one of the location subtypes (e.g. it could 
 be a File or could be a Group depending on run-time 
 conditions), so it kind of feels natural to use polymorphism 
 and classes for that, but what would you do with the struct 
 approach? The only thing that comes to mind is Variant, but 
 it's quite meh to use in practice.


 Void unlink(File f){}
 Void unlink(Group g){}

 For simple cases maybe one can keep it simple, and despite 
 the Byzantine interface what one is trying to do when using 
 HDF5 is not intrinsically so complex.


 So your solution is copying and pasting the code?

 But now repeat that for 200 other functions and a dozen more 
 types that can be polymorphic in weirdest ways possible...


 If you are simply have a few lines calling the API and the 
 validation is different enough for file and group (I haven't 
 written unlink yet) then why not (and move proper shared code 
 out into helper functions).  The alternative is a long method 
 with lots of conditions, which may be the best in some cases 
 but may be harder to follow.

 I do like the h5py and pytables approaches.  One doesn't need 
 to bother too much with the implementation when using their 
 library.
  However, what I am doing is quite simple from a data 
 perspective - a decent amount of it, but it is not an 
 interesting problem from a theoretical perspective - just 
 execution.  Now if you are higher octane as a user you may be 
 able to see what I cannot.  But on the other hand, the Pareto 
 principle applies, and in my view a library should make it 
 simple to do simple things.  One can't get there if the primary 
 interface is a direct mapping of the HDF5 hierarchy, and I also 
 think that is unnecessary with D.

 But I very much appreciate your work as the final result is 
 better for everyone that way, and you are evidently a much 
 longer running user of D than me.  I never used C++ as it just 
 seemed too ugly! and I suspect the difference in backgrounds is 
 shaping perspectives.

 What do you think the trickiest parts are with HDF5?  (You 
 mention weird polymorphism).



 Laeeth


I don't think you've read h5py source in enough detail :) It's 
based HEAVILY on duck typing. In addition, it has way MORE 
classes than the C++ hierarchy does. E.g., the high-level File 
object actually has these parents: File : Group, Group : 
HLObject, MutableMappingWithLock, HLObject : CommonStateObject 
and internally the File also keeps a reference to file id which 
is an instance of FileID which inherits from GroupID which 
inherits from ObjectID, do I need to continue? :) PyTables, on 
the contrary is quite badly written (although it works quite well 
and there are brilliant folks on the dev team like francesc 
alted) and looks like a dump of C code interweaved with hackish 
Python code.

In h5py you can do things like file["/dataset"].write(...) --> 
this just wouldn't work as is in a strictly typed language since 
the indexing operator generally returns you something of a 
Location type (or an interface, rather) which can be a 
group/datatype/dataset which is only known at runtime. Out of all 
of them, only the dataset supports the write method but you don't 
know it's going to be a dataset. See the problem? I don't want 
the user code to deal with any of the HDF5 C API and/or have a 
bunch of if conditions or explicit casts which is outright ugly. 
Ideally, it would work kind of like H5PY, abstracting the user 
away from refcounting, error code checking after each operation, 
object type checking and all that stuff.
Jan 14 2015
parent "Laeeth Isharc" <laeethnospam nospamlaeeth.com> writes:
 I don't think you've read h5py source in enough detail :)


You're right - I haven't done more than browsed it.


 It's based HEAVILY on duck typing.


There is a question here about what to do in D.  On the one hand, 
the flexibility of being able to open a foreign HDF5 file where 
you don't know beforehand the dataset type is very nice.  On the 
other, the adaptations needed to handle this flexibly get in the 
way when you are dealing with your own data that has a set format 
and where recompilation is acceptable if it changes.  Looking at 
the 'ease' of processing JSON, even using vibed, I think that one 
will need to implement both eventually, but perhaps starting with 
static typing.



 In addition, it has way MORE classes than the C++ hierarchy 
 does. E.g., the high-level File object actually has these 
 parents: File : Group, Group : HLObject, 
 MutableMappingWithLock, HLObject : CommonStateObject and 
 internally the File also keeps a reference to file id which is 
 an instance of FileID which inherits from GroupID which 
 inherits from ObjectID, do I need to continue?


Okay - I guess there is a distinction between the interface to 
the outside world (where I think the h5py etc way is superior for 
most uses) and the implementation.  Is not the reason h5py has 
lots of classes primarily because that is how you write good code 
in python, whereas in many cases this is not true in D (not that 
you should ban classes, but often structs + free floating 
functions are more suitable).


 PyTables, on the contrary is quite badly written (although it 
 works quite well and there are brilliant folks on the dev team 
 like francesc alted) and looks like a dump of C code 
 interweaved with hackish Python code.


Interesting.  What do you think is low quality about the design?


 In h5py you can do things like file["/dataset"].write(...) --> 
 this just wouldn't work as is in a strictly typed language 
 since the indexing operator generally returns you something of 
 a Location type (or an interface, rather) which can be a 
 group/datatype/dataset which is only known at runtime.


Well, if you don't mind recompiling your code when the data set 
type changes (or you encounter a new data set) then you can do 
that (which is what I posted a link to earlier).

It depends on your use case.  It's hard to think of an 
application more dynamic than web sites, and yet people seem 
happy enough with vibed's use of compiled diet templates as the 
primary implementation.  They would like the option of dynamic 
ones too, and I think this would be useful in this domain too, 
since one does look at foreign data on occasion.  One could of 
course use the quick compilation of D to regenerate parts of the 
code when this happens.  Whether or not this is acceptable 
depends on your use case - for some it might be okay, but 
obviously it is no good if you are writing a generic H5 
browser/charting tool.

So I think if you don't allow static dataset typing it means the 
flexibility  of dynamic typing gets in the way for some uses 
(which might be most of them), but you need to add dynamic typing 
too.

Shall we move this to a different thread and/or email, as I am 
afraid I have hijacked the poor original poster's request.

On the refcounting question, I confess that I do not fully 
understand your concern, which may well reflect a lack of deep 
experience with D on my part.  Adam Ruppe suggests that it's 
generally okay to rely on a struct destructor to call C cleanup 
code.  I can appreciate this may not be true with h5 and, if you 
can spare the time, I would love to understand more precisely why 
not.


 Out of all of them, only the dataset supports the write method 
 but you don't know it's going to be a dataset. See the problem?


In this case I didn't quite follow.  Where does this fall down ?

void h5write(T)(Dataset x, T data)


I have your email somewhere and will drop you a line.  Or you can 
email me laeeth at laeeth.com.  And let's create a new thread.



Laeeth.
Jan 14 2015
prev sibling parent ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
On Mon, 12 Jan 2015 22:07:13 +0000
aldanor via Digitalmars-d-learn <digitalmars-d-learn puremagic.com>
wrote:


 I see, thanks! :) I've started liking structs more and more=20
 recently as well and been pondering on how to convert a=20
 class-based code that looks like this (only the base class has=20
 any data):


p.s. can't you convert most of that to free functions? thanks to UFCS
you'll be able to use them with `obj.func` notation. and by either
defining `package` protection for class fields, or simply writing that
functions in the same module they will have access to internal object
stuff.

and if you can return `obj` from each function, you can go with
templates and chaining. ;-)
Jan 13 2015
prev sibling parent reply "jmh530" <john.michael.hall gmail.com> writes:
I had seen some stuff on alias thing, but I hadn't bothered to
try to understand it until now. If I'm understanding the first
example <a href="http://dlang.org/class.html#AliasThis">here</a>,
alias this let's you refer to x in s by writing either s.x (as
normally) or just s. That didn't seem that interesting, but then
I found <a href="http://3d.benjamin-thaut.de/?p=90">example</a>
where they alias this'ed a struct method. That's pretty
interesting.

OOP seems like a good idea, but every time I've written a bunch
of classes in C++ or Python, I inevitably wonder to myself why I
just spent 5 times as long doing something I could do with
functions. Then there's endless discussion about pimpl.

On Monday, 12 January 2015 at 21:54:51 UTC, ketmar via
Digitalmars-d-learn wrote:

 On Mon, 12 Jan 2015 21:37:27 +0000
 aldanor via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com>
 wrote:


 On Monday, 12 January 2015 at 20:30:45 UTC, ketmar via 
 Digitalmars-d-learn wrote:

 it even has `RefCounted!`, but it doesn't play well with 
 classes yet
 (AFAIR).


 I wonder if it's possible to somehow make a version of 
 refcounted that would work with classes (even if 
 limited/restricted in some certain ways), or is it just 
 technically impossible because of reference semantics?


 it's hard. especially hard when you considering inheritance 
 (which is
 not playing well with templates) and yes, ref semantics.

 on the other side i found myself rarely using classes at all. i 
 mostly
 writing templates that checks if a passed "thing" has all the
 neccessary methods and properties in place and just using that. 
 with D
 metaprogramming abilities (and `alias this` trick ;-) 
 inheritance
 becomes not so important. and so classes too. sometimes i'm 
 using
 structs with delegate fields to simulate some sort of "virtual 
 methods"
 'cause i tend to constantly forgetting about that `class` 
 thingy. ;-)

 OOP is overrated. at least c++-like (should i say 
 "simula-like"?)
 OOP. ;-)
Jan 12 2015
parent ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
On Mon, 12 Jan 2015 23:06:16 +0000
jmh530 via Digitalmars-d-learn <digitalmars-d-learn puremagic.com>
wrote:


 I had seen some stuff on alias thing, but I hadn't bothered to
 try to understand it until now. If I'm understanding the first
 example <a href=3D"http://dlang.org/class.html#AliasThis">here</a>,
 alias this let's you refer to x in s by writing either s.x (as
 normally) or just s. That didn't seem that interesting, but then
 I found <a href=3D"http://3d.benjamin-thaut.de/?p=3D90">example</a>
 where they alias this'ed a struct method. That's pretty
 interesting.


there is nice page by p0nce and it shows nice and simple `alias this`
trick usage:
http://p0nce.github.io/d-idioms/#Extending-a-struct-with-alias-this

and i have "stream.d" module in my IV package
( http://repo.or.cz/w/iv.d.git/tree ), which works with "i/o streams"
by testing if passed struct/class has necessary methods (`isReadable!`,
`isWriteable!`, `isSeekable!`, etc.
Jan 13 2015
prev sibling parent "Mike" <none none.com> writes:
On Monday, 12 January 2015 at 19:29:54 UTC, jmh530 wrote:

 I'm new to D. I have some modest knowledge of C++, but am more 
 familiar with scripting languages (Matlab, Python, R). D seems 
 so much easier than C++ in a lot of ways (and I just learned 
 about rdmd today, which is pretty cool). I am concerned about 
 performance of D vs. C++, so I wanted to learn a little bit 
 more about manual memory management, in case I might ever need 
 it (not for any particular application).


There is a good article on the D Wiki that covers this topic with 
several different patterns and working examples:

http://wiki.dlang.org/Memory_Management

I hope you'll find it helpful.

Mike
Jan 12 2015