www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Is this range behaviour correct?

reply "Andrea Fontana" <nospam example.com> writes:
I'm trying to implement a db cursor as simple InputRange. I can't 
implement as forward range because using c-api I can't clone/save 
cursor.

I wrote popFront() front() and empty() method and my range works 
fine.

Using on a foreach() it browses all elements inside range until 
range is exausthed.

If i do another foreach() it doesn't restart (empty() is called 
and it returns true, of course).

Is it the correct behaviour of a inputrange or have i missed 
something?
Mar 14 2013
next sibling parent reply "Andrea Fontana" <nospam example.com> writes:
On Thursday, 14 March 2013 at 10:08:53 UTC, Andrea Fontana wrote:

 I'm trying to implement a db cursor as simple InputRange. I 
 can't implement as forward range because using c-api I can't 
 clone/save cursor.

 I wrote popFront() front() and empty() method and my range 
 works fine.

 Using on a foreach() it browses all elements inside range until 
 range is exausthed.

 If i do another foreach() it doesn't restart (empty() is called 
 and it returns true, of course).

 Is it the correct behaviour of a inputrange or have i missed 
 something?


Maybe I got it. I implemented this range as class, so it is 
passed as reference to foreach, usually ranges are struct and 
passed by value. I guess this is the problem.

Is there a way to make a class based range like a struct one?
Mar 14 2013
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/14/13 6:45 AM, Andrea Fontana wrote:

 On Thursday, 14 March 2013 at 10:08:53 UTC, Andrea Fontana wrote:

 I'm trying to implement a db cursor as simple InputRange. I can't
 implement as forward range because using c-api I can't clone/save cursor.

 I wrote popFront() front() and empty() method and my range works fine.

 Using on a foreach() it browses all elements inside range until range
 is exausthed.

 If i do another foreach() it doesn't restart (empty() is called and it
 returns true, of course).

 Is it the correct behaviour of a inputrange or have i missed something?


 Maybe I got it. I implemented this range as class, so it is passed as
 reference to foreach, usually ranges are struct and passed by value. I
 guess this is the problem.

 Is there a way to make a class based range like a struct one?


Add .save() to it.

Andrei
Mar 14 2013
parent reply "Andrea Fontana" <nospam example.com> writes:
On Thursday, 14 March 2013 at 12:29:26 UTC, Andrei Alexandrescu 
wrote:

 On 3/14/13 6:45 AM, Andrea Fontana wrote:

 On Thursday, 14 March 2013 at 10:08:53 UTC, Andrea Fontana 
 wrote:

 I'm trying to implement a db cursor as simple InputRange. I 
 can't
 implement as forward range because using c-api I can't 
 clone/save cursor.




 Add .save() to it.

 Andrei


I can't add save and create a forward range: c-api doesn't allow 
me to clone db cursor.
Mar 14 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 14 March 2013 at 13:20:51 UTC, Andrea Fontana wrote:

 On Thursday, 14 March 2013 at 12:29:26 UTC, Andrei Alexandrescu 
 wrote:

 On 3/14/13 6:45 AM, Andrea Fontana wrote:

 On Thursday, 14 March 2013 at 10:08:53 UTC, Andrea Fontana 
 wrote:

 I'm trying to implement a db cursor as simple InputRange. I 
 can't
 implement as forward range because using c-api I can't 
 clone/save cursor.




 Add .save() to it.

 Andrei


 I can't add save and create a forward range: c-api doesn't 
 allow me to clone db cursor.


But think about it. You say you can't save, but you want pass by 
value to preserve state. If you can preserve state, doesn't that 
mean you just saved?

Once you've passed your "can't preserve state" to a foreach, then 
your range is consumed. So yes, it is correct behavior.

Most input ranges should model reference semantics when 
passed/copied by value. If somebody modifies the range, all 
ranges are affected. The tough part is usually preserving correct 
state.
Mar 14 2013
parent reply "Andrea Fontana" <nospam example.com> writes:
On Thursday, 14 March 2013 at 13:58:56 UTC, monarch_dodra wrote:

 On Thursday, 14 March 2013 at 13:20:51 UTC, Andrea Fontana 
 wrote:

 On Thursday, 14 March 2013 at 12:29:26 UTC, Andrei 
 Alexandrescu wrote:

 On 3/14/13 6:45 AM, Andrea Fontana wrote:

 On Thursday, 14 March 2013 at 10:08:53 UTC, Andrea Fontana 
 wrote:

 I'm trying to implement a db cursor as simple InputRange. I 
 can't
 implement as forward range because using c-api I can't 
 clone/save cursor.




 Add .save() to it.

 Andrei


 I can't add save and create a forward range: c-api doesn't 
 allow me to clone db cursor.


 But think about it. You say you can't save, but you want pass 
 by value to preserve state. If you can preserve state, doesn't 
 that mean you just saved?

 Once you've passed your "can't preserve state" to a foreach, 
 then your range is consumed. So yes, it is correct behavior.

 Most input ranges should model reference semantics when 
 passed/copied by value. If somebody modifies the range, all 
 ranges are affected. The tough part is usually preserving 
 correct state.


I was hoping there's something I didn't know like rewind() to 
implement :)

Ok, I'll pray db developer to add a clone method or I'll use 
something like

auto res = cursor.array(); if needed
Mar 14 2013
next sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 14 March 2013 at 14:29:59 UTC, Andrea Fontana wrote:

 On Thursday, 14 March 2013 at 13:58:56 UTC, monarch_dodra wrote:

 On Thursday, 14 March 2013 at 13:20:51 UTC, Andrea Fontana 
 wrote:

 On Thursday, 14 March 2013 at 12:29:26 UTC, Andrei 
 Alexandrescu wrote:

 On 3/14/13 6:45 AM, Andrea Fontana wrote:

 On Thursday, 14 March 2013 at 10:08:53 UTC, Andrea Fontana 
 wrote:

 I'm trying to implement a db cursor as simple InputRange. 
 I can't
 implement as forward range because using c-api I can't 
 clone/save cursor.




 Add .save() to it.

 Andrei


 I can't add save and create a forward range: c-api doesn't 
 allow me to clone db cursor.


 But think about it. You say you can't save, but you want pass 
 by value to preserve state. If you can preserve state, doesn't 
 that mean you just saved?

 Once you've passed your "can't preserve state" to a foreach, 
 then your range is consumed. So yes, it is correct behavior.

 Most input ranges should model reference semantics when 
 passed/copied by value. If somebody modifies the range, all 
 ranges are affected. The tough part is usually preserving 
 correct state.


 I was hoping there's something I didn't know like rewind() to 
 implement :)

 Ok, I'll pray db developer to add a clone method or I'll use 
 something like

 auto res = cursor.array(); if needed


You *could* add rewind to your range, but it isn't quite the same 
as save: Rewind means you can move back and forth in your view of 
the file (or cursor/db), but it does not mean you can actually 
*duplicate* (save) your range, to have two distinct ranges with a 
distinct view of the your db.

For example, while the interface is different, a C "FILE*" is 
what you'd call a rewindable input range with reference semantics.
Mar 14 2013
parent "Andrea Fontana" <nospam example.com> writes:
On Thursday, 14 March 2013 at 14:51:42 UTC, monarch_dodra wrote:

 On Thursday, 14 March 2013 at 14:29:59 UTC, Andrea Fontana 
 wrote:

 On Thursday, 14 March 2013 at 13:58:56 UTC, monarch_dodra 
 wrote:

 On Thursday, 14 March 2013 at 13:20:51 UTC, Andrea Fontana 
 wrote:

 On Thursday, 14 March 2013 at 12:29:26 UTC, Andrei 
 Alexandrescu wrote:

 On 3/14/13 6:45 AM, Andrea Fontana wrote:

 On Thursday, 14 March 2013 at 10:08:53 UTC, Andrea Fontana 
 wrote:

 I'm trying to implement a db cursor as simple InputRange. 
 I can't
 implement as forward range because using c-api I can't 
 clone/save cursor.




 Add .save() to it.

 Andrei


 I can't add save and create a forward range: c-api doesn't 
 allow me to clone db cursor.


 But think about it. You say you can't save, but you want pass 
 by value to preserve state. If you can preserve state, 
 doesn't that mean you just saved?

 Once you've passed your "can't preserve state" to a foreach, 
 then your range is consumed. So yes, it is correct behavior.

 Most input ranges should model reference semantics when 
 passed/copied by value. If somebody modifies the range, all 
 ranges are affected. The tough part is usually preserving 
 correct state.


 I was hoping there's something I didn't know like rewind() to 
 implement :)

 Ok, I'll pray db developer to add a clone method or I'll use 
 something like

 auto res = cursor.array(); if needed


 You *could* add rewind to your range, but it isn't quite the 
 same as save: Rewind means you can move back and forth in your 
 view of the file (or cursor/db), but it does not mean you can 
 actually *duplicate* (save) your range, to have two distinct 
 ranges with a distinct view of the your db.

 For example, while the interface is different, a C "FILE*" is 
 what you'd call a rewindable input range with reference 
 semantics.


Yes it was just a way to avoid dead cursors locked at end of 
results :)
I add a feature request hoping they add a method to clone cursors 
(i think/hope it's possible)

In the meanwhile you have to do query again.
Mar 15 2013
prev sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 03/14/2013 07:29 AM, Andrea Fontana wrote:

 On Thursday, 14 March 2013 at 13:58:56 UTC, monarch_dodra wrote:



 Most input ranges should model reference semantics when passed/copied
 by value. If somebody modifies the range, all ranges are affected. The
 tough part is usually preserving correct state.


 I was hoping there's something I didn't know like rewind() to 


implement :)

 Ok, I'll pray db developer to add a clone method or I'll use 


something like

 auto res = cursor.array(); if needed


I had toyed with the idea of making a ForwardRange from an InputRange by 
caching the elements. Without any guarantees, :) here is the code:

   http://forum.dlang.org/thread/ifg5ei$2qc7$1 digitalmars.com

Ali
Mar 14 2013
parent reply "Jesse Phillips" <Jessekphillips+D gmail.com> writes:
On Thursday, 14 March 2013 at 21:40:34 UTC, Ali Çehreli wrote:

 I had toyed with the idea of making a ForwardRange from an 
 InputRange by caching the elements. Without any guarantees, :) 
 here is the code:

   http://forum.dlang.org/thread/ifg5ei$2qc7$1 digitalmars.com

 Ali


I attempted to create a sliceable forward range from an input 
range. I think I ran into a problem of keeping the buffer in sync 
when doing lookahead.

list2 = list1.save();

while(true) {
     list1.popFront();
     list2.popFront();

     assert(list1.front == list2.front);
}

What happens when your buffer needs to fill, how do you get the 
data in list 1 & 2? I'm not posing this question as impossible 
only something to make work.
Mar 14 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 03/14/2013 04:58 PM, Jesse Phillips wrote:

 On Thursday, 14 March 2013 at 21:40:34 UTC, Ali Çehreli wrote:

 I had toyed with the idea of making a ForwardRange from an InputRange
 by caching the elements. Without any guarantees, :) here is the code:

 http://forum.dlang.org/thread/ifg5ei$2qc7$1 digitalmars.com

 Ali


 I attempted to create a sliceable forward range from an input range. I
 think I ran into a problem of keeping the buffer in sync when doing
 lookahead.

 list2 = list1.save();

 while(true) {
 list1.popFront();
 list2.popFront();

 assert(list1.front == list2.front);
 }

 What happens when your buffer needs to fill, how do you get the data in
 list 1 & 2? I'm not posing this question as impossible only something to
 make work.


I had to study that old code of mine to understand how it works. :)

The popFront() on the range itself dispatches the call to 
ForwardRangeLazyCache by providing the id of this "view". In a sense, it 
says "the range with this id would like to pop an element":

      property void popFront()
     {
         lazyCache.popFront(myId);
     }

ForwardRangeLazyCache.popFront simply increments the pointer of the 
range that is associated with that id:

     void popFront(size_t id)
     {
         ++(viewPtrs[id]);
     }

I am not sure whether this is a good idea but the actual filling of the 
cache and synchronizing the "views" happen in front():

      property immutable(E) front(size_t id)
     {
         ensureRead(viewPtrs[id]);
         return *(viewPtrs[id]);
     }

The above says "ensure that the element that this view is trying to 
access is in the cache", and then returns that element. It looks like 
there is even some optimization if the array elements have not been moved:

     void ensureRead(immutable(E) * ptr)
     {
         if (ptr == endPtr) {
             immutable oldPtr = cache.ptr;
             cache ~= range.front.idup;
             range.popFront();

             if (cache.ptr != oldPtr) {
                 /* The cache has been relocated */
                 adjustPtrs(oldPtr, cache.ptr);

             } else {
                 ++endPtr;
             }
         }
     }

etc. :) I am really not sure whether it works correctly. :/

Ali
Mar 14 2013
parent reply "Andrea Fontana" <nospam example.com> writes:
On Friday, 15 March 2013 at 03:44:51 UTC, Ali Çehreli wrote:

 On 03/14/2013 04:58 PM, Jesse Phillips wrote:

 On Thursday, 14 March 2013 at 21:40:34 UTC, Ali Çehreli wrote:

 I had toyed with the idea of making a ForwardRange from an 
 InputRange
 by caching the elements. Without any guarantees, :) here is 
 the code:

 http://forum.dlang.org/thread/ifg5ei$2qc7$1 digitalmars.com

 Ali


 I attempted to create a sliceable forward range from an input 
 range. I
 think I ran into a problem of keeping the buffer in sync when 
 doing
 lookahead.

 list2 = list1.save();

 while(true) {
 list1.popFront();
 list2.popFront();

 assert(list1.front == list2.front);
 }

 What happens when your buffer needs to fill, how do you get 
 the data in
 list 1 & 2? I'm not posing this question as impossible only 
 something to
 make work.


 I had to study that old code of mine to understand how it 
 works. :)

 The popFront() on the range itself dispatches the call to 
 ForwardRangeLazyCache by providing the id of this "view". In a 
 sense, it says "the range with this id would like to pop an 
 element":

      property void popFront()
     {
         lazyCache.popFront(myId);
     }

 ForwardRangeLazyCache.popFront simply increments the pointer of 
 the range that is associated with that id:

     void popFront(size_t id)
     {
         ++(viewPtrs[id]);
     }

 I am not sure whether this is a good idea but the actual 
 filling of the cache and synchronizing the "views" happen in 
 front():

      property immutable(E) front(size_t id)
     {
         ensureRead(viewPtrs[id]);
         return *(viewPtrs[id]);
     }

 The above says "ensure that the element that this view is 
 trying to access is in the cache", and then returns that 
 element. It looks like there is even some optimization if the 
 array elements have not been moved:

     void ensureRead(immutable(E) * ptr)
     {
         if (ptr == endPtr) {
             immutable oldPtr = cache.ptr;
             cache ~= range.front.idup;
             range.popFront();

             if (cache.ptr != oldPtr) {
                 /* The cache has been relocated */
                 adjustPtrs(oldPtr, cache.ptr);

             } else {
                 ++endPtr;
             }
         }
     }

 etc. :) I am really not sure whether it works correctly. :/

 Ali


It *seems* to work :) Funny. The only problem is that caching db 
results maybe it's not a good idea if i read too much data.

I'm not sure if it's fine or not but:
- I've changed AsForwardRange from class to struct
- I've added a method (it's an "hack" to make it works):
this(this)
{
	auto dummy = lazyCache.newView(lazyCache.cache.ptr);
	myId = dummy.myId;
}
- I've add  property to save() -> isForwardRange gives false 
without this.

I did this to make it similar to other ranges (original topic!)
Shouldn't a view unregister pointer on its d-tor?

I'm not sure it's correct, is it?
Mar 15 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 03/15/2013 02:57 AM, Andrea Fontana wrote:

 On Friday, 15 March 2013 at 03:44:51 UTC, Ali Çehreli wrote:



 I had toyed with the idea of making a ForwardRange from an InputRange
 by caching the elements. Without any guarantees, :) here is the code:









 It *seems* to work :) Funny. The only problem is that caching db results
 maybe it's not a good idea if i read too much data.


Agreed but that's the only feature of the range. :) The way the data is 
"uncached" is by moving the slice forward by this code:

         /* Nobody is using the first part of the cache anymore */
         cache = cache[(minPtr - cache.ptr) .. $];

I think it is sufficient for the GC to free the previous first part of 
the array. The following program keeps running without consuming the memory:

void main()
{
     int[] a;
     while (true) {
         a ~= 42;
         a = a[1 .. $];
     }
}


 I'm not sure if it's fine or not but:
 - I've changed AsForwardRange from class to struct


Good idea. It looks like I had already made it a struct in my local copy 
of it.


 - I've added a method (it's an "hack" to make it works):
 this(this)
 {
 auto dummy = lazyCache.newView(lazyCache.cache.ptr);
 myId = dummy.myId;
 }


I am not sure why that was necessary. Additionally, some of the member 
functions like empty() can be made const.


 - I've add  property to save() -> isForwardRange gives false without 


this.

Agreed.


 I did this to make it similar to other ranges (original topic!)
 Shouldn't a view unregister pointer on its d-tor?


That's a good idea too. :)

Ali
Mar 16 2013
parent "Andrea Fontana" <nospam example.com> writes:
On Sunday, 17 March 2013 at 04:54:58 UTC, Ali Çehreli wrote:

 On 03/15/2013 02:57 AM, Andrea Fontana wrote:

 On Friday, 15 March 2013 at 03:44:51 UTC, Ali Çehreli wrote:
 - I've added a method (it's an "hack" to make it works):
 this(this)
 {
 auto dummy = lazyCache.newView(lazyCache.cache.ptr);
 myId = dummy.myId;
 }


 I am not sure why that was necessary. Additionally, some of the 
 member functions like empty() can be made const.


If not something like:

foreach(e; yourrange)  // <-- here this(this) is called
{
    writeln(e);
}

foreach(e; yourrange) // <-- here this(this) is called
{
    writeln(e);
}

Print:
1 2 3 4 5 6

instead of:
1 2 3 1 2 3  as "expected"

But it's an hack, you should know a better way to do it :)
When a struct is copied like in example above, every copy should 
retain its own pointer to data (restarted from begin).
Mar 17 2013
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, March 14, 2013 11:08:52 Andrea Fontana wrote:

 I'm trying to implement a db cursor as simple InputRange. I can't
 implement as forward range because using c-api I can't clone/save
 cursor.
 
 I wrote popFront() front() and empty() method and my range works
 fine.
 
 Using on a foreach() it browses all elements inside range until
 range is exausthed.
 
 If i do another foreach() it doesn't restart (empty() is called
 and it returns true, of course).
 
 Is it the correct behaviour of a inputrange or have i missed
 something?


foreach(e; range)
{
}

translates to something like

for(auto _range = range; !_range.empty; _range.popFront())
{
    auto e = _range.front;
}

That means that foreach will alter the original range unless it's a struct 
which implicitly saves via =. Pure input ranges can't be saved, and therefore 
if you iterate over one until it's empty, it's empty, and it won't magically 
have elements again if you use foreach on it again.

- Jonathan M Davis
Mar 14 2013