www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Introducing alid

reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
I am happy to publish on code.dlang.org for the first time:

   https://code.dlang.org/packages/alid

Thanks to everyone who made registering a dub package so easy! :)


1) Currently, the main module in `alid` is `cached`, which

- caches range elements to ensure each is executed once (most
   useful with generator ranges that use e.g. `map`),

- as a result, elevates InputRanges to ForwardRanges (plus
   opIndex).

2) But I like `errornogc` as well, which helps throw Error
    objects from  nogc code.


Copying from the README:

* `cached` is for executing elements only once while not holding
   on to already popFront'ed elements for too long. `cached` was
   the main topic of Ali's DConf 2022 lightning talk.

* `circularblocks` was written for storing range elements by
   cached. It can be used independently.

* `blockreusable` was written as storage blocks for
   circularblocks. It can be used independently.

* `errornogc` was needed for throwing Errors from  nogc code. It
   can be used independently.

* `test` contains some helper utilities for unit testing.

Ali
Sep 12 2022
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
Looks pretty well tested, nice!

But in other less nice things, I take it you did not test with GDC? GDC 
does not support cli args with the same names as dmd. One of these is -mv.

The file structure of subPackage/alid/subPackage will not require it and 
you will not have the cross import issues, where if you depend on 
errornogc you can also import (and then get linker errors) for 
circularblocks.
Sep 12 2022
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/12/22 07:43, rikki cattermole wrote:

 Looks pretty well tested, nice!


Thanks! Proud with 100% coverage. :)


 But in other less nice things, I take it you did not test with GDC? GDC 
 does not support cli args with the same names as dmd. One of these is -mv.


So far, I started learning by copying arsd's dub.json. (Thank you, Adam! 
The misunderstandings are mine. :) )


 The file structure of subPackage/alid/subPackage will not require it and 
 you will not have the cross import issues, where if you depend on 
 errornogc you can also import (and then get linker errors) for 
 circularblocks.


If I understand you correctly, the directory structure need to be the 
following (also introducing src, which is clearly missing :)):

alid/src/errornogc/alid/errornogc.d
      .../circularblocks/alid/circularblocks.d
[...]

Can I add a CI step to catch all such issues? It would be awesome if dub 
provided that.

Ali

P.S. Another issue is function attributes seemingly used inaccurately 
but I asked that question on the 'learn' newsgroup already. Ping! ;)
Sep 12 2022
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 13/09/2022 4:25 AM, Ali Çehreli wrote:

 On 9/12/22 07:43, rikki cattermole wrote:

 Looks pretty well tested, nice!


 
 Thanks! Proud with 100% coverage. :)


I was going to ask about coverage, that is awesome!


 But in other less nice things, I take it you did not test with GDC? 
 GDC does not support cli args with the same names as dmd. One of these 
 is -mv.


 
 So far, I started learning by copying arsd's dub.json. (Thank you, Adam! 
 The misunderstandings are mine. :) )


It has the same issues, which is why I recommended against copying it.


 The file structure of subPackage/alid/subPackage will not require it 
 and you will not have the cross import issues, where if you depend on 
 errornogc you can also import (and then get linker errors) for 
 circularblocks.


 
 If I understand you correctly, the directory structure need to be the 
 following (also introducing src, which is clearly missing :)):
 
 alid/src/errornogc/alid/errornogc.d
       .../circularblocks/alid/circularblocks.d
 [...]


No, you don't need the src directory, the subPackage directory functions 
as this.

So:

dub.json
errornogc/alid/errornogc.d
circularblocks/alid/circularblocks.d

That'll work.


 Can I add a CI step to catch all such issues? It would be awesome if dub 
 provided that.


There is:

https://github.com/dlang-community/setup-dlang

But it doesn't look to support gdc or have been updated in a little 
while. Guess I need to start pinging people about it.



 Ali
 
 P.S. Another issue is function attributes seemingly used inaccurately 
 but I asked that question on the 'learn' newsgroup already. Ping! ;)


I have no solution to that unfortunately beyond template all the things.
Sep 12 2022
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/12/22 09:34, rikki cattermole wrote:


 dub.json
 errornogc/alid/errornogc.d
 circularblocks/alid/circularblocks.d


Considering I may want to let the users import the entire package as 
well with

   import alid;

how can I achieve my goal of subpackages? Telling me to forget about 
subpackages altogether :) is an option because I've already spent hours 
trying to find my way through different directory structures, random 
dub.json edits, experimenting with ways of stopping dub from fetching 
and using the bad version of the repo repeatedly, and many other random 
things...

The main problem is likely I don't have an accurate mental model of the 
whole thing. For example, I don't know whether dub will take the state 
of my local directory as-is or will it copy from the committed state of 
the local repo? I ask because the add-local command puts a commit hash 
as well. If that's the case, I wouldn't want to commit every little 
change before seeing it's effect.

Yes, I did the path override thing as well...

Ali
Sep 14 2022
next sibling parent Adam D Ruppe <destructionator gmail.com> writes:
On Wednesday, 14 September 2022 at 08:44:48 UTC, Ali Çehreli 
wrote:

 Considering I may want to let the users import the entire 
 package as well with


dub is poorly designed and doesn't cooperate well with D 
features. There's no good solution - hence the hundreds of lines 
long dub.json trying to hack it in. Like you could have the 
top-level package depend on all of its own subpackages and then 
if you depend on it, you can import alid. Lots of spam for 
something that just works out of the box with stock dmd!
Sep 14 2022
prev sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 14/09/2022 8:44 PM, Ali Çehreli wrote:

 On 9/12/22 09:34, rikki cattermole wrote:
 
  > dub.json
  > errornogc/alid/errornogc.d
  > circularblocks/alid/circularblocks.d
 
 Considering I may want to let the users import the entire package as 
 well with
 
    import alid;
 
 how can I achieve my goal of subpackages? Telling me to forget about 
 subpackages altogether :) is an option because I've already spent hours 
 trying to find my way through different directory structures, random 
 dub.json edits, experimenting with ways of stopping dub from fetching 
 and using the bad version of the repo repeatedly, and many other random 
 things...


In your root package you can still have the package.d file.

You would use the version added by Dub to detect if you should public 
import the other modules.


 DUB provides version identifier of dependencies for conditional 


compilation with version conditions. Have_<dependency> version 
identifier can be used for conditional compilation.

https://dub.pm/advanced_usage
Sep 14 2022
prev sibling next sibling parent reply Salih Dincer <salihdb hotmail.com> writes:
On Monday, 12 September 2022 at 09:15:34 UTC, Ali Çehreli wrote:

 I am happy to publish on code.dlang.org for the first time:

   https://code.dlang.org/packages/alid

 Thanks to everyone who made registering a dub package so easy! 
 :)


Hoş geldin AliD :)

Thank you very much for bringing these codes over a thousand 
lines to the D World.  I was looking forward to trying it right 
away, and when I tried it with the classic iota() and its sister 
inclusiveRange(), I found that it didn't get along well with 
cycle().

For example:
```d
immutable size_t pageSize = 4096;

void main()
{
   "D Compiler v".writeln(__VERSION__*.001); // D Compiler v2.087
   {
     auto r = iota(3).cached(0);/*
     auto r = iota(3);//*/

     r.cycle.take(30).writeln;
   }

   {
     auto r = inclusiveRange(3).cached(0);/*
     auto r = inclusiveRange(3);//*/

     r.cycle.take(30).writeln;
   }
}/* Prints:

/usr/src/dmd/linux/bin64/../../src/phobos/std/range/package.d(3928): Error:
mutable method `source.CachedRange!(ElementCache!(Result)).CachedRange.front`
is not callable using a `const` object
/usr/src/dmd/linux/bin64/../../src/phobos/std/range/package.d(3928):       
Consider adding `const` or `inout` to
source.CachedRange!(ElementCache!(Result)).CachedRange.front
/usr/src/dmd/linux/bin64/../../src/phobos/std/range/package.d(4060): Error:
template instance `std.range.Cycle!(RefCounted!(CachedRange!(Ele
entCache!(Result)), cast(RefCountedAutoInitialize)0))` error instantiating
instantiated from here: 
`cycle!(RefCounted!(CachedRange!(ElementCache!(Result)), 
cast(RefCountedAutoInitialize)0))`

*/
```

Is it the same with the new D versions?

SDB 79
Sep 12 2022
next sibling parent Salih Dincer <salihdb hotmail.com> writes:
On Tuesday, 13 September 2022 at 07:25:18 UTC, SDB 79 wrote:

 Is it the same with the new D versions?


Now I replaced the cycle() to the leftward and tried it with the 
current version. It works great!

```d
0.iota!double(1,.1).cycle.cached.take(30).writeln;

// [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0, 0.1, 0.2, 
0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0, 0.1, 0.2, 0.3, 0.4, 0.5, 
0.6, 0.7, 0.8, 0.9]
```

Surely it's a useless implementation for cycle() I guess?

SDB 79
Sep 13 2022
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/12/22 22:24, Salih Dincer wrote:


 `source.CachedRange!(ElementCache!(Result)).CachedRange.front` is not
 callable using a `const` object


This exposes at least two issues:

1) The error was because I did not define cached's front as const 
because it has to expand the underlying buffer (a member) as needed. (I 
would have marked the buffer 'mutable' but we don't have that in D.)

This can be solved in some ways, crudest of which is the following:

         (cast()this).expandAsNeeded(id, 1);

Which is technically undefined behavior because the 'cached' range 
object could really be on read-only memory but since we are talking 
about a range, and such objects are supposed to be mutated, it becomes 
an interesting discussion at that point.

(There must be other ways like casting just the buffer but it is related 
to the discussion below.)

After fixing it that way (not pushed to github), I hit another 
compilation error.

2) This point is about a topic that I brought up recently: Types gain a 
'const' eagerly (and they have to, understandably).

For example, in your example you are caching an 'int', but my code sees 
const(int) just because std.range.Cycle.front chose to put a 'const' on 
itself. (With all good intentions: Cycle.front really does not mutate a 
Cycle object.)

However, as my range picks the element type with ElementType!T, I see 
const(int) as well. Again, all good so far... And here is my opApply 
funtion:

     int opApply(int delegate(ref EC.ET) func) scope
     {
         while(!empty)
         {
             auto f = front;

             int result = func(f);    // ERROR
             if (result)
             {
                 return result;
             }
             popFront();
         }

         return 0;
     }

ERROR: delegate `func(ref int)` is not callable using argument types 
`(const(int))`

Oh! See: I am passing an int... oh! by ref...  I guess I shouldn't 
expect my users to use 'const' in their foreach parameters. (I am not 
trying to see whether it is possible.)

So instead, I should have decided on my storage type as int (not 
const(int)) in this case. It can be done like this:

             static if (hasIndirections!(EC.ET))
             {
                 // Use the exact type
                 alias StorageT = T;
             }
             else
             {
                 alias StorageT = Unqual!T;
             }

But... Do I have the right to choose my storage type as 'int' even 
though I am caching const(int)? Perhaps I shouldn't because as discussed 
earlier in the general forum, my choice changes function overloading for 
my users. So, perhaps I should really store const(int)... (?)

I have to digest these issues more before jumping to a solution. :/

Ali

P.S. Related, I had to provide an opApply function because of this reason:

/**
     Support for `foreach` loops
  */
// This is needed to support foreach iteration because although
// RefCounted!CachedRange implicitly converts to CachedRange, which would be
// a range, the result gets copied to the foreach loop. Unfortunately, this
// type does not allow copying so we have to support foreach iteration
// explicitly here.
int opApply(int delegate(ref EC.ET) func) scope
{
     // ...
}

See that 'ref' there? Maybe I should have provided different opApply() 
functions to cover all cases. I am thinking... :)

Or maybe I should use something else other than RefCounted... Still 
thinking...
Sep 13 2022
parent reply Salih Dincer <salihdb hotmail.com> writes:
On Tuesday, 13 September 2022 at 15:24:20 UTC, Ali Çehreli wrote:

 On 9/12/22 22:24, Salih Dincer wrote:
 2) This point is about a topic that I brought up recently: 
 Types gain a 'const' eagerly (and they have to, understandably).

 For example, in your example you are caching an 'int', but my 
 code sees const(int) just because std.range.Cycle.front chose 
 to put a 'const' on itself. (With all good intentions: 
 Cycle.front really does not mutate a Cycle object.)

 However, as my range picks the element type with ElementType!T, 
 I see const(int) as well. Again, all good so far... And here is 
 my opApply funtion:

 ```d
     int opApply(int delegate(ref EC.ET) func) scope
     {
         while(!empty)
         {
             auto f = front;

             int result = func(f);    // ERROR
             if (result)
             {
                 return result;
             }
             popFront();
         }

         return 0;
     }
 ```

 ERROR: delegate `func(ref int)` is not callable using argument 
 types `(const(int))`


I'm far from making a solid recommendation.  Immutable with const 
still doesn't make sense to me.  I claim we can live without 
them. Immutable confuses me a lot.

I think we should take control by creating our own types.  D 
Language should be unornamented.

SDB 79
Sep 13 2022
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 14/09/2022 2:48 PM, Salih Dincer wrote:

 I'm far from making a solid recommendation.  Immutable with const still 
 doesn't make sense to me.  I claim we can live without them. Immutable 
 confuses me a lot.
 
 I think we should take control by creating our own types.  D Language 
 should be unornamented.


We have to have immutable.

You need a way in a systems language to say that a given chunk of memory 
is read only to prevent accidental writes.

Of course you are free to lie and say its mutable, but you can't lie to 
the cpu. It'll error if you try to write to it, resulting in the end of 
a process.
Sep 13 2022
parent Salih Dincer <salihdb hotmail.com> writes:
On Wednesday, 14 September 2022 at 02:58:07 UTC, rikki cattermole 
wrote:

 Of course you are free to lie and say its mutable, but you 
 can't lie to the cpu. It'll error if you try to write to it, 
 resulting in the end of a process.


I agree with what you said. Moreover, I sign it as an electronics 
engineer. However, we have to perform write protection with our 
own types. Anyway, I don't want to get into the discussions that 
are full of pages.

SDB 79
Sep 13 2022
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/12/22 02:15, Ali Çehreli wrote:

 I am happy to publish on code.dlang.org for the first time:

    https://code.dlang.org/packages/alid


I've fixed 2 of the 3 issues reported here:

   - Removed gratuitous function attributes

   - Fixed Salih's .cycle use case

   - Subpackage design (thinking about it...)

Let's continue with the issues at a more appropriate place:

   https://github.com/acehreli/alid/issues

Thanks! :)

Ali
Sep 14 2022
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/14/22 12:08, Ali Çehreli wrote:


    - Subpackage design (thinking about it...)


Ok, I think I fixed that one as well.

I think my main problem was trying to import 'alid':

   import alid;      // WRONG - Could not make it work
   import alid.alid; // Worked with package.d file

I think subpackages work as well:

   import alid.cached;
   // etc.

(Yes, I used many "I think"s. :) )

Ali
Sep 14 2022
parent reply Salih Dincer <salihdb hotmail.com> writes:
On Thursday, 15 September 2022 at 02:30:43 UTC, Ali Çehreli wrote:

 On 9/14/22 12:08, Ali Çehreli wrote:
   import alid;      // WRONG - Could not make it work
   import alid.alid; // Worked with package.d file


What's objection with combining all the code in the package into 
one module?

SDB 79
Sep 15 2022
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/15/22 02:18, Salih Dincer wrote:

 On Thursday, 15 September 2022 at 02:30:43 UTC, Ali Çehreli wrote:

 On 9/14/22 12:08, Ali Çehreli wrote:
   import alid;      // WRONG - Could not make it work
   import alid.alid; // Worked with package.d file


 
 What's objection with combining all the code in the package into one 
 module?
 
 SDB 79


As a general principal, we don't want to include more than necessary. 
One reason can be compilation speed and binary size: 'import std;' 
compiles slow and (likely) adds and initializes module variable my 
program does not need.

Ali
Sep 15 2022