www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Introducing alid

reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
I am happy to publish on code.dlang.org for the first time:

   https://code.dlang.org/packages/alid

Thanks to everyone who made registering a dub package so easy! :)


1) Currently, the main module in `alid` is `cached`, which

- caches range elements to ensure each is executed once (most
   useful with generator ranges that use e.g. `map`),

- as a result, elevates InputRanges to ForwardRanges (plus
   opIndex).

2) But I like `errornogc` as well, which helps throw Error
    objects from  nogc code.


Copying from the README:

* `cached` is for executing elements only once while not holding
   on to already popFront'ed elements for too long. `cached` was
   the main topic of Ali's DConf 2022 lightning talk.

* `circularblocks` was written for storing range elements by
   cached. It can be used independently.

* `blockreusable` was written as storage blocks for
   circularblocks. It can be used independently.

* `errornogc` was needed for throwing Errors from  nogc code. It
   can be used independently.

* `test` contains some helper utilities for unit testing.

Ali
Sep 12
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
Looks pretty well tested, nice!

But in other less nice things, I take it you did not test with GDC? GDC 
does not support cli args with the same names as dmd. One of these is -mv.

The file structure of subPackage/alid/subPackage will not require it and 
you will not have the cross import issues, where if you depend on 
errornogc you can also import (and then get linker errors) for 
circularblocks.
Sep 12
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/12/22 07:43, rikki cattermole wrote:
 Looks pretty well tested, nice!
Thanks! Proud with 100% coverage. :)
 But in other less nice things, I take it you did not test with GDC? GDC 
 does not support cli args with the same names as dmd. One of these is -mv.
So far, I started learning by copying arsd's dub.json. (Thank you, Adam! The misunderstandings are mine. :) )
 The file structure of subPackage/alid/subPackage will not require it and 
 you will not have the cross import issues, where if you depend on 
 errornogc you can also import (and then get linker errors) for 
 circularblocks.
If I understand you correctly, the directory structure need to be the following (also introducing src, which is clearly missing :)): alid/src/errornogc/alid/errornogc.d .../circularblocks/alid/circularblocks.d [...] Can I add a CI step to catch all such issues? It would be awesome if dub provided that. Ali P.S. Another issue is function attributes seemingly used inaccurately but I asked that question on the 'learn' newsgroup already. Ping! ;)
Sep 12
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 13/09/2022 4:25 AM, Ali Çehreli wrote:
 On 9/12/22 07:43, rikki cattermole wrote:
 Looks pretty well tested, nice!
Thanks! Proud with 100% coverage. :)
I was going to ask about coverage, that is awesome!
 But in other less nice things, I take it you did not test with GDC? 
 GDC does not support cli args with the same names as dmd. One of these 
 is -mv.
So far, I started learning by copying arsd's dub.json. (Thank you, Adam! The misunderstandings are mine. :) )
It has the same issues, which is why I recommended against copying it.
 The file structure of subPackage/alid/subPackage will not require it 
 and you will not have the cross import issues, where if you depend on 
 errornogc you can also import (and then get linker errors) for 
 circularblocks.
If I understand you correctly, the directory structure need to be the following (also introducing src, which is clearly missing :)): alid/src/errornogc/alid/errornogc.d      .../circularblocks/alid/circularblocks.d [...]
No, you don't need the src directory, the subPackage directory functions as this. So: dub.json errornogc/alid/errornogc.d circularblocks/alid/circularblocks.d That'll work.
 Can I add a CI step to catch all such issues? It would be awesome if dub 
 provided that.
There is: https://github.com/dlang-community/setup-dlang But it doesn't look to support gdc or have been updated in a little while. Guess I need to start pinging people about it.
 Ali
 
 P.S. Another issue is function attributes seemingly used inaccurately 
 but I asked that question on the 'learn' newsgroup already. Ping! ;)
I have no solution to that unfortunately beyond template all the things.
Sep 12
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/12/22 09:34, rikki cattermole wrote:

 dub.json
 errornogc/alid/errornogc.d
 circularblocks/alid/circularblocks.d
Considering I may want to let the users import the entire package as well with import alid; how can I achieve my goal of subpackages? Telling me to forget about subpackages altogether :) is an option because I've already spent hours trying to find my way through different directory structures, random dub.json edits, experimenting with ways of stopping dub from fetching and using the bad version of the repo repeatedly, and many other random things... The main problem is likely I don't have an accurate mental model of the whole thing. For example, I don't know whether dub will take the state of my local directory as-is or will it copy from the committed state of the local repo? I ask because the add-local command puts a commit hash as well. If that's the case, I wouldn't want to commit every little change before seeing it's effect. Yes, I did the path override thing as well... Ali
Sep 14
next sibling parent Adam D Ruppe <destructionator gmail.com> writes:
On Wednesday, 14 September 2022 at 08:44:48 UTC, Ali Çehreli 
wrote:
 Considering I may want to let the users import the entire 
 package as well with
dub is poorly designed and doesn't cooperate well with D features. There's no good solution - hence the hundreds of lines long dub.json trying to hack it in. Like you could have the top-level package depend on all of its own subpackages and then if you depend on it, you can import alid. Lots of spam for something that just works out of the box with stock dmd!
Sep 14
prev sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 14/09/2022 8:44 PM, Ali Çehreli wrote:
 On 9/12/22 09:34, rikki cattermole wrote:
 
  > dub.json
  > errornogc/alid/errornogc.d
  > circularblocks/alid/circularblocks.d
 
 Considering I may want to let the users import the entire package as 
 well with
 
    import alid;
 
 how can I achieve my goal of subpackages? Telling me to forget about 
 subpackages altogether :) is an option because I've already spent hours 
 trying to find my way through different directory structures, random 
 dub.json edits, experimenting with ways of stopping dub from fetching 
 and using the bad version of the repo repeatedly, and many other random 
 things...
In your root package you can still have the package.d file. You would use the version added by Dub to detect if you should public import the other modules.
 DUB provides version identifier of dependencies for conditional 
compilation with version conditions. Have_<dependency> version identifier can be used for conditional compilation. https://dub.pm/advanced_usage
Sep 14
prev sibling next sibling parent reply Salih Dincer <salihdb hotmail.com> writes:
On Monday, 12 September 2022 at 09:15:34 UTC, Ali Çehreli wrote:
 I am happy to publish on code.dlang.org for the first time:

   https://code.dlang.org/packages/alid

 Thanks to everyone who made registering a dub package so easy! 
 :)
Hoş geldin AliD :) Thank you very much for bringing these codes over a thousand lines to the D World. I was looking forward to trying it right away, and when I tried it with the classic iota() and its sister inclusiveRange(), I found that it didn't get along well with cycle(). For example: ```d immutable size_t pageSize = 4096; void main() { "D Compiler v".writeln(__VERSION__*.001); // D Compiler v2.087 { auto r = iota(3).cached(0);/* auto r = iota(3);//*/ r.cycle.take(30).writeln; } { auto r = inclusiveRange(3).cached(0);/* auto r = inclusiveRange(3);//*/ r.cycle.take(30).writeln; } }/* Prints: /usr/src/dmd/linux/bin64/../../src/phobos/std/range/package.d(3928): Error: mutable method `source.CachedRange!(ElementCache!(Result)).CachedRange.front` is not callable using a `const` object /usr/src/dmd/linux/bin64/../../src/phobos/std/range/package.d(3928): Consider adding `const` or `inout` to source.CachedRange!(ElementCache!(Result)).CachedRange.front /usr/src/dmd/linux/bin64/../../src/phobos/std/range/package.d(4060): Error: template instance `std.range.Cycle!(RefCounted!(CachedRange!(Ele entCache!(Result)), cast(RefCountedAutoInitialize)0))` error instantiating instantiated from here: `cycle!(RefCounted!(CachedRange!(ElementCache!(Result)), cast(RefCountedAutoInitialize)0))` */ ``` Is it the same with the new D versions? SDB 79
Sep 12
next sibling parent Salih Dincer <salihdb hotmail.com> writes:
On Tuesday, 13 September 2022 at 07:25:18 UTC, SDB 79 wrote:
 Is it the same with the new D versions?
Now I replaced the cycle() to the leftward and tried it with the current version. It works great! ```d 0.iota!double(1,.1).cycle.cached.take(30).writeln; // [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9] ``` Surely it's a useless implementation for cycle() I guess? SDB 79
Sep 13
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/12/22 22:24, Salih Dincer wrote:

 `source.CachedRange!(ElementCache!(Result)).CachedRange.front` is not
 callable using a `const` object
This exposes at least two issues: 1) The error was because I did not define cached's front as const because it has to expand the underlying buffer (a member) as needed. (I would have marked the buffer 'mutable' but we don't have that in D.) This can be solved in some ways, crudest of which is the following: (cast()this).expandAsNeeded(id, 1); Which is technically undefined behavior because the 'cached' range object could really be on read-only memory but since we are talking about a range, and such objects are supposed to be mutated, it becomes an interesting discussion at that point. (There must be other ways like casting just the buffer but it is related to the discussion below.) After fixing it that way (not pushed to github), I hit another compilation error. 2) This point is about a topic that I brought up recently: Types gain a 'const' eagerly (and they have to, understandably). For example, in your example you are caching an 'int', but my code sees const(int) just because std.range.Cycle.front chose to put a 'const' on itself. (With all good intentions: Cycle.front really does not mutate a Cycle object.) However, as my range picks the element type with ElementType!T, I see const(int) as well. Again, all good so far... And here is my opApply funtion: int opApply(int delegate(ref EC.ET) func) scope { while(!empty) { auto f = front; int result = func(f); // ERROR if (result) { return result; } popFront(); } return 0; } ERROR: delegate `func(ref int)` is not callable using argument types `(const(int))` Oh! See: I am passing an int... oh! by ref... I guess I shouldn't expect my users to use 'const' in their foreach parameters. (I am not trying to see whether it is possible.) So instead, I should have decided on my storage type as int (not const(int)) in this case. It can be done like this: static if (hasIndirections!(EC.ET)) { // Use the exact type alias StorageT = T; } else { alias StorageT = Unqual!T; } But... Do I have the right to choose my storage type as 'int' even though I am caching const(int)? Perhaps I shouldn't because as discussed earlier in the general forum, my choice changes function overloading for my users. So, perhaps I should really store const(int)... (?) I have to digest these issues more before jumping to a solution. :/ Ali P.S. Related, I had to provide an opApply function because of this reason: /** Support for `foreach` loops */ // This is needed to support foreach iteration because although // RefCounted!CachedRange implicitly converts to CachedRange, which would be // a range, the result gets copied to the foreach loop. Unfortunately, this // type does not allow copying so we have to support foreach iteration // explicitly here. int opApply(int delegate(ref EC.ET) func) scope { // ... } See that 'ref' there? Maybe I should have provided different opApply() functions to cover all cases. I am thinking... :) Or maybe I should use something else other than RefCounted... Still thinking...
Sep 13
parent reply Salih Dincer <salihdb hotmail.com> writes:
On Tuesday, 13 September 2022 at 15:24:20 UTC, Ali Çehreli wrote:
 On 9/12/22 22:24, Salih Dincer wrote:
 2) This point is about a topic that I brought up recently: 
 Types gain a 'const' eagerly (and they have to, understandably).

 For example, in your example you are caching an 'int', but my 
 code sees const(int) just because std.range.Cycle.front chose 
 to put a 'const' on itself. (With all good intentions: 
 Cycle.front really does not mutate a Cycle object.)

 However, as my range picks the element type with ElementType!T, 
 I see const(int) as well. Again, all good so far... And here is 
 my opApply funtion:

 ```d
     int opApply(int delegate(ref EC.ET) func) scope
     {
         while(!empty)
         {
             auto f = front;

             int result = func(f);    // ERROR
             if (result)
             {
                 return result;
             }
             popFront();
         }

         return 0;
     }
 ```

 ERROR: delegate `func(ref int)` is not callable using argument 
 types `(const(int))`
I'm far from making a solid recommendation. Immutable with const still doesn't make sense to me. I claim we can live without them. Immutable confuses me a lot. I think we should take control by creating our own types. D Language should be unornamented. SDB 79
Sep 13
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 14/09/2022 2:48 PM, Salih Dincer wrote:
 I'm far from making a solid recommendation.  Immutable with const still 
 doesn't make sense to me.  I claim we can live without them. Immutable 
 confuses me a lot.
 
 I think we should take control by creating our own types.  D Language 
 should be unornamented.
We have to have immutable. You need a way in a systems language to say that a given chunk of memory is read only to prevent accidental writes. Of course you are free to lie and say its mutable, but you can't lie to the cpu. It'll error if you try to write to it, resulting in the end of a process.
Sep 13
parent Salih Dincer <salihdb hotmail.com> writes:
On Wednesday, 14 September 2022 at 02:58:07 UTC, rikki cattermole 
wrote:
 Of course you are free to lie and say its mutable, but you 
 can't lie to the cpu. It'll error if you try to write to it, 
 resulting in the end of a process.
I agree with what you said. Moreover, I sign it as an electronics engineer. However, we have to perform write protection with our own types. Anyway, I don't want to get into the discussions that are full of pages. SDB 79
Sep 13
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/12/22 02:15, Ali Çehreli wrote:
 I am happy to publish on code.dlang.org for the first time:

    https://code.dlang.org/packages/alid
I've fixed 2 of the 3 issues reported here: - Removed gratuitous function attributes - Fixed Salih's .cycle use case - Subpackage design (thinking about it...) Let's continue with the issues at a more appropriate place: https://github.com/acehreli/alid/issues Thanks! :) Ali
Sep 14
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/14/22 12:08, Ali Çehreli wrote:

    - Subpackage design (thinking about it...)
Ok, I think I fixed that one as well. I think my main problem was trying to import 'alid': import alid; // WRONG - Could not make it work import alid.alid; // Worked with package.d file I think subpackages work as well: import alid.cached; // etc. (Yes, I used many "I think"s. :) ) Ali
Sep 14
parent reply Salih Dincer <salihdb hotmail.com> writes:
On Thursday, 15 September 2022 at 02:30:43 UTC, Ali Çehreli wrote:
 On 9/14/22 12:08, Ali Çehreli wrote:
   import alid;      // WRONG - Could not make it work
   import alid.alid; // Worked with package.d file
What's objection with combining all the code in the package into one module? SDB 79
Sep 15
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 9/15/22 02:18, Salih Dincer wrote:
 On Thursday, 15 September 2022 at 02:30:43 UTC, Ali Çehreli wrote:
 On 9/14/22 12:08, Ali Çehreli wrote:
   import alid;      // WRONG - Could not make it work
   import alid.alid; // Worked with package.d file
What's objection with combining all the code in the package into one module? SDB 79
As a general principal, we don't want to include more than necessary. One reason can be compilation speed and binary size: 'import std;' compiles slow and (likely) adds and initializes module variable my program does not need. Ali
Sep 15