www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Suggestion for DUB users: The vanity package

reply Mathias LANG <geod24 gmail.com> writes:
I was looking at a few packages recently, and noticed a bothering 
trend: Single-module libraries ending up at the top level.

Why is it bothering ? Well, here's a little puzzle: Does the 
following code compile?
```D
import std.stdio;

void main ()
{
     writeln(foo.stringof);
}
```

The answer is "maybe". In most case, it will not. However, if the 
file this resides in is named `foo.d`, it will compile. Because 
the module name accounts for an identifier that can be addressed 
in the scope of the module (and other modules that import it).

Now imagine what would happen if `core.atomic` or `core.time` 
were top-level modules ? Conflicts, conflicts everywhere. It only 
gets worse with imports, as `import atomic;` would insert the 
`atomic` name into the current scope, which would then shadow any 
symbol named `atomic` in any import (there used to be an evil 
hack in the compiler to make this special case work - it is gone 
since a few releases).

So what's the solution ? Well we could go with the Java-esque 
`com.foo`, but that'd be a bit ridiculous. Instead, for a few 
years now, I have started a habit of using the "vanity package", 
a top-level package with my username, for simple libraries I 
publish:
- 
https://github.com/Geod24/bitblob/blob/v2.2.0/source/geod24/bitblob.d#L16
- 
https://github.com/Geod24/localrest/blob/ddfa8cd3bc2f48e1dfc5edd803b990e5aaa69044/source/geod24/LocalRest.d#L102
- 
https://github.com/Geod24/minivariant/blob/d20b240d45289a857b4c0d6e7f842a92b3d4a19e/source/geod24/variant.d#L30

It "just works", is pretty simple to remember, easily extendable, 
and you can be pretty sure no user of your library will run into 
conflicts. I encourage anyone considering publishing a package to 
do the same.
Oct 18 2021
next sibling parent reply WebFreak001 <d.forum webfreak.org> writes:
On Tuesday, 19 October 2021 at 04:03:38 UTC, Mathias LANG wrote:

 I was looking at a few packages recently, and noticed a 
 bothering trend: Single-module libraries ending up at the top 
 level.

 Why is it bothering ? Well, here's a little puzzle: Does the 
 following code compile?
 ```D
 import std.stdio;

 void main ()
 {
     writeln(foo.stringof);
 }
 ```

 The answer is "maybe". In most case, it will not. However, if 
 the file this resides in is named `foo.d`, it will compile. 
 Because the module name accounts for an identifier that can be 
 addressed in the scope of the module (and other modules that 
 import it).

 Now imagine what would happen if `core.atomic` or `core.time` 
 were top-level modules ? Conflicts, conflicts everywhere. It 
 only gets worse with imports, as `import atomic;` would insert 
 the `atomic` name into the current scope, which would then 
 shadow any symbol named `atomic` in any import (there used to 
 be an evil hack in the compiler to make this special case work 
 - it is gone since a few releases).

 So what's the solution ? Well we could go with the Java-esque 
 `com.foo`, but that'd be a bit ridiculous. Instead, for a few 
 years now, I have started a habit of using the "vanity 
 package", a top-level package with my username, for simple 
 libraries I publish:
 - 
 https://github.com/Geod24/bitblob/blob/v2.2.0/source/geod24/bitblob.d#L16
 - 
 https://github.com/Geod24/localrest/blob/ddfa8cd3bc2f48e1dfc5edd803b990e5aaa69044/source/geod24/LocalRest.d#L102
 - 
 https://github.com/Geod24/minivariant/blob/d20b240d45289a857b4c0d6e7f842a92b3d4a19e/source/geod24/variant.d#L30

 It "just works", is pretty simple to remember, easily 
 extendable, and you can be pretty sure no user of your library 
 will run into conflicts. I encourage anyone considering 
 publishing a package to do the same.


I like this idea. We should put this on the dub documentation to 
make more users do this.

I'm not opposed to the Java style either though. (at least for me 
that would be org.webfreak instead of webfreak)

However some projects are quite generic, I think having some 
common idiom there like calling generic packages `util.atomic` or 
`util.money` would be good. Of course there might be name 
clashing with this, but it's unlikely to happen in the same 
project depending on them because packages with the same name are 
usually competing packages and not packages you would use 
together. In the case you do have an issue with overlapping 
package names you could make dub wrapper packages as well.
Oct 19 2021
parent Tobias Pankrath <tobias pankrath.net> writes:
On Tuesday, 19 October 2021 at 14:36:17 UTC, WebFreak001 wrote:

 It "just works", is pretty simple to remember, easily 
 extendable, and you can be pretty sure no user of your library 
 will run into conflicts. I encourage anyone considering 
 publishing a package to do the same.


 I like this idea. We should put this on the dub documentation 
 to make more users do this.


 However some projects are quite generic, I think having some 
 common idiom there like calling generic packages `util.atomic` 
 or `util.money` would be good.


What about `vanity`?
Oct 19 2021
prev sibling next sibling parent reply newbie <newbie1234 gmail.com> writes:
On Tuesday, 19 October 2021 at 04:03:38 UTC, Mathias LANG wrote:

 I was looking at a few packages recently, and noticed a 
 bothering trend: Single-module libraries ending up at the top 
 level.


Hi Mathias LANG,

I use your alpine LDC package and it work well, but the version 
is 1.26 and now 1.28 will release anytime.


Are your plan to upgrade it to 1.28?
Oct 19 2021
parent reply newbie <newbie1234 gmail.com> writes:
On Tuesday, 19 October 2021 at 15:15:29 UTC, newbie wrote:

 Hi Mathias LANG,

 I use your alpine LDC package and it work well, but the version 
 is 1.26 and now 1.28 will release anytime.


 Are your plan to upgrade it to 1.28?



Hi Mathias LANG,

It has been 6 month since latest alpine LDC release, and now 
alpine LLVM already upgrade to LLVM12. I hope there is any plan 
or roadmap to share with community.
Oct 26 2021
parent reply Mathias LANG <geod24 gmail.com> writes:
On Wednesday, 27 October 2021 at 06:30:43 UTC, newbie wrote:

 It has been 6 month since latest alpine LDC release, and now 
 alpine LLVM already upgrade to LLVM12. I hope there is any plan 
 or roadmap to share with community.


LDC v1.28.0 just hit edge today and will be part of 3.15.
Dub and rdmd have been rebuilt too. Currently working on DMD.
Oct 28 2021
parent newbie <newbie1234 gmail.com> writes:
On Thursday, 28 October 2021 at 09:18:51 UTC, Mathias LANG wrote:

 On Wednesday, 27 October 2021 at 06:30:43 UTC, newbie wrote:

 It has been 6 month since latest alpine LDC release, and now 
 alpine LLVM already upgrade to LLVM12. I hope there is any 
 plan or roadmap to share with community.


 LDC v1.28.0 just hit edge today and will be part of 3.15.
 Dub and rdmd have been rebuilt too. Currently working on DMD.


Thanks for the great work, now I am able to remove a lot 
workaround to keep backward compatible.
Oct 28 2021
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
On Tuesday, 19 October 2021 at 04:03:38 UTC, Mathias LANG wrote:

 So what's the solution ? Well we could go with the Java-esque 
 `com.foo`, but that'd be a bit ridiculous. Instead, for a few 
 years now, I have started a habit of using the "vanity 
 package", a top-level package with my username, for simple 
 libraries I publish:
 - 
 https://github.com/Geod24/bitblob/blob/v2.2.0/source/geod24/bitblob.d#L16
 - 
 https://github.com/Geod24/localrest/blob/ddfa8cd3bc2f48e1dfc5edd803b990e5aaa69044/source/geod24/LocalRest.d#L102
 - 
 https://github.com/Geod24/minivariant/blob/d20b240d45289a857b4c0d6e7f842a92b3d4a19e/source/geod24/variant.d#L30

 It "just works", is pretty simple to remember, easily 
 extendable, and you can be pretty sure no user of your library 
 will run into conflicts. I encourage anyone considering 
 publishing a package to do the same.


If we can have several libraries starting with com and org, why 
not have a default package `d` or `dlang` used by everyone, so it 
would be
module d.bitblob;
Oct 19 2021
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/19/21 12:37 PM, Kagamin wrote:

 On Tuesday, 19 October 2021 at 04:03:38 UTC, Mathias LANG wrote:

 So what's the solution ? Well we could go with the Java-esque 
 `com.foo`, but that'd be a bit ridiculous. Instead, for a few years 
 now, I have started a habit of using the "vanity package", a top-level 
 package with my username, for simple libraries I publish:
 - 
 https://github.com/Geod24/bitblob/blob/v2.2.0/source/geod24/bitblob.d#L16
 - 
 https://github.com/Geod24/localrest/blob/ddfa8cd3bc2f48e1dfc5edd803b990e5aaa69044/source/geo
24/LocalRest.d#L102 

 - 
 https://github.com/Geod24/minivariant/blob/d20b240d45289a857b4c0d6e7f842a92b3d4a19e/source/
eod24/variant.d#L30 


 It "just works", is pretty simple to remember, easily extendable, and 
 you can be pretty sure no user of your library will run into 
 conflicts. I encourage anyone considering publishing a package to do 
 the same.


 
 If we can have several libraries starting with com and org, why not have 
 a default package `d` or `dlang` used by everyone, so it would be
 module d.bitblob;


Note that the vanity package needs to be a name that isn't used anywhere 
else. Even with the module under a package, the problem is still present 
with the *top level package*. e.g. `std` is not a common name used by 
anyone, so it causes little grief.

Something like `geod24` is not going to be a common name. But `d` likely 
is. I don't think `dlang` would be a bad idea, but I suggest actually 
something really unlikely. But it is a good idea to have a specific 
language-sanctioned "put your one module projects here".

I've been using `schlib` for [my 
stuff](https://github.com/schveiguy/lookuptable).

-Steve
Oct 19 2021
parent Kagamin <spam here.lot> writes:
On Tuesday, 19 October 2021 at 17:04:04 UTC, Steven Schveighoffer 
wrote:

 Note that the vanity package needs to be a name that isn't used 
 anywhere else. Even with the module under a package, the 
 problem is still present with the *top level package*. e.g. 
 `std` is not a common name used by anyone, so it causes little 
 grief.


Does it prevent java package naming convention too? `com.sun` etc.
Oct 19 2021
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 19 October 2021 at 16:37:00 UTC, Kagamin wrote:

 If we can have several libraries starting with com and org, why 
 not have a default package `d` or `dlang` used by everyone, so 
 it would be
 module d.bitblob;


Then you have to get everyone using that package to agree not to 
create naming conflicts. E.g., if I decide to publish `d.foo`, 
nobody else is allowed to publish their own `d.foo`.

But if you are capable of doing that, then you don't need the 
package at all: you can just get everybody to agree to avoid 
naming conflicts at the top level.

The reason prefixes like `geod24` are necessary in the first 
place is because we *cannot* get the entire community to 
coordinate in the way that you're proposing.
Oct 19 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/19/21 3:31 PM, Paul Backus wrote:

 On Tuesday, 19 October 2021 at 16:37:00 UTC, Kagamin wrote:

 If we can have several libraries starting with com and org, why not 
 have a default package `d` or `dlang` used by everyone, so it would be
 module d.bitblob;


 
 Then you have to get everyone using that package to agree not to create 
 naming conflicts. E.g., if I decide to publish `d.foo`, nobody else is 
 allowed to publish their own `d.foo`.


How is this different from today where your no-package module can be 
named whatever you want? Seems like a separate problem.


 But if you are capable of doing that, then you don't need the package at 
 all: you can just get everybody to agree to avoid naming conflicts at 
 the top level.


Naming conflicts with *other package names* is not the problem. It's the 
fact that the no-package module conflicts with anything else in your 
namespace, including imports from other modules.

e.g. try to name a function `std`, then import `std.stdio`. You will get 
a failure.

Whereas if you name your function `stdio`, then everything works fine.

Note that I didn't think before about `com` and `org`, those can be 
reasonable names. My codebase has a lot of `org` variables, as I'm 
dealing with organizations in a database.

-Steve
Oct 20 2021
next sibling parent reply Mathias LANG <geod24 gmail.com> writes:
On Wednesday, 20 October 2021 at 13:39:22 UTC, Steven 
Schveighoffer wrote:

 Naming conflicts with *other package names* is not the problem. 
 It's the fact that the no-package module conflicts with 
 anything else in your namespace, including imports from other 
 modules.


IMO it *is* also a problem. As the number of packages grow, so 
will the risk of conflict.
Oct 20 2021
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/20/21 10:40 AM, Mathias LANG wrote:

 On Wednesday, 20 October 2021 at 13:39:22 UTC, Steven Schveighoffer wrote:

 Naming conflicts with *other package names* is not the problem. It's 
 the fact that the no-package module conflicts with anything else in 
 your namespace, including imports from other modules.


 
 IMO it *is* also a problem. As the number of packages grow, so will the 
 risk of conflict.
 


Yes, but it's more of a garden variety naming conflict that can happen 
in any ecosystem. Not the same as the weird status of top-level module 
or package names in D.

-Steve
Oct 20 2021
prev sibling parent Paul Backus <snarwin gmail.com> writes:
On Wednesday, 20 October 2021 at 13:39:22 UTC, Steven 
Schveighoffer wrote:

 e.g. try to name a function `std`, then import `std.stdio`. You 
 will get a failure.

 Whereas if you name your function `stdio`, then everything 
 works fine.


Huh. Yep, it sure does:

```d
import std.stdio;

// Error: variable `onlineapp.std` conflicts with import 
`onlineapp.std` at onlineapp.d(1)
int std;

void main()
{
     writeln("Hello world!");
}
```

Honestly I would classify this as a straight-up bug. According to 
the spec on [Symbol Name Lookups][1], this program should work 
fine.

[1]: https://dlang.org/spec/module.html#name_lookup
Oct 20 2021
prev sibling parent FeepingCreature <feepingcreature gmail.com> writes:
On Tuesday, 19 October 2021 at 04:03:38 UTC, Mathias LANG wrote:

 I was looking at a few packages recently, and noticed a 
 bothering trend: Single-module libraries ending up at the top 
 level.

 Why is it bothering ? Well, here's a little puzzle: Does the 
 following code compile?
 ```D
 import std.stdio;

 void main ()
 {
     writeln(foo.stringof);
 }
 ```

 The answer is "maybe". In most case, it will not. However, if 
 the file this resides in is named `foo.d`, it will compile. 
 Because the module name accounts for an identifier that can be 
 addressed in the scope of the module (and other modules that 
 import it).


This is ultimately because D modules just reuse the notion of 
import paths from C. I had to find a better way in my language in 
order to handle self-rebuilds, where there's name collisions 
between compiler runtime and compiled runtime packages, but the 
same approach would also solve the issue in D, so let me outline 
it.

Basically, we replace import paths (`-I~/.dub/packages/bla/src`) 
with package paths:

```
-Pbla:~/.dub/packages/bla/src
```

As you can see, each package is named. This allows us to 
explicitly specify which packages a package can see and access:

```
-Pmain:src:bla,bla2
```

As it walks imports, the compiler tracks each module's package. 
When evaluating an import statement, only files in the current 
package or a direct dependency of the current package are 
considered.

Thus, your 'foo' problem will only happen if you directly depend 
on package "foo" in your dub.json.

This approach is fully backwards compatible with include paths. 
It also allows other funky things, such as per-package settings 
(-i, -deps, -g, -debug etc) which solves a whole host of problems 
with dub. For instance, a static library can safely be used, even 
if the main program's source is being built with different flags.
Oct 28 2021