www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Snap packages for D compilers and core projects

reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
Hello all,

I thought it might be time to share more generally something I've 
been working on for a little while: snap packages for some of the 
core D projects.

For those who don't know, snap packages are a new format 
developed by Ubuntu to facilitate upstreams being able to provide 
the latest versions of their apps directly to users.  The format 
is also designed to provide effective confinement for apps, so 
that they can only access the parts of the host system that they 
need to.  While developed by Ubuntu, the format is gaining quite 
a bit of of cross-distro traction: see http://snapcraft.io/ for 
more information.

I started by trying to snap LDC, mainly because the cmake build 
system made for a very easy integration with the snapcraft 
package-build system.  The LDC developers have been kind enough 
to accept this as an official contribution, and a first 
submission is currently waiting for review in the Ubuntu snap 
store, based on the following package definition:
https://github.com/ldc-developers/ldc2.snap

However, I have also created two other snap package definitions, 
one for DUB, and one (just this afternoon!) for DMD:
https://github.com/WebDrake/dub.snap/tree/classic-snap
https://github.com/WebDrake/dmd.snap/tree/classic-snap

As you can probably see from the package definitions, one of the 
attractions of the format is how remarkably easy it is to define 
a snap package.  People running Ubuntu 16.04 or later may like to 
try installing snapcraft and building these packages for 
themselves.

However, now that they exist, I'd like to try publishing these to 
the official snap store.  Rather than do this as some random 
developer, I'd quite like to publish them as official D language 
packages.  Note that the snap store supports multiple 'channels' 
with different levels of stability -- 'edge' and 'beta' being the 
testing ones -- so any issues with the packages can be worked 
through before a stable release is made.

The question is, (i) is this a welcome proposal? and (ii) if it 
is welcome, what do people see as the best way to go about this?

I would also welcome feedback on the current package definitions, 
which can be provided in the associated PRs:
https://github.com/WebDrake/dub.snap/pull/3
https://github.com/WebDrake/dmd.snap/pull/1

In creating the DUB snap package I also had to write a snapcraft 
plugin for DUB itself.  This would be good to submit upstream, 
but I'd like to get some feedback on it before doing so.  The 
plugin is in the `dub.py` file included in the PR above: it 
sufficies for the current purpose, but would need more work 
before it covered all potential use-cases.

Lastly, I have to mention that in creating the above packages, I 
received a great deal of very warm and welcoming support from the 
folks on the Snapcraft mailing list.  They were very eager to 
help me solve problems I encountered, and to help me find the 
most straightforward way to get the results I wanted out of these 
packages.

Anyway, hope this is interesting to everyone -- any thoughts, 
questions, feedback ... ?

Thanks & best wishes,

     -- Joe
Jan 29
next sibling parent reply qznc <qznc web.de> writes:
On Sunday, 29 January 2017 at 20:07:50 UTC, Joseph Rushton 
Wakeling wrote:
 The question is, (i) is this a welcome proposal? and (ii) if it 
 is welcome, what do people see as the best way to go about this?
No comments? Well, there seems to be no downside (apart from the work). So far, I considered Snap an Ubuntu-only initiative of yet-another-package-format. If it really gains cross-distro support, this is a great way for better D support on Fedora, Arch, etc. What is the evidence for cross-distro support? Can it be measured somehow? Is it legally possible to distribute DMD this way? Afaik only dlang.org is allowed to distribute it publically due to the backend licence issue.
Jan 30
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 30 January 2017 at 14:40:13 UTC, qznc wrote:
 No comments? Well, there seems to be no downside (apart from 
 the work).
Yea, I'm a little sad to see the apparent lack of feedback/interest :-\ I had quite a lot of fun creating these packages and was hoping for a bit more curiosity.
 So far, I considered Snap an Ubuntu-only initiative of 
 yet-another-package-format. If it really gains cross-distro 
 support, this is a great way for better D support on Fedora, 
 Arch, etc. What is the evidence for cross-distro support? Can 
 it be measured somehow?
Snap packaging started as something Ubuntu were developing for their use-case, but started gaining cross-distro interest last year (probably because AFAICT its feature-set and simplicity of use is quite a bit ahead of alternatives like Flatpak). Installation instructions for various distros: http://snapcraft.io/docs/core/install
 Is it legally possible to distribute DMD this way? Afaik only 
 dlang.org is allowed to distribute it publically due to the 
 backend licence issue.
Yes, that's a very good point that I wanted to raise. There is a legal agreement that you're asked to make with respect to usage and distribution of packages. It looks like it ought to be compatible with constraints on DMD (re)distribution, but it would obviously need to be reviewed and agreed to by Walter. This is one reason why if I do take this forward, I'd like to do so with some sort of official backing. It doesn't stop me moving forward with DUB, of course, but it'd be nice to do that officially as well.
Jan 30
next sibling parent reply bachmeier <no spam.net> writes:
On Monday, 30 January 2017 at 16:53:34 UTC, Joseph Rushton 
Wakeling wrote:
 Yea, I'm a little sad to see the apparent lack of 
 feedback/interest :-\
I have interest, but as I've never heard of Snap before, I have no comments. Others may be in the same boat. Installation of D compilers is probably not an issue for most of us that read these messages.
Jan 30
parent Jon Degenhardt <jond noreply.com> writes:
On Monday, 30 January 2017 at 19:28:58 UTC, bachmeier wrote:
 On Monday, 30 January 2017 at 16:53:34 UTC, Joseph Rushton 
 Wakeling wrote:
 Yea, I'm a little sad to see the apparent lack of 
 feedback/interest :-\
I have interest, but as I've never heard of Snap before, I have no comments. Others may be in the same boat. Installation of D compilers is probably not an issue for most of us that read these messages.
Same for me. It'd be great to have simpler and more widespread distribution of D and D built applications. But I don't have enough background to comment on Snap per se. --Jon
Jan 30
prev sibling parent reply qznc <qznc web.de> writes:
On Monday, 30 January 2017 at 16:53:34 UTC, Joseph Rushton 
Wakeling wrote:
 Snap packaging started as something Ubuntu were developing for 
 their use-case, but started gaining cross-distro interest last 
 year (probably because AFAICT its feature-set and simplicity of 
 use is quite a bit ahead of alternatives like Flatpak).
I just tried FlatPak and Snap. Snap is actually useable. FlatPak might be superior technology with its sandboxing, but I'm no expert. FlatPak has no central repository, which makes it unuseable for the layman at the moment. It feels like the PPA situation in Ubuntu. Snap has stuff like pulseaudio (audio daemon) or ogre (rendering engine) in the repo. That seems weird, because these are not applications. The first belongs into the base system, the second into a development environment.
Feb 03
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Friday, 3 February 2017 at 08:51:38 UTC, qznc wrote:
 I just tried FlatPak and Snap. Snap is actually useable.
One of the first things that struck me about snap packaging was the ease of its syntax and how straightforward it was to get things working. I actually started creating snap packages as an easy way to build and install D compilers and other tools on my own system. It's both simpler and cleaner than doing those builds by hand or writing custom scripts. The fact that it's then trivially easy to share the results with everyone else is the cherry on the cake :-)
 FlatPak might be superior technology with its sandboxing, but 
 I'm no expert. FlatPak has no central repository, which makes 
 it unuseable for the layman at the moment. It feels like the 
 PPA situation in Ubuntu.
I'm no expert on sandboxing either, but snapcraft certainly also provides confinement -- and to be honest my impression is it does so in a more effective and flexible way than Flatpak. Some of the non-Ubuntu distros may however be disabling the confinement right now as some issues with different confinement systems are worked through.
 Snap has stuff like pulseaudio (audio daemon) or ogre 
 (rendering engine) in the repo. That seems weird, because these 
 are not applications. The first belongs into the base system, 
 the second into a development environment.
I think this is by design. For example, it makes perfect sense that a daemon like pulseaudio might live in an isolated, confined environment where only applications that specifically need access are allowed to talk to it. Probably its presence in the store is in order to allow people to explore working with it in that way. I can't speak to ogre, but I don't think the use-cases of snap packaging are intended to be limited to applications in the long run. It also makes sense that a development library could be provided via a read-only filesystem, so that while anyone might _read_ it to build or run a program, it couldn't be overwritten by another application. For an example of a library snap, I believe some core KDE frameworks are being distributed as a snap in their own right, which other KDE application snaps can then depend on.
Feb 03
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 01/30/2017 09:40 AM, qznc wrote:
 Is it legally possible to distribute DMD this way? Afaik only dlang.org
 is allowed to distribute it publically due to the backend licence issue.
I don't understand where people keep getting that idea. It very clearly states that all you need is to ask permission. It's always been that way, and no reasonable request (or any at all to my knowledge) has ever been denied.
Jan 30
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 30 January 2017 at 19:07:03 UTC, Nick Sabalausky wrote:
 I don't understand where people keep getting that idea. It very 
 clearly states that all you need is to ask permission. It's 
 always been that way, and no reasonable request (or any at all 
 to my knowledge) has ever been denied.
Yea, I don't see there being any particular issue here beyond that permission needs to be sought and obtained. In case it wasn't obvious, Walter: I'm asking ;-)
Jan 30
parent reply Joakim <dlang joakim.fea.st> writes:
On Monday, 30 January 2017 at 23:47:50 UTC, Joseph Rushton 
Wakeling wrote:
 On Monday, 30 January 2017 at 19:07:03 UTC, Nick Sabalausky 
 wrote:
 I don't understand where people keep getting that idea. It 
 very clearly states that all you need is to ask permission. 
 It's always been that way, and no reasonable request (or any 
 at all to my knowledge) has ever been denied.
Yea, I don't see there being any particular issue here beyond that permission needs to be sought and obtained. In case it wasn't obvious, Walter: I'm asking ;-)
He can't read every forum thread, you should email him. That's what I did when I got his permission to put dmd in FreeBSD ports.
Jan 30
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Tuesday, 31 January 2017 at 05:41:38 UTC, Joakim wrote:
 He can't read every forum thread, you should email him.  That's 
 what I did when I got his permission to put dmd in FreeBSD 
 ports.
Yes, I understand that, and I was going to do so anyway. But I was interested in any case in some more general discussion/feedback.
Jan 31
prev sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 29 January 2017 at 20:07:50 UTC, Joseph Rushton 
Wakeling wrote:
 I started by trying to snap LDC, mainly because the cmake build 
 system made for a very easy integration with the snapcraft 
 package-build system.  The LDC developers have been kind enough 
 to accept this as an official contribution, and a first 
 submission is currently waiting for review in the Ubuntu snap 
 store, based on the following package definition:
 https://github.com/ldc-developers/ldc2.snap
This LDC snap is now published in the official snap store. See: https://forum.dlang.org/thread/rkxyvsmgwhfkigybjpig forum.dlang.org ... for more details.
Feb 03