www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Low hanging fruit: dub git integration

reply Andre Pany <andre s-e-a-p.de> writes:
Hi,

While integrating the git protocol into dub is complex, there is 
a much much easier solution.

Github and bitbucket provides access to the source code, 
including releases, branches and commits as archive files using 
the http protocol.

Without counting the actual unzip/untar coding I assume more or 
less 100 lines of additional coding is needed in dub.

In dub.json in addition to the existing path attribute of the 
dependeny object a new attribute "url" has to be added. You can 
specify here an url to a zip/tar.gz. This file has to be 
downloaded and extracted. The extract path is then filled into 
the existing path attribute of the dependency object.

The only issue is the untar/unzip logic...

This little tweak would make dub a lot more flexible like npm.

Kind regards
André
May 16 2017
next sibling parent reply Brad Anderson <eco gnuk.net> writes:
On Tuesday, 16 May 2017 at 18:10:52 UTC, Andre Pany wrote:
 Hi,

 While integrating the git protocol into dub is complex, there 
 is a much much easier solution.

 Github and bitbucket provides access to the source code, 
 including releases, branches and commits as archive files using 
 the http protocol.

 Without counting the actual unzip/untar coding I assume more or 
 less 100 lines of additional coding is needed in dub.

 In dub.json in addition to the existing path attribute of the 
 dependeny object a new attribute "url" has to be added. You can 
 specify here an url to a zip/tar.gz. This file has to be 
 downloaded and extracted. The extract path is then filled into 
 the existing path attribute of the dependency object.

 The only issue is the untar/unzip logic...

 This little tweak would make dub a lot more flexible like npm.

 Kind regards
 André
This is actually exactly how dub fetches source code but the registry does it rather than dub itself (I'm not sure why)[1]. A bare URL dependency may be controversial though because it makes versioning more difficult to ensure. 1. https://github.com/dlang/dub-registry/blob/d825840770bb29356495f265480035ed7e3321b8/source/dubregistry/repositories/github.d#L89
May 16 2017
parent Andre Pany <andre s-e-a-p.de> writes:
On Tuesday, 16 May 2017 at 19:41:32 UTC, Brad Anderson wrote:
 On Tuesday, 16 May 2017 at 18:10:52 UTC, Andre Pany wrote:
 Hi,

 While integrating the git protocol into dub is complex, there 
 is a much much easier solution.

 Github and bitbucket provides access to the source code, 
 including releases, branches and commits as archive files 
 using the http protocol.

 Without counting the actual unzip/untar coding I assume more 
 or less 100 lines of additional coding is needed in dub.

 In dub.json in addition to the existing path attribute of the 
 dependeny object a new attribute "url" has to be added. You 
 can specify here an url to a zip/tar.gz. This file has to be 
 downloaded and extracted. The extract path is then filled into 
 the existing path attribute of the dependency object.

 The only issue is the untar/unzip logic...

 This little tweak would make dub a lot more flexible like npm.

 Kind regards
 André
This is actually exactly how dub fetches source code but the registry does it rather than dub itself (I'm not sure why)[1]. A bare URL dependency may be controversial though because it makes versioning more difficult to ensure. 1. https://github.com/dlang/dub-registry/blob/d825840770bb29356495f265480035ed7e3321b8/source/dubregistry/repositories/github.d#L89
In my scenario I have github repositories stored on the company github server but not on the public github server. Also running an own dub registry server is not an option as I would have to adapt the source code (settings are hard coded) and the license type of dub registry is lgpl. Kind regards André
May 16 2017
prev sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 16/05/2017 7:10 PM, Andre Pany wrote:
 Hi,

 While integrating the git protocol into dub is complex, there is a much
 much easier solution.

 Github and bitbucket provides access to the source code, including
 releases, branches and commits as archive files using the http protocol.

 Without counting the actual unzip/untar coding I assume more or less 100
 lines of additional coding is needed in dub.

 In dub.json in addition to the existing path attribute of the dependeny
 object a new attribute "url" has to be added. You can specify here an
 url to a zip/tar.gz. This file has to be downloaded and extracted. The
 extract path is then filled into the existing path attribute of the
 dependency object.

 The only issue is the untar/unzip logic...

 This little tweak would make dub a lot more flexible like npm.

 Kind regards
 André
I've built a prototype UI[1] for some code[0] to solve this exact problem. It may seem complex, but you can't rely on HTTP download options for easy access to repositories. I would appreciate anyone taking the time to do the survey[1] (the question mark) which has a couple of tasks to do. It'll give you the basic idea of what I'm thinking UI wise. [0] https://gist.github.com/rikkimax/4718740223748256d94b3b1474525012 [1] http://cattermole.co.nz/comp626/
May 16 2017
parent Andre Pany <andre s-e-a-p.de> writes:
On Wednesday, 17 May 2017 at 01:54:20 UTC, rikki cattermole wrote:
 I've built a prototype UI[1] for some code[0] to solve this 
 exact problem.

 It may seem complex, but you can't rely on HTTP download 
 options for easy access to repositories.

 I would appreciate anyone taking the time to do the survey[1] 
 (the question mark) which has a couple of tasks to do. It'll 
 give you the basic idea of what I'm thinking UI wise.

 [0] 
 https://gist.github.com/rikkimax/4718740223748256d94b3b1474525012
 [1] http://cattermole.co.nz/comp626/
Same for me;) I build a little console application (100 lines of coding) which I call instead of dub. It allows me to have following dub.json { "name": "test", "dependencies": { "sample1": {"url": "http://localhost:8080/zapp-sample1.zip"}, "sample2": {"path": "C:\\D\\projects\\test\\zapp-sample2.zip"}, } } The console application downloads/extracts the zip files, makes a backup of the original dub.json, creates a new dub.json with the adapted path attributes, calls dub and after that restoring the original dub.json. If you want to use dub with git repositories and you can't for one or the other reason use the public github / public dub registry this feature is really nice. Kind regards André
May 16 2017