www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - GitHub behind proxy servers among other questions

reply "JohnnyK" <johnnykinsey comcast.net> writes:
Hi all,
   I have searched everywhere over the Internet and I have yet to 
find a way to clone a project using git when my workstation is 
behind a company proxy.  Can you guys clone your projects to a 
single zip file that I can download?  This would be easier 
instead of working with some strange command-line tool that does 
not recognize modern networks.  Honestly I just need the DWT 
binary with the help files so I can use the api.  I have spent 
weeks searching for a way to download DWT to my windows 
workstation at work and have yet figured out how to make GIT work.
Jul 07 2013
next sibling parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Monday, 8 July 2013 at 02:42:50 UTC, JohnnyK wrote:
 Hi all,
   I have searched everywhere over the Internet and I have yet 
 to find a way to clone a project using git when my workstation 
 is behind a company proxy.  Can you guys clone your projects to 
 a single zip file that I can download?  This would be easier 
 instead of working with some strange command-line tool that 
 does not recognize modern networks.  Honestly I just need the 
 DWT binary with the help files so I can use the api.  I have 
 spent weeks searching for a way to download DWT to my windows 
 workstation at work and have yet figured out how to make GIT 
 work.
Git provides a download by zip; Right side, bottom. This is what I found on getting Git to work with a proxy: http://stackoverflow.com/questions/128035/how-do-i-pull-from-a-git-repository-through-an-http-proxy
Jul 07 2013
next sibling parent reply "JohnnyK" <johnnykinsey comcast.net> writes:
On Monday, 8 July 2013 at 03:35:03 UTC, Jesse Phillips wrote:
 On Monday, 8 July 2013 at 02:42:50 UTC, JohnnyK wrote:
 Hi all,
  I have searched everywhere over the Internet and I have yet 
 to find a way to clone a project using git when my workstation 
 is behind a company proxy.  Can you guys clone your projects 
 to a single zip file that I can download?  This would be 
 easier instead of working with some strange command-line tool 
 that does not recognize modern networks.  Honestly I just need 
 the DWT binary with the help files so I can use the api.  I 
 have spent weeks searching for a way to download DWT to my 
 windows workstation at work and have yet figured out how to 
 make GIT work.
Git provides a download by zip; Right side, bottom. This is what I found on getting Git to work with a proxy: http://stackoverflow.com/questions/128035/how-do-i-pull-from-a-git-repository-through-an-http-proxy
I appreciate your response. I have tried these. I think the real issue is that I am not sure on the IP and port needed for the proxy here at work. The company uses WPAD in the browser and I cannot figure out what the IP and port the browser is using to connect through the proxy. If I knew that I probably could make it work. It would be nice if GitHUB would change their Downlaod Zip button such that it does a recursive zip to include all the subfolders.
Jul 08 2013
next sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 8 July 2013 at 16:24:53 UTC, JohnnyK wrote:
 On Monday, 8 July 2013 at 03:35:03 UTC, Jesse Phillips wrote:
 On Monday, 8 July 2013 at 02:42:50 UTC, JohnnyK wrote:
 Hi all,
 I have searched everywhere over the Internet and I have yet 
 to find a way to clone a project using git when my 
 workstation is behind a company proxy.  Can you guys clone 
 your projects to a single zip file that I can download?  This 
 would be easier instead of working with some strange 
 command-line tool that does not recognize modern networks.  
 Honestly I just need the DWT binary with the help files so I 
 can use the api.  I have spent weeks searching for a way to 
 download DWT to my windows workstation at work and have yet 
 figured out how to make GIT work.
Git provides a download by zip; Right side, bottom. This is what I found on getting Git to work with a proxy: http://stackoverflow.com/questions/128035/how-do-i-pull-from-a-git-repository-through-an-http-proxy
I appreciate your response. I have tried these. I think the real issue is that I am not sure on the IP and port needed for the proxy here at work. The company uses WPAD in the browser and I cannot figure out what the IP and port the browser is using to connect through the proxy. If I knew that I probably could make it work. It would be nice if GitHUB would change their Downlaod Zip button such that it does a recursive zip to include all the subfolders.
I would just ssh to somewhere outside the firewall, git clone there, tar the folder, then scp it back. Assuming that is possible through your firewall. If you're on windows then there are the always useful putty and winscp to do your ssh and scp work respectively.
Jul 08 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 8 July 2013 at 16:49:05 UTC, John Colvin wrote:
 On Monday, 8 July 2013 at 16:24:53 UTC, JohnnyK wrote:
 On Monday, 8 July 2013 at 03:35:03 UTC, Jesse Phillips wrote:
 On Monday, 8 July 2013 at 02:42:50 UTC, JohnnyK wrote:
 Hi all,
 I have searched everywhere over the Internet and I have yet 
 to find a way to clone a project using git when my 
 workstation is behind a company proxy.  Can you guys clone 
 your projects to a single zip file that I can download?  
 This would be easier instead of working with some strange 
 command-line tool that does not recognize modern networks.  
 Honestly I just need the DWT binary with the help files so I 
 can use the api.  I have spent weeks searching for a way to 
 download DWT to my windows workstation at work and have yet 
 figured out how to make GIT work.
Git provides a download by zip; Right side, bottom. This is what I found on getting Git to work with a proxy: http://stackoverflow.com/questions/128035/how-do-i-pull-from-a-git-repository-through-an-http-proxy
I appreciate your response. I have tried these. I think the real issue is that I am not sure on the IP and port needed for the proxy here at work. The company uses WPAD in the browser and I cannot figure out what the IP and port the browser is using to connect through the proxy. If I knew that I probably could make it work. It would be nice if GitHUB would change their Downlaod Zip button such that it does a recursive zip to include all the subfolders.
I would just ssh to somewhere outside the firewall, git clone there, tar the folder, then scp it back. Assuming that is possible through your firewall. If you're on windows then there are the always useful putty and winscp to do your ssh and scp work respectively.
Well, the thing to remember is that he did say "company", so such approaches to bypass a firewall could very well be a violation of his IT policies. I know doing this would be *major* violation of my companie's policies, and could be cause for termination.
Jul 08 2013
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 8 July 2013 at 18:09:44 UTC, monarch_dodra wrote:
 On Monday, 8 July 2013 at 16:49:05 UTC, John Colvin wrote:
 On Monday, 8 July 2013 at 16:24:53 UTC, JohnnyK wrote:
 On Monday, 8 July 2013 at 03:35:03 UTC, Jesse Phillips wrote:
 On Monday, 8 July 2013 at 02:42:50 UTC, JohnnyK wrote:
 Hi all,
 I have searched everywhere over the Internet and I have yet 
 to find a way to clone a project using git when my 
 workstation is behind a company proxy.  Can you guys clone 
 your projects to a single zip file that I can download?  
 This would be easier instead of working with some strange 
 command-line tool that does not recognize modern networks.  
 Honestly I just need the DWT binary with the help files so 
 I can use the api.  I have spent weeks searching for a way 
 to download DWT to my windows workstation at work and have 
 yet figured out how to make GIT work.
Git provides a download by zip; Right side, bottom. This is what I found on getting Git to work with a proxy: http://stackoverflow.com/questions/128035/how-do-i-pull-from-a-git-repository-through-an-http-proxy
I appreciate your response. I have tried these. I think the real issue is that I am not sure on the IP and port needed for the proxy here at work. The company uses WPAD in the browser and I cannot figure out what the IP and port the browser is using to connect through the proxy. If I knew that I probably could make it work. It would be nice if GitHUB would change their Downlaod Zip button such that it does a recursive zip to include all the subfolders.
I would just ssh to somewhere outside the firewall, git clone there, tar the folder, then scp it back. Assuming that is possible through your firewall. If you're on windows then there are the always useful putty and winscp to do your ssh and scp work respectively.
Well, the thing to remember is that he did say "company", so such approaches to bypass a firewall could very well be a violation of his IT policies. I know doing this would be *major* violation of my companie's policies, and could be cause for termination.
I don't see how it is problematic? The firewall doesn't allow a particular type of connection, so you cache the result somewhere else and then access it via something that is allowed. It's the same as downloading the zip file from github, only you're doing the "zipping" yourself, then using scp to do the download instead of http. Is it common to have ssh connections banned in the IT policy but *not* blocked by the firewall?
Jul 08 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 8 July 2013 at 19:20:46 UTC, John Colvin wrote:
 I don't see how it is problematic? The firewall doesn't allow a 
 particular type of connection, so you cache the result 
 somewhere else and then access it via something that is allowed.

 It's the same as downloading the zip file from github, only 
 you're doing the "zipping" yourself, then using scp to do the 
 download instead of http.

 Is it common to have ssh connections banned in the IT policy 
 but *not* blocked by the firewall?
I'm just saying that maybe the firwall is blocking that protocol on purpose? Github can very easily be seen as "File Sharing" and/or "Online Storage". The sole act of downloading/uploading anything on there is possible violations of policy. You could download viruses, or be stealing your company's code to publish it online. Even if you *can* download the zip via http, it doesn't mean you are actually allowed to. If your company's firewall doesn't block a porn site, does that means its OK to surf it? Now, most companies wouldn't really care that much about it, if you connect every now and then, no one will care. Worst case scenario, you'll get a reminder not to download/upload crap on internet. But, if you start doing all this via ssh, or other alternative routes, then you are *activelly* bypassing protections, and in particular, doing something you *know* you shouldn't be doing. That's a more serious offense. Now, do what you want. I'm just saying before going it and deploying such tools to download the files on git, please make sure you aren't violating your company's policies.
Jul 09 2013
parent "JohnnyK" <johnnykinsey comcast.net> writes:
On Tuesday, 9 July 2013 at 08:55:29 UTC, monarch_dodra wrote:
 On Monday, 8 July 2013 at 19:20:46 UTC, John Colvin wrote:
 I don't see how it is problematic? The firewall doesn't allow 
 a particular type of connection, so you cache the result 
 somewhere else and then access it via something that is 
 allowed.

 It's the same as downloading the zip file from github, only 
 you're doing the "zipping" yourself, then using scp to do the 
 download instead of http.

 Is it common to have ssh connections banned in the IT policy 
 but *not* blocked by the firewall?
I'm just saying that maybe the firwall is blocking that protocol on purpose? Github can very easily be seen as "File Sharing" and/or "Online Storage". The sole act of downloading/uploading anything on there is possible violations of policy. You could download viruses, or be stealing your company's code to publish it online. Even if you *can* download the zip via http, it doesn't mean you are actually allowed to. If your company's firewall doesn't block a porn site, does that means its OK to surf it? Now, most companies wouldn't really care that much about it, if you connect every now and then, no one will care. Worst case scenario, you'll get a reminder not to download/upload crap on internet. But, if you start doing all this via ssh, or other alternative routes, then you are *activelly* bypassing protections, and in particular, doing something you *know* you shouldn't be doing. That's a more serious offense. Now, do what you want. I'm just saying before going it and deploying such tools to download the files on git, please make sure you aren't violating your company's policies.
I do not encourage anyone to bypass network security protections and I do encourage letting your conscious be your guide on this topic. The issue is that it is just more complicating than I think it should be to download libraries and such. Authors of helper libraries and other D related code should take their queue from dlang.org and actually have binaries that users can download and just use without the need to compile before they can use the wares. It is not a firewall issue or anything like that. It was just GIT not supporting modern proxy authorization schemes. Problem resolved but it could have been avoided all together for those of us that are not to the point of contributing to these projects and are just trying to get their feet wet without drowning first. On another note I was able to successfully compile the DWT lib and I was able to compile the snippet example programs to test DWT. I did notice for those that are interested DWT exe's are over 2MB which is really what I was wanted to find out. I will redirect DWT questions to the appropriate forum. The workaround for GIT via proxy is just not as pretty as I would have liked it. However as I posted previously I found a suitable workaround for the issue. No other discussion is needed for this issue.
Jul 09 2013
prev sibling parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Monday, 8 July 2013 at 16:24:53 UTC, JohnnyK wrote:

 It would be nice if GitHUB would change their Downlaod Zip 
 button such that it does a recursive zip to include all the 
 subfolders.
Ehhh, well that is useless. Sorry I couldn't hook you up with a solution.
Jul 08 2013
parent reply "JohnnyK" <johnnykinsey comcast.net> writes:
On Monday, 8 July 2013 at 18:34:33 UTC, Jesse Phillips wrote:
 On Monday, 8 July 2013 at 16:24:53 UTC, JohnnyK wrote:

 It would be nice if GitHUB would change their Downlaod Zip 
 button such that it does a recursive zip to include all the 
 subfolders.
Ehhh, well that is useless. Sorry I couldn't hook you up with a solution.
Thanks for all the input. I found a tool called NTLMAPS at http://ntlmaps.sourceforge.net/ which allows one to create a pass through proxy which understands NTLM proxy servers and are able to connect and forward HTTP requests and return responses so long story short I can now git clone as I need to NTLMAPS config file is pretty easy to configure with your proxy authentication information. However I did have to change the following named/value pairs. from this LM_PART:1 NT_PART:0 # Highly experimental option. See research.txt for details. # LM - 06820000 # NT - 05820000 # LM + NT - 07820000 NTLM_FLAGS: 06820000 to this LM_PART:1 NT_PART:1 # Highly experimental option. See research.txt for details. # LM - 06820000 # NT - 05820000 # LM + NT - 07820000 NTLM_FLAGS: 07820000 The proxy that I am behind was not happy until I did that. Anyway it is working now and I hope this post serves to help others with the same issue. also note that I had to use http:// in place of git:// in the git paths and in the case of DWT I had to go to every major folder and git clone them individually because for some reason git wanted to use git:// for subsequent requests. It is strange that it did not do that for the Win-Res folder just for the others. So I had to do commands like the following C:\GITClones>cd dwt C:\GITClones\dwt>git clone --recursive http://github.com/d-widget-toolkit/base.g it Cloning into 'base'... remote: Counting objects: 806, done. remote: Compressing objects: 100% (326/326), done. remote: Total 806 (delta 402), reused 803 (delta 401) Receiving objects: 100% (806/806), 310.62 KiB | 261.00 KiB/s, done. Resolving deltas: 100% (402/402), done. C:\GITClones\dwt>git clone --recursive http://github.com/d-widget-toolkit/org.ec lipse.swt.gtk.linux.x86.git Cloning into 'org.eclipse.swt.gtk.linux.x86'... remote: Counting objects: 1691, done. remote: Compressing objects: 100% (634/634), done. remote: Total 1691 (delta 951), reused 1659 (delta 944) Receiving objects: 100% (1691/1691), 1.71 MiB | 361.00 KiB/s, done. Resolving deltas: 100% (951/951), done. C:\GITClones\dwt>git clone --recursive http://github.com/d-widget-toolkit/org.ec lipse.swt.win32.win32.x86.git Cloning into 'org.eclipse.swt.win32.win32.x86'... The only other thing I wish git could do is compile it for me so that I wouldn't have too but until we have HDgit I guess I am stuck with all this command line busy work. Thanks again for every ones time.
Jul 08 2013
parent reply "JohnnyK" <johnnykinsey comcast.net> writes:
On Monday, 8 July 2013 at 19:37:08 UTC, JohnnyK wrote:
Another thing about GitHub's Download Zip button and this process 
as a whole.  While the Download Zip button does allow you to 
download the master folders with recursive directories I do find 
it somewhat cumbersome or should I say awkward since you have to 
extract these in their correct locations upon download.  This is 
not good when you are trying to follow someones instructions on 
how to download, build, and install their wares.  I just don't 
get git?  Just compile the code and put a binary up there for 
people to use.  If I want to contribute to the code base then I 
will I guess go through all this hassle but for those of us not 
smart enough to contribute and just want to use it show me the 
binary please.  In the amount of time it is taking to get this 
stuff I could have made major strides at building my own.  Also I 
don't know what this fragmented downloading is going to do in the 
future when I want an updated version of the software.  Please 
remember many of us are simpletons and don't use all these fancy 
version control systems and just need the binary so that we can 
use it.  I guess you could say we are your customers and not your 
contributors.  Please everyone in the D community try and follow 
the KISS methodology and if you do I bet you could get many more 
users of your wares and not people banging their heads until they 
give up.  Yeah I was only 2 clicks away from giving up on this 
whole D thing thinking it was just too darned hard to make it 
worth my while.  Who knows I still may give up since I have not 
attempted to compile it yet.  I think I will wait until tomorrow 
before I try to tackle that part of the destruction's.  For now I 
think I have it downloaded.  BTW did I mention that I really hate 
git and compiling other people's code just to use an edit box and 
button on a dialog???
Jul 08 2013
parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 8 July 2013 at 19:57:52 UTC, JohnnyK wrote:
 On Monday, 8 July 2013 at 19:37:08 UTC, JohnnyK wrote:
 Another thing about GitHub's Download Zip button and this 
 process as a whole.  While the Download Zip button does allow 
 you to download the master folders with recursive directories I 
 do find it somewhat cumbersome or should I say awkward since 
 you have to extract these in their correct locations upon 
 download.  This is not good when you are trying to follow 
 someones instructions on how to download, build, and install 
 their wares.  I just don't get git?  Just compile the code and 
 put a binary up there for people to use.  If I want to 
 contribute to the code base then I will I guess go through all 
 this hassle but for those of us not smart enough to contribute 
 and just want to use it show me the binary please.  In the 
 amount of time it is taking to get this stuff I could have made 
 major strides at building my own.  Also I don't know what this 
 fragmented downloading is going to do in the future when I want 
 an updated version of the software.  Please remember many of us 
 are simpletons and don't use all these fancy version control 
 systems and just need the binary so that we can use it.  I 
 guess you could say we are your customers and not your 
 contributors.  Please everyone in the D community try and 
 follow the KISS methodology and if you do I bet you could get 
 many more users of your wares and not people banging their 
 heads until they give up.  Yeah I was only 2 clicks away from 
 giving up on this whole D thing thinking it was just too darned 
 hard to make it worth my while.  Who knows I still may give up 
 since I have not attempted to compile it yet.  I think I will 
 wait until tomorrow before I try to tackle that part of the 
 destruction's.  For now I think I have it downloaded.  BTW did 
 I mention that I really hate git and compiling other people's 
 code just to use an edit box and button on a dialog???
Ar you talking about D specifically? You *can* just download the package you know: http://dlang.org/download.html I'm actually wondering how you even reached git?
Jul 09 2013
prev sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 8 July 2013 at 03:35:03 UTC, Jesse Phillips wrote:
 On Monday, 8 July 2013 at 02:42:50 UTC, JohnnyK wrote:
 Hi all,
  I have searched everywhere over the Internet and I have yet 
 to find a way to clone a project using git when my workstation 
 is behind a company proxy.  Can you guys clone your projects 
 to a single zip file that I can download?  This would be 
 easier instead of working with some strange command-line tool 
 that does not recognize modern networks.  Honestly I just need 
 the DWT binary with the help files so I can use the api.  I 
 have spent weeks searching for a way to download DWT to my 
 windows workstation at work and have yet figured out how to 
 make GIT work.
Git provides a download by zip; Right side, bottom.
Holly crap! Is this new? I'm in need of this.
Jul 08 2013
prev sibling parent "Dicebot" <public dicebot.lv> writes:
On Monday, 8 July 2013 at 02:42:50 UTC, JohnnyK wrote:
 Hi all,
   I have searched everywhere over the Internet and I have yet 
 to find a way to clone a project using git when my workstation 
 is behind a company proxy.  Can you guys clone your projects to 
 a single zip file that I can download?  This would be easier 
 instead of working with some strange command-line tool that 
 does not recognize modern networks.  Honestly I just need the 
 DWT binary with the help files so I can use the api.  I have 
 spent weeks searching for a way to download DWT to my windows 
 workstation at work and have yet figured out how to make GIT 
 work.
git can clone via HTTP(S): git clone http://github.com/D-Programming-Language/dmd.git
Jul 08 2013