www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Download a file into array (using std.net.curl.download)

reply BoQsc <vaidas.boqsc gmail.com> writes:
Let's say I can't store information into files.
Is it possible to download a file into an array.


This is what I have now. It saves a download into a file named: 
file.txt
 import std;
 import std.net.curl;
 
 
 void main()
 {
     writeln("Hello D");
     
 download("https://gist.githubusercontent.com/deekayen/4148741/raw/98d35708fa344717d8eee15d11987de6c
e26d7d/1-1000.txt", "file.txt");
 }
Also run.dlang.io seems to be blocking and not letting me to run curl requests. I'm feeling sad right now. https://run.dlang.io/?compiler=dmd&source=import%20std;%0Aimport%20std.net.curl;%0A%0A%0Avoid%20main()%0A%7B%0A%20%20%20%20writeln(%22Hello%20D%22);%0A%20%20%20%20download(%22https:%2F%2Fgist.githubusercontent.com%2Fdeekayen%2F4148741%2Fraw%2F98d35708fa344717d8eee15d11987de6c8e26d7d%2F1-1000.txt%22,%20%22.%22);%0A%7D
Jul 05 2021
parent reply MoonlightSentinel <moonlightsentinel disroot.org> writes:
On Monday, 5 July 2021 at 14:57:20 UTC, BoQsc wrote:
 Let's say I can't store information into files.
 Is it possible to download a file into an array.
Yes, use [`get`](https://dlang.org/phobos/std_net_curl.html#.get): ```d import std.net.curl; void main() { char[] content = get("https://gist.githubusercontent.com/deekayen/4148741/raw/98d35708fa344717d8eee15d11987de6c8e26d7d/1-1000.txt"); } ```
Jul 05 2021
parent reply notna <notna.remove.this ist-einmalig.de> writes:
On Monday, 5 July 2021 at 15:08:45 UTC, MoonlightSentinel wrote:
 On Monday, 5 July 2021 at 14:57:20 UTC, BoQsc wrote:
 Let's say I can't store information into files.
 Is it possible to download a file into an array.
Yes, use [`get`](https://dlang.org/phobos/std_net_curl.html#.get): ```d import std.net.curl; void main() { char[] content = get("https://gist.githubusercontent.com/deekayen/4148741/raw/98d35708fa344717d8eee15d11987de6c8e26d7d/1-1000.txt"); } ```
On Windows: ``` ::> dmd curl_get.d ::> .\curl_get.exe object.Error (0): Access Violation ---------------- 0x0283CA66 0x0041DE8D 0x004023A2 0x00402308 0x00414D33 0x00414CAD 0x00414B48 0x0040D41F 0x00402363 0x74B96359 in BaseThreadInitThunk 0x773887A4 in RtlGetAppContainerNamedObjectPath 0x77388774 in RtlGetAppContainerNamedObjectPath ``` Nice and helpful Error messages is on the top of our desires list, right?
Jul 07 2021
next sibling parent reply z <z z.com> writes:
On Wednesday, 7 July 2021 at 10:27:47 UTC, notna wrote:
 On Windows:
 ```
 ::> dmd curl_get.d
 ::> .\curl_get.exe

 object.Error (0): Access Violation
 ----------------
 0x0283CA66
 0x0041DE8D
 0x004023A2
 0x00402308
 0x00414D33
 0x00414CAD
 0x00414B48
 0x0040D41F
 0x00402363
 0x74B96359 in BaseThreadInitThunk
 0x773887A4 in RtlGetAppContainerNamedObjectPath
 0x77388774 in RtlGetAppContainerNamedObjectPath
 ```

 Nice and helpful Error messages is on the top of our desires 
 list, right?
On 64 bits you don't even get a stack trace or description. Sad, i know. If you want better i could recommend you to compile with `-g` and hook up a debugger, then just let it run and it should triger a breakpoint on 0xC0000009(access violation).
Jul 07 2021
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 07/07/2021 11:54 PM, z wrote:
 On Wednesday, 7 July 2021 at 10:27:47 UTC, notna wrote:
 On Windows:
 ```
 ::> dmd curl_get.d
 ::> .\curl_get.exe

 object.Error (0): Access Violation
 ----------------
 0x0283CA66
 0x0041DE8D
 0x004023A2
 0x00402308
 0x00414D33
 0x00414CAD
 0x00414B48
 0x0040D41F
 0x00402363
 0x74B96359 in BaseThreadInitThunk
 0x773887A4 in RtlGetAppContainerNamedObjectPath
 0x77388774 in RtlGetAppContainerNamedObjectPath
 ```

 Nice and helpful Error messages is on the top of our desires list, right?
On 64 bits you don't even get a stack trace or description. Sad, i know. If you want better i could recommend you to compile with `-g` and hook up a debugger, then just let it run and it should triger a breakpoint on 0xC0000009(access violation).
With ldc you can turn on address sanitizer which will give you that information (and a LOT more!) without a debugger (but you still need -g). http://johanengelen.github.io/ldc/2017/12/25/LDC-and-AddressSanitizer.html
Jul 07 2021
parent notna <notna.remove.this ist-einmalig.de> writes:
On Wednesday, 7 July 2021 at 12:07:26 UTC, rikki cattermole wrote:
 With ldc you can turn on address sanitizer which will give you 
 that information (and a LOT more!) without a debugger (but you 
 still need -g).

 http://johanengelen.github.io/ldc/2017/12/25/LDC-and-AddressSanitizer.html
compiling with ldc2 "just works"! thanks for the hint ;)
Jul 07 2021
prev sibling parent notna <notna.remove.this ist-einmalig.de> writes:
On Wednesday, 7 July 2021 at 11:54:39 UTC, z wrote:
 On 64 bits you don't even get a stack trace or description.
 Sad, i know.
 If you want better i could recommend you to compile with `-g` 
 and hook up a debugger, then just let it run and it should 
 triger a breakpoint on 0xC0000009(access violation).
Well, that's an option, for those who want and are knowledgeable to debug ;) Anyhow, compiling it with ```dmd -v -g``` results in ``` λ .\curl_get.exe object.Error (0): Access Violation ---------------- 0x028DCA66 0x004211E1 in ref void* std.concurrency.initOnce!(std.net.curl.CurlAPI._handle).initOnce(lazy void*) 0x00402449 in char[] std.net.curl.get!(std.net.curl.AutoProtocol, char).get(const(char)[], std.net.curl.AutoProtocol) at c:\d\dmd2\windows\bin\..\..\src\phobos\std\net\curl.d(588) 0x00402308 in _Dmain at C:\Programming\D_playground\curl_get.d(6) 0x00417E0B in void rt.dmain2._d_run_main2(char[][], uint, extern (C) int function(char[][])*).runAll().__lambda1() 0x00417D85 in void rt.dmain2._d_run_main2(char[][], uint, extern (C) int function(char[][])*).runAll() 0x00417C20 in _d_run_main2 0x0040FD63 in _d_run_main 0x00402404 in main at c:\d\dmd2\windows\bin\..\..\src\druntime\import\core\internal\entrypoint.d(29) 0x0048ADC9 in mainCRTStartup 0x74B96359 in BaseThreadInitThunk 0x773887A4 in RtlGetAppContainerNamedObjectPath 0x77388774 in RtlGetAppContainerNamedObjectPath ```
Jul 07 2021
prev sibling parent reply MoonlightSentinel <moonlightsentinel disroot.org> writes:
On Wednesday, 7 July 2021 at 10:27:47 UTC, notna wrote:
 On Windows:
 [...]
 Nice and helpful Error messages is on the top of our desires 
 list, right?
It's hard to give proper error backtraces without debug information (-g). Anyways, I can reproduce the error when compiling with `-m32` (the default), using `-m32mscoff` and `-m64` works as expected. You should probably avoid `-m32` in general (as dub does btw).
Jul 07 2021
parent notna <notna.remove.this ist-einmalig.de> writes:
On Wednesday, 7 July 2021 at 13:49:50 UTC, MoonlightSentinel 
wrote:
 On Wednesday, 7 July 2021 at 10:27:47 UTC, notna wrote:
 On Windows:
 [...]
 Nice and helpful Error messages is on the top of our desires 
 list, right?
It's hard to give proper error backtraces without debug information (-g). Anyways, I can reproduce the error when compiling with `-m32` (the default), using `-m32mscoff` and `-m64` works as expected. You should probably avoid `-m32` in general (as dub does btw).
for what it's worth... due to license restrictions and costs we are not allowed to have the MS toolchain installed on our office clients... tried it now HOME with the '-m64' switch anabled, as suggested by you, and can confirm, it also "just works"...
Jul 08 2021