www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Need help to get OpenSSL 64 work on Windows x64 | I hate D's GC!

reply Suliman <evermind live.ru> writes:
It's look that GC in D is really suxx. There is already second 
toy-project where I am getting stuck on Windows with D for last 3 
month.

I'm using 32-bit build, because I can't understand which libs I 
should use to get OpenSSL 64 bit work with dlang-request.

32-bit version compile and works fine, but it's fail during 
downloading 300MB file with next error:
core.exception.OutOfMemoryError src\core\exception.d(696): Memory 
allocation failed

code:

import std.stdio;
import std.conv;
import std.file;
import std.path;
import std.string;
import std.algorithm;
import std.regex;

import requests;
pragma(lib, "ssl");
pragma(lib, "eay");


void main()
{
     string login = "Suliman1";
     string pass = "Infinity8"; //*
     string url = 
"https://n5eil01u.ecs.nsidc.org/SMAP/SPL3SMP_E.001/2017.07.11/";
	
	Request request;
	request.addHeaders(["User-Agent": "Mozilla/5.0 (Windows NT 6.1; 
Win64; x64; rv:54.0) Gecko/20100101 Firefox/54.0"]);
	request.authenticator = new BasicAuthentication(login, pass);
	auto rs = request.get(url);
	string content = to!string(rs.responseBody);

	string fname = content.matchFirst(`(SMAP_[^"]+)`).hit;

	string h5_file = url ~ fname;
	string isoXML_file = url ~ fname ~ `.iso.xml`;
	string qa_file = url ~ fname.replace(`h5`, `qa`);

	string [] arrayFullURLs;
	arrayFullURLs ~= h5_file;
	arrayFullURLs ~= isoXML_file;
	arrayFullURLs ~= qa_file;

	foreach(f; arrayFullURLs)
	{
		writeln("Loading file: ", f);
		auto rs1 = request.get(f);
		File file = File(baseName(f), "w");
		file.write(rs1.responseBody);
	}
	
}

dub.sdl:
name "parser"
dependency "requests" version="~>0.4.2"
dependency "progress" version="~>4.0.3"


--
* please do now hack account because I publish this pass to get 
it's reproduce error easier.

So could anybody explain step-by-step how to get OpenSSL 64 work 
on 64-bit Windows.

On my PC this app eat 500MB of RAM and then failure with error 
above.
Jul 14 2017
parent reply bauss <jj_1337 live.dk> writes:
On Friday, 14 July 2017 at 13:16:17 UTC, Suliman wrote:

 It's look that GC in D is really suxx. There is already second 
 toy-project where I am getting stuck on Windows with D for last 
 3 month.

 I'm using 32-bit build, because I can't understand which libs I 
 should use to get OpenSSL 64 bit work with dlang-request.

 32-bit version compile and works fine, but it's fail during 
 downloading 300MB file with next error:
 core.exception.OutOfMemoryError src\core\exception.d(696): 
 Memory allocation failed


You might wanna read the file in chunks and write in chunks.
Jul 14 2017
parent reply Suliman <evermind live.ru> writes:
On Friday, 14 July 2017 at 14:50:04 UTC, bauss wrote:

 On Friday, 14 July 2017 at 13:16:17 UTC, Suliman wrote:

 It's look that GC in D is really suxx. There is already second 
 toy-project where I am getting stuck on Windows with D for 
 last 3 month.

 I'm using 32-bit build, because I can't understand which libs 
 I should use to get OpenSSL 64 bit work with dlang-request.

 32-bit version compile and works fine, but it's fail during 
 downloading 300MB file with next error:
 core.exception.OutOfMemoryError src\core\exception.d(696): 
 Memory allocation failed


 You might wanna read the file in chunks and write in chunks.


It's helped, but it's obviously that D on Windows should have 
64-bit linker out of the box. Because GC on 32-bit machine show a 
lot of bugs.
Jul 14 2017
parent reply tetyys <tetyys tetyys.com> writes:
On Friday, 14 July 2017 at 17:24:05 UTC, Suliman wrote:

 GC on 32-bit machine show a lot of bugs.


such as..?
Jul 15 2017
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 07/15/2017 03:05 PM, tetyys wrote:

 On Friday, 14 July 2017 at 17:24:05 UTC, Suliman wrote:

 GC on 32-bit machine show a lot of bugs.


 such as..?


D's GC is conservative i.e. it cannot assume an integer is not a 
pointer. There are ways around this such as marking the memory block as 
not containing pointers but in general, if the language allows you to 
cast a pointer to integer, it's possible that any integer may be the 
only reference to a GC memory.

The problem with 32-bit is, it's very likely that an integer value may 
happen to have a value that equals to an address into GC memory. This 
happens when you allocate e.g. half gigabyte memory, read a file into 
it. So far so good. Unfortunately, it's likely that there is some 
integer in the program that points into this half a gig memory and the 
GC will never release it.

Ali
Jul 15 2017