digitalmars.D.learn - What is the Correct way to Malloc in nogc section?
- Kitt (35/35) Feb 12 2015 I'm currently trying to write a personal Associate Array class
- Kitt (6/6) Feb 12 2015 I realized the above "_new" has an issue with emplace depending
- Foo (5/40) Feb 12 2015 This is something I've done recently.
- anonymous (13/17) Feb 13 2015 /* Class and Struct */
- Foo (6/23) Feb 13 2015 Ah, that is nice to know. I will fix this soon. Or would you help
- anonymous (2/4) Feb 13 2015 As it is right now, I fear it may do more harm than good.
- Foo (6/10) Feb 13 2015 Always the same in this newsgroup. You want to help as good as
- Kitt (5/16) Feb 13 2015 Please don't do that. Even if the code has some errors, it was a
- bearophile (7/9) Feb 13 2015 Language communities aren't perfect, but the success of a
- ketmar (9/11) Feb 13 2015 code that you throwing at newcomer may not be perfect, but it at least=2...
- weaselcat (5/22) Feb 13 2015 Low level memory management isn't easy.
- ketmar (3/28) Feb 13 2015 how "smart pointers" can help here? they don't magically hiding all the=...
- Foo (4/21) Feb 21 2015 Finally, I tried to take your criticism objectively and, as far
- Jakob Ovrum (16/28) Feb 12 2015 this one, are related to the loss of guarantee attributes when
- Kitt (8/12) Feb 13 2015 I see. So, in theory, after some patching emplace(),
- Benjamin Thaut (9/9) Feb 16 2015 Hi,
I'm currently trying to write a personal Associate Array class that can be used in nogc sections. Obviously, this means I'll want to use malloc/free, however I'm not sure what the "Correct" way to do this is. In a few places online I've seen code for custom "_new" and "_delete" functions similar to this: [code] T _new(T, Args...) (Args args) { size_t objSize; static if(is (ValType == class)) objSize = __traits(classInstanceSize, T); else objSize = T.sizeof; void* tmp = core.stdc.stdlib.malloc(objSize); if (!tmp) throw new Exception("Memory allocation failed"); void[] mem = tmp[0..objSize]; T obj = emplace!(T, Args)(mem, args); return obj; } void _delete(T)(T obj) { clear(obj); core.stdc.stdlib.free(cast(void*)obj); } [/code] The Exception obviously uses the GC, and would need to be replaced with a printf or something; however, emplace doesn't have a nogc replacement that I know of. What I'd like to know, from people much more knowledgeable about the ins and outs of D, is what the "Best" or "Correct" way to allocate and deallocate within nogc sections? PS: Tangentially related, what hashing algorithm does D use? I've seen people suggest using typeid(object).toHash(&object); however, toHash isn't nogc, so I need to write my own Hashing function. For the sake of consistency and peace of mind, I'd really like to match mine as closely to Ds internal one as possible.
Feb 12 2015
I realized the above "_new" has an issue with emplace depending on whether it's a class (reference type) or not. Class c = emplace() works, but something like string s = emplace() does not. Is doing string s = *emplace() safe and okay? Or is there a better way to handle the differences between allocating a Class with malloc vs a non-Class?
Feb 12 2015
On Thursday, 12 February 2015 at 23:27:51 UTC, Kitt wrote:I'm currently trying to write a personal Associate Array class that can be used in nogc sections. Obviously, this means I'll want to use malloc/free, however I'm not sure what the "Correct" way to do this is. In a few places online I've seen code for custom "_new" and "_delete" functions similar to this: [code] T _new(T, Args...) (Args args) { size_t objSize; static if(is (ValType == class)) objSize = __traits(classInstanceSize, T); else objSize = T.sizeof; void* tmp = core.stdc.stdlib.malloc(objSize); if (!tmp) throw new Exception("Memory allocation failed"); void[] mem = tmp[0..objSize]; T obj = emplace!(T, Args)(mem, args); return obj; } void _delete(T)(T obj) { clear(obj); core.stdc.stdlib.free(cast(void*)obj); } [/code] The Exception obviously uses the GC, and would need to be replaced with a printf or something; however, emplace doesn't have a nogc replacement that I know of. What I'd like to know, from people much more knowledgeable about the ins and outs of D, is what the "Best" or "Correct" way to allocate and deallocate within nogc sections? PS: Tangentially related, what hashing algorithm does D use? I've seen people suggest using typeid(object).toHash(&object); however, toHash isn't nogc, so I need to write my own Hashing function. For the sake of consistency and peace of mind, I'd really like to match mine as closely to Ds internal one as possible.This is something I've done recently. Would be glad if my code will help you: https://github.com/Dgame/m3 Especially the m3.d module could be useful for you.
Feb 12 2015
On Thursday, 12 February 2015 at 23:52:41 UTC, Foo wrote:This is something I've done recently. Would be glad if my code will help you: https://github.com/Dgame/m3 Especially the m3.d module could be useful for you./* Class and Struct */ emplace: You can't assume zero-initialization for structs. destruct: Is not memory-safe, and must not be marked trusted. /* Array */ make: You can't assume zero-initialization. T.sizeof is not the size of an element. reserve: Not safe (you're freeing the old memory), must not be trusted. append: T.sizeof is not the size of an element. You're multiplying twice with T.sizeof; in `append`, and also in `reserve`. destruct: Not safe, must not be trusted.
Feb 13 2015
On Friday, 13 February 2015 at 22:55:27 UTC, anonymous wrote:On Thursday, 12 February 2015 at 23:52:41 UTC, Foo wrote:I'm aware of that.This is something I've done recently. Would be glad if my code will help you: https://github.com/Dgame/m3 Especially the m3.d module could be useful for you./* Class and Struct */ emplace: You can't assume zero-initialization for structs. destruct: Is not memory-safe, and must not be marked trusted. /* Array */ make: You can't assume zero-initialization. T.sizeof is not the size of an element.reserve: Not safe (you're freeing the old memory), must not be trusted. append: T.sizeof is not the size of an element. You're multiplying twice with T.sizeof; in `append`, and also in `reserve`. destruct: Not safe, must not be trusted.Ah, that is nice to know. I will fix this soon. Or would you help me out with an PR? Don't understand me wrong. My code is not perfect, but maybe it can be helpful for someone.
Feb 13 2015
On Friday, 13 February 2015 at 23:04:25 UTC, Foo wrote:Don't understand me wrong. My code is not perfect, but maybe it can be helpful for someone.As it is right now, I fear it may do more harm than good.
Feb 13 2015
On Friday, 13 February 2015 at 23:13:05 UTC, anonymous wrote:On Friday, 13 February 2015 at 23:04:25 UTC, Foo wrote:Always the same in this newsgroup. You want to help as good as you can (even if it's not perfect) and all you get are subliminal messages... :) I'm regret that I tried to help, I will delete this repo as far as possible. :)Don't understand me wrong. My code is not perfect, but maybe it can be helpful for someone.As it is right now, I fear it may do more harm than good.
Feb 13 2015
On Friday, 13 February 2015 at 23:29:11 UTC, Foo wrote:On Friday, 13 February 2015 at 23:13:05 UTC, anonymous wrote:Please don't do that. Even if the code has some errors, it was a helpful and useful module I can work with. I'm sure others may feel the same way =) Stick and Stones.On Friday, 13 February 2015 at 23:04:25 UTC, Foo wrote:Always the same in this newsgroup. You want to help as good as you can (even if it's not perfect) and all you get are subliminal messages... :) I'm regret that I tried to help, I will delete this repo as far as possible. :)Don't understand me wrong. My code is not perfect, but maybe it can be helpful for someone.As it is right now, I fear it may do more harm than good.
Feb 13 2015
Foo:I'm regret that I tried to help, I will delete this repo as far as possible. :)Language communities aren't perfect, but the success of a language comes from the help of many little hands :) Perhaps Rust will "win" over D in the end, but there's no reason to throw away your work just for a comment... be tolerant and be good. Bye, bearophile
Feb 13 2015
On Fri, 13 Feb 2015 23:29:10 +0000, Foo wrote:Always the same in this newsgroup. You want to help as good as you can (even if it's not perfect) and all you get are subliminal messages... :)code that you throwing at newcomer may not be perfect, but it at least=20 should not be wrong. your code is obvious wrong, as anonymous said. you may make your own question like "what is wrong with this code?".=20 would you be glad to get the wrong answer? anonymous is right: your code does more harm than good *as* *an*=20 *answer*. but it seems that you don't care about the quality, "it works=20 for me so it's good", yeah? 'cause if you *do* care, you're able to see=20 why anonymous said what he's said.=
Feb 13 2015
On Friday, 13 February 2015 at 22:55:27 UTC, anonymous wrote:On Thursday, 12 February 2015 at 23:52:41 UTC, Foo wrote:Low level memory management isn't easy. Bugs like these wouldn't exist if D had decent smart pointer implementations in its standard library. You seem knowledgeable enough, care to lend a hand? ; )This is something I've done recently. Would be glad if my code will help you: https://github.com/Dgame/m3 Especially the m3.d module could be useful for you./* Class and Struct */ emplace: You can't assume zero-initialization for structs. destruct: Is not memory-safe, and must not be marked trusted. /* Array */ make: You can't assume zero-initialization. T.sizeof is not the size of an element. reserve: Not safe (you're freeing the old memory), must not be trusted. append: T.sizeof is not the size of an element. You're multiplying twice with T.sizeof; in `append`, and also in `reserve`. destruct: Not safe, must not be trusted.
Feb 13 2015
On Sat, 14 Feb 2015 00:57:33 +0000, weaselcat wrote:On Friday, 13 February 2015 at 22:55:27 UTC, anonymous wrote:how "smart pointers" can help here? they don't magically hiding all the=20 low-level mechanics.=On Thursday, 12 February 2015 at 23:52:41 UTC, Foo wrote:=20 Low level memory management isn't easy. Bugs like these wouldn't exist if D had decent smart pointer implementations in its standard library. You seem knowledgeable enough, care to lend a hand? ; )This is something I've done recently. Would be glad if my code will help you: https://github.com/Dgame/m3 Especially the m3.d module could be useful for you./* Class and Struct */ emplace: You can't assume zero-initialization for structs. destruct: Is not memory-safe, and must not be marked trusted. /* Array */ make: You can't assume zero-initialization. T.sizeof is not the size of an element. reserve: Not safe (you're freeing the old memory), must not be trusted. append: T.sizeof is not the size of an element. You're multiplying twice with T.sizeof; in `append`, and also in `reserve`. destruct: Not safe, must not be trusted.
Feb 13 2015
On Saturday, 14 February 2015 at 06:38:19 UTC, ketmar wrote:On Sat, 14 Feb 2015 00:57:33 +0000, weaselcat wrote:The file he was referring to was a smart pointer implementation.On Friday, 13 February 2015 at 22:55:27 UTC, anonymous wrote:how "smart pointers" can help here? they don't magically hiding all the low-level mechanics.On Thursday, 12 February 2015 at 23:52:41 UTC, Foo wrote:Low level memory management isn't easy. Bugs like these wouldn't exist if D had decent smart pointer implementations in its standard library. You seem knowledgeable enough, care to lend a hand? ; )This is something I've done recently. Would be glad if my code will help you: https://github.com/Dgame/m3 Especially the m3.d module could be useful for you./* Class and Struct */ emplace: You can't assume zero-initialization for structs. destruct: Is not memory-safe, and must not be marked trusted. /* Array */ make: You can't assume zero-initialization. T.sizeof is not the size of an element. reserve: Not safe (you're freeing the old memory), must not be trusted. append: T.sizeof is not the size of an element. You're multiplying twice with T.sizeof; in `append`, and also in `reserve`. destruct: Not safe, must not be trusted.
Feb 13 2015
On Sat, 14 Feb 2015 07:59:51 +0000, weaselcat wrote:*one* of the files.=how "smart pointers" can help here? they don't magically hiding all the low-level mechanics.=20 The file he was referring to was a smart pointer implementation.
Feb 14 2015
On Friday, 13 February 2015 at 22:55:27 UTC, anonymous wrote:On Thursday, 12 February 2015 at 23:52:41 UTC, Foo wrote:Finally, I tried to take your criticism objectively and, as far as I can tell, I resolved the issues. Is there still any objections or misconduct?This is something I've done recently. Would be glad if my code will help you: https://github.com/Dgame/m3 Especially the m3.d module could be useful for you./* Class and Struct */ emplace: You can't assume zero-initialization for structs. destruct: Is not memory-safe, and must not be marked trusted. /* Array */ make: You can't assume zero-initialization. T.sizeof is not the size of an element. reserve: Not safe (you're freeing the old memory), must not be trusted. append: T.sizeof is not the size of an element. You're multiplying twice with T.sizeof; in `append`, and also in `reserve`. destruct: Not safe, must not be trusted.
Feb 21 2015
On Saturday, 21 February 2015 at 13:41:41 UTC, Foo wrote:Finally, I tried to take your criticism objectively and, as far as I can tell, I resolved the issues. Is there still any objections or misconduct?Nice. I think you fixed everything I had pointed out.
Feb 21 2015
On Saturday, 21 February 2015 at 14:39:47 UTC, anonymous wrote:On Saturday, 21 February 2015 at 13:41:41 UTC, Foo wrote:Yeah, I hope so too. Sorry for my annoyed behaviour, that wasn't my week and you where the the straw to break the camel's back. ;) I hope I have the phrase translated correctly. :DFinally, I tried to take your criticism objectively and, as far as I can tell, I resolved the issues. Is there still any objections or misconduct?Nice. I think you fixed everything I had pointed out.
Feb 21 2015
On Thursday, 12 February 2015 at 23:27:51 UTC, Kitt wrote:The Exception obviously uses the GC, and would need to be replaced with a printf or something; however, emplace doesn't have a nogc replacement that I know of. What I'd like to know, from people much more knowledgeable about the ins and outs of D, is what the "Best" or "Correct" way to allocate and deallocate within nogc sections?This issue, as well asPS: Tangentially related, what hashing algorithm does D use? I've seen people suggest using typeid(object).toHash(&object); however, toHash isn't nogc, so I need to write my own Hashing function. For the sake of consistency and peace of mind, I'd really like to match mine as closely to Ds internal one as possible.this one, are related to the loss of guarantee attributes when using TypeInfo methods, most recently discussed in this thread[1]. It is essentially a bug, and a work in progress. The correct way to handle OOM in D is to call core.exception.onOutOfMemoryError: --- if (auto p = malloc(...)) /* use p */ else onOutOfMemoryError(); // Throws OutOfMemoryError without allocating memory --- [1] http://forum.dlang.org/post/krccldftrbbczahasvbi forum.dlang.org
Feb 12 2015
On Friday, 13 February 2015 at 05:56:37 UTC, Jakob Ovrum wrote:This issue, ... [is] related to the loss of guarantee attributes when using TypeInfo methods, most recently discussed in this thread[1]. It is essentially a bug, and a work in progress.I see. So, in theory, after some patching emplace(), onOutOfMemoryError() and toHash() should all work in nogc sections? If this is the case, Foo's m3.d module looks rather helpful, and should tide me over until this fix is released; though, I'll be honest when I say that I don't know how idiomatic or "correct" Foo's implementation is.
Feb 13 2015
Hi, you can also take a look at my implementation: https://github.com/Ingrater/druntime/blob/master/src/core/allocator.d Look at AllocatorNew and AllocatorDelete Especially important is, that you correctly handle the constructor throwing an exception. You have to catch that exception in your new function, release the memory and rethrow. Kind Regards Benjamin Thaut
Feb 16 2015