www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - What have I missed?

reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
  It's been a little while since I've been back. Got burned out 
and stuff. Currently watching all the Dconf 2014 videos (although 
I never finished the 2013 ones).

  So, I don't suppose there's a short quick & dirty summary of 
what's happened in the last 18 months?

  Also somewhat curious if anyone took my BitArray updates and got 
them implemented in phobos or not. Quite annoying I still haven't 
gotten a full understanding of GitHub yet, and at this rate 
probably won't.
Aug 05 2014
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 5 August 2014 at 09:30:45 UTC, Era Scarecrow wrote:

  It's been a little while since I've been back. Got burned out 
 and stuff. Currently watching all the Dconf 2014 videos 
 (although I never finished the 2013 ones).

  So, I don't suppose there's a short quick & dirty summary of 
 what's happened in the last 18 months?


Big things that directly come to my mind : DDMD, shared 
libraries,  nogc


  Also somewhat curious if anyone took my BitArray updates and 
 got them implemented in phobos or not. Quite annoying I still 
 haven't gotten a full understanding of GitHub yet, and at this 
 rate probably won't.


Check commit history 
https://github.com/D-Programming-Language/phobos/commits/master/std/bitmanip.d 
;)
Aug 05 2014
parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Tuesday, 5 August 2014 at 09:37:10 UTC, Dicebot wrote:

 Check commit history


  It's confusing. Glancing at code snippets the code doesn't look 
like mine. I'll just have to assume my work was junked.
Aug 05 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 5 August 2014 at 09:47:16 UTC, Era Scarecrow wrote:

 On Tuesday, 5 August 2014 at 09:37:10 UTC, Dicebot wrote:

 Check commit history


  It's confusing. Glancing at code snippets the code doesn't 
 look like mine. I'll just have to assume my work was junked.


I have meant that I have no idea what changes you are speaking 
about and change list for std.bitmanip is too long to look 
through unless you know exactly what to look for :(
Aug 05 2014
parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Tuesday, 5 August 2014 at 09:52:13 UTC, Dicebot wrote:

 I have meant that I have no idea what changes you are speaking 
 about and change list for std.bitmanip is too long to look 
 through unless you know exactly what to look for :(


  I remember i almost re-wrote the entire thing from scratch, and 
it had so many bug fixes it was crazy, something like 30 pulls 
and bug numbers were referenced; Not just for BitArray but for 
the BitManip template as well.

  Trust me, it would be recognizable, and long too. I think at one 
point Andre or Walter wanted it broken down to simpler and fewer 
fixes so they are more modularized, and to do so would be so 
counter productive since it was basically an entire rewrite.

  Worse is the error of why it wouldn't merge with phobos back in 
2012/2013... It was like 13 newlines didn't match up so it didn't 
know how to merge it.... 13 whitespaces... I don't know...
Aug 05 2014
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Tuesday, 5 August 2014 at 09:30:45 UTC, Era Scarecrow wrote:

  So, I don't suppose there's a short quick & dirty summary of 
 what's happened in the last 18 months?


Too much to list.

http://dlang.org/changelog.html
Aug 05 2014
prev sibling next sibling parent "Wyatt" <wyatt.epp gmail.com> writes:
On Tuesday, 5 August 2014 at 09:30:45 UTC, Era Scarecrow wrote:

  So, I don't suppose there's a short quick & dirty summary of 
 what's happened in the last 18 months?


The bikeshed is now a very pleasing red, but some people think it 
should be a different shade of red and the rest think it should 
be green.
Aug 05 2014
prev sibling next sibling parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
  Well it looks like i got a lot of work ahead of me. Is Ali's 
online book good and up-to-date on D's current behavior as well 
as it's features?

  I'll look over the bugzilla listing of stuff for BitManip and 
see if i can't help with that library once more, but so far 
personal experience trying to contribute to phobos & D has been a 
sub-par experience. Probably due to not being familiar with Git 
or GitHub.


  At the DConf2014 Walter Bright commented on using LZW 
compression on name mangling to manage template expansion to keep 
it from going overboard. Has anyone done that? If not i could do 
that...
Aug 06 2014
parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
  So I guess I need to ask: Should I try to resume work on the 
BitManip library? (So far it seems none of my code has been 
integrated to phobos)

  Assuming I do, should I try to fix lots of small bugs and make 
smaller pulls or should I try to do several at once? When I 
re-wrote the BitArray I personally think it is an improvement in 
many ways, and being a re-write you can't just do bug #xxxx and 
then bug #xxxx and then bug... etc etc.

  Also to ask, how many people tried out the rewrite I proposed, 
and do they think it was actually an improvement for ease of use, 
speed, fewer bugs, etc?
Aug 06 2014
next sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Thursday, 7 August 2014 at 01:40:13 UTC, Era Scarecrow wrote:

  So I guess I need to ask: Should I try to resume work on the 
 BitManip library? (So far it seems none of my code has been 
 integrated to phobos)

  Assuming I do, should I try to fix lots of small bugs and make 
 smaller pulls or should I try to do several at once? When I 
 re-wrote the BitArray I personally think it is an improvement 
 in many ways, and being a re-write you can't just do bug #xxxx 
 and then bug #xxxx and then bug... etc etc.

  Also to ask, how many people tried out the rewrite I proposed, 
 and do they think it was actually an improvement for ease of 
 use, speed, fewer bugs, etc?


I haven't looked at your rewrite, but from what I've seen the 
current implementation is indeed very awkward and full of bugs. 
There were also some discussions about the behaviour on resizing 
with respect to stomping, IIRC. This seems to imply that not even 
the semantics are completely clear. I guess a complete rewrite 
would be appropriate in such a situation.
Aug 07 2014
parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Thursday, 7 August 2014 at 10:08:30 UTC, Marc Schütz wrote:

 I haven't looked at your rewrite, but from what I've seen the 
 current implementation is indeed very awkward and full of bugs. 
 There were also some discussions about the behavior on resizing 
 with respect to stomping, IIRC. This seems to imply that not 
 even the semantics are completely clear. I guess a complete 
 rewrite would be appropriate in such a situation.


  Fun... I'll go through for the most recent version of DMD and 
see about porting my changes over in smaller bite-sized chunks 
and see if it won't be accepted. This might be easier than 
figuring out all the changes in the D language and figuring them 
out as I come across them (not that I used many of the features 
beyond templates and operator overloading).

  That is assuming this is still the case (it's been a year 
afterall). Hmmm. Off hand I'm remembering there was the issue of 
'sometimes a var sometimes a value sometimes a ref' that was 
popping up when it switched over to allocated memory when the 
array got beyond a certain size. There's something I still need 
to figure out.
Jan 24 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/6/14, 6:40 PM, Era Scarecrow wrote:

 When I re-wrote the BitArray I personally think it is an improvement in
 many ways, and being a re-write you can't just do bug #xxxx and then bug
 #xxxx and then bug... etc etc.


Welcome back!

Regarding BitArray, when I implemented HeapBlock in 
https://github.com/andralex/phobos/blob/allocator/std/allocator.d I 
needed a way to "see" a ulong[] as an array of bits, with primitives 
such as "find a run of at least n zeros" and "set n bit to zero or one" 
etc. A "bit range" abstraction built on top of a ulong[] would have been 
quite useful. It would have the classic range primitives but also 
functions specific to runs of bits, which are unique to it.

I wanted to sit down and extract the abstraction at some point, but 
didn't get to it yet. Sounds like a great thing to include in std.bitmanip.


Andrei
Aug 07 2014
parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Thursday, 7 August 2014 at 10:08:30 UTC, Marc Schütz wrote:

 I haven't looked at your rewrite, but from what I've seen the 
 current implementation is indeed very awkward and full of bugs.
 ... This seems to imply that not even the semantics are 
 completely clear. I guess a complete rewrite would be 
 appropriate in such a situation.


On Thursday, 7 August 2014 at 14:24:13 UTC, Andrei Alexandrescu 
wrote:

 Welcome back!


  Thanks i guess :)


 I wanted to sit down and extract the abstraction at some point, 
 but didn't get to it yet. Sounds like a great thing to include 
 in std.bitmanip.



  So the gist of it, is there's still interest in the rewrite. 
Interesting...

  As for being able to find x number of bits that are 0 or 1 in a 
row, that both sounds easy and hard at the same time (easy if you 
don't mind it being slow). In my rewrite there was a bulk 
template I created that was intended to do huge speedups with 
binary data (when the data was naturally aligned correctly and 
large enough to use the binary operators on normal types). Some 
of the unittests and example code also used them in a read-only 
fashion that could be specialized for finding a certain type of 
pattern...


  As I'm trying to recall, I think the largest bug in my 
implementation was the confusion of when it should be reference 
vs value type, since when it was small enough it didn't need to 
allocate any data (64/128 bits or less) but larger than that it 
had to allocate data. Naturally this causes problems...

  Of course I also implemented it to be able to pass around a 
slice of a BitArray which was always reference...

  I guess the things to do is make sure the current bug fixes are 
included in my implementation (including BitFields, I had it 
programmed to accept default values), get it to merge with 
phobos, afterwards continue forward and try to make improvements 
and further bug fixes...
Aug 07 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/7/14, 12:40 PM, Era Scarecrow wrote:

   As for being able to find x number of bits that are 0 or 1 in a row,
 that both sounds easy and hard at the same time (easy if you don't mind
 it being slow). In my rewrite there was a bulk template I created that
 was intended to do huge speedups with binary data (when the data was
 naturally aligned correctly and large enough to use the binary operators
 on normal types). Some of the unittests and example code also used them
 in a read-only fashion that could be specialized for finding a certain
 type of pattern...


A thought: if whatever work on bit arrays you do isn't fast, it's 
probably not worth doing; people who opt for that kind of packing are 
most often performance motivated.

Alignment is often not an issue - you handle the setup/teardown 
misalignments separately and to the bulk 64 bits at a time.

Instead of a full-blown abstraction you may want to instead opt for 
defining some simple primitives using ulong[] that are accessible to 
people having data embedded in odd places. The bit operations in 
std.allocator would be good and practical.


Andrei
Aug 08 2014
parent reply "Mehrdad" <wfunction hotmail.com> writes:
On Friday, 8 August 2014 at 22:43:38 UTC, Andrei Alexandrescu 
wrote:

 On 8/7/14, 12:40 PM, Era Scarecrow wrote:

  As for being able to find x number of bits that are 0 or 1 in 
 a row, that both sounds easy and hard at the same time (easy 
 if you don't mind it being slow). In my rewrite there was a 
 bulk template I created that was intended to do huge speedups 
 with binary data (when the data was naturally aligned 
 correctly and large enough to use the binary operators on 
 normal types). Some of the unittests and example code also 
 used them in a read-only fashion that could be specialized for 
 finding a certain type of pattern...


 A thought: if whatever work on bit arrays you do isn't fast, 
 it's probably not worth doing; people who opt for that kind of 
 packing are most often performance motivated.

 Alignment is often not an issue - you handle the setup/teardown 
 misalignments separately and to the bulk 64 bits at a time.


What kind of performance are you looking for? I have some very 
basic bit-manipulation code written in C++ that operates on whole 
words at a time, not sure if it's what you need but if it is then 
it should be trivial to port this to D:

template<class Diff, class It>
It setbits(It i, Diff j, Diff n)
{
	typedef typename std::iterator_traits<It>::value_type T;
	T ones = static_cast<T>(~T());
	Diff const bits = static_cast<Diff>(sizeof(*i) * CHAR_BIT);
	i += j / bits;
	j %= bits;
	if (j)
	{
		*i |= (ones << j) &
			(ones >> max(0, static_cast<int>(bits - j - n)));
		++i;
		n -= min(n, bits - j);
	}
	fill_ni, n / bits, ones);
	i += static_cast<ptrdiff_t>(n / bits);
	n %= bits;
	if (n)
	{
		*i |= ones >> max(0, static_cast<int>(bits - n));
		++i;
	}
	return i;
}
Aug 09 2014
next sibling parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Saturday, 9 August 2014 at 09:34:38 UTC, Mehrdad wrote:

 On Friday, 8 August 2014 at 22:43:38 UTC, Andrei Alexandrescu 
 wrote:

 Alignment is often not an issue - you handle the 
 setup/teardown misalignments separately and to the bulk 64 
 bits at a time.


 What kind of performance are you looking for? I have some very 
 basic bit-manipulation code written in C++ that operates on 
 whole words at a time, not sure if it's what you need but if it 
 is then it should be trivial to port this to D:


  Glancing at it, it looks like it would probably do what i'd 
want. Although i'd have to study it closer while converting it. 
But these would probably only take effect when you have at least 
a certain number of bits or more to make it worth it. Probably as 
a guess, 8 bits or more.

  Most of the speedup would probably be seen while doing matrix 
operations. Still if they don't align perfectly the penalty based 
on that C++ algo is probably 8x-64x slower (mostly do to 
modulus/division, if it's a power of 2 than it drops to 4x-8x 
slower) but it would be considerably faster than doing each bit 
individually.
Aug 09 2014
parent reply "Mehrdad" <wfunction hotmail.com> writes:
On Saturday, 9 August 2014 at 09:44:17 UTC, Era Scarecrow wrote:

 On Saturday, 9 August 2014 at 09:34:38 UTC, Mehrdad wrote:

 On Friday, 8 August 2014 at 22:43:38 UTC, Andrei Alexandrescu 
 wrote:

 Alignment is often not an issue - you handle the 
 setup/teardown misalignments separately and to the bulk 64 
 bits at a time.


 What kind of performance are you looking for? I have some very 
 basic bit-manipulation code written in C++ that operates on 
 whole words at a time, not sure if it's what you need but if 
 it is then it should be trivial to port this to D:


  Glancing at it, it looks like it would probably do what i'd 
 want. Although i'd have to study it closer while converting it.


Oh in that case, I'll just send you the code I have right now. 
I'd double-check it to see if it works though -- it may have 
bugs, especially in the SIMD.

Here: http://pastebin.com/UG9wb0s5
Aug 09 2014
parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Saturday, 9 August 2014 at 09:54:09 UTC, Mehrdad wrote:

 Oh in that case, I'll just send you the code I have right now. 
 I'd double-check it to see if it works though -- it may have 
 bugs, especially in the SIMD.


  I probably won't be touching this for a while... makes me wonder 
if the 30 day expiring is a bad idea for if/when i return to this.
Aug 09 2014
parent "Mehrdad" <wfunction hotmail.com> writes:
On Saturday, 9 August 2014 at 10:02:24 UTC, Era Scarecrow wrote:

 On Saturday, 9 August 2014 at 09:54:09 UTC, Mehrdad wrote:

 Oh in that case, I'll just send you the code I have right now. 
 I'd double-check it to see if it works though -- it may have 
 bugs, especially in the SIMD.


  I probably won't be touching this for a while... makes me 
 wonder if the 30 day expiring is a bad idea for if/when i 
 return to this.


Oh, feel free to make a copy for yourself and use it whenever you 
might need it. :) I might change the expiration date later, 
though currently I don't want to leave it on there forever.
Aug 09 2014
prev sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Saturday, 9 August 2014 at 09:34:38 UTC, Mehrdad wrote:

 (snip)


Sorry, just to clarify, that was just one example of what I had, 
not the only one. I also have another (monstrous) function

long long GetRunLength(
	void const *bitmap, unsigned long long bitmap_bits,
	long long start, long long end,
	bool reverse, /*out*/ bool *bit);

but before I'd post it I'd need to make sure it's bug-free (I 
even SIMD'd this one at one point, but it's been a long time 
since I last used it and I don't remember if it's buggy or not, 
so I'd have to check first). If these are the types of bit 
operations you're looking for let me know if you're interested 
and I can try to fix any bugs in my implementation and post them 
here so you can port them to D.
Aug 09 2014
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
05-Aug-2014 13:30, Era Scarecrow пишет:

   It's been a little while since I've been back. Got burned out and
 stuff. Currently watching all the Dconf 2014 videos (although I never
 finished the 2013 ones).

   So, I don't suppose there's a short quick & dirty summary of what's
 happened in the last 18 months?


E-hm, not easy. Lots of good stuff...
 nogc, tons of bug fixes, better packages. And we still await on 
allocators ;)


   Also somewhat curious if anyone took my BitArray updates and got them
 implemented in phobos or not. Quite annoying I still haven't gotten a
 full understanding of GitHub yet, and at this rate probably won't.


Glad to see you're back!

FYI
https://github.com/D-Programming-Language/phobos/pull/2248
https://github.com/D-Programming-Language/phobos/pull/2249

These were mostly bugfixes not trying to fix any design flaws.

-- 
Dmitry Olshansky
Aug 08 2014
parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Friday, 8 August 2014 at 21:35:59 UTC, Dmitry Olshansky wrote:

 E-hm, not easy. Lots of good stuff...
  nogc, tons of bug fixes, better packages. And we still await 
 on allocators ;)


  Hmm good allocation fixes would be nice... I know i'd like to 
see as much of the phobos library adjusted not to use the GC as 
possible, or custom allocators for bare metal programming and 
stuff where the OS can't help you.

 Era Scarecrow wrote:

  Also somewhat curious if anyone took my BitArray updates and 
 got them implemented in phobos or not. Quite annoying I still 
 haven't gotten a
 full understanding of GitHub yet, and at this rate probably 
 won't.


 Glad to see you're back!


  Thanks..


 FYI
 https://github.com/D-Programming-Language/phobos/pull/2248
 https://github.com/D-Programming-Language/phobos/pull/2249

 These were mostly bugfixes not trying to fix any design flaws.


  Yeah doesn't look like any of my code. Unfortunately due to the 
huge re-write i did most of those fixes go in the garbage (my 
rewrite probably covered most of them anyways).

  I think i'll concentrate on bitfields first, then when that's 
good i'll try and get BitArray in. As i recall my BitArray 
requires default assignment on bitfields, so i had to add that 
before my BitArray worked right. Depending on how much the CTFE 
has improved, i may be able to cut out extra bit/type enum i 
added which gives useful static information on the side i used 
for my BitArray. Far more likely the log2 can be 
reduced/removed...
Aug 08 2014
parent reply "safety0ff" <safety0ff.dev gmail.com> writes:
On Friday, 8 August 2014 at 22:28:56 UTC, Era Scarecrow wrote:

 On Friday, 8 August 2014 at 21:35:59 UTC, Dmitry Olshansky 
 wrote:

 FYI
 https://github.com/D-Programming-Language/phobos/pull/2248
 https://github.com/D-Programming-Language/phobos/pull/2249

 These were mostly bugfixes not trying to fix any design flaws.


  Yeah doesn't look like any of my code. Unfortunately due to 
 the huge re-write i did most of those fixes go in the garbage 
 (my rewrite probably covered most of them anyways).


Yea, one of those was my code and the other was somebody else's 
PR that I revived since they weren't responding.

I was moving forward with the philosophy that we should make the 
existing implementation as correct as possible and leave new 
features to new designs.

I think it will be difficult to make a "one size fits all" 
BitArray that satisfies everybody's wishes.
E.g.:
Bit level slice operations versus performance.
Value semantics versus D slice semantics.
Having compatibility with other parts of phobos versus having a 
maximum of 2^35-1 bits on a 32 bit system.

This is not as bad making a one size fits all fixed point 
integer, but it's not pleasant either.
Aug 08 2014
parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Friday, 8 August 2014 at 22:43:38 UTC, Andrei Alexandrescu 
wrote:

 A thought: if whatever work on bit arrays you do isn't fast, 
 it's probably not worth doing; people who opt for that kind of 
 packing are most often performance motivated.

 Alignment is often not an issue - you handle the setup/teardown 
 misalignments separately and to the bulk 64 bits at a time.

 Instead of a full-blown abstraction you may want to instead opt 
 for defining some simple primitives using ulong[] that are 
 accessible to people having data embedded in odd places. The 
 bit operations in std.allocator would be good and practical.


  And now i'm confused. There's enhancement requests for making it 
so you can prepend or append to the bitarray really quick, which 
ensures it's not aligned right unless you rebuild the entire 
thing. Quite often you'll be working with only a few bits at a 
time vs large bulk operations, and if they don't align right 
somehow, then either bit-shifting has to be present and sort of a 
leap-frog approach, or they align at the same place so you can 
treat it more like an array...

  Of course you can always expand it so 1 bit becomes 1 byte, and 
then afterwards repack it into something smaller for storage 
purposes...




On Friday, 8 August 2014 at 22:46:44 UTC, safety0ff wrote:

 Yea, one of those was my code and the other was somebody else's 
 PR that I revived since they weren't responding.

 I was moving forward with the philosophy that we should make 
 the existing implementation as correct as possible and leave 
 new features to new designs.


  Not surprising. We want to move forward/advance as much as 
possible. Kinda my internal motto when playing chess :P



 I think it will be difficult to make a "one size fits all" 
 BitArray that satisfies everybody's wishes.
 E.g.:
 Bit level slice operations versus performance. Value semantics 
 versus D slice semantics. Having compatibility with other parts 
 of phobos versus having a  maximum of 2^35-1 bits on a 32 bit 
 system.

 This is not as bad making a one size fits all fixed point 
 integer, but it's not pleasant either.


  I recall trying to satisfy everyone's requests. The only thing 
that really came up was it was sometimes a reference (>64 bits) 
and sometimes a value type (<=64 bits) and that was confusing 
only because it wasn't consistent. That and someone wants to be 
able to have a static size that probably doesn't rely on 
allocation at all. Possible but not sure how to make it...

  As someone mentioned before, i think it comes down that we 
really aren't sure what it is really for, it hasn't been fully 
defined. Is it a storage container? An array? Is it mostly so we 
can foreach over the elements for something? Is it for bit 
packing/unpacking for compression? (Which is how i see it 
mostly). Encryption?


  Suddenly the idea of having it attached to a portion of memory, 
then that's converted to a byte array that can be treated as a 
normal array works fine, then just having it update the original 
array on a sync call or something comes to mind... That might 
work... But who owns the data? If the GC owns it, then you create 
a portion of memory and assign it, or you tell the BitArray this 
memory goes to this slice of the data, and 'do this' with it.

  Hmmm new ideas... but doesn't seem easy to do...
Aug 08 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Aug 08, 2014 at 11:30:07PM +0000, Era Scarecrow via Digitalmars-d wrote:
[...]

  I recall trying to satisfy everyone's requests. The only thing that
 really came up was it was sometimes a reference (>64 bits) and
 sometimes a value type (<=64 bits) and that was confusing only because
 it wasn't consistent.  That and someone wants to be able to have a
 static size that probably doesn't rely on allocation at all. Possible
 but not sure how to make it...
 
  As someone mentioned before, i think it comes down that we really
 aren't sure what it is really for, it hasn't been fully defined. Is it
 a storage container? An array? Is it mostly so we can foreach over the
 elements for something? Is it for bit packing/unpacking for
 compression? (Which is how i see it mostly). Encryption?


[...]

I think this indicates that perhaps there is not a single BitArray type
that will satisfy everybody, but perhaps 2-3 related types that share
some algorithms.  At least the following types come to mind (these are
all temporary names so please don't bust out the rainbow bikeshed just
yet):

- SmallBitArray: fits inside a ulong, and has value semantics. Behaves
  like a ulong: minimal cost for passing it around, making multiple
  copies of it, etc.. Basically for juggling bits in a ulong.

- StaticBitArray: a value type that fits inside n ulong's. Appends only
  work up to the max capacity of n ulong's. This one is mainly for
  people who need to manipulate bitmasks, perform bit operations en
  masse, etc.. Backed by an embedded static array, so it has value
  semantics, and you can pass it around, copy it, etc., without worrying
  about aliasing.

- DynamicBitArray: a reference type backed by the GC. This one is for
  people who are after something that behaves like D dynamic arrays;
  slicing is supported (by means of extra info about how many bits at
  the beginning/end of the array is actually part of an array, so you
  can slice it at arbitrary bit positions not aligned with word
  boundaries), as is arbitrary appending, etc.. The emphasis for this
  one is memory efficiency (e.g. for people who need to store massive
  numbers of bools without wasting an entire byte per bool), at the cost
  of potentially slower copying / overhead of maintaining start/end bit
  indices, etc..

Obviously, these types are closely related, so they will probably share
quite a good number of common algorithms. So potentially they can be
implemented as a core of common functions that can work on all 3
BitArray types, with a few customized methods for each type.


T

-- 
I've been around long enough to have seen an endless parade of magic new
techniques du jour, most of which purport to remove the necessity of
thought about your programming problem.  In the end they wind up
contributing one or two pieces to the collective wisdom, and fade away
in the rearview mirror. -- Walter Bright
Aug 08 2014
parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Friday, 8 August 2014 at 23:57:17 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 - SmallBitArray: fits inside a ulong, and has value semantics. 
 Behaves
   like a ulong: minimal cost for passing it around, making 
 multiple
   copies of it, etc.. Basically for juggling bits in a ulong.


  Easy enough to make a struct that doesn't care, would be able to 
foreach over it and do direct access as long as you don't have to 
worry about where it starts/ends (the length). Just forcibly 
convert a type to a small bitarray and it works... slicing and 
other advanced features would be missing...


 - StaticBitArray: a value type that fits inside n ulong's. 
 Appends only
   work up to the max capacity of n ulong's. This one is mainly 
 for
   people who need to manipulate bitmasks, perform bit 
 operations en
   masse, etc.. Backed by an embedded static array, so it has 
 value
   semantics, and you can pass it around, copy it, etc., without 
 worrying
   about aliasing.


  I remember starting something like this, it started to become a 
template hell... And code bloat... I ended up deciding it was 
simpler to have one type that allowed slicing as part of it's 
design.


 - DynamicBitArray: a reference type backed by the GC. This one 
 is for
   people who are after something that behaves like D dynamic 
 arrays;
   slicing is supported (by means of extra info about how many 
 bits at
   the beginning/end of the array is actually part of an array, 
 so you
   can slice it at arbitrary bit positions not aligned with word
   boundaries), as is arbitrary appending, etc.. The emphasis 
 for this
   one is memory efficiency (e.g. for people who need to store 
 massive
   numbers of bools without wasting an entire byte per bool), at 
 the cost
   of potentially slower copying / overhead of maintaining 
 start/end bit
   indices, etc..

 Obviously, these types are closely related, so they will 
 probably share quite a good number of common algorithms. So 
 potentially they can be implemented as a core of common 
 functions that can work on all 3 BitArray types, with a few 
 customized methods for each type.


  So UFCS rather than be template functions inside of template 
structs... that would simplify separating the storage vs shared 
algorithm/design. Yeah i think i can do that... And hopefully 
satisfy all three basic types.

  So to decide how it identifies these, what makes up a BitArray. 
For simplicity probably readable fields, being bitslicestart and 
bitslicelength. Finally a flag that says if it's dynamic or not. 
bitslice_isdynamic... Probably those three things (Unless someone 
else has a better idea).

  After that what operators do they all support... Definitely 
basic opIndex get/set. And being able to foreach over them, or a 
wrapper to make it an input/output range.

  If anyone wants to help with the exact details for what the 
requirements of all the BitArray types are, and a good list of 
what kind of unittests we should use to verify it's usage and 
correctness then i'm all ears.
Aug 08 2014
parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
  But naturally before i can do anything i need to re-familiarize 
myself with Git and GitHub... and immediately remember why i 
pretty much rage-quit 18 months ago...

  Is there anyone who i can refer to via IM/PM or some quick 
method when i start using this stuff to get it sorted out (rather 
than these forums)?

  Although it's pretty likely i'll either start a brand new 
fork/repository of phobos or destroy my current saved changes 
(commits, fixes, everything) as they will be a pain to figure out 
since i never could get it to work last time... even after i 
rebased it last time...
Aug 08 2014
parent reply "eles" <eles215 gzk.dot> writes:
On Saturday, 9 August 2014 at 02:53:34 UTC, Era Scarecrow wrote:

  But naturally before i can do anything i need to 
 re-familiarize myself with Git and GitHub... and immediately 
 remember why i pretty much rage-quit 18 months ago...

  Is there anyone who i can refer to via IM/PM or some quick 
 method when i start using this stuff to get it sorted out 
 (rather than these forums)?

  Although it's pretty likely i'll either start a brand new 
 fork/repository of phobos or destroy my current saved changes 
 (commits, fixes, everything) as they will be a pain to figure 
 out since i never could get it to work last time... even after 
 i rebased it last time...


I am not a heavy github user and I use git basically only for 
myself and collaboration outside my workplace (there, we use the 
"professional" ClearCase...), but for git, I could recommend you 
these resources:

https://rogerdudler.github.io/git-guide/
https://try.github.io/levels/1/challenges/1

If you have at least 30 minutes to read about git, then this:

http://git-scm.com/book/en/Getting-Started-Git-Basics

About branching (but a bit complex) for large projects, there is 
this much acclaimed resource:

http://nvie.com/posts/a-successful-git-branching-model/

But that may wait till later.

One word of caution about git: once you learn it, you'll pretty 
much dump anything else and you'll become addicted to it to the 
point that you'd dream to git-ify all your projects and stuff. 
Personal experience speaking here. I went through CVS (just a 
little), then quite heavily SVN (wow, what a wonder, I thought!), 
had an attempt at bazaar...

All these were missing one crucial feature at that time and, one 
day, I stumbled upon git (I was avoiding git on purpose, just to 
not join the acclaiming chorus) and had the "a-ha" moment. I have 
found it. The feature was:

"so I can change the version of the code that I work on without 
having to change my folder, paths, put in plae symbolic links for 
my IDE and stuff? It takes just a 'git checkout the_other_branch' 
and it is exactly at the point where I did left it? and I am able 
to switch branches for just 2 minutes, every 3 minutes, just to 
compare results and so on? WOW!"

I forgot how the world was looking before the advent of git. I 
forgot on purpose.
Aug 09 2014
next sibling parent "eles" <eles215 gzk.dot> writes:
On Saturday, 9 August 2014 at 07:12:45 UTC, eles wrote:

 On Saturday, 9 August 2014 at 02:53:34 UTC, Era Scarecrow wrote:



 "so I can change the version of the code that I work on without 
 having to change my folder, paths, put in plae symbolic links 
 for my IDE and stuff? It takes just a 'git checkout 
 the_other_branch' and it is exactly at the point where I did 
 left it? and I am able to switch branches for just 2 minutes, 
 every 3 minutes, just to compare results and so on? WOW!" (*)


(*) yes, there is a cobazaar or some plugin like that for bazaar, 
but basically they try to replicate what is for me the truest 
feature of git (yes, I am aware of its power, but this one is 
*the* feature).
Aug 09 2014
prev sibling parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Saturday, 9 August 2014 at 07:12:45 UTC, eles wrote:

 I am not a heavy github user and I use git basically only for 
 myself and collaboration outside my workplace (there, we use 
 the "professional" ClearCase...), but for git, I could 
 recommend you these resources:


  I'm watching a few tutorials on GitHub and i have a book on Git. 
Still doesn't explain about the problems i had. I'm sure Git will 
sink into my brain very soon at the rate i'm consuming the 
material.

  Clearcase... i remember that... used it once...


 One word of caution about git: once you learn it, you'll pretty 
 much dump anything else and you'll become addicted to it to the 
 point that you'd dream to git-ify all your projects and stuff.


  Yeah thought about that myself. Most of my personal project are 
'duplicate the entire directory and rename the new one to reflect 
the version' and use diff and patch... it works... although it's 
not pretty and uses a lot of space... thankfully 7zip does a good 
job compacting all of them together.


 All these were missing one crucial feature at that time and, 
 one day, I stumbled upon git (I was avoiding git on purpose, 
 just to not join the acclaiming chorus) and had the "a-ha" 
 moment. I have found it. The feature was:

 "so I can change the version of the code that I work on without 
 having to change my folder, paths, put in plae symbolic links 
 for my IDE and stuff? It takes just a 'git checkout 
 the_other_branch' and it is exactly at the point where I did 
 left it? and I am able to switch branches for just 2 minutes, 
 every 3 minutes, just to compare results and so on? WOW!"

 I forgot how the world was looking before the advent of git. I 
 forgot on purpose.


  Things will be easier when it clicks for me i'm sure. Re-reading 
part of the book, a portion of it, the designs behind Git makes 
sense, but there's no 'ah ha!' moment, not yet. I think mostly 
it's the terminology i'm confused on since i'm often a blank 
state (self taught).

  I'm seriously tempted to set up a VM or separate machine with a 
linux distro and programming environment. My main computer 
doesn't feel like a programming station anymore.
Aug 09 2014
next sibling parent reply "eles" <eles215 gzk.dot> writes:
On Saturday, 9 August 2014 at 07:21:28 UTC, Era Scarecrow wrote:

 On Saturday, 9 August 2014 at 07:12:45 UTC, eles wrote:



  Things will be easier when it clicks for me i'm sure. 
 Re-reading part of the book, a portion of it, the designs 
 behind Git makes sense, but there's no 'ah ha!' moment, not 
 yet. I think mostly it's the terminology i'm confused on since 
 i'm often a blank state (self taught).



That's because many git tutorial start with praising its 
distributed repository model and bragging about why that feature 
is cool (which is very true, but it is confusing for beginners). 
Ignore that part for the time being, it will click in later.

For now, just consider that your current folder has some source 
code in it and that executing

$ git init

it transforms it in a "git-managed project". That's all

$ git add -A
$ git commit -m "added files"

will add all files to that project (why two commands, it will 
become clear later)

$ git branch test_idea01
$ git checkout test_idea01

will put you in a new git-subproject (also known as "branch") 
called "test_idea01" where you wan to test your new idea

Subsequent

$ git add -A
$ git commit -m "these modifications are added to the test_idea01 
subproject"

will add modifications to the test_idea01 subproject.

once you finish with test_idea01, you go back to the original 
state with:

$ git checkout master

(the initial sub-project in git it is just called "master")

and start, maybe, a new idea with:

$ git branch test_idea02
$ git checkout test_idea02

then etc., then back to the original to master

This will allow you to switch between several versionas and to 
test them in the same folder, with all the rest on your computer 
being equal.

Then, you will discover merging (you could bring the 
modifications from test_idea01 and test_idea02 back t master) and 
so on, but to start, this is it.

It will succesfully replace your archive-and-rename strategy.

Not to even tell that

$ git diff master test_idea01

will give you a nice view of the differences...
Aug 09 2014
next sibling parent "eles" <eles215 gzk.dot> writes:
On Saturday, 9 August 2014 at 07:48:28 UTC, eles wrote:

 On Saturday, 9 August 2014 at 07:21:28 UTC, Era Scarecrow wrote:

 On Saturday, 9 August 2014 at 07:12:45 UTC, eles wrote:





 will give you a nice view of the differences...


Oh, yes, and your subprojects "master", "test_idea01" and 
"test_idea02" won't be lost.

You could bring any of them back to life in the current folder by 
executing:

$ git checkout <subproject name here>

and start from where you left it.

Just note that no longer need the command

$ git branch <subproject name here>

if the subproject (well, maye i should be called "version" or 
something like that, but the proper name is "branch") already 
exists.
Aug 09 2014
prev sibling parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Saturday, 9 August 2014 at 07:48:28 UTC, eles wrote:

 That's because many git tutorial start with praising its 
 distributed repository model and bragging about why that 
 feature is cool (which is very true, but it is confusing for 
 beginners). Ignore that part for the time being, it will click 
 in later.


  Yeah, when it starts getting into non-changing data the SHA1 and 
160bit hashing makes perfect sense, even for identifying 
renamed/moved files based on how much matching code they have.

  I understand every line you entered because i just ended up 
watching 2-3 hours of Github help videos regarding the various 
features, otherwise i'd be completely lost, even with a book to 
guide me. But perhaps what we lack is sorta this nice 
introduction into D using GitHub. My first experience is still 
very jarring.



  I'd like to see offered a standard environment i can download 
and run as a LiceCD on a VM that contains all the tools and 
everything ready-to-use. Recommended or ready-to-use doesn't mean 
it's the only way to use it :P

  Course i'd also like to see some videos. Andrei and Walter 
talking about phobos and the approaches to design and why they 
took various directions. Talking about how to use phobos, and the 
various tools or development steps to contribute. Sure you can 
read from the documentation but sometimes a good explanation of 
why things are named or how they are named and their use would 
make it a lot easier as a type of tutorial of the library and 
language.

  I believe Channel-9's videos on the C++ STL taught me more of 
how to use it than i could get from a book due to the huge 
jarring and confusing differences between C and C++'s approach to 
it. I'd hope Facebook or Google could provide a public green-room 
where some of these informational videos could be captured.

  Of course i'd also like to see Richard Stallman give his 
blessing and provide some of his skills and support as well. But 
most of this is wishful thinking...
Aug 09 2014
parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Saturday, 9 August 2014 at 09:15:10 UTC, Era Scarecrow wrote:

 and run as a LiceCD on a VM that contains all the tools and


   LiveCD LIVE!!!.... Wish i had an edit button for quick edits...
Aug 09 2014
parent David Gileadi <gileadis NSPMgmail.com> writes:
On 8/9/14, 2:16 AM, Era Scarecrow wrote:

 On Saturday, 9 August 2014 at 09:15:10 UTC, Era Scarecrow wrote:

 and run as a LiceCD on a VM that contains all the tools and


    LiveCD LIVE!!!.... Wish i had an edit button for quick edits...


Yeah, NNTP is lousy for editing.
Aug 09 2014
prev sibling parent "eles" <eles eles.com> writes:
On Saturday, 9 August 2014 at 07:21:28 UTC, Era Scarecrow wrote:

 On Saturday, 9 August 2014 at 07:12:45 UTC, eles wrote:



  Things will be easier when it clicks for me i'm sure. 
 Re-reading part of the book, a portion of it, the designs 
 behind Git makes sense, but there's no 'ah ha!' moment, not yet.


How is this going?
Aug 27 2014