## digitalmars.D - colour lib needs reviewers

• Manu via Digitalmars-d (14/14) Sep 11 2016 I think I'm about as happy with my colour lib as I'm going to be.
• Guillaume Chatelet (7/24) Sep 12 2016 Looks pretty good to me.
• Manu via Digitalmars-d (14/42) Sep 12 2016 I thought about it, but it added (even more) complexity, and I'm not
• Guillaume Chatelet (19/35) Sep 12 2016 Yep I totally understand the rationale - I've been there too :)
• Ilya Yaroshenko (5/8) Sep 12 2016 Looks great!
• Basile B. (10/19) Sep 12 2016 1. Maybe that the each package ddoc comment should mention that
• Marco Leise (32/51) Sep 12 2016 Nice work. What's u7e3? 3*10-bit-floats + 2-bit padding?
• Guillaume Chatelet (15/32) Sep 12 2016 And also, do you handle limited (16-235) to full dynamic range
• Marco Leise (13/44) Sep 12 2016 Practically all video content is encoded with the limited
• Manu via Digitalmars-d (13/37) Sep 12 2016 Actually, I spent quite some time stressing about dynamic range.
• Guillaume Chatelet (3/20) Sep 12 2016 Yes I agree, as you and Marco said: I think it's best to leave it
• Guillaume Piolat (8/12) Sep 12 2016 As someone who had to write lots of code involving such video
• Andrei Alexandrescu (12/14) Sep 12 2016 Upon a quick look it seems to have the right amount of "notes", as
• Manu via Digitalmars-d (35/50) Sep 12 2016 Thanks. It has taken a lot more work than I actually expected to make
• Adam D. Ruppe (13/15) Sep 12 2016 Indeed. I know there's a window library that was waiting on this
• Edwin van Leeuwen (4/7) Sep 12 2016 Just for your information. ggplotd currently uses an old version
• Guillaume Chatelet (17/71) Sep 12 2016 +1
• Walter Bright (4/35) Sep 12 2016 Thanks for the rationale. I'm glad to see you're planning follow-on libr...
• jmh530 (3/7) Sep 12 2016 You might try
• Guillaume Piolat (9/12) Sep 12 2016 My thoughts:
• Edwin van Leeuwen (3/6) Sep 12 2016 What happens when the string is invalid? Does it throw an error?
• Guillaume Piolat (3/9) Sep 12 2016 It throws an Exception, like it should.
• John Colvin (4/10) Sep 12 2016 It should be integrated with std.conv.parse and std.conv.to and
• Walter Bright (3/11) Sep 12 2016 I'm not so sure. Follow the conventions, ok, but integrating them in - t...
• Manu via Digitalmars-d (6/25) Sep 12 2016 std.conv.to seems to 'just work' for color type conversion.
• Andrei Alexandrescu (3/7) Sep 13 2016 That was the purpose of toImpl, which I recall has been recently
• ketmar (3/8) Sep 13 2016 yes. just define a ctor that accepts string, and you're there. it
• Random D user (47/50) Sep 12 2016 Hi. I'm just a random forum lurker, but here's my feedback.
• Walter Bright (25/27) Sep 12 2016 I have zero expertise in color programming, and have no idea what a colo...
• Andrei Alexandrescu (3/5) Sep 12 2016 Great question and well asked. It was mine as well but I could not put
• Manu via Digitalmars-d (9/37) Sep 12 2016 Okay. I'll work on this feedback.
• Walter Bright (7/16) Sep 12 2016 Ok, but I have no idea how. Perhaps include something as an example? Pho...
• Walter Bright (4/7) Sep 12 2016 BTW, linking to authoritative reference material has the effect of reass...
• Andrea Fontana (7/9) Sep 13 2016 I think it's easy to do some image processing using netbpm [1] as
• Guillaume Piolat (18/20) Sep 13 2016 Having correct conversions.
• Meta (6/23) Sep 14 2016 Looks like a nice and simple module. My only complaint is that
• Manu via Digitalmars-d (8/34) Sep 14 2016 Cheers.
• Meta (2/6) Sep 14 2016 I can't say I'm a fan either.
• Marco Leise (7/15) Sep 14 2016 You can replace $(D =E2=80=A6) with =E2=80=A6 nowadays. It made me a b... • Andrei Alexandrescu (2/9) Sep 14 2016 Why? -- Andrei • Jacob Carlborg (54/55) Sep 14 2016 I've recently worked on a theme for Ddoc that will be used in TextMate. • Manu via Digitalmars-d (6/60) Sep 15 2016 Do we mind moving this discussion to the other topic I started • Andrei Alexandrescu (13/58) Sep 15 2016 Yah, this is terrible (it happens with http: all the time). We need to • Jacob Carlborg (20/26) Sep 15 2016 It's useful for many of the sections to allow the section name on the • Andrei Alexandrescu (8/30) Sep 15 2016 That's fine because "Returns" is special-cased. Random words at the • Jacob Carlborg (10/15) Sep 16 2016 Ah, so you mean all non-standard non-special sections? Basically any • Meta (12/23) Sep 15 2016 The syntax is clunky and it's poorly documented. I've been • Andrei Alexandrescu (40/42) Sep 14 2016 I was using ddoc today for the Greeks in DIP1000 and was musing about • Andrei Alexandrescu (12/17) Sep 14 2016 I meant to add here: we can improve this in a backward-compatible way by... • bachmeier (11/13) Sep 14 2016 One thing that I didn't like when I used DDOC was that I couldn't • Andrei Alexandrescu (3/17) Sep 14 2016 Do you mean like a macro$(VERBATIM ... text with balanced parens ...)?
• bachmeier (4/6) Sep 15 2016 Yes. Unfortunately I cannot find any mention of it on this page:
• Andrei Alexandrescu (3/8) Sep 15 2016 It doesn't exist :o). I agree it's needed, I've hit the matter a few
• Manu via Digitalmars-d (32/73) Sep 14 2016 I haven't had enough experience for meaningful input. I have
• Andrei Alexandrescu (22/38) Sep 14 2016 I don't see those workarounds to ddoc shortcomings. m4 and all macro
• Manu via Digitalmars-d (72/109) Sep 15 2016 That's absolutely what I'm saying, but not only that, given that I'm
• Patrick Schluter (13/65) Sep 15 2016 doxygen is mostly javadoc compatible and that one is the standard.
• Bastiaan Veelo (7/11) Sep 15 2016 Not only that, by using tag files [1] Doxygen even generates
• Walter Bright (2/4) Sep 14 2016 The $(MACRO a1, a2, a3) syntax comes from makefiles. • Walter Bright (20/24) Sep 14 2016 The original idea was to be able to write documentation without needing ... • Manu via Digitalmars-d (3/32) Sep 15 2016 +1 • Paolo Invernizzi (5/9) Sep 15 2016 I **really** love the pragmatism of this man! • Marco Leise (5/7) Sep 15 2016 +1 • Nemanja Boric (6/10) Sep 15 2016 https://github.com/kiith-sa/harbored-mod provides support for • Walter Bright (3/14) Sep 15 2016 It looks like you've done some great work with this! And Boost licensed,... • Jacob Carlborg (11/30) Sep 15 2016 Yeah, if you already have defined all the standard ddoc macros because • Andrei Alexandrescu (7/9) Sep 15 2016 Glad to hear that. It becomes a matter of improving the default-provided... • Jacob Carlborg (5/10) Sep 15 2016 The oldest bugzilla issue I referred to is more than four years old and • Walter Bright (3/5) Sep 15 2016 The default styling was set up to produce legible output while not requi... • Jacob Carlborg (11/14) Sep 16 2016 I don't see how that's relevant. Currently one needs to have knowledge • Andrei Alexandrescu (6/8) Sep 16 2016 Jacob, would you want to try your hand at tweaking the defaults? They • Jacob Carlborg (13/18) Sep 16 2016 My design skills are quite limited. The theme I'm using for TextMate is • Andrei Alexandrescu (9/26) Sep 16 2016 I'd say we start without a separate CSS file. When the need for it • Andrei Alexandrescu (20/25) Sep 15 2016 In the words of Seinfeld, "And that doesn't help either!" • Walter Bright (2/2) Sep 15 2016 Started a new thread on this, "Ddoc macro syntax". Please direct further... • Walter Bright (2/3) Sep 14 2016 s/Args/Params/ • Johan Engelen (11/13) Sep 16 2016 I don't know enough about colors and what people do with it to Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. I added packed-RGB support, including weird micro-float and shared-exponent formats. They're important for interacting with any real-time rendering libraries. There is only one texture format I'm aware of that isn't supported, and that is u7e3 floats, only available on Xbox360 ;) Chromatic adaptation functions are in. I've done a tidy/reorg pass. I'm not sure what else should be in the scope of this lib. PR: https://github.com/dlang/phobos/pull/2845 dub: https://code.dlang.org/packages/color docs: http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html - Manu  Sep 11 2016 Guillaume Chatelet <chatelet.guillaume gmail.com> writes: On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. I added packed-RGB support, including weird micro-float and shared-exponent formats. They're important for interacting with any real-time rendering libraries. There is only one texture format I'm aware of that isn't supported, and that is u7e3 floats, only available on Xbox360 ;) Chromatic adaptation functions are in. I've done a tidy/reorg pass. I'm not sure what else should be in the scope of this lib. PR: https://github.com/dlang/phobos/pull/2845 dub: https://code.dlang.org/packages/color docs: http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html - Manu Looks pretty good to me. Quick question though. Do you handle endianness? DPX might be a dead file format but it's main usage relies on 10bits packed RGB into 32 bits (last two bits are discarded or alpha) but it can be either little or big endian. Good job on the library though!  Sep 12 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: I thought about it, but it added (even more) complexity, and I'm not really sure it's the MO of a color lib to worry about the endian-ness of storage. I could support "_le"/"_be" suffix on the PackedRGB format string, but then you have to also specify the word size, ie, is the packed color 32bit words (eg, 11,11,10) or 16bit words (16f,16f,16f,16f)? It's only really practically applicable to PackedRGB, so I wonder why just one color type should worry about it, but not all the others... Consider an unpacked RGB with 16bit elements... that may also suffer endian problems, but RGB is a runtime type, not a storage type, which means it doesn't have a pack/unpack process... so there's no meaningful palace to do it. On 12 September 2016 at 18:09, Guillaume Chatelet via Digitalmars-d <digitalmars-d puremagic.com> wrote: On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. I added packed-RGB support, including weird micro-float and shared-exponent formats. They're important for interacting with any real-time rendering libraries. There is only one texture format I'm aware of that isn't supported, and that is u7e3 floats, only available on Xbox360 ;) Chromatic adaptation functions are in. I've done a tidy/reorg pass. I'm not sure what else should be in the scope of this lib. PR: https://github.com/dlang/phobos/pull/2845 dub: https://code.dlang.org/packages/color docs: http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html - Manu Looks pretty good to me. Quick question though. Do you handle endianness? DPX might be a dead file format but it's main usage relies on 10bits packed RGB into 32 bits (last two bits are discarded or alpha) but it can be either little or big endian. Good job on the library though!  Sep 12 2016 Guillaume Chatelet <chatelet.guillaume gmail.com> writes: On Monday, 12 September 2016 at 09:09:29 UTC, Manu wrote: I thought about it, but it added (even more) complexity, and I'm not really sure it's the MO of a color lib to worry about the endian-ness of storage. I could support "_le"/"_be" suffix on the PackedRGB format string, but then you have to also specify the word size, ie, is the packed color 32bit words (eg, 11,11,10) or 16bit words (16f,16f,16f,16f)? It's only really practically applicable to PackedRGB, so I wonder why just one color type should worry about it, but not all the others... Consider an unpacked RGB with 16bit elements... that may also suffer endian problems, but RGB is a runtime type, not a storage type, which means it doesn't have a pack/unpack process... so there's no meaningful palace to do it. Yep I totally understand the rationale - I've been there too :) I don't think it belong to Color but maybe Buffer(1) could be templated on endianness? Buffer is indeed the place where you deal with storage. If you don't want to add it to the library I think you should add a paragraph in the documentation explaining why it's not in there and how people can overcome it (copy + change endianness or use an adapter?). Also, it would be nice to have some code snippets in the documentation. Last thing, for the white points you didn't specify which CIE standard observer(2) version you used: CIE 1931 2° Standard Observer or CIE 1964 10° Standard Observer? IIRC the standard illuminant are not the same depending on the observer. 1. http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color/packedrgb/buffer.html 2. https://en.wikipedia.org/wiki/CIE_1931_color_space#CIE_standard_observer  Sep 12 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 12 September 2016 at 21:13, Guillaume Chatelet via Digitalmars-d <digitalmars-d puremagic.com> wrote: On Monday, 12 September 2016 at 09:09:29 UTC, Manu wrote: I thought about it, but it added (even more) complexity, and I'm not really sure it's the MO of a color lib to worry about the endian-ness of storage. I could support "_le"/"_be" suffix on the PackedRGB format string, but then you have to also specify the word size, ie, is the packed color 32bit words (eg, 11,11,10) or 16bit words (16f,16f,16f,16f)? It's only really practically applicable to PackedRGB, so I wonder why just one color type should worry about it, but not all the others... Consider an unpacked RGB with 16bit elements... that may also suffer endian problems, but RGB is a runtime type, not a storage type, which means it doesn't have a pack/unpack process... so there's no meaningful palace to do it. Yep I totally understand the rationale - I've been there too :) I don't think it belong to Color but maybe Buffer(1) could be templated on endianness? Buffer is indeed the place where you deal with storage. If you don't want to add it to the library I think you should add a paragraph in the documentation explaining why it's not in there and how people can overcome it (copy + change endianness or use an adapter?). Also, it would be nice to have some code snippets in the documentation. Last thing, for the white points you didn't specify which CIE standard observer(2) version you used: CIE 1931 2° Standard Observer or CIE 1964 10° Standard Observer? IIRC the standard illuminant are not the same depending on the observer. That's very true. The data is all CIE 1931 reference. I haven't seen deployed use of the updated standard... I don't know why.  Sep 12 2016 Ilya Yaroshenko <ilyayaroshenko gmail.com> writes: On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. [...] Looks great! 1. packedrgb has not enough docs to understand what it is. 2. colorFromString should have scope argument. - Ilya  Sep 12 2016 Basile B. <b2.temp gmx.com> writes: On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. I'm not sure what else should be in the scope of this lib. PR: https://github.com/dlang/phobos/pull/2845 dub: https://code.dlang.org/packages/color docs: http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html [...] - Manu 1. Maybe that the each package ddoc comment should mention that everything is always safe nothrow nogc...For me it wasn't directly clear: - seen that there's opCast: Q: safe ? - seen in the sources that opCast call a templatized function - seen that the unittest for this function are not annotated - then only seen that the leading " safe nothrow nogc:" at the beg of the package. 2. Q: is there anything to convert a color to grey scale ?  Sep 12 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 12 September 2016 at 19:30, Basile B. via Digitalmars-d <digitalmars-d puremagic.com> wrote: On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. I'm not sure what else should be in the scope of this lib. PR: https://github.com/dlang/phobos/pull/2845 dub: https://code.dlang.org/packages/color docs: http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html [...] - Manu 1. Maybe that the each package ddoc comment should mention that everything is always safe nothrow nogc...For me it wasn't directly clear: - seen that there's opCast: Q: safe ? - seen in the sources that opCast call a templatized function - seen that the unittest for this function are not annotated - then only seen that the leading " safe nothrow nogc:" at the beg of the package. How do you think this should this look? (it's best to leave these comments on the PR) 2. Q: is there anything to convert a color to grey scale ? This isn't a short-answer question :) .. There are many ways depending on the job. A simple way might be: L8 grey = cast(L8)RGB8(r, g, b); // L8 is luminance only, casting will do the right thing This is the 'normal' way, but there are better ways... A superior way if you want a top-quality result would be to use the Lab luminance value, since Lab is perceptually uniform, when you desaturate, it maintains the colors relative luminance correctly, something like: Lab!float lab = cast(Lab!float)RGB8(r, g, b); // cast your color to Lab lab.a = 0; // eliminate saturation on a axis lab.b = 0; // eliminate saturation on b axis // lab.L is now pure luminance RGB8 c = cast(RGB8)lab; // cast it back to RGB, and your RGB will be greyscale, but with more perceptually correct conversion. Lots of pow's in that conversion, so it's slower. This link demonstrates some competing techniques: https://en.wikipedia.org/wiki/HSL_and_HSV#/media/File:Fire-breather_CIELAB_L*.jpg Press left/right to cycle through them. The first method (cast to L8) is: Rec. 601 luma Y′. (or more accurately, my code above would be: sRGB luma Y') The second method (Lab.L) is: CIELAB L*. L* performs much better in low light.  Sep 12 2016 John Colvin <john.loughran.colvin gmail.com> writes: On Monday, 12 September 2016 at 13:04:49 UTC, Manu wrote: 2. Q: is there anything to convert a color to grey scale ? This isn't a short-answer question :) .. There are many ways depending on the job. A simple way might be: L8 grey = cast(L8)RGB8(r, g, b); // L8 is luminance only, casting will do the right thing This is the 'normal' way, but there are better ways... A superior way if you want a top-quality result would be to use the Lab luminance value, since Lab is perceptually uniform, when you desaturate, it maintains the colors relative luminance correctly, something like: Lab!float lab = cast(Lab!float)RGB8(r, g, b); // cast your color to Lab lab.a = 0; // eliminate saturation on a axis lab.b = 0; // eliminate saturation on b axis // lab.L is now pure luminance RGB8 c = cast(RGB8)lab; // cast it back to RGB, and your RGB will be greyscale, but with more perceptually correct conversion. Lots of pow's in that conversion, so it's slower. This link demonstrates some competing techniques: https://en.wikipedia.org/wiki/HSL_and_HSV#/media/File:Fire-breather_CIELAB_L*.jpg Press left/right to cycle through them. The first method (cast to L8) is: Rec. 601 luma Y′. (or more accurately, my code above would be: sRGB luma Y') The second method (Lab.L) is: CIELAB L*. L* performs much better in low light. Is there some uniform policy w.r.t. casts here? If I take one colour type and cast it to another will it always do something unambiguously right or not compile? Seems to me that a function that takes a target type would be more obvious for the user. I think the multiple different meanings of cast are a mistake in D. We can't go back now, but I wish there was only reinterpet_cast and everything else was a function.  Sep 12 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 13 September 2016 at 01:30, John Colvin via Digitalmars-d <digitalmars-d puremagic.com> wrote: On Monday, 12 September 2016 at 13:04:49 UTC, Manu wrote: 2. Q: is there anything to convert a color to grey scale ? This isn't a short-answer question :) .. There are many ways depending on the job. A simple way might be: L8 grey = cast(L8)RGB8(r, g, b); // L8 is luminance only, casting will do the right thing This is the 'normal' way, but there are better ways... A superior way if you want a top-quality result would be to use the Lab luminance value, since Lab is perceptually uniform, when you desaturate, it maintains the colors relative luminance correctly, something like: Lab!float lab = cast(Lab!float)RGB8(r, g, b); // cast your color to Lab lab.a = 0; // eliminate saturation on a axis lab.b = 0; // eliminate saturation on b axis // lab.L is now pure luminance RGB8 c = cast(RGB8)lab; // cast it back to RGB, and your RGB will be greyscale, but with more perceptually correct conversion. Lots of pow's in that conversion, so it's slower. This link demonstrates some competing techniques: https://en.wikipedia.org/wiki/HSL_and_HSV#/media/File:Fire-breather_CIELAB_L*.jpg Press left/right to cycle through them. The first method (cast to L8) is: Rec. 601 luma Y′. (or more accurately, my code above would be: sRGB luma Y') The second method (Lab.L) is: CIELAB L*. L* performs much better in low light. Is there some uniform policy w.r.t. casts here? If I take one colour type and cast it to another will it always do something unambiguously right or not compile? I think, yes? So, the logic here is, if you cast, and then cast back, you will have the same thing barring loss of precision. Casts in this lib just change format (+precision), they don't change value at all. I feel it's compatible with casting an int to a float... it's the same sort of conversion. Seems to me that a function that takes a target type would be more obvious for the user. Really? I like the cast this way. There is an explicit function in package.d though...  Sep 12 2016 Marco Leise <Marco.Leise gmx.de> writes: Am Mon, 12 Sep 2016 14:14:27 +1000 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. I added packed-RGB support, including weird micro-float and shared-exponent formats. They're important for interacting with any real-time rendering libraries. There is only one texture format I'm aware of that isn't supported, and that is u7e3 floats, only available on Xbox360 ;) Chromatic adaptation functions are in. I've done a tidy/reorg pass. I'm not sure what else should be in the scope of this lib. PR: https://github.com/dlang/phobos/pull/2845 dub: https://code.dlang.org/packages/color docs: http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html - Manu Nice work. What's u7e3? 3*10-bit-floats + 2-bit padding? In the example for convert color you show that converting from a color space without alpha (XYZ) to one with alpha (RGBA) turns the pixel transparent. I believe you meant this as something to lookout for when converting colors. What's the reasoning for not setting maximum alpha as a more useful and natural default? Cool stuff there in the colorspace module, to get started with getting photos or DV/MPEG into a linear color space. Should BT.709 also have prefab conversion methods? It would seem HD video is more wide spread nowadays. There is a typo in "Function that converts a linear luminance to gamme space." (colorspace.d) I like how RGB color spaces are described as primaries in xyY + gamma functions. It seems inconsistent that you can specify all kinds of exponents when the exponent is shared but are forced to 5 bit exponents when mixed into each component, while the mantissa can still be adjusted. Would m7e3 have worked? (With f* being the IEEE standard formats.) The final package should contain a fair amount of documentation, in my opinion also on the topics of sRGB vs. linear and what "hybrid gamma" is. What is your stance on pixel formats that are arranged in planes? I assume that e.g. YUV is handled by a single channel "sRGB luminance" + 2 signed 8-bit color planes? It may be worth adding some hints in that direction. All in all good work with a scientific feel to it. -- Marco  Sep 12 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 12 September 2016 at 21:28, Marco Leise via Digitalmars-d <digitalmars-d puremagic.com> wrote: Am Mon, 12 Sep 2016 14:14:27 +1000 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. I added packed-RGB support, including weird micro-float and shared-exponent formats. They're important for interacting with any real-time rendering libraries. There is only one texture format I'm aware of that isn't supported, and that is u7e3 floats, only available on Xbox360 ;) Chromatic adaptation functions are in. I've done a tidy/reorg pass. I'm not sure what else should be in the scope of this lib. PR: https://github.com/dlang/phobos/pull/2845 dub: https://code.dlang.org/packages/color docs: http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html - Manu Nice work. What's u7e3? 3*10-bit-floats + 2-bit padding? Yup. In the example for convert color you show that converting from a color space without alpha (XYZ) to one with alpha (RGBA) turns the pixel transparent. I believe you meant this as something to lookout for when converting colors. What's the reasoning for not setting maximum alpha as a more useful and natural default? I flip-flopped on this multiple times. It's not so simple. 1. Alpha doesn't necessarily mean transparently, that's just one (extremely common) use 2. 1 is the multiplication identity, but 0 is the additive identity. I cant find either addition or multiplication to take precedence over eachother. 3. With those 2 points above already making me feel a bit worried about it, I think that appearing numbers out of nowhere is just generally bad, so I went with a=0. If alpha defaults to 1, then a*b works as expected (object does not become transparent by this operation). If alpha defaults to 0, then b-a works as expected (object does not become transparent by this operation). With that in mind, you realise alpha will *always* require explicit handling... so I think 0 is a better default at that stage. It may be possible to convince me otherwise ;) Cool stuff there in the colorspace module, to get started with getting photos or DV/MPEG into a linear color space. Should BT.709 also have prefab conversion methods? It would seem HD video is more wide spread nowadays. BT.709's gamma func isn't changed from 601. Rec.2020 introduced a higher precision version. There is a typo in "Function that converts a linear luminance to gamme space." (colorspace.d) Thx. But please log comments like this on the PR so I don't lose track of them? :) I like how RGB color spaces are described as primaries in xyY + gamma functions. Seems natural :) So, I was initially planning to make that struct the RGB template argument, but then I realised that would make the symbols gigantic (heaps of CS data in the symbol names), so I introduced the enum... sadly though, this means you can't define a custom colour without introducing a new custom-colour type to do what I originally intended. I'm upset about this either way... I think I should leave it how it is... custom colours are probably more rare than gigantic symbol names ;) It seems inconsistent that you can specify all kinds of exponents when the exponent is shared but are forced to 5 bit exponents when mixed into each component, while the mantissa can still be adjusted. Would m7e3 have worked? (With f* being the IEEE standard formats.) I was thinking that, and mentioned as such in the comments near those lines ;) Thing is, there are no realtime small floats with exponent != 5bits except the unique 7e3 format on xbox 360. I can extend this in an unbreaking way. I might: f## = ieee-like, s##e# = signed float, (u)##e# = unsgned float. It's just pretty verbose is all: "rgb_11e5_11e5_10e5". I mean, it's fine... but maybe people wouldn't find this intuitive? Many programmers don't know what an 'exponent' or 'mantissa' is... :/ The final package should contain a fair amount of documentation, in my opinion also on the topics of sRGB vs. linear and what "hybrid gamma" is. Yeah, but is it really my job to teach someone about colour space? Wikipedia can do that much better than I can... :/ I agree, but I don't really know where to draw the line, or how to organise it. Perhaps people with strong opinions on how this should be presented can create a PR? What is your stance on pixel formats that are arranged in planes? I assume that e.g. YUV is handled by a single channel "sRGB luminance" + 2 signed 8-bit color planes? It may be worth adding some hints in that direction. I haven't supported YUV, but it is as you say. Nitpick: most of those YUV sinals are probably broadcast signals, with rec.601 luminance ;) YUV is a pretty big family, I'll do it if it's requested... There are a lot of micro-permutations. It's easy to get carried away (ie; as you can see in HSx). I don't think 'types' can really express bitplanes well. We need to express array operations to support them. I want to extend this to support efficient array operations, but I don't really know how to work this into D's ranges efficiently, and my other topic on the matter died before anything worthwhile emerged. All in all good work with a scientific feel to it. Thanks, I struggled with this balance a lot. I'm fairly happy with it now.  Sep 12 2016 Marco Leise <Marco.Leise gmx.de> writes: Am Tue, 13 Sep 2016 00:37:22 +1000 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>: I flip-flopped on this multiple times. It's not so simple. 1. Alpha doesn't necessarily mean transparently, that's just one (extremely common) use 2. 1 is the multiplication identity, but 0 is the additive identity. I cant find either addition or multiplication to take precedence over eachother. 3. With those 2 points above already making me feel a bit worried about it, I think that appearing numbers out of nowhere is just generally bad, so I went with a=0. If alpha defaults to 1, then a*b works as expected (object does not become transparent by this operation). If alpha defaults to 0, then b-a works as expected (object does not become transparent by this operation). Additive color components can work like that, but the alpha component just doesn't represent an amount of light. Don't try too hard to make the arithmetics make sense, when the practical use lies in 100% alpha by default. Converting every color to full transparent is not helpful when you want to save a 24-bit BMP as 32-bit PNG or get an RGBA color from an RGB texture. I don't know who declared that 100% alpha means opaque. For what it's worth it could have been the other way around and all the (1-a) and (a) in blending functions swapped. And a blending function is what is needed here to give any meaning at all to arithmetics on alpha. An additive one may be: color = color_dst + color_src * alpha_src alpha = alpha_dst With that in mind, you realise alpha will *always* require explicit handling... so I think 0 is a better default at that stage. It may be possible to convince me otherwise ;) I fully agree with the first line, I just think that for practical uses alpha is considered to be 100% in pixel formats without alpha channel. Otherwise the most popular blending functions would do nothing on RGB colors. SDL for example also uses 100% alpha by default: "If the specified pixel format has an alpha component it will be returned as all 1 bits (fully opaque)." Cool stuff there in the colorspace module, to get started with getting photos or DV/MPEG into a linear color space. Should BT.709 also have prefab conversion methods? It would seem HD video is more wide spread nowadays. BT.709's gamma func isn't changed from 601. Rec.2020 introduced a higher precision version. There is a typo in "Function that converts a linear luminance to gamme space." (colorspace.d) Thx. But please log comments like this on the PR so I don't lose track of them? :) Will do! It seems inconsistent that you can specify all kinds of exponents when the exponent is shared but are forced to 5 bit exponents when mixed into each component, while the mantissa can still be adjusted. Would m7e3 have worked? (With f* being the IEEE standard formats.) I was thinking that, and mentioned as such in the comments near those lines ;) Thing is, there are no realtime small floats with exponent != 5bits except the unique 7e3 format on xbox 360. I can extend this in an unbreaking way. I might: f## = ieee-like, s##e# = signed float, (u)##e# = unsgned float. It's just pretty verbose is all: "rgb_11e5_11e5_10e5". I mean, it's fine... but maybe people wouldn't find this intuitive? Many programmers don't know what an 'exponent' or 'mantissa' is... :/ Yeah I thought that the common formats would be the IEEE ones, especially f16 and that anything less are very niche custom formats that come and go and need verbose descriptions. Well, either way works. The final package should contain a fair amount of documentation, in my opinion also on the topics of sRGB vs. linear and what "hybrid gamma" is. Yeah, but is it really my job to teach someone about colour space? Wikipedia can do that much better than I can... :/ I agree, but I don't really know where to draw the line, or how to organise it. Perhaps people with strong opinions on how this should be presented can create a PR? Alright, but hybrid gamma is really not something that can be googled. Or rather I end up at Toyota's Gamma Hybrid product page. :D And as always, don't let the turkeys get you down. -- Marco  Sep 12 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 13 September 2016 at 07:00, Marco Leise via Digitalmars-d <digitalmars-d puremagic.com> wrote: Am Tue, 13 Sep 2016 00:37:22 +1000 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>: I flip-flopped on this multiple times. It's not so simple. 1. Alpha doesn't necessarily mean transparently, that's just one (extremely common) use 2. 1 is the multiplication identity, but 0 is the additive identity. I cant find either addition or multiplication to take precedence over eachother. 3. With those 2 points above already making me feel a bit worried about it, I think that appearing numbers out of nowhere is just generally bad, so I went with a=0. If alpha defaults to 1, then a*b works as expected (object does not become transparent by this operation). If alpha defaults to 0, then b-a works as expected (object does not become transparent by this operation). Additive color components can work like that, but the alpha component just doesn't represent an amount of light. Don't try too hard to make the arithmetics make sense, when the practical use lies in 100% alpha by default. I understand the argument, I have stressed over this to no end. I still think it's just not 'right'. :/ Converting every color to full transparent is not helpful when you want to save a 24-bit BMP as 32-bit PNG or get an RGBA color from an RGB texture. What is the worth of storing alpha data if it's uniform 0xFF anyway? It sounds like you mean rgbx, not rgba (ie, 32bit, but no alpha). There should only be an alpha channel if there's actually alpha data... right? I don't know who declared that 100% alpha means opaque. For what it's worth it could have been the other way around and all the (1-a) and (a) in blending functions swapped. And a blending function is what is needed here to give any meaning at all to arithmetics on alpha. An additive one may be: color = color_dst + color_src * alpha_src alpha = alpha_dst I thought about adding blend's, but I stopped short on that. I think that's firmly entering image processing territory, and I felt that was one step beyond the MO of this particular lib... no? Blending opens up a whole world. With that in mind, you realise alpha will *always* require explicit handling... so I think 0 is a better default at that stage. It may be possible to convince me otherwise ;) I fully agree with the first line, I just think that for practical uses alpha is considered to be 100% in pixel formats without alpha channel. 'rgbx' is for image formats without alpha channels... Otherwise the most popular blending functions would do nothing on RGB colors. 'rgbx' colors fed through a blend function would always properly imply alpha=1. SDL for example also uses 100% alpha by default: "If the specified pixel format has an alpha component it will be returned as all 1 bits (fully opaque)." From which functions? Link me? I'd love to see more precedents. It seems inconsistent that you can specify all kinds of exponents when the exponent is shared but are forced to 5 bit exponents when mixed into each component, while the mantissa can still be adjusted. Would m7e3 have worked? (With f* being the IEEE standard formats.) I was thinking that, and mentioned as such in the comments near those lines ;) Thing is, there are no realtime small floats with exponent != 5bits except the unique 7e3 format on xbox 360. I can extend this in an unbreaking way. I might: f## = ieee-like, s##e# = signed float, (u)##e# = unsgned float. It's just pretty verbose is all: "rgb_11e5_11e5_10e5". I mean, it's fine... but maybe people wouldn't find this intuitive? Many programmers don't know what an 'exponent' or 'mantissa' is... :/ Yeah I thought that the common formats would be the IEEE ones, especially f16 and that anything less are very niche custom formats that come and go and need verbose descriptions. Well, either way works. The verbose idea I said above is my best idea. I'll roll with that if enough people agree. There was the other fellow who posted before who feels like it's already too complicated for him though. I've been very carefully trying to put the complicated bits just one level back from arms reach, so a casual user isn't confronted with those details. Putting that complexity in the format string makes it very user-facing. That's the part I'm not wild about. The final package should contain a fair amount of documentation, in my opinion also on the topics of sRGB vs. linear and what "hybrid gamma" is. Yeah, but is it really my job to teach someone about colour space? Wikipedia can do that much better than I can... :/ I agree, but I don't really know where to draw the line, or how to organise it. Perhaps people with strong opinions on how this should be presented can create a PR? Alright, but hybrid gamma is really not something that can be googled. Or rather I end up at Toyota's Gamma Hybrid product page. :D True. I'm not even sure what the technical term for this sort of gamma function is... I just made that up! :/ As Walter and others have asked, I'll have to start adding links to reference material I guess, although that still feels really weird to me for some reason.  Sep 12 2016 Random D user <no email.com> writes: On Tuesday, 13 September 2016 at 02:00:44 UTC, Manu wrote: On 13 September 2016 at 07:00, Marco Leise via Digitalmars-d <digitalmars-d puremagic.com> wrote: Am Tue, 13 Sep 2016 00:37:22 +1000 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>: Alright, but hybrid gamma is really not something that can be googled. Or rather I end up at Toyota's Gamma Hybrid product page. :D True. I'm not even sure what the technical term for this sort of gamma function is... I just made that up! :/ As Walter and others have asked, I'll have to start adding links to reference material I guess, although that still feels really weird to me for some reason. FWIW I stumbled into this while double-checking HDR standards for my previous post. (I'm not a HDR expert, only somewhat interested because it's the future of displays/graphics) https://en.wikipedia.org/wiki/Hybrid_Log-Gamma  Sep 13 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 14 September 2016 at 04:25, Random D user via Digitalmars-d <digitalmars-d puremagic.com> wrote: On Tuesday, 13 September 2016 at 02:00:44 UTC, Manu wrote: On 13 September 2016 at 07:00, Marco Leise via Digitalmars-d <digitalmars-d puremagic.com> wrote: Am Tue, 13 Sep 2016 00:37:22 +1000 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>: Alright, but hybrid gamma is really not something that can be googled. Or rather I end up at Toyota's Gamma Hybrid product page. :D True. I'm not even sure what the technical term for this sort of gamma function is... I just made that up! :/ As Walter and others have asked, I'll have to start adding links to reference material I guess, although that still feels really weird to me for some reason. FWIW I stumbled into this while double-checking HDR standards for my previous post. (I'm not a HDR expert, only somewhat interested because it's the future of displays/graphics) https://en.wikipedia.org/wiki/Hybrid_Log-Gamma Perfect, thanks!  Sep 13 2016 Marco Leise <Marco.Leise gmx.de> writes: Am Tue, 13 Sep 2016 12:00:44 +1000 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>: What is the worth of storing alpha data if it's uniform 0xFF anyway? It sounds like you mean rgbx, not rgba (ie, 32bit, but no alpha). There should only be an alpha channel if there's actually alpha data... r= ight? I don't mean RGBX. JavaScript's canvas works that way for example. I.e. the only pixel format is RGBA for simplicity's sake and I'm not surprised it actually draws something if I load it with a 24-bit graphic. ;) [=E2=80=A6] An additive one may be: color =3D color_dst + color_src * alpha_src alpha =3D alpha_dst =20 =20 I thought about adding blend's, but I stopped short on that. I think that's firmly entering image processing territory, and I felt that was one step beyond the MO of this particular lib... no? Blending opens up a whole world. I agree with that decision, and that it entails that arithmetics are undefined for alpha channels. :-( Yeah bummer. The idea that basic (saturating) arithmetics work on colors is a great simplification that works for the most part, but let's be fair, multiplying two HSV colors isn't exactly going to yield a well defined hue either, just as multiplying two angles doesn't give you a new angle. I.e.: http://math.stackexchange.com/a/47880 [=E2=80=A6] [=E2=80=A6] From which functions? Link me? =20 I'd love to see more precedents. Yep, that's better than arguing :) So here are all graphics APIs I know and what they do when they encounter colors without alpha and need a default value: SDL: https://wiki.libsdl.org/SDL_MapRGB "If the specified pixel format has an alpha component it will be returned a= s all 1 bits (fully opaque)." Allegro: https://github.com/liballeg/allegro5/blob/master/include/allegro5/internal/= aintern_pixels.h#L59 (No docs, just source code that defaults to 255 for alpha when converting a color from a bitmap with non-alpha pixel format to an ALLEGRO_COLOR.) Cairo: https://www.cairographics.org/manual/cairo-cairo-t.html#cairo-set-source-rgb "Sets the source pattern within cr to an opaque color." Microsoft GDI+: https://msdn.microsoft.com/de-de/library/windows/desktop/ms536255%28v=3Dvs.= 85%29.aspx "The default value of the alpha component for this Color object is 255." Gnome GDK: https://developer.gnome.org/gdk-pixbuf/2.33/gdk-pixbuf-Utilities.html#gdk-p= ixbuf-add-alpha "[=E2=80=A6] the alpha channel is initialized to 255 (full opacity)." Qt: http://doc.qt.io/qt-4.8/qcolor.html#alpha-blended-drawing "By default, the alpha-channel is set to 255 (opaque)." OpenGL: https://www.opengl.org/sdk/docs/man2/xhtml/glColor.xml "glColor3 variants specify new red, green, and blue values explicitly and set the current alpha value to 1.0 (full intensity) implicitly." (Note: The color can be queried and shows a=3D1.0 without blending operations setting it internally if needed.) Java (AWT): https://docs.oracle.com/javase/7/docs/api/java/awt/Color.html#Color%28int,%= 20boolean%29 "If the hasalpha argument is false, alpha is defaulted to 255." Apple's Quartz does not seem to provide color space conversions and always requires the user to give the alpha value for new colors, so there is no default: https://developer.apple.com/library/tvos/documentation/GraphicsImaging/Refe= rence/CGColor/index.html#//apple_ref/c/func/CGColorCreate One thing I noticed is that many of those strictly separate color spaces from alpha as concepts. For example in Quartz *all* color spaces have alpha. In Allegro color space conversions are ignorant of alpha. That begs the question what should happen when you convert RGBA to a HLx color space and back to RGBA. Can you retain the alpha value? CSS3 for example has HSLA colors that raise the bar a bit. --=20 Marco  Sep 13 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 14 September 2016 at 04:34, Marco Leise via Digitalmars-d <digitalmars-d puremagic.com> wrote: Am Tue, 13 Sep 2016 12:00:44 +1000 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>: What is the worth of storing alpha data if it's uniform 0xFF anyway? It sounds like you mean rgbx, not rgba (ie, 32bit, but no alpha). There should only be an alpha channel if there's actually alpha data... right? I don't mean RGBX. JavaScript's canvas works that way for example. I.e. the only pixel format is RGBA for simplicity's sake and I'm not surprised it actually draws something if I load it with a 24-bit graphic. ;) Given this example, isn't it the job of the image loader to populate the image with data? If you ask for an RGBA image loaded from a 24bit BMP or something, sure, it should put 0xFF in there, but I don't know if it's right that that logic belongs here...? […] An additive one may be: color = color_dst + color_src * alpha_src alpha = alpha_dst I thought about adding blend's, but I stopped short on that. I think that's firmly entering image processing territory, and I felt that was one step beyond the MO of this particular lib... no? Blending opens up a whole world. I agree with that decision, and that it entails that arithmetics are undefined for alpha channels. :-( Yeah bummer. The idea that basic (saturating) arithmetics work on colors is a great simplification that works for the most part, but let's be fair, multiplying two HSV colors isn't exactly going to yield a well defined hue either, You'll notice I didn't add arithmetic operators to the HSx type ;) If you have HSx colors, and want to do arithmetic, cast it to RGB first. just as multiplying two angles doesn't give you a new angle. I.e.: http://math.stackexchange.com/a/47880 I went through a brief phase where I thought about adding an angle type (beside normint), but I felt it was overkill. I still wonder if it's the right thing to do though... some type that understands a circle, and making angle arithmetic work as expected. I like my solution of not providing arithmetic operators for HSx and LCh for now though ;) […] […] From which functions? Link me? I'd love to see more precedents. Yep, that's better than arguing :) So here are all graphics APIs I know and what they do when they encounter colors without alpha and need a default value: SDL: https://wiki.libsdl.org/SDL_MapRGB "If the specified pixel format has an alpha component it will be returned as all 1 bits (fully opaque)." Allegro: https://github.com/liballeg/allegro5/blob/master/include/allegro5/internal/aintern_pixels.h#L59 (No docs, just source code that defaults to 255 for alpha when converting a color from a bitmap with non-alpha pixel format to an ALLEGRO_COLOR.) Cairo: https://www.cairographics.org/manual/cairo-cairo-t.html#cairo-set-source-rgb "Sets the source pattern within cr to an opaque color." Microsoft GDI+: https://msdn.microsoft.com/de-de/library/windows/desktop/ms536255%28v=vs.85%29.aspx "The default value of the alpha component for this Color object is 255." Gnome GDK: https://developer.gnome.org/gdk-pixbuf/2.33/gdk-pixbuf-Utilities.html#gdk-pixbuf-add-alpha "[…] the alpha channel is initialized to 255 (full opacity)." Qt: http://doc.qt.io/qt-4.8/qcolor.html#alpha-blended-drawing "By default, the alpha-channel is set to 255 (opaque)." OpenGL: https://www.opengl.org/sdk/docs/man2/xhtml/glColor.xml "glColor3 variants specify new red, green, and blue values explicitly and set the current alpha value to 1.0 (full intensity) implicitly." (Note: The color can be queried and shows a=1.0 without blending operations setting it internally if needed.) Java (AWT): https://docs.oracle.com/javase/7/docs/api/java/awt/Color.html#Color%28int,%20boolean%29 "If the hasalpha argument is false, alpha is defaulted to 255." Apple's Quartz does not seem to provide color space conversions and always requires the user to give the alpha value for new colors, so there is no default: https://developer.apple.com/library/tvos/documentation/GraphicsImaging/Reference/CGColor/index.html#//apple_ref/c/func/CGColorCreate Touche! ;) One thing I noticed is that many of those strictly separate color spaces from alpha as concepts. For example in Quartz *all* color spaces have alpha. In Allegro color space conversions are ignorant of alpha. That begs the question what should happen when you convert RGBA to a HLx color space and back to RGBA. Can you retain the alpha value? CSS3 for example has HSLA colors that raise the bar a bit. I've actually had the thought that I should remove alpha from RGB, and instead provide an aggregate colour type where you could make RGBA by RGB + alpha, and each element in the aggregate would follow its own natural arithmetic rules... again, it feels like massive overkill though, and RGBA 8 is just way simpler for the 99% case. That aggregate thing can appear, or be added in the future, or by users. The library and conversions are all extensible.  Sep 13 2016 Marco Leise <Marco.Leise gmx.de> writes: Am Wed, 14 Sep 2016 11:36:12 +1000 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>: On 14 September 2016 at 04:34, Marco Leise via Digitalmars-d <digitalmars-d puremagic.com> wrote: JavaScript's canvas works that way for example. I.e. the only pixel format is RGBA for simplicity's sake and I'm not surprised it actually draws something if I load it with a 24-bit graphic. ;) =20 =20 Given this example, isn't it the job of the image loader to populate the image with data?2 [=E2=80=A6] I admit is was a constructed example. You can't load an image directly into a HTML5 canvas. Shame on me. You'll notice I didn't add arithmetic operators to the HSx type ;) If you have HSx colors, and want to do arithmetic, cast it to RGB first. Now that you say it, yes I was wondering how the arithmetics work since I couldn't find any opBinary. =20 I went through a brief phase where I thought about adding an angle type (beside normint), but I felt it was overkill. I still wonder if it's the right thing to do though... some type that understands a circle, and making angle arithmetic work as expected. I see. I'm also wondering what equality means for a color with floating point hue 10=C2=B0 and another with hue 730=C2=B0. I guess it is ok the way it is now, because wrapping the values doesn't guarantee they map to the same value either. Floating point equality is flawed either way. --=20 Marco  Sep 14 2016 Basile B. <b2.temp gmx.com> writes: On Tuesday, 13 September 2016 at 02:00:44 UTC, Manu wrote: What is the worth of storing alpha data if it's uniform 0xFF anyway? He he, that's the big problem with classic image formats. When a graphic is described in term of vector, fill, stroke, gradient, layer, etc (not to name SVG) there's no more waste.  Sep 13 2016 Guillaume Chatelet <chatelet.guillaume gmail.com> writes: On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. I added packed-RGB support, including weird micro-float and shared-exponent formats. They're important for interacting with any real-time rendering libraries. There is only one texture format I'm aware of that isn't supported, and that is u7e3 floats, only available on Xbox360 ;) Chromatic adaptation functions are in. I've done a tidy/reorg pass. I'm not sure what else should be in the scope of this lib. PR: https://github.com/dlang/phobos/pull/2845 dub: https://code.dlang.org/packages/color docs: http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html - Manu And also, do you handle limited (16-235) to full dynamic range (0-255) conversions? It's a setting some acquisition/graphic card allow. https://wiki.videolan.org/VSG:Video:Color_washed_out/ From this page as well http://www.digitalpreservation.gov/formats/fdd/fdd000352.shtml "Video and wide range. The ITU-R specifications support values for Y, Cb, and Cr that conform to what is sometimes called "video range," "legal range," or "studio swing" levels. Expressed in terms of 8-bit tonal range values, video range has a 16-235 levels for Y and 16-240 levels for Cr and Cb. The term video range is used to contrast with "wide range" or "super white " values from 0 to 255, sometimes used when video signals are created using computer-based graphics applications."  Sep 12 2016 Marco Leise <Marco.Leise gmx.de> writes: Am Mon, 12 Sep 2016 11:31:13 +0000 schrieb Guillaume Chatelet <chatelet.guillaume gmail.com>: On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. I added packed-RGB support, including weird micro-float and shared-exponent formats. They're important for interacting with any real-time rendering libraries. There is only one texture format I'm aware of that isn't supported, and that is u7e3 floats, only available on Xbox360 ;) Chromatic adaptation functions are in. I've done a tidy/reorg pass. I'm not sure what else should be in the scope of this lib. PR: https://github.com/dlang/phobos/pull/2845 dub: https://code.dlang.org/packages/color docs: http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html - Manu And also, do you handle limited (16-235) to full dynamic range (0-255) conversions? It's a setting some acquisition/graphic card allow. https://wiki.videolan.org/VSG:Video:Color_washed_out/ [...] Practically all video content is encoded with the limited range, from VHS to HDTV, IIRC and HiFi equipment uses it as well, only computers are not so familiar with it and when connected to a TV over HDMI one or the other may think that the other part thinks ... well colors are often washed out, in effect. On the other hand this luminance compression can apply to any color format, so it is probably best to leave it to the user to compress colors if the output is a video file. -- Marco  Sep 12 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 12 September 2016 at 21:31, Guillaume Chatelet via Digitalmars-d <digitalmars-d puremagic.com> wrote: On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. I added packed-RGB support, including weird micro-float and shared-exponent formats. They're important for interacting with any real-time rendering libraries. There is only one texture format I'm aware of that isn't supported, and that is u7e3 floats, only available on Xbox360 ;) Chromatic adaptation functions are in. I've done a tidy/reorg pass. I'm not sure what else should be in the scope of this lib. PR: https://github.com/dlang/phobos/pull/2845 dub: https://code.dlang.org/packages/color docs: http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html - Manu And also, do you handle limited (16-235) to full dynamic range (0-255) conversions? Actually, I spent quite some time stressing about dynamic range. Trouble is, there's no real format spec's relating to dynamic range, it tends to be by convention. I'm not sure what the API would look like, so I haven't approached that problem. It's very easy to do the transform yourself, I don't really think it needs to be part of the type, particularly since there's no conventional way to express it. Note also that I didn't do any YUV formats (this case you discuss is usually related to older broadcast formats)... didn't know if they'd be particularly useful to people. At least, it's not in my initial offering.  Sep 12 2016 Guillaume Chatelet <chatelet.guillaume gmail.com> writes: On Monday, 12 September 2016 at 14:12:35 UTC, Manu wrote: On 12 September 2016 at 21:31, Guillaume Chatelet via Digitalmars-d <digitalmars-d puremagic.com> wrote: [...] Actually, I spent quite some time stressing about dynamic range. Trouble is, there's no real format spec's relating to dynamic range, it tends to be by convention. I'm not sure what the API would look like, so I haven't approached that problem. It's very easy to do the transform yourself, I don't really think it needs to be part of the type, particularly since there's no conventional way to express it. Note also that I didn't do any YUV formats (this case you discuss is usually related to older broadcast formats)... didn't know if they'd be particularly useful to people. At least, it's not in my initial offering. Yes I agree, as you and Marco said: I think it's best to leave it to clients of the API.  Sep 12 2016 Guillaume Piolat <first.last gmail.com> writes: On Monday, 12 September 2016 at 14:12:35 UTC, Manu wrote: Note also that I didn't do any YUV formats (this case you discuss is usually related to older broadcast formats)... didn't know if they'd be particularly useful to people. At least, it's not in my initial offering. As someone who had to write lots of code involving such video color spaces, I'd say to don't bother. - Video engineers will write their own highly-optimized version anyway, - Dynamic range is indeed very often 16-235 / 16-240 but often with no meta-date allowing to know if this is the case, - you often don't want clamping on each intermediate results  Sep 12 2016 Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes: On 09/12/2016 12:14 AM, Manu via Digitalmars-d wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. Upon a quick look it seems to have the right amount of "notes", as Emperor Joseph II would say. Well balanced, nothing jumping at you. Nice work. Regarding Phobos suitability: One thing I'd like to hear from the community is the applicability of this work. I know very little about colors (only the RGB was familiar from HTML etc, but then there was no YUL), but that doesn't mean much. The library is quite isolated from the rest of Phobos, e.g. a stronger case would be made if there were some std.experimental.canvas package which uses the color package. Or at least a std.experimental.terminal package or something. A good case for inclusion would motivate the presence of the library as a catalyst for many user-level abstractions. -- Andrei  Sep 12 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 13 September 2016 at 00:37, Andrei Alexandrescu via Digitalmars-d <digitalmars-d puremagic.com> wrote: On 09/12/2016 12:14 AM, Manu via Digitalmars-d wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. Upon a quick look it seems to have the right amount of "notes", as Emperor Joseph II would say. Well balanced, nothing jumping at you. Nice work. Thanks. It has taken a lot more work than I actually expected to make this not suck! Regarding Phobos suitability: One thing I'd like to hear from the community is the applicability of this work. I know very little about colors (only the RGB was familiar from HTML etc, but then there was no YUL), but that doesn't mean much. The library is quite isolated from the rest of Phobos, e.g. a stronger case would be made if there were some std.experimental.canvas package which uses the color package. Or at least a std.experimental.terminal package or something. A good case for inclusion would motivate the presence of the library as a catalyst for many user-level abstractions. -- Andrei So, the main reason I wanted to push for this in phobos is because it's an enabler. There is no multimedia in phobos at all. I think that's a shame, and this is a starting point. Any conceivable image based work depends on this as foundation. If pixel format types are phobos-supplied, then D image libraries will naturally be compatible, as opposed to naturally incompatible since every image library will otherwise re-implement their own colour types (and almost always incorrectly). The thing about colours is, they're HARD to get right, but more importantly, most programmers have no idea that it's actually a hard problem and won't tend to go looking for a lib for a colour type (Ie, struct { ubyte r, g, b;}... simple! *cough*). They probably will use it if it's in phobos though for compatibility reasons. Image processing is super easy by contrast. One of my first port-of-calls after this work would be a graph/plot library... I'd use the hell out of that. I constantly want to output data visualisations! It also enables seamless interaction with realtime rendering api's. Ie, if OpenGL/D3D/Vulkan bindings support phobos supplied colour types, then image decoding libraries, and other sources of image data become easy to render without any requirement for adaptation. Rendering libs and image/video data source libs are almost always separate, independent libraries. We need to maximise probability that they are compatible out of the gate. I didn't want to touch 'images' in this work, because there are countless subjective designs to present those API's, but I don't think there's many competing designs for a colour library, so it felt it was the best and most bang-for-buck starting place overall. Also, I think this has very high synergy with ndslice. I think ndslice + color is the foundation for image processing, or at least image data transport.  Sep 12 2016 Adam D. Ruppe <destructionator gmail.com> writes: On Monday, 12 September 2016 at 15:06:29 UTC, Manu wrote: So, the main reason I wanted to push for this in phobos is because it's an enabler. Indeed. I know there's a window library that was waiting on this work to try for Phobos. Also, worth noting that the one module I import more than any other in my libs is.... color.d. Image load/save depends on it. Windowing display depends on it. Graphing depends on one or the other, GUI depends on windowing, terminal I managed to keep independent... but it is REALLY tempting to make it depend on color too. It is a very useful base building block, and moreover, having it in Phobos means User X's graphing lib and my image libs could conceivably be able to interchange data with ease since we both use the same underlying data structure!  Sep 12 2016 Edwin van Leeuwen <edder tkwsping.nl> writes: On Monday, 12 September 2016 at 15:06:29 UTC, Manu wrote: One of my first port-of-calls after this work would be a graph/plot library... I'd use the hell out of that. I constantly want to output data visualisations! Just for your information. ggplotd currently uses an old version of your library on the backend. I'm planning on porting it soon to the latest (dub) version.  Sep 12 2016 Guillaume Chatelet <chatelet.guillaume gmail.com> writes: On Monday, 12 September 2016 at 15:06:29 UTC, Manu wrote: On 13 September 2016 at 00:37, Andrei Alexandrescu via Regarding Phobos suitability: One thing I'd like to hear from the community is the applicability of this work. I know very little about colors (only the RGB was familiar from HTML etc, but then there was no YUL), but that doesn't mean much. The library is quite isolated from the rest of Phobos, e.g. a stronger case would be made if there were some std.experimental.canvas package which uses the color package. Or at least a std.experimental.terminal package or something. A good case for inclusion would motivate the presence of the library as a catalyst for many user-level abstractions. -- Andrei So, the main reason I wanted to push for this in phobos is because it's an enabler. There is no multimedia in phobos at all. I think that's a shame, and this is a starting point. Any conceivable image based work depends on this as foundation. If pixel format types are phobos-supplied, then D image libraries will naturally be compatible, as opposed to naturally incompatible since every image library will otherwise re-implement their own colour types (and almost always incorrectly). The thing about colours is, they're HARD to get right, but more importantly, most programmers have no idea that it's actually a hard problem and won't tend to go looking for a lib for a colour type (Ie, struct { ubyte r, g, b;}... simple! *cough*). They probably will use it if it's in phobos though for compatibility reasons. Image processing is super easy by contrast. One of my first port-of-calls after this work would be a graph/plot library... I'd use the hell out of that. I constantly want to output data visualisations! It also enables seamless interaction with realtime rendering api's. Ie, if OpenGL/D3D/Vulkan bindings support phobos supplied colour types, then image decoding libraries, and other sources of image data become easy to render without any requirement for adaptation. Rendering libs and image/video data source libs are almost always separate, independent libraries. We need to maximise probability that they are compatible out of the gate. I didn't want to touch 'images' in this work, because there are countless subjective designs to present those API's, but I don't think there's many competing designs for a colour library, so it felt it was the best and most bang-for-buck starting place overall. Also, I think this has very high synergy with ndslice. I think ndslice + color is the foundation for image processing, or at least image data transport. +1 It's simply amazing how the concept of color is well understood by everyone and yet utterly hard to understand in the details. Color is a Universe in itself! Just have a look at Metamerism(1) (colors that look the same but aren't), Human Tetrachromacy(2) (some people - mostly women - perceive more colors than others). I'm not even scratching the surface: it connects with everything: Physics, Physiology, Art, Entertainement, Evolution(3)... Having a high quality Color API in Phobos is a differentiator and an enabler: think Raytracer, image batch processing, scientific analysis, interactions with GUI or rendering APIs. I think it's an important addition to Phobos - I might be biased though :-P 1. https://en.wikipedia.org/wiki/Metamerism_(color) 2. https://en.wikipedia.org/wiki/Tetrachromacy#Humans 3. https://www.youtube.com/watch?v=qrKZBh8BL_U  Sep 12 2016 Walter Bright <newshound2 digitalmars.com> writes: On 9/12/2016 8:06 AM, Manu via Digitalmars-d wrote: So, the main reason I wanted to push for this in phobos is because it's an enabler. There is no multimedia in phobos at all. I think that's a shame, and this is a starting point. Any conceivable image based work depends on this as foundation. If pixel format types are phobos-supplied, then D image libraries will naturally be compatible, as opposed to naturally incompatible since every image library will otherwise re-implement their own colour types (and almost always incorrectly). The thing about colours is, they're HARD to get right, but more importantly, most programmers have no idea that it's actually a hard problem and won't tend to go looking for a lib for a colour type (Ie, struct { ubyte r, g, b;}... simple! *cough*). They probably will use it if it's in phobos though for compatibility reasons. Image processing is super easy by contrast. One of my first port-of-calls after this work would be a graph/plot library... I'd use the hell out of that. I constantly want to output data visualisations! It also enables seamless interaction with realtime rendering api's. Ie, if OpenGL/D3D/Vulkan bindings support phobos supplied colour types, then image decoding libraries, and other sources of image data become easy to render without any requirement for adaptation. Rendering libs and image/video data source libs are almost always separate, independent libraries. We need to maximise probability that they are compatible out of the gate. I didn't want to touch 'images' in this work, because there are countless subjective designs to present those API's, but I don't think there's many competing designs for a colour library, so it felt it was the best and most bang-for-buck starting place overall. Also, I think this has very high synergy with ndslice. I think ndslice + color is the foundation for image processing, or at least image data transport. Thanks for the rationale. I'm glad to see you're planning follow-on libraries! I'd use a graph/plot library myself. Some of the rationale above should go into the documentation for the library.  Sep 12 2016 jmh530 <john.michael.hall gmail.com> writes: On Monday, 12 September 2016 at 22:55:01 UTC, Walter Bright wrote: Thanks for the rationale. I'm glad to see you're planning follow-on libraries! I'd use a graph/plot library myself. Some of the rationale above should go into the documentation for the library. You might try https://github.com/BlackEdder/ggplotd  Sep 12 2016 Guillaume Piolat <first.last gmail.com> writes: On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. - Manu My thoughts: - I've wanted a function like colorFromString many times. It's especially nice with the added #RGBA and #RRGGBBAA syntax that eg. SVG lacks. - The knowledge encoded in this library is great in itself. - This could form the basis of the next-gen DbI image library so it is much needed. => I would certainly vote for inclusion.  Sep 12 2016 Edwin van Leeuwen <edder tkwsping.nl> writes: On Monday, 12 September 2016 at 19:55:57 UTC, Guillaume Piolat wrote: - I've wanted a function like colorFromString many times. It's especially nice with the added #RGBA and #RRGGBBAA syntax that eg. SVG lacks. What happens when the string is invalid? Does it throw an error?  Sep 12 2016 Guillaume Piolat <first.last gmail.com> writes: On Monday, 12 September 2016 at 19:59:58 UTC, Edwin van Leeuwen wrote: On Monday, 12 September 2016 at 19:55:57 UTC, Guillaume Piolat wrote: - I've wanted a function like colorFromString many times. It's especially nice with the added #RGBA and #RRGGBBAA syntax that eg. SVG lacks. What happens when the string is invalid? Does it throw an error? It throws an Exception, like it should.  Sep 12 2016 John Colvin <john.loughran.colvin gmail.com> writes: On Monday, 12 September 2016 at 19:59:58 UTC, Edwin van Leeuwen wrote: On Monday, 12 September 2016 at 19:55:57 UTC, Guillaume Piolat wrote: - I've wanted a function like colorFromString many times. It's especially nice with the added #RGBA and #RRGGBBAA syntax that eg. SVG lacks. What happens when the string is invalid? Does it throw an error? It should be integrated with std.conv.parse and std.conv.to and follow their conventions.  Sep 12 2016 Walter Bright <newshound2 digitalmars.com> writes: On 9/12/2016 2:08 PM, John Colvin wrote: On Monday, 12 September 2016 at 19:59:58 UTC, Edwin van Leeuwen wrote: On Monday, 12 September 2016 at 19:55:57 UTC, Guillaume Piolat wrote: - I've wanted a function like colorFromString many times. It's especially nice with the added #RGBA and #RRGGBBAA syntax that eg. SVG lacks. What happens when the string is invalid? Does it throw an error? It should be integrated with std.conv.parse and std.conv.to and follow their conventions. I'm not so sure. Follow the conventions, ok, but integrating them in - they are already large and complex.  Sep 12 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 13 September 2016 at 12:05, Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote: On 9/12/2016 2:08 PM, John Colvin wrote: On Monday, 12 September 2016 at 19:59:58 UTC, Edwin van Leeuwen wrote: On Monday, 12 September 2016 at 19:55:57 UTC, Guillaume Piolat wrote: - I've wanted a function like colorFromString many times. It's especially nice with the added #RGBA and #RRGGBBAA syntax that eg. SVG lacks. What happens when the string is invalid? Does it throw an error? It should be integrated with std.conv.parse and std.conv.to and follow their conventions. I'm not so sure. Follow the conventions, ok, but integrating them in - they are already large and complex. std.conv.to seems to 'just work' for color type conversion. I'd like to support string -> color conversion via std.conv.to; ie: auto c = "#FFFF00".to!RGB8; Is there a simple way to do that?  Sep 12 2016 Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes: On 9/12/16 10:50 PM, Manu via Digitalmars-d wrote: std.conv.to seems to 'just work' for color type conversion. I'd like to support string -> color conversion via std.conv.to; ie: auto c = "#FFFF00".to!RGB8; Is there a simple way to do that? That was the purpose of toImpl, which I recall has been recently removed... -- Andrei  Sep 13 2016 ketmar <ketmar ketmar.no-ip.org> writes: On Tuesday, 13 September 2016 at 02:50:02 UTC, Manu wrote: std.conv.to seems to 'just work' for color type conversion. I'd like to support string -> color conversion via std.conv.to; ie: auto c = "#FFFF00".to!RGB8; Is there a simple way to do that? yes. just define a ctor that accepts string, and you're there. it works both for classes and structs.  Sep 13 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 13 September 2016 at 20:19, ketmar via Digitalmars-d <digitalmars-d puremagic.com> wrote: On Tuesday, 13 September 2016 at 02:50:02 UTC, Manu wrote: std.conv.to seems to 'just work' for color type conversion. I'd like to support string -> color conversion via std.conv.to; ie: auto c = "#FFFF00".to!RGB8; Is there a simple way to do that? yes. just define a ctor that accepts string, and you're there. it works both for classes and structs. Oh... it already works! :P  Sep 13 2016 Random D user <no email.com> writes: On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. - Manu Hi. I'm just a random forum lurker, but here's my feedback. It needs more docs/examples for the basic usage cases (i.e. how to skip docs and start working). Now at first glance it looks like working with RGBA (most common case (with srgb)) is complicated and while reading the docs I kind of felt like I'd just write my own. Few basic things I'd be interested in (examples): 1. How to pack/unpack color into uint/rgba channels. Also can I swizzle channels directly? How would I extract a single channel from color? 2. Can I just cast an array of bytes into a RGBA array and operate on that? 3. How to scale color channels e.g. 5 bits to 8 bits and vice versa. How to convert color formats? Can I combine scale/conversion with single channel or channel swizzling? 4. Which way are RGBA format names defined ("byte array order" r is first byte or "uint order" r is in high bits)? 5. Is it the same for packed rgba formats? I'm not sure if packedrgb is a necessary distinction. Aren't they all kind of packed (small structs)? There's no guarantee that 1 byte is not just a part of a bigger field. Btw. Your RGBA8 example is a bit weird: static assert(RGBA8("0x908000FF") == RGBA8(0x80,0x00,0xFF,0x90)); I'd assume RGBA8(r,g,b,a) would be either "uint order" r(0x90), g(0x80), b(0x00), a(0xFF) or "byte array order" (le) r(0xFF), g(0x00), b(0x80), a(0x90) 6. Perhaps you should define all the usual color formats, since everybody and their cat has their own preference for rgba channel orders. In my experience all these 4 variants for RGBA8888 are pretty common: RGBA8888 BGRA8888 ARGB8888 ABGR8888 and their X and 24bit variants. For 16 bits fairly common are: RGB565 and RGBA5551, also sometimes you see one of RGBA4444 permutations (like RGBA8 above). Also L16 is pretty common for heightfields or other data images/textures. I guess there are also planar formats, but I think those are more of a Image problem space. Just as a completely random idea - How about aliases for HDR formats like HDR10 and Dolby Vision? Kind of looks like they're just combinations what you already have.  Sep 12 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 13 September 2016 at 06:29, Random D user via Digitalmars-d <digitalmars-d puremagic.com> wrote: On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. - Manu Hi. I'm just a random forum lurker, but here's my feedback. It needs more docs/examples for the basic usage cases (i.e. how to skip docs and start working). Now at first glance it looks like working with RGBA (most common case (with srgb)) is complicated and while reading the docs I kind of felt like I'd just write my own. You'll get it wrong though! Can you describe what you perceive to be hard? import std.experimental.color; RGBA8 a(0xFF808000); RGBA8 b("GoldenRodYellow"); auto r = a + b; // do normal operations Or: auto image = new RGBA8[height][width]; image[x][y] = RGBA8("Aquamarine"); Few basic things I'd be interested in (examples): 1. How to pack/unpack color into uint/rgba channels. auto c = RGBA8(r, g, b, a); uint r = c.r.value; // 'r' is a 'NormalizedInt", so you need '.value' to get the raw integer value float g = cast(float)c.g; // cast to float knows about the normalized 0-1 range Also can I swizzle channels directly? I could add something like: auto brg = c.swizzle!"brg"; The result would be strongly typed to the new arrangement. How would I extract a single channel from color? uint r = c.r.value; // like this? 2. Can I just cast an array of bytes into a RGBA array and operate on that? Yes. Of course, if the image library returned an array of colors, you wouldn't have an array of raw bytes in the first place. 3. How to scale color channels e.g. 5 bits to 8 bits and vice versa. How to convert color formats? Can I combine scale/conversion with single channel or channel swizzling? alias CustomPackedColor = PackedRGB!("BGR_5_6_5", ubyte); // <- ordered bgr CustomPackedColor[] image = loadPackedImage(); auto unpacked = cast(RGB8)image[0]; // unpack and swizzle I'm not sure what you mean by "with single channel"? 4. Which way are RGBA format names defined ("byte array order" r is first byte or "uint order" r is in high bits)? I describe formats in byte (bit) order. It's the only reasonable way that works outside of a uint. 5. Is it the same for packed rgba formats? Yes. I'm not sure if packedrgb is a necessary distinction. Aren't they all kind of packed (small structs)? There's no guarantee that 1 byte is not just a part of a bigger field. RGBA8 has 4 ubytes, and you can operate on them. ubyte a, b; a += b; // <- is valid D code Unpacked types can do maths. Packed types can't do maths, they need to be unpacked before you can do anything, and you don't generally want the result to be immediately packed again. If I allowed maths on PackedRGB for instance, it would be expected that the result is re-packed at the end of every operation, and that would be useless. PackedRGB is for storage, not for doing work. This is why they're distinct. Btw. Your RGBA8 example is a bit weird: static assert(RGBA8("0x908000FF") == RGBA8(0x80,0x00,0xFF,0x90)); I'd assume RGBA8(r,g,b,a) would be either "uint order" r(0x90), g(0x80), b(0x00), a(0xFF) or "byte array order" (le) r(0xFF), g(0x00), b(0x80), a(0x90) I'm showing that parsing colors from string's follow the web convention. I can't redefine how colours are expressed on the web for decades. I think you've just highlighted that I need to remove the '0x' notation though, that is definitely misleading! Use '#', that should be clear. 6. Perhaps you should define all the usual color formats, since everybody and their cat has their own preference for rgba channel orders. In my experience all these 4 variants for RGBA8888 are pretty common: RGBA8888 BGRA8888 ARGB8888 ABGR8888 and their X and 24bit variants. I don't think they're common at all. Only 2 are common (one of which is deprecated), the other 2 are very niche. I was trying to balance a tasteful amount of instantiations. I don't really want to provide an instantiation of every single realtime format I've ever encountered out of the box. For 16 bits fairly common are: RGB565 and RGBA5551, also sometimes you see one of RGBA4444 permutations (like RGBA8 above). Nobody uses 16bit textures anymore. Compressed textures are both MUCH smaller, and generally higher quality. Also L16 is pretty common for heightfields or other data images/textures. I could probably put that one. I guess there are also planar formats, but I think those are more of a Image problem space. I think if people want to provide instantiations for convenience, it should be in the libs that interact with those formats. For instance, I imagine in OpenGL binding there might be: alias C = ColorFor!GL_R11F_G11F_B10F; // <- instantiation of the proper type for the GL enum. Most people really just want RGB8, or RGBA8. If you're interacting with PackedRGB, you're already in specialist territory. Just as a completely random idea - How about aliases for HDR formats like HDR10 and Dolby Vision? Kind of looks like they're just combinations what you already have. This is very specialist. Instantiations are expensive. If they're not used, it's a waste of compiler effort.  Sep 12 2016 John Colvin <john.loughran.colvin gmail.com> writes: On Tuesday, 13 September 2016 at 01:05:56 UTC, Manu wrote: Also can I swizzle channels directly? I could add something like: auto brg = c.swizzle!"brg"; The result would be strongly typed to the new arrangement. Perfect use-case for opDispatch like in gl3n.  Sep 13 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 13 September 2016 at 17:47, John Colvin via Digitalmars-d <digitalmars-d puremagic.com> wrote: On Tuesday, 13 September 2016 at 01:05:56 UTC, Manu wrote: Also can I swizzle channels directly? I could add something like: auto brg = c.swizzle!"brg"; The result would be strongly typed to the new arrangement. Perfect use-case for opDispatch like in gl3n. One trouble with arbitrary swizzling is that people often want to write expressions like "c.xyzy", or whatever, but repeating components... and that's not something my colours can do. It's useful in realtime code, but it doesn't mean anything, and you can't interpret that value as a colour anymore after you do that. This sort of thing is used when you're not actually storing colours in textures, but instead just some arbitrary data. Don't use a colour type for that, use a vector type instead. What that sort of swizzling really is, are vector operations, not colour operations. Maybe I could add an API to populate a vector from the components of colours, in-order... but then we don't actually have linear algebra functions in phobos either! So colour->vectors and arbitrary swizzling is probably not something we need immediately. In my lib, colours are colours. If you have BGR8 x and RGB8 y, and add them, you don't get x.b+y.r, x.g+y.g, x.r+y.b... that's not a colour operation, that's an element-wise vector operation.  Sep 13 2016 Guillaume Piolat <first.last gmail.com> writes: On Tuesday, 13 September 2016 at 09:31:53 UTC, Manu wrote: In my lib, colours are colours. If you have BGR8 x and RGB8 y, and add them, you don't get x.b+y.r, x.g+y.g, x.r+y.b... that's not a colour operation, that's an element-wise vector operation. I think swizzling is way overrated, especially swizzling assignment. Which is the most readable? color.rb = color.br; // swizzling swap(color.r, color.b); // no swizzling auto color = RGBA8(c.rgb, 255); // swizzling auto color = RGBA8(c.r, c.g, c.b, 255); // no swizzling And finally: auto rb = color.rgrb; // When do you ever need that?  Sep 13 2016 John Colvin <john.loughran.colvin gmail.com> writes: On Tuesday, 13 September 2016 at 09:31:53 UTC, Manu wrote: On 13 September 2016 at 17:47, John Colvin via Digitalmars-d <digitalmars-d puremagic.com> wrote: On Tuesday, 13 September 2016 at 01:05:56 UTC, Manu wrote: Also can I swizzle channels directly? I could add something like: auto brg = c.swizzle!"brg"; The result would be strongly typed to the new arrangement. Perfect use-case for opDispatch like in gl3n. One trouble with arbitrary swizzling is that people often want to write expressions like "c.xyzy", or whatever, but repeating components... and that's not something my colours can do. It's useful in realtime code, but it doesn't mean anything, and you can't interpret that value as a colour anymore after you do that. This sort of thing is used when you're not actually storing colours in textures, but instead just some arbitrary data. Don't use a colour type for that, use a vector type instead. What that sort of swizzling really is, are vector operations, not colour operations. Maybe I could add an API to populate a vector from the components of colours, in-order... but then we don't actually have linear algebra functions in phobos either! So colour->vectors and arbitrary swizzling is probably not something we need immediately. In my lib, colours are colours. If you have BGR8 x and RGB8 y, and add them, you don't get x.b+y.r, x.g+y.g, x.r+y.b... that's not a colour operation, that's an element-wise vector operation. Fair enough, you know much better than me how useful it would be. I was just suggesting that if you do support some sorts of swizzling then opDispatch would allow you to avoid users having to use strings. It would be as simple to implement as alias opDispatch = swizzle; given the swizzle function you were using before.  Sep 13 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 14 September 2016 at 00:56, John Colvin via Digitalmars-d <digitalmars-d puremagic.com> wrote: On Tuesday, 13 September 2016 at 09:31:53 UTC, Manu wrote: On 13 September 2016 at 17:47, John Colvin via Digitalmars-d <digitalmars-d puremagic.com> wrote: On Tuesday, 13 September 2016 at 01:05:56 UTC, Manu wrote: Also can I swizzle channels directly? I could add something like: auto brg = c.swizzle!"brg"; The result would be strongly typed to the new arrangement. Perfect use-case for opDispatch like in gl3n. One trouble with arbitrary swizzling is that people often want to write expressions like "c.xyzy", or whatever, but repeating components... and that's not something my colours can do. It's useful in realtime code, but it doesn't mean anything, and you can't interpret that value as a colour anymore after you do that. This sort of thing is used when you're not actually storing colours in textures, but instead just some arbitrary data. Don't use a colour type for that, use a vector type instead. What that sort of swizzling really is, are vector operations, not colour operations. Maybe I could add an API to populate a vector from the components of colours, in-order... but then we don't actually have linear algebra functions in phobos either! So colour->vectors and arbitrary swizzling is probably not something we need immediately. In my lib, colours are colours. If you have BGR8 x and RGB8 y, and add them, you don't get x.b+y.r, x.g+y.g, x.r+y.b... that's not a colour operation, that's an element-wise vector operation. Fair enough, you know much better than me how useful it would be. I was just suggesting that if you do support some sorts of swizzling then opDispatch would allow you to avoid users having to use strings. It would be as simple to implement as alias opDispatch = swizzle; given the swizzle function you were using before. Oh, it would be super-useful! Just that it's not a colour operation, its liner algebra... I think it's a job for another lib. Getting a colour as a vector is all that's needed.  Sep 13 2016 Random D user <no email.com> writes: On Tuesday, 13 September 2016 at 01:05:56 UTC, Manu wrote: Can you describe what you perceive to be hard? Well, I just skimmed through the docs and I didn't look at the code, so that sense it was an "honest" view for phobos proposal. Also I was trying to convey that based on the docs it "looks hard", although I was suspecting that it isn't really. So the list of things was a list of examples that I would've wanted to see in the docs. Maybe in an overview or "here's how you use it" page. I can read the details if I really need them, but I think it's important to have a cook book examples for quick start. In general, I think basics should be dead simple (even over simplified for the very basic case), but the API and the docs should provide me layers of increasing detail and sophistication, which I can study if (or when) I really need control and care about the details. Few basic things I'd be interested in (examples): 1. How to pack/unpack color into uint/rgba channels. auto c = RGBA8(r, g, b, a); uint r = c.r.value; // 'r' is a 'NormalizedInt", so you need '.value' to get the raw integer value float g = cast(float)c.g; // cast to float knows about the normalized 0-1 range Let's put these in the docs. Also can I swizzle channels directly? I could add something like: auto brg = c.swizzle!"brg"; It's just a nicety. I guess I could just define exact color formats and convert (let the lib do the rest), or just unpack and repack. Also the docs might want to mention which way is the preferred/designed way. 2. Can I just cast an array of bytes into a RGBA array and operate on that? Yes. Of course, if the image library returned an array of colors, you wouldn't have an array of raw bytes in the first place. Cool. I just wanted to make sure something like this possible. Also should be in the docs as an example, so that people don't have to try this out themselves. Or search through github for details. 3. How to scale color channels e.g. 5 bits to 8 bits and vice versa. How to convert color formats? Can I combine scale/conversion with single channel or channel swizzling? alias CustomPackedColor = PackedRGB!("BGR_5_6_5", ubyte); // <- ordered bgr CustomPackedColor[] image = loadPackedImage(); auto unpacked = cast(RGB8)image[0]; // unpack and swizzle I'm not sure what you mean by "with single channel"? For example if you first unpack RGBA_uint32 into R,G,B,A then you want to take G (or just swizzle color.g) and scale that to 5 bits or 16 bits for what ever use case you might have. Looks to me like this can be handled with something like PackedRGB!("G_5") and then do color conversion into that. 4. Which way are RGBA format names defined ("byte array order" r is first byte or "uint order" r is in high bits)? I describe formats in byte (bit) order. It's the only reasonable way that works outside of a uint. Agreed. This should be in the docs, so that it's clear to users. (especially, since RGBA8888 is the common case and it does fit into uint) I'm showing that parsing colors from string's follow the web convention. I can't redefine how colours are expressed on the web for decades. I think you've just highlighted that I need to remove the '0x' notation though, that is definitely misleading! Use '#', that should be clear. Right. I was interpreting 0x........ as c hex literal. You should document that it's following the web convention. 6. Perhaps you should define all the usual color formats, since everybody and their cat has their own preference for rgba channel orders. In my experience all these 4 variants for RGBA8888 are pretty common: RGBA8888 BGRA8888 ARGB8888 ABGR8888 and their X and 24bit variants. I don't think they're common at all. Only 2 are common (one of which is deprecated), the other 2 are very niche. Deprecated by who? Shouldn't phobos grade lib include all reasonable platforms? I agree that you probably don't see too much variation within windows APIs, images (BMP etc.) or with D3D GPU textures (they're compressed anyway and asset pipelines to do the conversions beforehand), but I was more of thinking image libraries and the numerous, sometimes old and quirky, image formats. Or perhaps someone wants to implement a software blitter for their favorite toy device/embedded project. I was trying to balance a tasteful amount of instantiations. I don't really want to provide an instantiation of every single realtime format I've ever encountered out of the box. Seems reasonable. For 16 bits fairly common are: RGB565 and RGBA5551, also sometimes you see one of RGBA4444 permutations (like RGBA8 above). Nobody uses 16bit textures anymore. Compressed textures are both MUCH smaller, and generally higher quality. Sure, agreed. These are not too useful as GPU textures these days (even on mobile), but if you do software 2d, load older image formats (image viewer etc.) or something even more specialized, these are pretty useful. I guess I was going for comprehensive (within reason), where as you were going for "90% of 2016 colors/image data", which is fine. Anyways, it would be nice to see color (and hopefully image + simple blitter) in phobos. There's no need to write the same convert/copy/stretch/blend loop over and over again. And cpu-side code is nice to have for quick prototyping and small-scale work. Just as a completely random idea - How about aliases for HDR formats like HDR10 and Dolby Vision? Kind of looks like they're just combinations what you already have. This is very specialist. Instantiations are expensive. If they're not used, it's a waste of compiler effort. HDR TVs and displays are coming fast and with that I'm sure HDR editing and photography will be common, which of course means HDR color formats and apps. But these can be added later.  Sep 13 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 14 September 2016 at 04:10, Random D user via Digitalmars-d <digitalmars-d puremagic.com> wrote: In general, I think basics should be dead simple (even over simplified for the very basic case) I think they are: import std.experimental.color; RGB8 color("red"); I don't think i's possible to make it simpler than that? but the API and the docs should provide me layers of increasing detail and sophistication, which I can study if (or when) I really need control and care about the details. I'll work on the docs some more. Deprecated by who? Shouldn't phobos grade lib include all reasonable platforms? They're all comprehensively supported, I just didn't pre-instantiate every possible format imaginable. That would be devastating to compile times, and have absolutely no value. BGR really just exists because WINAPI. Microsoft haven't recommended people write GDI based software for decades. Look at DX10(,11,12), BGR isn't even supported in modern DirectX. Microsoft left BGR to die years ago (and good riddance). That said, there is an instantiation for BGR in package.d... the reversed versions (ie, alpha first) are completely unusual; just some big-endian games consoles. Modern formats have the channels in order; RGB, this is true for all data types, ubyte, short, float, etc. I agree that you probably don't see too much variation within windows APIs, images (BMP etc.) or with D3D GPU textures (they're compressed anyway and asset pipelines to do the conversions beforehand), but I was more of thinking image libraries and the numerous, sometimes old and quirky, image formats. Or perhaps someone wants to implement a software blitter for their favorite toy device/embedded project. "favorite toy device/embedded project" <- this is the definition of specialist task. I don't mind if that person has to type 'alias' at the top of their app. For 16 bits fairly common are: RGB565 and RGBA5551, also sometimes you see one of RGBA4444 permutations (like RGBA8 above). Nobody uses 16bit textures anymore. Compressed textures are both MUCH smaller, and generally higher quality. Sure, agreed. These are not too useful as GPU textures these days (even on mobile), but if you do software 2d, load older image formats (image viewer etc.) or something even more specialized, these are pretty useful. I guess I was going for comprehensive (within reason), where as you were going for "90% of 2016 colors/image data", which is fine. There is comprehensive support for all these formats... I just didn't pre-instantiate the templates. It's not hard to do at the top of your program: alias MyRGBType = RGB!(...details...); Anyways, it would be nice to see color (and hopefully image + simple blitter) in phobos. I think that'll come. This is a pre-requisite. There's no need to write the same convert/copy/stretch/blend loop over and over again. And cpu-side code is nice to have for quick prototyping and small-scale work. Sure. Just as a completely random idea - How about aliases for HDR formats like HDR10 and Dolby Vision? Kind of looks like they're just combinations what you already have. This is very specialist. Instantiations are expensive. If they're not used, it's a waste of compiler effort. HDR TVs and displays are coming fast and with that I'm sure HDR editing and photography will be common, which of course means HDR color formats and apps. But these can be added later. There is already support for HDR. Use a floating point RGB with UHDTV colour space: alias HDR = RGB!("rgb", float, linear(?), RGBColorSpace.UHDTV); CompressedRGB also has comprehensive support for all the HDR formats I've ever encountered. My point was, I don't want to provide instantiations for these types... most people don't need it. People writing HDR rendering software will instantiate the types they need themselves. It's very specialist work.  Sep 13 2016 Walter Bright <newshound2 digitalmars.com> writes: On 9/11/2016 9:14 PM, Manu via Digitalmars-d wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. I have zero expertise in color programming, and have no idea what a color library might do or what it would be used for. I looked at the front page of the documentation. I would like to see some sort of explanation of who, what, where, why and how of a color library. Like what is a color space, etc. Maybe add some links to explanatory Wikipedia pages? "Set of colors defined by X11, adopted by the W3C, SVG, and other popular libraries." Would be a great place to add clickable links for the relevant X11, W3C, SVG, etc. specification pages. It really, really needs a "References:" section, with links like: https://en.wikipedia.org/wiki/RGB_color_model Are there any definitive books on the topic? What is a BGR color type? What is a UV color type? What is a UVW color type? What is an alpha color type? What is luminance? "This module implements HSV, HSL, HSI, HCY, HWB, HCG color types." "angular color spaces" Some references to what these are would be nice. Contrast with: http://dlang.org/phobos/std_regex.html where general information is given and links for more are provided (although that is weak and could be improved). ------- My last comment is what can I do with a color library that would not require some additional library?  Sep 12 2016 Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes: On 9/12/16 6:51 PM, Walter Bright wrote: My last comment is what can I do with a color library that would not require some additional library? Great question and well asked. It was mine as well but I could not put it as succinctly. -- Andrei  Sep 12 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 13 September 2016 at 08:51, Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote: On 9/11/2016 9:14 PM, Manu via Digitalmars-d wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. I have zero expertise in color programming, and have no idea what a color library might do or what it would be used for. I looked at the front page of the documentation. I would like to see some sort of explanation of who, what, where, why and how of a color library. Like what is a color space, etc. Maybe add some links to explanatory Wikipedia pages? "Set of colors defined by X11, adopted by the W3C, SVG, and other popular libraries." Would be a great place to add clickable links for the relevant X11, W3C, SVG, etc. specification pages. It really, really needs a "References:" section, with links like: https://en.wikipedia.org/wiki/RGB_color_model Are there any definitive books on the topic? What is a BGR color type? What is a UV color type? What is a UVW color type? What is an alpha color type? What is luminance? "This module implements HSV, HSL, HSI, HCY, HWB, HCG color types." "angular color spaces" Some references to what these are would be nice. Contrast with: http://dlang.org/phobos/std_regex.html where general information is given and links for more are provided (although that is weak and could be improved). Okay. I'll work on this feedback. My last comment is what can I do with a color library that would not require some additional library? With color, ndslice, and std.algorithm, you can do quite a lot of useful image processing already. But my point before was that I see this primarily as an enabler. We really need a graph/plot library for instance! Also, multimedia libraries will all become naturally compatible when the primitive working type is defined in phobos.  Sep 12 2016 Walter Bright <newshound2 digitalmars.com> writes: On 9/12/2016 6:11 PM, Manu via Digitalmars-d wrote: Okay. I'll work on this feedback. Wonderful! My last comment is what can I do with a color library that would not require some additional library? With color, ndslice, and std.algorithm, you can do quite a lot of useful image processing already. Ok, but I have no idea how. Perhaps include something as an example? Phobos in general lacks short, pithy, and *useful* examples. Such can go a long way towards selling the color library to users. But my point before was that I see this primarily as an enabler. We really need a graph/plot library for instance! Also, multimedia libraries will all become naturally compatible when the primitive working type is defined in phobos. I had read your post on this topic after making my post. I think it's a strong answer.  Sep 12 2016 Walter Bright <newshound2 digitalmars.com> writes: On 9/12/2016 7:03 PM, Walter Bright wrote: On 9/12/2016 6:11 PM, Manu via Digitalmars-d wrote: Okay. I'll work on this feedback. Wonderful! BTW, linking to authoritative reference material has the effect of reassuring the user that the library is designed by someone who knows his stuff. It has the same effect as notes and references appendices have in any good technical book.  Sep 12 2016 Andrea Fontana <nospam example.com> writes: On Monday, 12 September 2016 at 22:51:14 UTC, Walter Bright wrote: My last comment is what can I do with a color library that would not require some additional library? I think it's easy to do some image processing using netbpm [1] as format. Reading/Writing images doesn't require any other library. And using ndslices a lot of things can be done :) Andrea [1] https://en.wikipedia.org/wiki/Netpbm_format  Sep 13 2016 Guillaume Piolat <first.last gmail.com> writes: On Monday, 12 September 2016 at 22:51:14 UTC, Walter Bright wrote: My last comment is what can I do with a color library that would not require some additional library? Having correct conversions. Converting sRGB to linear RGB is typically not done correctly, pow isn't enough. It also dissipates most misunderstanding about color space (yes Lab also are dependent on the color point, it was something I wasn't sure about and this library says it clearly). Right now I rely on the OS for Apple RGB to sRGB conversion, this library would allow me to do it without using a system call. Other uses I can imagine: - Downsampling an image preserving colors (compressed RGB isn't necessarily the right sep to do those) - Perceptually correct color gradients. In font rendering, - HSV is very useful for colouring stuff in video games, it's what the color selection does in painter-programs like GIMP. So lots of library out there have a HSV <-> RGB conversions - converting to packed RGB is useful for programs using GPUs. Same reason than half-float support is useful.  Sep 13 2016 Meta <jared771 gmail.com> writes: On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. I added packed-RGB support, including weird micro-float and shared-exponent formats. They're important for interacting with any real-time rendering libraries. There is only one texture format I'm aware of that isn't supported, and that is u7e3 floats, only available on Xbox360 ;) Chromatic adaptation functions are in. I've done a tidy/reorg pass. I'm not sure what else should be in the scope of this lib. PR: https://github.com/dlang/phobos/pull/2845 dub: https://code.dlang.org/packages/color docs: http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html - Manu Looks like a nice and simple module. My only complaint is that DDOC sections such as Returns, Args, etc. don't seem to be used at all. Also maybe a bit more detailed of an overview in package.d, which I assume is the entry point for the documentation.  Sep 14 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 14 September 2016 at 22:47, Meta via Digitalmars-d <digitalmars-d puremagic.com> wrote: On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote: I think I'm about as happy with my colour lib as I'm going to be. It really needs reviews. I added packed-RGB support, including weird micro-float and shared-exponent formats. They're important for interacting with any real-time rendering libraries. There is only one texture format I'm aware of that isn't supported, and that is u7e3 floats, only available on Xbox360 ;) Chromatic adaptation functions are in. I've done a tidy/reorg pass. I'm not sure what else should be in the scope of this lib. PR: https://github.com/dlang/phobos/pull/2845 dub: https://code.dlang.org/packages/color docs: http://dtest.thecybershadow.net/artifact/website-04a64e024cf75be39700bebd3a50d26f6c7bd163-7185c8ec7b15c9e785880cab6d512e6f/web/library-prerelease/std/experimental/color.html - Manu Looks like a nice and simple module. My only complaint is that DDOC sections such as Returns, Args, etc. don't seem to be used at all. Also maybe a bit more detailed of an overview in package.d, which I assume is the entry point for the documentation. Cheers. Yeah, I need to do better with ddoc. ... I'm just gonna go on the record and say that I am really, really not enjoying ddoc ;) If I could get people starting to review the code, I'll polish up the docs over the next few days.  Sep 14 2016 Meta <jared771 gmail.com> writes: On Wednesday, 14 September 2016 at 13:28:23 UTC, Manu wrote: Cheers. Yeah, I need to do better with ddoc. ... I'm just gonna go on the record and say that I am really, really not enjoying ddoc ;) I can't say I'm a fan either.  Sep 14 2016 Marco Leise <Marco.Leise gmx.de> writes: Am Wed, 14 Sep 2016 13:47:43 +0000 schrieb Meta <jared771 gmail.com>: On Wednesday, 14 September 2016 at 13:28:23 UTC, Manu wrote: Cheers. Yeah, I need to do better with ddoc. ... I'm just gonna go on the record and say that I am really,=20 really not enjoying ddoc ;) =20 =20 I can't say I'm a fan either. You can replace$(D =E2=80=A6) with =E2=80=A6 nowadays. It made me a bit =
more
of a fan. :)

--=20
Marco

Sep 14 2016
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/14/2016 09:47 AM, Meta wrote:
On Wednesday, 14 September 2016 at 13:28:23 UTC, Manu wrote:
Cheers.
Yeah, I need to do better with ddoc.

... I'm just gonna go on the record and say that I am really, really
not enjoying ddoc ;)

I can't say I'm a fan either.

Why? -- Andrei

Sep 14 2016
Jacob Carlborg <doob me.com> writes:
On 2016-09-14 22:14, Andrei Alexandrescu wrote:

Why? -- Andrei

I've recently worked on a theme for Ddoc that will be used in TextMate.
For convince I started by adding the CSS inline in the head tag. Turns
out if you have CSS looking like this:

.foo {
font-weight: bold;
background-color: white;
}

Everything works perfectly fine. Then I added text color as well:

color: black;

Guess what happens? It cuts, it cuts everything after "color". The
solution is to use a colon macro:

Macros:
COLON = :

color$(COLON) black; It took me quite a while and a lot of frustration until I figured that out. But I guess that's an inherit problem with a macro system. That is what I'm missing from Ddoc: * Inline source code for each symbol [1]. The [1] example both has a button to show the source code inline and a direct link to GitHub * Link to GitHub for each symbol [1] and for the whole module. The way it's done now with$(PHOBOSSRC std/net/_isemail.d) is not standardized
and feels like a hack. Why slashes and why the underscore? If anything
it should be the fully qualified name. Ideally it should be automatic,
or a compiler recognized macro that inserts the link

* Ddoc should organize symbols by visibility [2]
* Ddoc should generate index table [3]
* Ddoc should organize a module by symbol category [4]
* Add a way to inherit documentation from the parent class [5]

* Ddoc should give a list of inherited members [6]. JavaDoc does this by
default [7]

* Consistent way of cross-referencing. Ideally one would just pass the
fully qualified name (or local name for symbols in the same module) to a
compiler recognized macro and it would generate the appropriate links

* An index page with all symbols, including a filter [8]. This is super
convenient when you know the symbol to use but you need to double check
how it's used or the behavior. I use it almost every day with Ruby

All this should be done within Ddoc without the need for any post
processing.

In general I feel that Ddoc is too much focused on a generic macro
system instead of focusing on being a good tool for writing
documentation for code.

[1]
http://api.rubyonrails.org/classes/ActiveRecord/FinderMethods.html#method-i-first

[2] https://issues.dlang.org/show_bug.cgi?id=11893
[3] https://issues.dlang.org/show_bug.cgi?id=11891
[4] https://issues.dlang.org/show_bug.cgi?id=11892
[5] https://issues.dlang.org/show_bug.cgi?id=10399
[6] https://issues.dlang.org/show_bug.cgi?id=7688

[7]
http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.jdt.doc.isv%2Freference%2Fapi%2Forg%2Feclipse%2Fjdt%2Fcore%2Fdom%2FSimpleName.html

[8] http://ruby-doc.org/core-2.3.1

--
/Jacob Carlborg

Sep 14 2016
Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
Do we mind moving this discussion to the other topic I started
actually on this topic?

It's slightly upsetting that this discussion about ddoc is more
popular than the colour lib I spent months workinng on :(

On 15 September 2016 at 16:55, Jacob Carlborg via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
On 2016-09-14 22:14, Andrei Alexandrescu wrote:

Why? -- Andrei

I've recently worked on a theme for Ddoc that will be used in TextMate. For
convince I started by adding the CSS inline in the head tag. Turns out if
you have CSS looking like this:

.foo {
font-weight: bold;
background-color: white;
}

Everything works perfectly fine. Then I added text color as well:

color: black;

Guess what happens? It cuts, it cuts everything after "color". The solution
is to use a colon macro:

Macros:
COLON = :

color$(COLON) black; It took me quite a while and a lot of frustration until I figured that out. But I guess that's an inherit problem with a macro system. That is what I'm missing from Ddoc: * Inline source code for each symbol [1]. The [1] example both has a button to show the source code inline and a direct link to GitHub * Link to GitHub for each symbol [1] and for the whole module. The way it's done now with$(PHOBOSSRC std/net/_isemail.d) is not standardized and feels
like a hack. Why slashes and why the underscore? If anything it should be
the fully qualified name. Ideally it should be automatic, or a compiler
recognized macro that inserts the link

* Ddoc should organize symbols by visibility [2]
* Ddoc should generate index table [3]
* Ddoc should organize a module by symbol category [4]
* Add a way to inherit documentation from the parent class [5]

* Ddoc should give a list of inherited members [6]. JavaDoc does this by
default [7]

* Consistent way of cross-referencing. Ideally one would just pass the fully
qualified name (or local name for symbols in the same module) to a compiler
recognized macro and it would generate the appropriate links

* An index page with all symbols, including a filter [8]. This is super
convenient when you know the symbol to use but you need to double check how
it's used or the behavior. I use it almost every day with Ruby

All this should be done within Ddoc without the need for any post
processing.

In general I feel that Ddoc is too much focused on a generic macro system
instead of focusing on being a good tool for writing documentation for code.

[1]
http://api.rubyonrails.org/classes/ActiveRecord/FinderMethods.html#method-i-first

[2] https://issues.dlang.org/show_bug.cgi?id=11893
[3] https://issues.dlang.org/show_bug.cgi?id=11891
[4] https://issues.dlang.org/show_bug.cgi?id=11892
[5] https://issues.dlang.org/show_bug.cgi?id=10399
[6] https://issues.dlang.org/show_bug.cgi?id=7688

[7]
http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.jdt.doc.isv%2Freference%2Fapi%2Forg%2Feclipse%2Fjdt%2Fcore%2Fdom%2FSimpleName.html

[8] http://ruby-doc.org/core-2.3.1

--
/Jacob Carlborg


Sep 15 2016
Jacob Carlborg <doob me.com> writes:
On 2016-09-15 09:11, Manu via Digitalmars-d wrote:
Do we mind moving this discussion to the other topic I started
actually on this topic?

It's slightly upsetting that this discussion about ddoc is more
popular than the colour lib I spent months workinng on :(

Sorry.

Is it the "ddoc latex/formulas?" topic? That's the only recent topic by
you related to ddoc. But I have no interest in either latex or formulas.

--
/Jacob Carlborg

Sep 15 2016
Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 15 September 2016 at 19:23, Jacob Carlborg via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
On 2016-09-15 09:11, Manu via Digitalmars-d wrote:
Do we mind moving this discussion to the other topic I started
actually on this topic?

It's slightly upsetting that this discussion about ddoc is more
popular than the colour lib I spent months workinng on :(

Sorry.

Is it the "ddoc latex/formulas?" topic? That's the only recent topic by you
related to ddoc. But I have no interest in either latex or formulas.

Yeah, sorry I forgot that detail. That thread meandered in the same
direction though.

Sep 15 2016
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/15/2016 02:55 AM, Jacob Carlborg wrote:
On 2016-09-14 22:14, Andrei Alexandrescu wrote:

Why? -- Andrei

I've recently worked on a theme for Ddoc that will be used in TextMate.
For convince I started by adding the CSS inline in the head tag. Turns
out if you have CSS looking like this:

.foo {
font-weight: bold;
background-color: white;
}

Everything works perfectly fine. Then I added text color as well:

color: black;

Guess what happens? It cuts, it cuts everything after "color". The
solution is to use a colon macro:

Macros:
COLON = :

color$(COLON) black; It took me quite a while and a lot of frustration until I figured that out. But I guess that's an inherit problem with a macro system. Yah, this is terrible (it happens with http: all the time). We need to suppress the special meaning of ":" to only the cases when it's preceded by only one word and followed by a newline, e,g, "\ncolor:\n". That is what I'm missing from Ddoc: * Inline source code for each symbol [1]. The [1] example both has a button to show the source code inline and a direct link to GitHub Can this be done by defining the expansion macros appropriately? * Link to GitHub for each symbol [1] and for the whole module. The way it's done now with$(PHOBOSSRC std/net/_isemail.d) is not standardized
and feels like a hack. Why slashes and why the underscore?

Slashes are tolerable. The underscore I agree it's a quirk.

If anything
it should be the fully qualified name.

That'd be nice.

Ideally it should be automatic,
or a compiler recognized macro that inserts the link

Ironically the underscore is necessary because ddoc does too much
automatically.

* Ddoc should organize symbols by visibility [2]
* Ddoc should generate index table [3]
* Ddoc should organize a module by symbol category [4]
* Add a way to inherit documentation from the parent class [5]

* Ddoc should give a list of inherited members [6]. JavaDoc does this by
default [7]

* Consistent way of cross-referencing. Ideally one would just pass the
fully qualified name (or local name for symbols in the same module) to a
compiler recognized macro and it would generate the appropriate links

* An index page with all symbols, including a filter [8]. This is super
convenient when you know the symbol to use but you need to double check
how it's used or the behavior. I use it almost every day with Ruby

All this should be done within Ddoc without the need for any post
processing.

what capabilities need to be added to obviate the need for postprocessing.

In general I feel that Ddoc is too much focused on a generic macro
system instead of focusing on being a good tool for writing
documentation for code.

That's a fair assessment, and an actionable one.

Andrei

Sep 15 2016
Jacob Carlborg <doob me.com> writes:
On 2016-09-15 14:54, Andrei Alexandrescu wrote:

Yah, this is terrible (it happens with http: all the time). We need to
suppress the special meaning of ":" to only the cases when it's preceded
by only one word and followed by a newline, e,g, "\ncolor:\n".

It's useful for many of the sections to allow the section name on the
same line as the content:

/// Returns: some useful value
int foo();

See the Ddoc documentation for more examples [1].

Can this be done by defining the expansion macros appropriately?

What are "the expansion macros" ?

Slashes are tolerable. The underscore I agree it's a quirk.

There are several (other) problems:

* How do I know I should use slashes? There's no documentation
* It's not a standard macro. If it's not here [2] nobody will find it or
use it

That's a fair assessment, and an actionable one.

Perhaps the current implementation of Ddoc could be moved to a separate
tool that would focus on being a macro system. That can still be used
for the site and other documents. Ddoc in the compiler could be improved
to be a better tool for documenting code. I think these are two separate
use cases that might not be best accomplished with the same tool.

[1] http://dlang.org/spec/ddoc.html#standard_srction
[2] http://dlang.org/spec/ddoc.html#predefined_macros

--
/Jacob Carlborg

Sep 15 2016
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/15/2016 02:34 PM, Jacob Carlborg wrote:
On 2016-09-15 14:54, Andrei Alexandrescu wrote:

Yah, this is terrible (it happens with http: all the time). We need to
suppress the special meaning of ":" to only the cases when it's preceded
by only one word and followed by a newline, e,g, "\ncolor:\n".

It's useful for many of the sections to allow the section name on the
same line as the content:

/// Returns: some useful value
int foo();

See the Ddoc documentation for more examples [1].

That's fine because "Returns" is special-cased. Random words at the
beginning of a line followed by ":" should not receive special treatment.

Can this be done by defining the expansion macros appropriately?

What are "the expansion macros" ?

Those DDOC_XXX things that are used for formatting declarations.

Slashes are tolerable. The underscore I agree it's a quirk.

There are several (other) problems:

* How do I know I should use slashes? There's no documentation

"Monkey see, monkey do." :o)

* It's not a standard macro. If it's not here [2] nobody will find it or
use it

Guess we can improve the (d)docs.

That's a fair assessment, and an actionable one.

Perhaps the current implementation of Ddoc could be moved to a separate
tool that would focus on being a macro system. That can still be used
for the site and other documents. Ddoc in the compiler could be improved
to be a better tool for documenting code. I think these are two separate
use cases that might not be best accomplished with the same tool.

That'd be neat. As always, somebody's got to do it...

Andrei

Sep 15 2016
Jacob Carlborg <doob me.com> writes:
On 2016-09-15 20:55, Andrei Alexandrescu wrote:

That's fine because "Returns" is special-cased. Random words at the
beginning of a line followed by ":" should not receive special treatment.

Ah, so you mean all non-standard non-special sections? Basically any
section that is not listed in the ddoc documentation.

Those DDOC_XXX things that are used for formatting declarations.

Hmm, I don't think there's a DDOC_ macro that evaluates to the full
source code a given symbol. For a function that would include the source
code of the body.

Guess we can improve the (d)docs.

But then we need to have a standardized macro for that.

That'd be neat. As always, somebody's got to do it...

As always.

--
/Jacob Carlborg

Sep 16 2016
Meta <jared771 gmail.com> writes:
On Wednesday, 14 September 2016 at 20:14:48 UTC, Andrei
Alexandrescu wrote:
On 09/14/2016 09:47 AM, Meta wrote:
On Wednesday, 14 September 2016 at 13:28:23 UTC, Manu wrote:
Cheers.
Yeah, I need to do better with ddoc.

... I'm just gonna go on the record and say that I am really,
really
not enjoying ddoc ;)

I can't say I'm a fan either.

Why? -- Andrei

The syntax is clunky and it's poorly documented. I've been
lurking this mailing list since late 2012 and sporadically
contributing since late 2013, and I had no idea you could do
something like your example for Manu using DDOC. It seems like a
real case of NiH syndrome; there are plenty of more widely used
systems that would probably fit D just fine, but instead we have
our own special macro documentation language. Really the only
thing DDOC has going for it in comparison to something like
Doxygen, IMO, is momentum. It's just too much of a pain to switch
now.

Sep 15 2016
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/14/2016 09:28 AM, Manu via Digitalmars-d wrote:
... I'm just gonna go on the record and say that I am really, really
not enjoying ddoc ;)

I was using ddoc today for the Greeks in DIP1000 and was musing about
what makes it annoying. I think a major part is the macro invocation
syntax: $(MACRO a1, a2, a3). After writing it for a few dozen times today, I figured it's objectively awkward to type (at least on the US keyboard): the dollar sign followed by the left paren is just unpleasant for mechanical reasons. The resulting look is alien too, as opposed to the more familiar$MACRO(a1, a2, a3) or the CPP-style MACRO(a1, a2, a3).

There are a couple of un(der)documented tricks without which life is
very difficult. For example, people commonly believe that in order to
embed a literal comma they need to go like this:

COMMA = ,

and then $(MACRO Hello$(COMMA) world, a2). That scales poorly with the
size of the argument. In fact, there's a much better solution:

ARGS = $0 and then$(MACRO $(ARGS Hello, world), a2) allows any amount of text with commas. Sadly no similar solution for unpaired parens. Also, the fact that the doc opens with the text and then has an obscure "Macros:" line at a point is unpleasant. The macro definitions should come first, followed by a separator and then the content. That would make for a much nicer flow. There are a few other macros I never leave home without: ARGS =$0
COMMA = ,
DOLLAR = $LPAREN = ( RPAREN = ) TAIL =$+
COLON = :
LF = $(LF)$(LF)
COMMENT =
SP = $(COMMENT space coming)$(COMMENT that was a space)

Overall: I'm trying to build a list of ddoc grievances that can be
fixed. It's the de jure and de facto standard for D documentation (with
the ddoc processor or ddox), so we'd do good to improve it.

Andrei

Sep 14 2016
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/14/2016 05:31 PM, Andrei Alexandrescu wrote:
After writing it for a few dozen times today, I figured it's objectively
awkward to type (at least on the US keyboard): the dollar sign followed
by the left paren is just unpleasant for mechanical reasons.

The resulting look is alien too, as opposed to the more familiar
$MACRO(a1, a2, a3) or the CPP-style MACRO(a1, a2, a3). I meant to add here: we can improve this in a backward-compatible way by allowing users to choose syntax by means of some notation, e.g. this thing as the first macro: DD0C_SYNTAX =$(X)

for the existing syntax, vs.

DD0C_SYNTAX = $X() for the$MACRO(a1, a2) syntax. We may also allow users to define the
argument separator, also on a per-document basis. By default:

DDOC_ARGUMENT_SEPARATOR = ,

We may also allow custom parens and so on.

Andrei

Sep 14 2016
bachmeier <no spam.net> writes:
On Wednesday, 14 September 2016 at 21:31:51 UTC, Andrei
Alexandrescu wrote:

Andrei

One thing that I didn't like when I used DDOC was that I couldn't
find a way to add a block of text directly to the output file. I
know that can be done with html, but it's not the same. When I am
programming I want to make notes (the way I make comments) but
not get bogged down with all the macros/html stuff. Unformatted
text is way better than unwritten text. Blocks of markdown would
be better, but blocks of text would have helped.

[Maybe this is possible but I was a new D user at that time and
haven't had a reason to look into it.]

Sep 14 2016
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/14/16 7:04 PM, bachmeier wrote:
On Wednesday, 14 September 2016 at 21:31:51 UTC, Andrei Alexandrescu wrote:

Andrei

One thing that I didn't like when I used DDOC was that I couldn't find a
way to add a block of text directly to the output file. I know that can
be done with html, but it's not the same. When I am programming I want
to make notes (the way I make comments) but not get bogged down with all
the macros/html stuff. Unformatted text is way better than unwritten
text. Blocks of markdown would be better, but blocks of text would have
helped.

[Maybe this is possible but I was a new D user at that time and haven't
had a reason to look into it.]

Do you mean like a macro $(VERBATIM ... text with balanced parens ...)? -- Andrei  Sep 14 2016 bachmeier <no spam.net> writes: On Thursday, 15 September 2016 at 00:42:57 UTC, Andrei Alexandrescu wrote: Do you mean like a macro$(VERBATIM ... text with balanced
parens ...)? -- Andrei

Yes. Unfortunately I cannot find any mention of it on this page:
http://dlang.org/spec/ddoc.html

Sep 15 2016
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/15/2016 11:25 AM, bachmeier wrote:
On Thursday, 15 September 2016 at 00:42:57 UTC, Andrei Alexandrescu wrote:

Do you mean like a macro $(VERBATIM ... text with balanced parens ...)? -- Andrei Yes. Unfortunately I cannot find any mention of it on this page: http://dlang.org/spec/ddoc.html It doesn't exist :o). I agree it's needed, I've hit the matter a few times. See https://issues.dlang.org/show_bug.cgi?id=16498. -- Andrei  Sep 15 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 15 September 2016 at 07:31, Andrei Alexandrescu via Digitalmars-d <digitalmars-d puremagic.com> wrote: On 09/14/2016 09:28 AM, Manu via Digitalmars-d wrote: ... I'm just gonna go on the record and say that I am really, really not enjoying ddoc ;) I was using ddoc today for the Greeks in DIP1000 and was musing about what makes it annoying. I think a major part is the macro invocation syntax:$(MACRO a1, a2, a3). After writing it for a few dozen times today, I figured
it's objectively awkward to type (at least on the US keyboard): the dollar
sign followed by the left paren is just unpleasant for mechanical reasons.

The resulting look is alien too, as opposed to the more familiar $MACRO(a1, a2, a3) or the CPP-style MACRO(a1, a2, a3). There are a couple of un(der)documented tricks without which life is very difficult. For example, people commonly believe that in order to embed a literal comma they need to go like this: COMMA = , and then$(MACRO Hello$(COMMA) world, a2). That scales poorly with the size of the argument. In fact, there's a much better solution: ARGS =$0

and then $(MACRO$(ARGS Hello, world), a2) allows any amount of text with
commas. Sadly no similar solution for unpaired parens.

Also, the fact that the doc opens with the text and then has an obscure
"Macros:" line at a point is unpleasant. The macro definitions should come
first, followed by a separator and then the content. That would make for a
much nicer flow.

There are a few other macros I never leave home without:

ARGS = $0 COMMA = , DOLLAR =$
LPAREN = (
RPAREN = )
TAIL = $+ COLON = : LF =$(LF)
$(LF) COMMENT = SP =$(COMMENT space coming) $(COMMENT that was a space) TAIL sadly doesn't work properly. Overall: I'm trying to build a list of ddoc grievances that can be fixed. It's the de jure and de facto standard for D documentation (with the ddoc processor or ddox), so we'd do good to improve it. Please add your own ideas. I haven't had enough experience for meaningful input. I have experienced the comma and parens things myself, and I generally just hate that everything is a macro, like you said yourself above. It didn't occur to me to attempt the sort of workarounds you illustrated here, I rather just factored the commas and parens out of my docs and moved on ;) Like, I really just don't care enough to try and understand ddoc sufficiently to have a bag of tricks like those you demonstrated above to workaround these issues. It's not a skill I *want* to possess, rather, in this case, it's a skill I'm being forced into. You're welcome to call me lazy, I'd suggest I'm being realistic. Perhaps a phobos contributor will be required to work it out (as seems to be a requirement for me right now), but normal programmers wouldn't. In 7 years, I've never been motivated to find workarounds to ddoc shortcomings before now, and now, it's only because people are hassling me. There's no intrinsic motivation here... one reason I use D is because I hate the C preprocessor ;) If I had to suggest, I'd introduce doxygen style \tags alongside the macros, then when people try and type docs in the way they've been doing for decades, it'll just work, and they can get on with their code. Nobody likes writing documentation, it needs to have the minimum possible friction or people just won't. Added bonus; when you cut&paste C code to adapt bindings, the existing docs on the C code would remain valid documentation without additional amendment. I bind C code regularly, and my goto 'solution' is to just remove the doxygen doco comments from the translated code and point them to the C/C++ docs instead. Again, call me what you like, I think my behaviour is probably typical though. I'm just being honest.  Sep 14 2016 Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes: On 9/14/16 9:28 PM, Manu via Digitalmars-d wrote: Like, I really just don't care enough to try and understand ddoc sufficiently to have a bag of tricks like those you demonstrated above to workaround these issues. It's not a skill I *want* to possess, rather, in this case, it's a skill I'm being forced into. You're welcome to call me lazy, I'd suggest I'm being realistic. Perhaps a phobos contributor will be required to work it out (as seems to be a requirement for me right now), but normal programmers wouldn't. In 7 years, I've never been motivated to find workarounds to ddoc shortcomings before now, and now, it's only because people are hassling me. There's no intrinsic motivation here... one reason I use D is because I hate the C preprocessor ;) I don't see those workarounds to ddoc shortcomings. m4 and all macro systems I know of have similar idioms. It's the nature of macro processing. Are you simply saying you're familiar with other documentation tools and are not motivated to get into another one? That's entirely fair. If I had to suggest, I'd introduce doxygen style \tags alongside the macros, then when people try and type docs in the way they've been doing for decades, it'll just work, and they can get on with their code. Nobody likes writing documentation, it needs to have the minimum possible friction or people just won't. I have difficulty understanding this. I haven't looked at Doxygen in a long time and never really used it, but from what I see at https://www.stack.nl/~dimitri/doxygen/manual/commands.html it seems the \tags you refer to are just a form of macros. The syntax is different, i.e. you'd write \a hello whereas in html you'd write <i>hello</i>, in latex \textit{hello}, in ddoc$(I hello). It's a matter of syntax and
though I agree syntax matters (and it would be nice to make ddoc's more
configurable), is it right to assume you simply want a syntax you're
more familiar with?

How do you mean people "type docs in the way they've been doing for
decades"? Are you implying doxygen not only has been around for decades,
but it's been some sort of ubiquitous standard? Honest question, I'm
definitely not getting that.

Does this boil down to - if we replace the macro syntax with one closer
to doxygen things will just click? (That may as well be the case.)

Andrei

Sep 14 2016
Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 15 September 2016 at 13:49, Andrei Alexandrescu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
On 9/14/16 9:28 PM, Manu via Digitalmars-d wrote:
Like, I really just don't care enough to try and understand ddoc
sufficiently to have a bag of tricks like those you demonstrated above
to workaround these issues. It's not a skill I *want* to possess,
rather, in this case, it's a skill I'm being forced into.
You're welcome to call me lazy, I'd suggest I'm being realistic.
Perhaps a phobos contributor will be required to work it out (as seems
to be a requirement for me right now), but normal programmers
wouldn't. In 7 years, I've never been motivated to find workarounds to
ddoc shortcomings before now, and now, it's only because people are
hassling me. There's no intrinsic motivation here... one reason I use
D is because I hate the C preprocessor ;)

I don't see those workarounds to ddoc shortcomings. m4 and all macro systems
I know of have similar idioms. It's the nature of macro processing. Are you
simply saying you're familiar with other documentation tools and are not
motivated to get into another one? That's entirely fair.

That's absolutely what I'm saying, but not only that, given that I'm
being forced into it now, I'm actively not enjoying it.
I understand it's a macro system, and I'm saying that's the problem as
I perceive it :)

I clearly recall when I learned doxygen. I generally liked it, and I
was motivated to populate my code with docs.
Perhaps there's an obvious counter-pressure in this case since I
already know doxygen, but I think that's insignificant compared to the
fact that I just don't like ddoc.
This macro system is not what I would have done, and it's just not
compelling. I don't want documentation-realted macro definitions in my
code. I also hate the paren spam.

If I had to suggest, I'd introduce doxygen style \tags alongside the
macros, then when people try and type docs in the way they've been
doing for decades, it'll just work, and they can get on with their
code. Nobody likes writing documentation, it needs to have the minimum
possible friction or people just won't.

I have difficulty understanding this. I haven't looked at Doxygen in a long
time and never really used it, but from what I see at
https://www.stack.nl/~dimitri/doxygen/manual/commands.html it seems the
\tags you refer to are just a form of macros. The syntax is different, i.e.
you'd write \a hello whereas in html you'd write <i>hello</i>, in latex
\textit{hello}, in ddoc $(I hello). It's a matter of syntax and though I agree syntax matters (and it would be nice to make ddoc's more configurable), is it right to assume you simply want a syntax you're more familiar with? I think of those examples, doxygen is objectively the least annoying syntax! It's 2 bytes, and doesn't introduce delimiters or parens of any kind. Sure, that's definitely an influence, I hate paren spam, but it's not all. If in practise I was able to make my docs be how I want as easily as I was able to the first time I encountered doxygen, I wouldn't be bothered. But I've had a number of contacts with ddoc now, and it's just not intuitive how to make my docs how I want, whereas with doxygen, I was walking within minutes, seconds even. I think I have a sense of uneasy-ness that comes from my intuition that macros are macros, and I don't know which ones are 'special', or which are just macros. Ie, some macros are obviously not actually macros, but keywords in disguise. Doxygen says things like "\a word" (style this as an argument), ddoc seems to be$(I word) (italic this), which says nothing about the
semantic meaning of the thing allowing styling to be deferred.
There's nothing there that says how to style a template argument for
instance, it's all 'italic, big, small, bold' etc, and skimming other
ddoc code, I see people using raw styling like $(I ..) (as you just demonstrated above). At face value, it feels clumsy and poorly defined. I'm sure I'm incorrect, but I just didn't have this same first-impression with doxygen, or C#'s doco standard (they're the only 2 others I've used before). The only reference I've ever looked at for doxygen is this page: https://www.stack.nl/~dimitri/doxygen/manual/commands.html I never read another line from their manuals, everything else emerged via osmosis, and I think osmosis is the correct way a programmer should learn this stuff, which means it needs to be as obvious and intuitive as possible. How do you mean people "type docs in the way they've been doing for decades"? Are you implying doxygen not only has been around for decades, but it's been some sort of ubiquitous standard? Honest question, I'm definitely not getting that. Do you feel that doxygen is not a ubiquitous standard? It's been around for like 20 years. I can't think of any non-doxygen docs I interact with regularly other than Microsoft docs... Maybe this is just a C/C++ ecosystem thing? Does this boil down to - if we replace the macro syntax with one closer to doxygen things will just click? (That may as well be the case.) Well, I think the core of my uneasy-ness is that the macros have no clear semantic meaning. Obviously I can design my own set of macros that express my semantic meaning... to hell with standards! ;) Your prior post said "here's a bunch of macros I never leave home without", that's exactly the problem as I see it. I don't want too see ddoc macro definition blocks appearing at random locations in my source code. And I'll also remind you of my past criticism that my solution to porting C code to D, is to block delete the doxygen comment blocks rather than bothering to port them to D. If I had a reason to say what you said above "if we replace the macro syntax with one closer to doxygen...", this is that reason, but it's the fact that they're macros rather than keywords that makes me upset, so replacing the macros with a different syntax probably won't make me feel less uneasy. Perhaps my uneasy-ness would subside if the list of 'builtins' (https://dlang.org/spec/ddoc.html) was more comprehensive (comparable to doxygen commands), and I never saw a ddoc macro definition in code anywhere. My gripe with the syntax would probably have passed without comment if that were the case and I would have just gotten used to it. And don't get me wrong, I think it's absolutely fantastic that D has a built-in documentation system. I just don't really like it.  Sep 15 2016 Walter Bright <newshound2 digitalmars.com> writes: On 9/15/2016 12:06 AM, Manu via Digitalmars-d wrote: I think I have a sense of uneasy-ness that comes from my intuition that macros are macros, and I don't know which ones are 'special', or which are just macros. That's easy, they are all just macros, and can all be redefined.  Sep 15 2016 Manu via Digitalmars-d <digitalmars-d puremagic.com> writes: On 15 September 2016 at 17:38, Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote: On 9/15/2016 12:06 AM, Manu via Digitalmars-d wrote: I think I have a sense of uneasy-ness that comes from my intuition that macros are macros, and I don't know which ones are 'special', or which are just macros. That's easy, they are all just macros, and can all be redefined. Yeah... I suspected this was the case ;)  Sep 15 2016 Patrick Schluter <Patrick.Schluter bbox.fr> writes: On Thursday, 15 September 2016 at 03:49:55 UTC, Andrei Alexandrescu wrote: On 9/14/16 9:28 PM, Manu via Digitalmars-d wrote: Like, I really just don't care enough to try and understand ddoc sufficiently to have a bag of tricks like those you demonstrated above to workaround these issues. It's not a skill I *want* to possess, rather, in this case, it's a skill I'm being forced into. You're welcome to call me lazy, I'd suggest I'm being realistic. Perhaps a phobos contributor will be required to work it out (as seems to be a requirement for me right now), but normal programmers wouldn't. In 7 years, I've never been motivated to find workarounds to ddoc shortcomings before now, and now, it's only because people are hassling me. There's no intrinsic motivation here... one reason I use D is because I hate the C preprocessor ;) I don't see those workarounds to ddoc shortcomings. m4 and all macro systems I know of have similar idioms. It's the nature of macro processing. Are you simply saying you're familiar with other documentation tools and are not motivated to get into another one? That's entirely fair. If I had to suggest, I'd introduce doxygen style \tags alongside the macros, then when people try and type docs in the way they've been doing for decades, it'll just work, and they can get on with their code. Nobody likes writing documentation, it needs to have the minimum possible friction or people just won't. I have difficulty understanding this. I haven't looked at Doxygen in a long time and never really used it, but from what I see at https://www.stack.nl/~dimitri/doxygen/manual/commands.html it seems the \tags you refer to are just a form of macros. The syntax is different, i.e. you'd write \a hello whereas in html you'd write <i>hello</i>, in latex \textit{hello}, in ddoc$(I
hello). It's a matter of syntax and though I agree syntax
matters (and it would be nice to make ddoc's more
configurable), is it right to assume you simply want a syntax
you're more familiar with?

How do you mean people "type docs in the way they've been doing
for decades"? Are you implying doxygen not only has been around
for decades, but it's been some sort of ubiquitous standard?
Honest question, I'm definitely not getting that.

doxygen is mostly javadoc compatible and that one is the standard.
(in fact the \ can be replaced by   every where) and it allows
direct html tags. The good thing about doxygen is that it
supports also markdown.
There are also some niceties like auto brief and autoreferencing
(when a symbol is written in a text will be clickable if its body
was reachable during generation of the doc).

Does this boil down to - if we replace the macro syntax with
one closer to doxygen things will just click? (That may as well
be the case.)

Probably. But be reminded of your remark in the SDLang vs. JSON
"...this is a strategic error. Please throw s/SDL/DDOC/ away and
use a standardized file format. -- Andrei

Sep 15 2016
Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Thursday, 15 September 2016 at 09:46:07 UTC, Patrick Schluter
wrote:
There are also some niceties like auto brief and
autoreferencing (when a symbol is written in a text will be
clickable if its body was reachable during generation of the
doc).

Not only that, by using tag files [1] Doxygen even generates
clickable symbols to third party libraries and other online

[1]
http://www.stack.nl/~dimitri/doxygen/manual/config.html#config_external

Sep 15 2016
Walter Bright <newshound2 digitalmars.com> writes:
On 9/14/2016 2:31 PM, Andrei Alexandrescu wrote:
The resulting look is alien too, as opposed to the more familiar $MACRO(a1, a2, a3) or the CPP-style MACRO(a1, a2, a3). The$(MACRO a1, a2, a3) syntax comes from makefiles.

Sep 14 2016
Walter Bright <newshound2 digitalmars.com> writes:
On 9/14/2016 11:11 PM, Walter Bright wrote:
On 9/14/2016 2:31 PM, Andrei Alexandrescu wrote:
The resulting look is alien too, as opposed to the more familiar $MACRO(a1, a2, a3) or the CPP-style MACRO(a1, a2, a3). The$(MACRO a1, a2, a3) syntax comes from makefiles.

The original idea was to be able to write documentation without needing many
macros, if any. Such as this I wrote recently:

/**
* Provides a safe interface to the Posix stat() function.
*
* Detailed description that I didn't write for this function,
* but this is where it goes.
*
* Another paragraph.
*
* Params:
*	path = file name
*	buf = stat instance to be filled in
* Returns:
*	0 for success, system errno for error
*/

I suspect that adding Markdown would help a lot (yes, I changed my mind about
that). Also auto-detecting email addresses and urls and turning them into

Sep 14 2016
Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 15 September 2016 at 16:44, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
On 9/14/2016 11:11 PM, Walter Bright wrote:
On 9/14/2016 2:31 PM, Andrei Alexandrescu wrote:
The resulting look is alien too, as opposed to the more familiar
$MACRO(a1, a2, a3) or the CPP-style MACRO(a1, a2, a3). The$(MACRO a1, a2, a3) syntax comes from makefiles.

The original idea was to be able to write documentation without needing many
macros, if any. Such as this I wrote recently:

/**
* Provides a safe interface to the Posix stat() function.
*
* Detailed description that I didn't write for this function,
* but this is where it goes.
*
* Another paragraph.
*
* Params:
*      path = file name
*      buf = stat instance to be filled in
* Returns:
*      0 for success, system errno for error
*/

I suspect that adding Markdown would help a lot (yes, I changed my mind

+1

Sep 15 2016
On Thursday, 15 September 2016 at 06:44:16 UTC, Walter Bright
wrote:
On 9/14/2016 11:11 PM, Walter Bright wrote:

I suspect that adding Markdown would help a lot (yes, I changed
urls and turning them into clickable links would help.

I **really** love the pragmatism of this man!

;-)

/Paolo

Sep 15 2016
Marco Leise <Marco.Leise gmx.de> writes:
Am Wed, 14 Sep 2016 23:44:16 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

I suspect that adding Markdown would help a lot (yes, I changed my mind about
that).

+1

--
Marco

Sep 15 2016
Nemanja Boric <4burgos gmail.com> writes:
On Thursday, 15 September 2016 at 09:45:26 UTC, Marco Leise wrote:
Am Wed, 14 Sep 2016 23:44:16 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

I suspect that adding Markdown would help a lot (yes, I changed

https://github.com/kiith-sa/harbored-mod provides support for
mixing ddoc and markdown. That's what we're using, and it's nice
(especially because docs in source files look very close to
generated ones), but it would be nicer if we could just use dmd
for that, of course.

Sep 15 2016
Walter Bright <newshound2 digitalmars.com> writes:
On 9/15/2016 4:59 AM, Nemanja Boric wrote:
On Thursday, 15 September 2016 at 09:45:26 UTC, Marco Leise wrote:
Am Wed, 14 Sep 2016 23:44:16 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

I suspect that adding Markdown would help a lot (yes, I changed my mind about
that).

https://github.com/kiith-sa/harbored-mod provides support for mixing ddoc and
markdown. That's what we're using, and it's nice (especially because docs in
source files look very close to generated ones), but it would be nicer if we
could just use dmd for that, of course.

It looks like you've done some great work with this! And Boost licensed, too! I
think we can leverage this for Ddoc markdown support. Thanks!

Sep 15 2016
Jacob Carlborg <doob me.com> writes:
On 2016-09-15 08:44, Walter Bright wrote:

The original idea was to be able to write documentation without needing
many macros, if any. Such as this I wrote recently:

/**
* Provides a safe interface to the Posix stat() function.
*
* Detailed description that I didn't write for this function,
* but this is where it goes.
*
* Another paragraph.
*
* Params:
*    path = file name
*    buf = stat instance to be filled in
* Returns:
*    0 for success, system errno for error
*/

I suspect that adding Markdown would help a lot (yes, I changed my mind
them into clickable links would help.

Yeah, if you already have defined all the standard ddoc macros because
the default styling is not useful at all.

This is an example of the documentation generator for Ruby called Yard
[1]. The default output view/style might not be the best in the world
but I think it's pretty good. Here's an example of how to write the
documentation [2].

[1] http://www.rubydoc.info/gems/httparty/0.14.0/HTTParty/ClassMethods
[2] http://www.rubydoc.info/gems/yard/file/docs/GettingStarted.md

--
/Jacob Carlborg

Sep 15 2016
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/15/2016 10:37 AM, Jacob Carlborg wrote:
Yeah, if you already have defined all the standard ddoc macros because
the default styling is not useful at all.

Glad to hear that. It becomes a matter of improving the default-provided
macros and/or distribute some .ddoc batteries.

Jacob, I'd also like to thank for the great constructive ideas for
improving things and for formulating them as enhancement requests on
bugsilla. This is the kind of communication that pushes things forward.

Andrei

Sep 15 2016
Jacob Carlborg <doob me.com> writes:
On 2016-09-15 16:45, Andrei Alexandrescu wrote:

Glad to hear that. It becomes a matter of improving the default-provided
macros and/or distribute some .ddoc batteries.

Jacob, I'd also like to thank for the great constructive ideas for
improving things and for formulating them as enhancement requests on
bugsilla. This is the kind of communication that pushes things forward.

The oldest bugzilla issue I referred to is more than four years old and
this is not the first discussion on the topic :(

--
/Jacob Carlborg

Sep 15 2016
Walter Bright <newshound2 digitalmars.com> writes:
On 9/15/2016 7:37 AM, Jacob Carlborg wrote:
Yeah, if you already have defined all the standard ddoc macros because the
default styling is not useful at all.

The default styling was set up to produce legible output while not requiring
any
configuration file, nor a .css style file nor any knowledge of HTML at all.

Sep 15 2016
Jacob Carlborg <doob me.com> writes:
On 2016-09-16 02:02, Walter Bright wrote:

The default styling was set up to produce legible output while not
requiring any configuration file, nor a .css style file nor any
knowledge of HTML at all.

I don't see how that's relevant. Currently one needs to have knowledge
of HTML and CSS because the default output is not usable.

With Yard (documentation generator for Ruby) one just invokes "yard -f
html -o output intput.rb" and it generates all files and put them into
"output". The result looks something like this [1].

How is that requiring any HTML or CSS knowledge?

[1]
http://www.rubydoc.info/gems/rails-bootstrap-helpers/0.1.0/RailsBootstrapHelpers/Helpers/ButtonHelper

--
/Jacob Carlborg

Sep 16 2016
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/16/16 7:30 AM, Jacob Carlborg wrote:
I don't see how that's relevant. Currently one needs to have knowledge
of HTML and CSS because the default output is not usable.

Jacob, would you want to try your hand at tweaking the defaults? They
are in a string in https://github.com/dlang/dmd/blob/master/src/doc.d.
Clearly there's agreement these can be improved, and it's incredibly
more productive to just improve the content instead of arguing how it
could. Thanks!! -- Andrei

Sep 16 2016
Jacob Carlborg <doob me.com> writes:
On 2016-09-16 16:39, Andrei Alexandrescu wrote:

Jacob, would you want to try your hand at tweaking the defaults? They
are in a string in https://github.com/dlang/dmd/blob/master/src/doc.d.
Clearly there's agreement these can be improved, and it's incredibly
more productive to just improve the content instead of arguing how it
could. Thanks!! -- Andrei

My design skills are quite limited. The theme I'm using for TextMate is
basically the same as Apple is using for their documentation. But now
when I tried to find a link to it, I see that they've changed the design.

If I can use what I already have it might not be that much work.

Two questions:

* Can we have DMD output a separate CSS file? What will avoid the colon
issue mentioned before and avoid the need to have 300-500 lines of CSS
code in each HTML document

* Can we create a new macro which will be used for each source code line
in examples enclosed in "---"? That would allow to add line numbers with CSS

--
/Jacob Carlborg

Sep 16 2016
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/16/2016 12:07 PM, Jacob Carlborg wrote:
On 2016-09-16 16:39, Andrei Alexandrescu wrote:

Jacob, would you want to try your hand at tweaking the defaults? They
are in a string in https://github.com/dlang/dmd/blob/master/src/doc.d.
Clearly there's agreement these can be improved, and it's incredibly
more productive to just improve the content instead of arguing how it
could. Thanks!! -- Andrei

My design skills are quite limited. The theme I'm using for TextMate is
basically the same as Apple is using for their documentation. But now
when I tried to find a link to it, I see that they've changed the design.

If I can use what I already have it might not be that much work.

That would be awesome!

Two questions:

* Can we have DMD output a separate CSS file? What will avoid the colon
issue mentioned before and avoid the need to have 300-500 lines of CSS
code in each HTML document

I'd say we start without a separate CSS file. When the need for it
becomes obvious, we can do that. What would be great is that your
initiative isn't forestalled by the need for some other work.

* Can we create a new macro which will be used for each source code line
in examples enclosed in "---"? That would allow to add line numbers with
CSS

We can, and we should (there are js solutions too). Again, let's get the
ball rolling and amplify this as needed.

Thanks!

Andrei

Sep 16 2016
Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/15/16 2:11 AM, Walter Bright wrote:
On 9/14/2016 2:31 PM, Andrei Alexandrescu wrote:
The resulting look is alien too, as opposed to the more familiar
$MACRO(a1, a2, a3) or the CPP-style MACRO(a1, a2, a3). The$(MACRO a1, a2, a3) syntax comes from makefiles.

In the words of Seinfeld, "And that doesn't help either!"

I was thinking more about it and figured we could introduce a special
macro at the very beginning of a .ddoc file or "Macros:" section called
DDOC_MACRO_CALL_SYNTAX. It may be defined one of the following:

DDOC_MACRO_CALL_SYNTAX = DDOC

That's today's syntax.

DDOC_MACRO_CALL_SYNTAX = CPP

A macro will be invoked as MACRO(a1, a2).

DDOC_MACRO_CALL_SYNTAX = $CPP A macro will be invoked as$MACRO(a1, a2)

DDOC_MACRO_CALL_SYNTAX = LaTeX

Macro invocation is \MACRO{a1}{a2}.

This definition is modular, i.e. it doesn't carry beyond the current
document (otherwise the semantics of a .dd file depends on the .ddoc
file(s) it's used with).

Macro definitions stay the same using $1,$2 etc. but macros invoked by
them would need to obey the DDOC_MACRO_CALL_SYNTAX defined at the top.

Thoughts?

Andrei

Sep 15 2016
Walter Bright <newshound2 digitalmars.com> writes:
Started a new thread on this, "Ddoc macro syntax". Please direct further
discussion there.

Sep 15 2016
Walter Bright <newshound2 digitalmars.com> writes:
On 9/14/2016 5:47 AM, Meta wrote:
DDOC sections such as Returns, Args, etc.

s/Args/Params/

Sep 14 2016
Johan Engelen <j j.nl> writes:
On Monday, 12 September 2016 at 04:14:27 UTC, Manu wrote:
I think I'm about as happy with my colour lib as I'm going to
be. It really needs reviews.

I don't know enough about colors and what people do with it to
give you helpful feedback, but I was wondering if you had looked
at the (auto) vectorization of the code.
I see some matrix math code, which perhaps would benefit from
LDC's  fastmath (or similar). It's just a thought. You can shoot
me some example that you think should vectorize.

Anyway, I think you are mostly interested in API feedback,
implementation optimization can come later.

(others: let's stop depressing Manu further and move all ddoc and
other irrelevant discussions to other topics! :)

Sep 16 2016