www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - How D could gain more traction?

reply "XavierAP" <n3minis-git yahoo.es> writes:
Hi people. I've followed D for many years, although I haven't 
used it for anything big or even have a good knowledge of Phobos. 
I currently work in C++ a lot, although I am convinced that in 
theory the only reason not to run away from C++ is being tied 
down to a large existing codebase; and yet I'm also convinced 
that I couldn't sell D to my team or manager. So why's that?

In my opinion the biggest and only mistake is to preach D to the 
C++ choir only. Of course some will say, we can't avoid this, the 
native choir is the only one D should or could preach to. Even if 
this is true or not, I said C++, not native, choir. Nowadays 
those two continue to be synonyms, but the point of D is that 
they shouldn't. More on this below.

All the many and severe flaws in the C++ language are fixed in D, 
but the shortcomings of the C++ standard library are not. Some 
will say there are no fundamental flaws in it that don't come 
from the language, and I agree. However it has the shortcoming of 
being too narrow. The D standard library has followed the C++ 
paradigm, but C++ is quite useless without Qt, Boost etc. The 
only reason why this hasn't caused more detriment to C++ is 
(besides these 3rd-party libraries being by now quite established 
and thus standard de facto), that there is no native alternative 
to C++, except D, which has the same library problem. I've heard 
people say that things such as Qt or whatever not belonging in 
the standard library because a cross-platform language should not 
assume the existence of a screen, mouse or keyboard in the 
device. That's good and all as a statement (embedded devices can 
implement only part of the standard library, as is the case in 
.NET and Java) but it's not even a purist point of view, it's 
just outdated. In theory and practice nowadays it makes almost 
less sense in comparison to assume the existence (or relevance) 
of a text console.

Nowadays a standard library should include classes or functions, 
not only for data structures, algorithms etc., but also for: GUI 
cross-platform creation, graphics, multi-threading at low and 
high level, SQL, XML, JSON, networking on all layers from raw 
sockets to TPC and HTTP, FTP, etc... etc.; and since the reason 
for native is performance, I would also throw in some advanced 
math (linear algebra, Newton-Raphson, etc.) If you don't agree 
that this should be included in a standard library, I think 
you're kind of still sitting in the C++ choir even if you prefer 
D.

At this point I would of course not propose to expand the std 
namespace. It would be as simple as dlang.org endorsing chosen 
existing libraries for these core functionalities, and afterwards 
some publicity work.

Otherwise try to convince a _business_ guy that switching to D 
will have any significant advantage (he will implicitly expect 
you to "prove it in advance"). These all-in-one solution standard 
libraries are in my opinion the main reason of the huge success 
of .NET, Java, Visual Basic 1-6, huge in absolute terms and 
certainly compared to D. In my company (SKF, 50k employees 
worldwide) they use C++ but they are expanding the use of WinDev! 
(I won't blame you if you don't know what the latter is, just 
google it.) Managers know only when a project is late.

Your thoughts?
Apr 15 2015
next sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 15 April 2015 at 09:39:09 UTC, XavierAP wrote:

 Nowadays a standard library should include classes or 
 functions, not only for data structures, algorithms etc., but 
 also for: GUI cross-platform creation, graphics, 
 multi-threading at low and high level, SQL, XML, JSON, 
 networking on all layers from raw sockets to TPC and HTTP, FTP, 
 etc... etc.; and since the reason for native is performance, I 
 would also throw in some advanced math (linear algebra, 
 Newton-Raphson, etc.) If you don't agree that this should be 
 included in a standard library, I think you're kind of still 
 sitting in the C++ choir even if you prefer D.


Andrei Alexandrescu has said that he wants the kitchen sink 
approach for the standard library. While it might not go quite as 
far as you would like, the direction seems the same.
Apr 15 2015
parent reply "weaselcat" <weaselcat gmail.com> writes:
On Wednesday, 15 April 2015 at 09:57:29 UTC, John Colvin wrote:

 On Wednesday, 15 April 2015 at 09:39:09 UTC, XavierAP wrote:

 Nowadays a standard library should include classes or 
 functions, not only for data structures, algorithms etc., but 
 also for: GUI cross-platform creation, graphics, 
 multi-threading at low and high level, SQL, XML, JSON, 
 networking on all layers from raw sockets to TPC and HTTP, 
 FTP, etc... etc.; and since the reason for native is 
 performance, I would also throw in some advanced math (linear 
 algebra, Newton-Raphson, etc.) If you don't agree that this 
 should be included in a standard library, I think you're kind 
 of still sitting in the C++ choir even if you prefer D.


 Andrei Alexandrescu has said that he wants the kitchen sink 
 approach for the standard library. While it might not go quite 
 as far as you would like, the direction seems the same.


The problem with a kitchen sink approach is that you have to make 
sure the libraries stay up to date - and phobos already has a few 
rotting modules.

See: python, many people actively avoid using the standard 
library in favor of third party libraries that accomplish the 
same task.
Apr 15 2015
next sibling parent reply "Panke" <tobias pankrath.net> writes:
 The problem with a kitchen sink approach is that you have to 
 make sure the libraries stay up to date - and phobos already 
 has a few rotting modules.


Which modules do you mean? There are some modules that are 
considered to have a bad design or lacking implementation, but 
none that do not work anymore with the newest DMD.


 See: python, many people actively avoid using the standard 
 library in favor of third party libraries that accomplish the 
 same task.


There will always be a better solution in the future than the 
implementation included in the standard library today. However 
that is no argument against the kitchen sink.
Apr 15 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 15 April 2015 at 13:18:55 UTC, Panke wrote:

 There will always be a better solution in the future than the 
 implementation included in the standard library today. However 
 that is no argument against the kitchen sink.


That's dodging the foundational issue which is that different 
application areas have different needs. So then you either have 
to settle for a standard library that is heavily biased or that 
is only suitable for prototyping.
Apr 15 2015
parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
On Wednesday, 15 April 2015 at 13:40:36 UTC, Ola Fosheim Grøstad 
wrote:

 On Wednesday, 15 April 2015 at 13:18:55 UTC, Panke wrote:

 There will always be a better solution in the future than the 
 implementation included in the standard library today. However 
 that is no argument against the kitchen sink.


 That's dodging the foundational issue which is that different 
 application areas have different needs. So then you either have 
 to settle for a standard library that is heavily biased or that 
 is only suitable for prototyping.


No. We were talking about parts of e.g. the python standard 
library that have been completely superseded by third party 
solutions at least for new projects.
Apr 15 2015
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 15 April 2015 at 14:03:13 UTC, Tobias Pankrath 
wrote:

 No. We were talking about parts of e.g. the python standard 
 library that have been completely superseded by third party 
 solutions at least for new projects.


Well, that has not happend. What has happend is that people use a 
mismash of urllib, urlib2, cgi, (standard pythhon), webob, ++ web 
framework specific libraries for even the simplest web 
programming tasks.

It is a mess! But seasoned Python developers either joined the 
ride (thus already know where to look) or have their own 
abstractions, and if not (almost) EVERYTHING has one or several 
standard lib based solutions on stackoverflow.

That is a completely different scenario.
Apr 15 2015
prev sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 15 April 2015 at 13:06:26 UTC, weaselcat wrote:

 See: python, many people actively avoid using the standard 
 library in favor of third party libraries that accomplish the 
 same task.


Some third party libaries are considered defacto standard and are 
shipped with distributions or even App Engine that only allow 
pure Python. But the list is small and is the result of natural 
selection/evolution:

https://cloud.google.com/appengine/docs/python/tools/libraries27
Apr 15 2015
parent reply "XavierAP" <n3minis-git yahoo.es> writes:
On Wednesday, 15 April 2015 at 13:19:32 UTC, Ola Fosheim Grøstad 
wrote:

 On Wednesday, 15 April 2015 at 13:06:26 UTC, weaselcat wrote:

 See: python, many people actively avoid using the standard 
 library in favor of third party libraries that accomplish the 
 same task.


 Some third party libaries are considered defacto standard and 
 are shipped with distributions or even App Engine that only 
 allow pure Python. But the list is small and is the result of 
 natural selection/evolution:

 https://cloud.google.com/appengine/docs/python/tools/libraries27


That list is small compared to the whole of available Python 
libraries, but what you can do with it is already enormously more 
than you can do with Phobos or CRT+STL. If D had half of it 
people would be using it...
Apr 15 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 15 April 2015 at 14:01:54 UTC, XavierAP wrote:

 That list is small compared to the whole of available Python 
 libraries, but what you can do with it is already enormously 
 more than you can do with Phobos or CRT+STL. If D had half of 
 it people would be using it...


Yes, it is the stuff you need for web programming. I assume they 
were selected based on user pressure. (numpy,  lxml, PIL).

I think maybe a modular approach is better, to have different 
profiles:

1. foundational libraries (basic types)

2. architecture related libraries (embedded/cloud/desktop/mobile)

3. application level libraries (image processing etc)

Then you enforce dependencies in a way where lower level numbers 
don't depend on higher levels and give priority to the 
implementation of lower level (greater stability guarantees).
Apr 15 2015
parent reply "XavierAP" <n3minis-git yahoo.es> writes:
So many good ideas and points posted. Something should come out 
after this discussion...

On Wednesday, 15 April 2015 at 14:07:11 UTC, Ola Fosheim Grøstad 
wrote:

 I think maybe a modular approach is better, to have different 
 profiles:

 1. foundational libraries (basic types)

 2. architecture related libraries 
 (embedded/cloud/desktop/mobile)

 3. application level libraries (image processing etc)

 Then you enforce dependencies in a way where lower level 
 numbers don't depend on higher levels and give priority to the 
 implementation of lower level (greater stability guarantees).


That looks like good architecture. I'm not sure if 2 and 3 
wouldn't be at the same level of (non-) dependency? Or maybe it's 
the image processing example you've used. In any case 2 would 
have more priority because of practical reasons (necessity for a 
real-world standard library, and core-ness) and lack of 3rd-party 
alternatives.
Apr 15 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 15 April 2015 at 15:36:44 UTC, XavierAP wrote:

 That looks like good architecture. I'm not sure if 2 and 3 
 wouldn't be at the same level of (non-) dependency? Or maybe 
 it's the image processing example you've used. In any case 2 
 would have more priority because of practical reasons 
 (necessity for a real-world standard library, and core-ness) 
 and lack of 3rd-party alternatives.


I am not sure either, one will have to map out a graph with 
concrete examples to get it right.

But for instance in Python the problem is that the type system 
isn't "conceptual" so a library like numpy provides separate 
types for arrays of numbers etc. You don't replace "an interface 
abstraction" with an "optimized implementation" which would be 
the ideal high level thing to do. I think a system level language 
will suffer a similar faith, because fast image processing 
depends on OpenCL or SSE or Metal (iOS) that could affect the 
API/memory handling.

Desktop and unix-servers are fairly stable "architectures" due to 
the cost of reimplementation/installed base, but GPU/CPU 
integration is new. Mobile less stable since they are more like 
"fashion items" and being "new" is a selling point. Cloud is too 
heterogenous. Embedded is very much in flux (SoC etc). But even 
on the desktop/server how you handle network traffic has changed 
over the past few decades (despite no big changes in the 
networking protocols).

Image processing is also a hard nut to crack. Think just about 
height fields in 3D. Satellite images with N layer of high 
resolution or too large to fit in memory etc. Then you have run 
length encoding...
Apr 15 2015
parent reply "CraigDillabaugh" <craig.dillabaugh gmail.com> writes:
On Wednesday, 15 April 2015 at 15:49:10 UTC, Ola Fosheim Grøstad 
wrote:

 On Wednesday, 15 April 2015 at 15:36:44 UTC, XavierAP wrote:

 That looks like good architecture. I'm not sure if 2 and 3 
 wouldn't be at the same level of (non-) dependency? Or maybe 
 it's the image processing example you've used. In any case 2 
 would have more priority because of practical reasons 
 (necessity for a real-world standard library, and core-ness) 
 and lack of 3rd-party alternatives.




clip

 Image processing is also a hard nut to crack. Think just about 
 height fields in 3D. Satellite images with N layer of high 
 resolution or too large to fit in memory etc. Then you have run 
 length encoding...


For Satellite image processing at least GDAL can take care of 
most of the memory/image handling stuff. I have some D Bindings 
(largely untested).

https://github.com/craig-dillabaugh/gdal
Apr 15 2015
next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 15 April 2015 at 16:19:07 UTC, CraigDillabaugh 
wrote:

 For Satellite image processing at least GDAL can take care of 
 most of the memory/image handling stuff. I have some D Bindings 
 (largely untested).


Maybe a good idea would be to collect bindings for a single 
domain (like image processing) then see if it is possible to 
build an easy to use abstraction over it. If done for one domain 
at the time it might lead to something.

The basic idea being that all the registered external libraries 
can be used for simpler applications with an easier to use 
interface.

Obvious use case would be format conversion/feature extraction.
Apr 15 2015
prev sibling parent "XavierAP" <n3minis-git yahoo.es> writes:
At any rate I believe this modular / one domain at a time and 
layered approach would be the correct one, and setting 
priorities. Then we'd have to eventually start sooner than later 
filling out implementations, because the optimal previous design 
time is sadly always less than infinite. :o) The result would be 
useful and successful way long before it were half complete.

I would be willing to get in touch to see where this goes, if 
someone else wants. As I said my D knowledge is not so good yet, 
but I am interested and willing to invest limited amounts of time.

Also if you know of existing initiatives in this direction... For 
example, how is this webpage maintained? Is it curated in any 
way, or an all-included purely wiki list?
http://code.dlang.org/

On Wednesday, 15 April 2015 at 16:19:07 UTC, CraigDillabaugh 
wrote:

 On Wednesday, 15 April 2015 at 15:49:10 UTC, Ola Fosheim 
 Grøstad wrote:

 Image processing is also a hard nut to crack. Think just about 
 height fields in 3D. Satellite images with N layer of high 
 resolution or too large to fit in memory etc. Then you have 
 run length encoding...


 For Satellite image processing at least GDAL can take care of 
 most of the memory/image handling stuff. I have some D Bindings 
 (largely untested).

 https://github.com/craig-dillabaugh/gdal


3D image processing, now this is beyond the kitchen sink for 
standard library ideas.  ;-) Heck if D had a not too fancy but 
established standard GUI libary its popularity would already soar 
in relative terms.
Though a friend of mine happens to be a big specialist in 
precisely this field. :-)
Apr 15 2015
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 15 April 2015 at 09:39:09 UTC, XavierAP wrote:

 Otherwise try to convince a _business_ guy that switching to D 
 will have any significant advantage (he will implicitly expect 
 you to "prove it in advance"). These all-in-one solution 
 standard libraries are in my opinion the main reason of the 
 huge success of .NET, Java, Visual Basic 1-6, huge in absolute 
 terms and certainly compared to D.


Yeah, the notion of a standard library has shifted: in .net it's 
preset project items, on which IDE starts the package manager, 
which fetches the corresponding library and the item becomes 
ready to use. This way the user sees little difference to 
traditional notion of a standard library. I suppose java has even 
longer history of usage of 3rd party libraries.
Apr 15 2015
parent reply "XavierAP" <n3minis-git yahoo.es> writes:
I understand such a library collection would have many holes 
right now, but movement also creates its own momentum. I just 
think it would be good that dlang.org provided some more guidance.

I don't know, I hope to get some time on lazy Sundays to finally 
read Alexandrescu's book which I bought quite long ago, then 
start using D for more hobby projects and get to know more 
libraries and maybe get involved in the projects.

On Wednesday, 15 April 2015 at 10:05:37 UTC, Kagamin wrote:

 I suppose java has even longer history of usage of 3rd party 
 libraries.


Yeah but Java's standard library itself includes many such 
utilities, and I think it was the key ingredient for its success 
in the 90s, together with the multi-platform support, but imo 
even more instrumental.

But probably the best example to follow for D both in general and 
at this point, which is forgot to mention, is Python. Its success 
story is almost unbelievable: the language design is godawful, so 
many people like it, but I can't help thinking that they just 
love what they can do with it, because "there's one package for 
that" always which is standard de facto.

If D had got the same amount of community involvement, it would 
have a complete kitchen sink standard library collection by now. 
But I think those two things have to go in sync.
Apr 15 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 15 April 2015 at 11:45:37 UTC, XavierAP wrote:

 I understand such a library collection would have many holes 
 right now, but movement also creates its own momentum. I just 
 think it would be good that dlang.org provided some more 
 guidance.


Yes, there is a lot of overlap in the D community: 4 D compiler 
projects, a bunch of IDE projects, a bunch of GUI library 
projects, a bunch of (basic) game engine projects... All rather 
large in scope if you want to reach a complete state.

What is needed is a committee ;). But seriously, if you had a 
nice public  API design on paper then maybe people would flock 
behind the same project rather than creating their own. Unless 
they do it for educational reasons/entertainment.


 But probably the best example to follow for D both in general 
 and at this point, which is forgot to mention, is Python. Its 
 success story is almost unbelievable: the language design is 
 godawful, so many people like it, but I can't help thinking 
 that they just love what they can do with it, because "there's 
 one package for that" always which is standard de facto.


Python is pretty close to pseduo code and has managed to grow a 
community that cares about API design which the language supports 
fairly well. Since the language is inefficient and provides no 
static typing good programmers tend to write cleaner code in 
it... Bad language qualities can lead to better coding ;)
Apr 15 2015
next sibling parent "XavierAP" <n3minis-git yahoo.es> writes:
Exactly, and with the overlapping efforts the result is less than 
the division among the parts, because none of them reach critical 
mass.

But this has already been kind of done in the past regarding 
Tango vs. Phobos. I think it would be good to extend the same 
guidance to a gradually growing set of necessary libraries.

I was thinking less ambitiously about seeing what promising 
already existing projects can be used, but it may be worth it to 
design the APIs we'd like as you say and contrast them.
Apr 15 2015
prev sibling next sibling parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 16/04/2015 12:00 a.m., "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= 
<ola.fosheim.grostad+dlang gmail.com>" wrote:

 On Wednesday, 15 April 2015 at 11:45:37 UTC, XavierAP wrote:

 I understand such a library collection would have many holes right
 now, but movement also creates its own momentum. I just think it would
 be good that dlang.org provided some more guidance.


 Yes, there is a lot of overlap in the D community: 4 D compiler
 projects, a bunch of IDE projects, a bunch of GUI library projects, a
 bunch of (basic) game engine projects... All rather large in scope if
 you want to reach a complete state.

 What is needed is a committee ;). But seriously, if you had a nice
 public  API design on paper then maybe people would flock behind the
 same project rather than creating their own. Unless they do it for
 educational reasons/entertainment.


Nope. Good API isn't good enough. For things like window creation, image 
library ext. If they ain't in phobos, it ain't gonna get used.
That's my experience with Devisualization anyway.
Apr 15 2015
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 15 April 2015 at 12:36:16 UTC, Rikki Cattermole 
wrote:

 Nope. Good API isn't good enough. For things like window 
 creation, image library ext. If they ain't in phobos, it ain't 
 gonna get used.
 That's my experience with Devisualization anyway.


Go, Dart, Python and Javascript are good counter examples. But 
they are high level so they don't demand performance. C/C++ are 
also good counter examples, but as you can see, different 
system-level-applications need different kinds of libraries 
because there are unsurpassable 
API-complexity/performance/features trade offs.

People do use libraries if they are:
- high quality
- have existed for many years OR have major backing/support
- have a reasonable user base
- are well documented
- demonstrate ease of use in tutorials
- no lock in

Creating excellent idiomatic wrappers to already popular, mature 
and well supported C/C++ libraries is probably more "marketable" 
than selling home grown new borns...

Keep in mind that designing at generally useful library that also 
performs well in unknown scenarios is VERY difficult when you go 
beyond the leaves in the call-tree. Performance APIs for anything 
complex tend to demand nasty APIs or a DSL with builtin compiler. 
Finding the right trade offs take many iterations with use in 
real world applications.

Designing "leaf libraries" (mostly independent of the 
application) like an image loader, xml/json parser or compression 
library is relatively easy (but still difficult).  But still, 
even compression libraries have to provide fine grained buffering 
and control in order to work in many projects...

Basic stuff like a generic DB abstraction or (non posix) file i/o 
abstraction is impossible to get right due to differences in the 
underlying system.

(you also need a skilled benevolent dictator/janitor/custodian 
which takes both motivation, the right personality, experience... 
and time)
Apr 15 2015
prev sibling parent "CraigDillabaugh" <craig.dillabaugh gmail.com> writes:
On Wednesday, 15 April 2015 at 12:00:54 UTC, Ola Fosheim Grøstad 
wrote:
clip

 Yes, there is a lot of overlap in the D community: 4 D compiler 
 projects, a bunch of IDE projects, a bunch of GUI library 
 projects, a bunch of (basic) game engine projects... All rather 
 large in scope if you want to reach a complete state.

 What is needed is a committee ;). But seriously, if you had a 
 nice public  API design on paper then maybe people would flock 
 behind the same project rather than creating their own. Unless 
 they do it for educational reasons/entertainment.


clip

D biggest problem is that it is so fun to use it leads people to 
want to create their own _ insert project here _
Apr 15 2015
prev sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 15 April 2015 at 09:39:09 UTC, XavierAP wrote:

 also for: GUI cross-platform creation, graphics, 
 multi-threading at low and high level, SQL, XML, JSON, 
 networking on all layers from raw sockets to TPC and HTTP, FTP, 
 etc... etc.; and since the reason for native is performance, I 
 would also throw in some advanced math (linear algebra, 
 Newton-Raphson, etc.) If you don't agree that this should be 
 included in a standard library, I think you're kind of still 
 sitting in the C++ choir even if you prefer D.


It is a good idea fix the pavement before you build bikes and 
wheelchairs. C++ is still fixing the pavement... Large sections 
of Python 2's pavement is largely unused. If anything D should 
focus on the basics.
Apr 15 2015