www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - For those ready to take the challenge

reply "eles" <eles eles.com> writes:
https://codegolf.stackexchange.com/questions/44278/debunking-stroustrups-debunking-of-the-myth-c-is-for-large-complicated-pro
Jan 09 2015
next sibling parent reply Justin Whear <justin economicmodeling.com> writes:
On Fri, 09 Jan 2015 13:50:28 +0000, eles wrote:

 https://codegolf.stackexchange.com/questions/44278/debunking-
stroustrups-debunking-of-the-myth-c-is-for-large-complicated-pro Was excited to give it a try, then remembered...std.xml :(
Jan 09 2015
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Friday, 9 January 2015 at 16:55:30 UTC, Justin Whear wrote:
 Was excited to give it a try, then remembered...std.xml  :(
Well, as the author of my dom.d, I think it counts as a first party library when I use it! --- import arsd.dom; import std.net.curl; import std.stdio, std.algorithm; void main() { auto document = new Document(cast(string) get("http://www.stroustrup.com/C++.html")); writeln(document.querySelectorAll("a[href]").map!(a=>a.href)); } --- prints: [snip ... "http://www.morganstanley.com/", "http://www.cs.columbia.edu/", "http://www.cse.tamu.edu", "index.html", "C++.html", "bs_faq.html", "bs_faq2.html", "C++11FAQ.html", "papers.html", "4th.html", "Tour.html", "programming.html", "dne.html", "bio.html", "interviews.html", "applications.html", "glossary.html", "compilers.html"] Or perhaps better yet: import arsd.dom; import std.net.curl; import std.stdio; void main() { auto document = new Document(cast(string) get("http://www.stroustrup.com/C++.html")); foreach(a; document.querySelectorAll("a[href]")) writeln(a.href); } Which puts each one on a separate line.
Jan 09 2015
next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
Huh, looking at the answers on the website, they're mostly using 
regular expressions. Weaksauce. And wrong - they don't find ALL 
the links, they find the absolute HTTP urls!
Jan 09 2015
next sibling parent Justin Whear <justin economicmodeling.com> writes:
On Fri, 09 Jan 2015 17:18:42 +0000, Adam D. Ruppe wrote:

 Huh, looking at the answers on the website, they're mostly using regular
 expressions. Weaksauce. And wrong - they don't find ALL the links, they
 find the absolute HTTP urls!
Yes, I noticed that. `<script src="http://app.js"`></script>` isn't a "hyperlink". Wake up sheeple!
Jan 09 2015
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 9 January 2015 at 17:18:43 UTC, Adam D. Ruppe wrote:
 Huh, looking at the answers on the website, they're mostly 
 using regular expressions. Weaksauce. And wrong - they don't 
 find ALL the links, they find the absolute HTTP urls!
Yeah... Surprising, since languages like python includes a HTML parser in the standard library. Besides, if you want all resource links you have to do a lot better, since the following attributes can contain resource addresses: href, src, data, cite, xlink:href… You also need to do entity expansion since the links can contain html entities like "&amp;". Depressing.
Jan 10 2015
prev sibling parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
On Friday, 9 January 2015 at 17:18:43 UTC, Adam D. Ruppe wrote:
 Huh, looking at the answers on the website, they're mostly 
 using regular expressions. Weaksauce. And wrong - they don't 
 find ALL the links, they find the absolute HTTP urls!
Since it is a comparison of languages it's okay to match the original behaviour.
Jan 10 2015
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 10 January 2015 at 12:34:42 UTC, Tobias Pankrath 
wrote:
 Since it is a comparison of languages it's okay to match the 
 original behaviour.
I don't think this is really a great comparison of languages either though because it is gluing together a couple library tasks. Only a few bits about the actual language are showing through. In the given regex solutions, C++ has an advantage over C wherein the regex structure can be freed automatically in a destructor and a raw string literal in here, but that's about all from the language itself. The original one is kinda long because he didn't use a http get library, not because the language couldn't do one. There are bits where the language can make those libraries nicer too: dom.d uses operator overloading and opDispatch to support things like .attribute and also .attr.X and .style.foo and element["selector"].addClass("foo") and so on implemented in very very little code - I didn't have to manually list methods for the collection or properties for the attributes - ...but a library *could* do it that way and get similar results for the end user; the given posts wouldn't show that.
Jan 10 2015
parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
On Saturday, 10 January 2015 at 15:13:27 UTC, Adam D. Ruppe wrote:
 On Saturday, 10 January 2015 at 12:34:42 UTC, Tobias Pankrath 
 wrote:
 Since it is a comparison of languages it's okay to match the 
 original behaviour.
I don't think this is really a great comparison of languages either though because it is gluing together a couple library tasks. Only a few bits about the actual language are showing through. In the given regex solutions, C++ has an advantage over C wherein the regex structure can be freed automatically in a destructor and a raw string literal in here, but that's about all from the language itself. The original one is kinda long because he didn't use a http get library, not because the language couldn't do one. There are bits where the language can make those libraries nicer too: dom.d uses operator overloading and opDispatch to support things like .attribute and also .attr.X and .style.foo and element["selector"].addClass("foo") and so on implemented in very very little code - I didn't have to manually list methods for the collection or properties for the attributes - ...but a library *could* do it that way and get similar results for the end user; the given posts wouldn't show that.
I agree and one of the answers says:
 I think the "no third-party" assumption is a fallacy. And is a 
 specific fallacy that afflicts C++ developers, since it's so 
 hard to make reusable code in C++. When you are developing 
 anything at all, even if it's a small script, you will always 
 make use of whatever pieces of reusable code are available to 
 you.
 The thing is, in languages like Perl, Python, Ruby (to name a 
 few), reusing
 someone else's code is not only easy, but it is how most people 
 actually write code most of the time.
I think he's wrong, because it spoils the comparison. Every answer should delegate those tasks to a library that Stroustroup used as well, e.g. regex matching, string to number conversion and some kind of TCP sockets. But it must do the same work that he's solution does: Create and parse HTML header and extract the html links, probably using regex, but I wouldn't mind another solution. Everyone can put a libdo_the_stroustroup_thing on dub and then call do_the_stroustroup_thing() in main. To compare what the standard libraries (and libraries easily obtained or quasi standard) offer is another challenge.
Jan 10 2015
next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 10 January 2015 at 15:52:21 UTC, Tobias Pankrath 
wrote:
 But it must do the same work that he's solution does: Create 
 and parse HTML header and extract the html links, probably 
 using regex, but I wouldn't mind another solution.
Yeah, that would be best. BTW interesting line here: s << "GET " << "http://" + server + "/" + file << " HTTP/1.0\r\n"; s << "Host: " << server << "\r\n"; Why + instead of <<? C++'s usage of << is totally blargh to me anyway, but seeing both is even stranger. Weird language, weird library.
 Everyone can put a libdo_the_stroustroup_thing on dub and then 
 call do_the_stroustroup_thing() in main. To compare what the 
 standard libraries (and libraries easily obtained or quasi 
 standard) offer is another challenge.
Yeah.
Jan 10 2015
prev sibling next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Saturday, 10 January 2015 at 15:52:21 UTC, Tobias Pankrath 
wrote:
 ...

 The thing is, in languages like Perl, Python, Ruby (to name a 
 few), reusing
 someone else's code is not only easy, but it is how most 
 people actually write code most of the time.
I think he's wrong, because it spoils the comparison. Every answer should delegate those tasks to a library that Stroustroup used as well, e.g. regex matching, string to number conversion and some kind of TCP sockets. But it must do the same work that he's solution does: Create and parse HTML header and extract the html links, probably using regex, but I wouldn't mind another solution. Everyone can put a libdo_the_stroustroup_thing on dub and then call do_the_stroustroup_thing() in main. To compare what the standard libraries (and libraries easily obtained or quasi standard) offer is another challenge.
I disagree. The great thing about comes with batteries runtimes is that I have the guarantee the desired features exist in all platforms supported by the language. If the libraries are dumped into a repository, there is always a problem if the library works across all OS supported by the language or even if they work together at all. Specially if they depend on common packages with incompatible versions. This is the cause of so many string and vector types across all C++ libraries as most of those libraries were developed before C++98 was even done. Or why C runtime isn't nothing more than a light version of UNIX as it was back in 1989, without any worthwhile feature since then, besides some extra support for numeric types and a little more secure libraries. Nowadays, unless I am doing something very OS specific, I hardly care which OS I am using, thanks to such "comes with batteries" runtimes. -- Paulo
Jan 10 2015
prev sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Saturday, 10 January 2015 at 15:52:21 UTC, Tobias Pankrath 
wrote:
 I think he's wrong, because it spoils the comparison. Every 
 answer should delegate those tasks to a library that 
 Stroustroup used as well, e.g. regex matching, string to number 
 conversion and some kind of TCP sockets. But it must do the 
 same work that he's solution does: Create and parse HTML header 
 and extract the html links, probably using regex, but I 
 wouldn't mind another solution.
The challenge is completely pointless. Different languages have different ways of hacking together a compact incorrect solution. How to directly translate a C++ hack into another language is a task for people who are drunk. For the challenge to make sense it would entail parsing all legal HTML5 documents, extracting all resource links, converting them into absolute form and printing them one per line. With no hickups.
Jan 10 2015
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 10 January 2015 at 17:23:31 UTC, Ola Fosheim Grøstad 
wrote:
 For the challenge to make sense it would entail parsing all 
 legal HTML5 documents, extracting all resource links, 
 converting them into absolute form and printing them one per 
 line. With no hickups.
Though, that's still a library thing rather than a language thing. dom.d and the Url struct in cgi.d should be able to do all that, in just a few lines even, but that's just because I've done a *lot* of web scraping with the libs before so I made them work for that. In fact.... let me to do it. I'll use my http2.d instead of cgi.d, actually, it has a similar Url struct just more focused on client requests. import arsd.dom; import arsd.http2; import std.stdio; void main() { auto base = Uri("http://www.stroustrup.com/C++.html"); // http2 is a newish module of mine that aims to imitate // a browser in some ways (without depending on curl btw) auto client = new HttpClient(); auto request = client.navigateTo(base); auto document = new Document(); // and http2 provides an asynchonous api but you can // pretend it is sync by just calling waitForCompletion auto response = request.waitForCompletion(); // parseGarbage uses a few tricks to fixup invalid/broken HTML // tag soup and auto-detect character encodings, including when // it lies about being UTF-8 but is actually Windows-1252 document.parseGarbage(response.contentText); // Uri.basedOn returns a new absolute URI based on something else foreach(a; document.querySelectorAll("a[href]")) writeln(Uri(a.href).basedOn(base)); } Snippet of the printouts: [...] http://www.computerhistory.org http://www.softwarepreservation.org/projects/c_plus_plus/ http://www.morganstanley.com/ http://www.cs.columbia.edu/ http://www.cse.tamu.edu http://www.stroustrup.com/index.html http://www.stroustrup.com/C++.html http://www.stroustrup.com/bs_faq.html http://www.stroustrup.com/bs_faq2.html http://www.stroustrup.com/C++11FAQ.html http://www.stroustrup.com/papers.html [...] The latter are relative links that it based on and the first few are absolute. Seems to have worked. There's other kinds of links than just a[href], but fetching them is as simple as adding them to the selector or looping for them too separately: foreach(a; document.querySelectorAll("script[src]")) writeln(Uri(a.src).basedOn(base)); none on that page, no <link>s either, but it is easy enough to do with the lib. Looking at the source of that page, I find some invalid HTML and lies about the character set. How did Document.parseGarbage do? Pretty well, outputting the parsed DOM tree shows it auto-corrected the problems I see by eye.
Jan 10 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Saturday, 10 January 2015 at 17:39:17 UTC, Adam D. Ruppe wrote:
 Though, that's still a library thing rather than a language 
 thing.
It is a language-library-platform thing, things like how composable the eco system is would be interesting to compare. But it would be unfair to require a minimalistic language to not use third party libraries. One should probably require that the library used is generic (not a spider-framework), not using FFI, mature and maintained?
 	document.parseGarbage(response.contentText);

         // Uri.basedOn returns a new absolute URI based on 
 something else
 	foreach(a; document.querySelectorAll("a[href]"))
 		writeln(Uri(a.href).basedOn(base));
 }
Nice and clean code; does it expand html entities ("&amp")? The HTML5 standard has improved on HTML4 by now being explicit on how incorrect documents shall be interpreted in section 8.2. That ought to be sufficient, since that is what web browsers are supposed to do. http://www.w3.org/TR/html5/syntax.html#html-parser
Jan 10 2015
parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 10 January 2015 at 19:17:22 UTC, Ola Fosheim Grøstad 
wrote:
 Nice and clean code; does it expand html entities ("&amp")?
Of course. It does it both ways: <span>a &amp;</span> span.innerText == "a &" span.innerText = "a \" b"; assert(span.innerHTML == "a &quot; b"); parseGarbage also tries to fix broken entities, so like & standing alone it will translate to &amp; for you. there's also parseStrict which just throws an exception in cases like that. That's one thing a lot of XML parsers don't do in the name of speed, but I do since it is pretty rare that I don't want them translated. One thing I did for a speedup though was scan the string for & and if it doesn't find one, return a slice of the original, and if it does, return a new string with the entity translated. Gave a surprisingly big speed boost without costing anything in convenience.
 The HTML5 standard has improved on HTML4 by now being explicit 
 on how incorrect documents shall be interpreted in section 8.2. 
 That ought to be sufficient, since that is what web browsers 
 are supposed to do.

 http://www.w3.org/TR/html5/syntax.html#html-parser
Huh, I never read that, my thing just did what looked right to me over hundreds of test pages that were broken in various strange and bizarre ways.
Jan 10 2015
prev sibling parent reply =?UTF-8?B?Ik5vcmRsw7Z3Ig==?= <per.nordlow gmail.com> writes:
On Friday, 9 January 2015 at 17:15:43 UTC, Adam D. Ruppe wrote:
 import arsd.dom;
 import std.net.curl;
 import std.stdio, std.algorithm;

 void main() {
 	auto document = new Document(cast(string) 
 get("http://www.stroustrup.com/C++.html"));
 	writeln(document.querySelectorAll("a[href]").map!(a=>a.href));
 }

 Or perhaps better yet:

 import arsd.dom;
 import std.net.curl;
 import std.stdio;

 void main() {
 	auto document = new Document(cast(string) 
 get("http://www.stroustrup.com/C++.html"));
 	foreach(a; document.querySelectorAll("a[href]"))
 		writeln(a.href);
 }

 Which puts each one on a separate line.
Both these code examples triggers the same assert() dmd: expression.c:3761: size_t StringExp::length(int): Assertion `encSize == 1 || encSize == 2 || encSize == 4' failed. on dmd git master. Ideas anyone?
Jan 10 2015
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 10 January 2015 at 13:22:57 UTC, Nordlöw wrote:
 on dmd git master. Ideas anyone?
Don't use git master :P Definitely another regression. That line was just pushed to git like two weeks ago and the failing assertion is pretty obviously a pure dmd code bug, it doesn't know the length of char apparently.
Jan 10 2015
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Adam D. Ruppe:

 Don't use git master :P
Is the issue in Bugzilla? Bye, bearophile
Jan 10 2015
next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 10 January 2015 at 15:24:45 UTC, bearophile wrote:
 Is the issue in Bugzilla?
I don't know, bugzilla is extremely difficult to search. I guess I'll post it again and worst case it will be closed as a duplicate.
Jan 10 2015
prev sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Saturday, 10 January 2015 at 15:24:45 UTC, bearophile wrote:
 Is the issue in Bugzilla?
https://issues.dlang.org/show_bug.cgi?id=13966
Jan 10 2015
prev sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Saturday, 10 January 2015 at 14:56:09 UTC, Adam D. Ruppe wrote:
 On Saturday, 10 January 2015 at 13:22:57 UTC, Nordlöw wrote:
 on dmd git master. Ideas anyone?
Don't use git master :P
Do use git master. The more people do, the fewer regressions will slip into the final release. You can use Dustmite to reduce the code to a simple example, and Digger to find the exact pull request which introduced the regression. (Yes, shameless plug, preaching to the choir, etc.)
Jan 10 2015
prev sibling next sibling parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Friday, 9 January 2015 at 13:50:29 UTC, eles wrote:
 https://codegolf.stackexchange.com/questions/44278/debunking-stroustrups-debunking-of-the-myth-c-is-for-large-complicated-pro
Link to answer in D: http://codegolf.stackexchange.com/a/44417/13362
Jan 09 2015
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/9/15 6:10 PM, Jesse Phillips wrote:
 On Friday, 9 January 2015 at 13:50:29 UTC, eles wrote:
 https://codegolf.stackexchange.com/questions/44278/debunking-stroustrups-debunking-of-the-myth-c-is-for-large-complicated-pro
Link to answer in D: http://codegolf.stackexchange.com/a/44417/13362
Nailed it. -- Andrei
Jan 09 2015
prev sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Saturday, 10 January 2015 at 02:10:04 UTC, Jesse Phillips 
wrote:
 On Friday, 9 January 2015 at 13:50:29 UTC, eles wrote:
 https://codegolf.stackexchange.com/questions/44278/debunking-stroustrups-debunking-of-the-myth-c-is-for-large-complicated-pro
Link to answer in D: http://codegolf.stackexchange.com/a/44417/13362
I think byLine is not necessary. By default . will not match line breaks. One statement solution: import std.net.curl, std.stdio; import std.algorithm, std.regex; void main() { get("http://www.stroustrup.com/C++.html") .matchAll(`<a.*?href="(.*)"`) .map!(m => m[1]) .each!writeln(); } Requires Phobos PR#2024 ;)
Jan 09 2015
parent Daniel Kozak via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
Vladimir Panteleev via Digitalmars-d-learn píše v So 10. 01. 2015 v
07:42 +0000:
 On Saturday, 10 January 2015 at 02:10:04 UTC, Jesse Phillips 
 wrote:
 On Friday, 9 January 2015 at 13:50:29 UTC, eles wrote:
 https://codegolf.stackexchange.com/questions/44278/debunking-stroustrups-debunking-of-the-myth-c-is-for-large-complicated-pro
Link to answer in D: http://codegolf.stackexchange.com/a/44417/13362
I think byLine is not necessary. By default . will not match line breaks. One statement solution: import std.net.curl, std.stdio; import std.algorithm, std.regex; void main() { get("http://www.stroustrup.com/C++.html") .matchAll(`<a.*?href="(.*)"`) .map!(m => m[1]) .each!writeln(); } Requires Phobos PR#2024 ;)
Oh here is it, I was looking for each. I think it is allready in a phobos but I can not find. Now I know why :D
Jan 10 2015
prev sibling parent "MattCoder" <stop spam.com> writes:
On Friday, 9 January 2015 at 13:50:29 UTC, eles wrote:
 https://codegolf.stackexchange.com/questions/44278/debunking-stroustrups-debunking-of-the-myth-c-is-for-large-complicated-pro
From the link: "Let's show Stroustrup what small and readable program actually is." Alright, there are a lot a examples in many languagens, but those examples doesn't should handle exceptions like the original code does? Matheus.
Jan 10 2015