www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - "Comment processing conceptually happens before tokenization."

reply Stewart Gordon <smjg_1998 yahoo.com> writes:
http://www.digitalmars.com/d/lex.html

"Comment processing conceptually happens before tokenization. This means 
that embedded strings and comments do not prevent recognition of comment 
openings and closings:"

But the first sentence, the way it's written, means that comments that 
begin within strings will be treated as comments, which is contrary to 
my experience.  That is,

     a = "Here is a string with a /+ comment delimiter"; // +/ in it";

would be treated as

     a = "Here is a string with a  in it";

when, in fact, the opening "/+", being within a string, is treated as 
part of the string; therefore a will contain

     "Here is a string with a /+ comment delimiter"

The second sentence of the quotation is also open to interpretation. 
Claiming that they "do not prevent recognition of comment openings" 
tempts one to think that arbitrary comment openings within a string or 
comment will be processed.

A more accurate way to put it (assuming that what the compiler does is 
what was intended) is:

The contents of strings and comments are not tokenized.  Consequently, 
comment openings occurring within a string do not begin a comment, and 
string delimiters within a comment do not affect the recognition of 
comment closings and nested "/+" comment openings.  With the exception 
of "/+" occurring within a "/+" comment, comment openings within a 
comment are ignored.

Stewart.

-- 
-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GCS/M d- s:-  C++  a->--- UB  P+ L E  W++  N+++ o K-  w++  O? M V? PS- 
PE- Y? PGP- t- 5? X? R b DI? D G e++++ h-- r-- !y
------END GEEK CODE BLOCK------

My e-mail is valid but not my primary mailbox.  Please keep replies on 
the 'group where everyone may benefit.
Oct 17 2006
parent reply Walter Bright <newshound digitalmars.com> writes:
Yours is better. I'll make the change. Thanks.
Oct 17 2006
parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Walter Bright wrote:
 Yours is better. I'll make the change. Thanks.

What's the reason for this behavior BTW? Does it make parsing/tokenizing more easy? Because it seems to me the ideal behaviour is otherwise. -- Bruno Medeiros - MSc in CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Oct 18 2006
parent reply Walter Bright <newshound digitalmars.com> writes:
Bruno Medeiros wrote:
 Walter Bright wrote:
 Yours is better. I'll make the change. Thanks.

What's the reason for this behavior BTW? Does it make parsing/tokenizing more easy? Because it seems to me the ideal behaviour is otherwise.

I think the examples in www.digitalmars.com/d/lex.html should make it clear.
Oct 18 2006
parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Walter Bright wrote:
 Bruno Medeiros wrote:
 Walter Bright wrote:
 Yours is better. I'll make the change. Thanks.

What's the reason for this behavior BTW? Does it make parsing/tokenizing more easy? Because it seems to me the ideal behaviour is otherwise.

I think the examples in www.digitalmars.com/d/lex.html should make it clear.

Huh? They make clear what the behavior is, but that I already know. Instead, I wanted to know the reasoning behind that. -- Bruno Medeiros - MSc in CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Oct 22 2006
parent "Unknown W. Brackets" <unknown simplemachines.org> writes:
What if you want a string literal which contains the following characters?

/* comment */

For whatever reason?  Do you propose that you should have to do the 
following?

char[] comment_example = "/" "* comment *" "/";

Or something?  I think it makes sense to not parse comments inside of 
strings, which is what every other programming language I've used does.

-[Unknown]


 Walter Bright wrote:
 Bruno Medeiros wrote:
 Walter Bright wrote:
 Yours is better. I'll make the change. Thanks.

What's the reason for this behavior BTW? Does it make parsing/tokenizing more easy? Because it seems to me the ideal behaviour is otherwise.

I think the examples in www.digitalmars.com/d/lex.html should make it clear.

Huh? They make clear what the behavior is, but that I already know. Instead, I wanted to know the reasoning behind that.

Oct 22 2006