www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D Tokens

reply Markus <Markus_member pathlink.com> writes:
Hi, guys!

I would like to ask about D Lexical.

As I understand the D Source Text consists of the following:

*Source Text 
**white spaces 
**end of lines 
**comments 
**tokens 
**special token sequences 
**end of file 

Then, D Tokens consist of the following:

*tokens 
**Identifiers 
**String Literals 
**Character Literals 
**Integer Literals 
**Float Literals 
**Keywords 
**Some Characters

My question is this: 
How do you name those characters (delimiters, operators or somehow else)? I know
that in Ada programming language the following characters are called
“delimiters”:

/
/=
Mar 29 2006
next sibling parent Hasan Aljudy <hasan.aljudy gmail.com> writes:
Markus wrote:
 Hi, guys!
 
 I would like to ask about D Lexical.
 
 As I understand the D Source Text consists of the following:
 
 *Source Text 
 **white spaces 
 **end of lines 
 **comments 
 **tokens 
 **special token sequences 
 **end of file 
 
 Then, D Tokens consist of the following:
 
 *tokens 
 **Identifiers 
 **String Literals 
 **Character Literals 
 **Integer Literals 
 **Float Literals 
 **Keywords 
 **Some Characters
 
 My question is this: 
 How do you name those characters (delimiters, operators or somehow else)? I
know
 that in Ada programming language the following characters are called
 “delimiters”:
 
 /
 /=

I called them 'symbols' I think a more appropriatre name might be 'operators' I think it doesn't really matter, though!
Mar 29 2006
prev sibling parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Markus" <Markus_member pathlink.com> wrote in message 
news:e0fj9n$1d41$1 digitaldaemon.com...
 My question is this:
 How do you name those characters (delimiters, operators or somehow else)?

I don't know Ada, but do you mean things like braces and semicolons ({, }, ;)? I wrote a C-style scripting language parser thing, and I called those "particles." This is to parallel the use of particles in Japanese, which do not have any inherent meaning, but which simply define the grammatical structure of the sentence. In the same way, braces and semicolons simply describe the structure of the program, and really only have meaning to the compiler and aren't directly represented in the final machine code which is generated.
Mar 29 2006
parent reply Markus <Markus_member pathlink.com> writes:
In article <e0fo44$1h57$1 digitaldaemon.com>, Jarrett Billingsley says...
"Markus" <Markus_member pathlink.com> wrote in message 
news:e0fj9n$1d41$1 digitaldaemon.com...
 My question is this:
 How do you name those characters (delimiters, operators or somehow else)?

I don't know Ada, but do you mean things like braces and semicolons ({, }, ;)? I wrote a C-style scripting language parser thing, and I called those "particles." This is to parallel the use of particles in Japanese, which do not have any inherent meaning, but which simply define the grammatical structure of the sentence. In the same way, braces and semicolons simply describe the structure of the program, and really only have meaning to the compiler and aren't directly represented in the final machine code which is generated.

<= << <<= <> <>= > >= >>= >>>= >> >>> ! != !== !<> !<>= !< !<= !> !>= !~ ( ) [ ] { } ? , ; : $ = == === * *= % %= ^ ^= ~ ~= ~~ go to http://www.digitalmars.com/d/lex.html and look at 'Tokens'. As i understand, D Tokens are Identifiers, Sring Literals, Charachter Literals, Integer Literals, Float Literals, Keywords, and above-mentioned symbols. How do you call those symbols - Symbols, Operators, or Delimiters? Look at http://en.wikibooks.org/wiki/Ada_Programming/Lexical_elements http://en.wikibooks.org/wiki/Ada_Programming/Operators http://en.wikibooks.org/wiki/Programming:C_plus_plus/Operators_Table http://en.wikibooks.org/wiki/C_Sharp_Programming/Operators I think that there should be a standard name for it. What do you think?
Mar 30 2006
parent reply Derek Parnell <derek psych.ward> writes:
On Thu, 30 Mar 2006 20:26:59 +0000 (UTC), Markus wrote:

 In article <e0fo44$1h57$1 digitaldaemon.com>, Jarrett Billingsley says...
"Markus" <Markus_member pathlink.com> wrote in message 
news:e0fj9n$1d41$1 digitaldaemon.com...
 My question is this:
 How do you name those characters (delimiters, operators or somehow else)?

I don't know Ada, but do you mean things like braces and semicolons ({, }, ;)? I wrote a C-style scripting language parser thing, and I called those "particles." This is to parallel the use of particles in Japanese, which do not have any inherent meaning, but which simply define the grammatical structure of the sentence. In the same way, braces and semicolons simply describe the structure of the program, and really only have meaning to the compiler and aren't directly represented in the final machine code which is generated.

<= << <<= <> <>= > >= >>= >>>= >> >>> ! != !== !<> !<>= !< !<= !> !>= !~ ( ) [ ] { } ? , ; : $ = == === * *= % %= ^ ^= ~ ~= ~~ go to http://www.digitalmars.com/d/lex.html and look at 'Tokens'. As i understand, D Tokens are Identifiers, Sring Literals, Charachter Literals, Integer Literals, Float Literals, Keywords, and above-mentioned symbols. How do you call those symbols - Symbols, Operators, or Delimiters? Look at http://en.wikibooks.org/wiki/Ada_Programming/Lexical_elements http://en.wikibooks.org/wiki/Ada_Programming/Operators http://en.wikibooks.org/wiki/Programming:C_plus_plus/Operators_Table http://en.wikibooks.org/wiki/C_Sharp_Programming/Operators I think that there should be a standard name for it. What do you think?

Operators. -- Derek (skype: derek.j.parnell) Melbourne, Australia "Down with mediocracy!" 31/03/2006 9:35:21 AM
Mar 30 2006
parent Sean Kelly <sean f4.ca> writes:
Derek Parnell wrote:
 On Thu, 30 Mar 2006 20:26:59 +0000 (UTC), Markus wrote:
 
 In article <e0fo44$1h57$1 digitaldaemon.com>, Jarrett Billingsley says...
 "Markus" <Markus_member pathlink.com> wrote in message 
 news:e0fj9n$1d41$1 digitaldaemon.com...
 My question is this:
 How do you name those characters (delimiters, operators or somehow else)?

;)? I wrote a C-style scripting language parser thing, and I called those "particles." This is to parallel the use of particles in Japanese, which do not have any inherent meaning, but which simply define the grammatical structure of the sentence. In the same way, braces and semicolons simply describe the structure of the program, and really only have meaning to the compiler and aren't directly represented in the final machine code which is generated.

<= << <<= <> <>= > >= >>= >>>= >> >>> ! != !== !<> !<>= !< !<= !> !>= !~ ( ) [ ] { } ? , ; : $ = == === * *= % %= ^ ^= ~ ~= ~~ go to http://www.digitalmars.com/d/lex.html and look at 'Tokens'. As i understand, D Tokens are Identifiers, Sring Literals, Charachter Literals, Integer Literals, Float Literals, Keywords, and above-mentioned symbols. How do you call those symbols - Symbols, Operators, or Delimiters? Look at http://en.wikibooks.org/wiki/Ada_Programming/Lexical_elements http://en.wikibooks.org/wiki/Ada_Programming/Operators http://en.wikibooks.org/wiki/Programming:C_plus_plus/Operators_Table http://en.wikibooks.org/wiki/C_Sharp_Programming/Operators I think that there should be a standard name for it. What do you think?

Operators.

It's worth noting that operators are only delimiters because the language spec says they are. Some languages actually require whitespace between identifiers, literals, and operators. Sean
Mar 30 2006