www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - End Of Line character and others not working.

reply "Alan" <alanpotteiger gmail.com> writes:
Hello!  The past few hours I've been working on some things and I 
came accross a small bug.

I'm essentially practicing lexing and parsing by implementing a 
(very) simple language.  Everything is going great so far 
(variable declarations, writing to stdout etc...) but I have a 
small problem with my string literals.  I've got them working 
fine, quotes can be escaped etc... But when they include the new 
line character (\n) for example and it's written out it doesn't 
create a new line but prints out those characters raw.  Does 
anyone have any idea why?
Any help is very much appreciated!
Aug 16 2013
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, Aug 17, 2013 at 06:53:16AM +0200, Alan wrote:

 Hello!  The past few hours I've been working on some things and I
 came accross a small bug.
 
 I'm essentially practicing lexing and parsing by implementing a
 (very) simple language.  Everything is going great so far (variable
 declarations, writing to stdout etc...) but I have a small problem
 with my string literals.  I've got them working fine, quotes can be
 escaped etc... But when they include the new line character (\n) for
 example and it's written out it doesn't create a new line but prints
 out those characters raw.  Does anyone have any idea why?
 Any help is very much appreciated!


Which quotation marks did you use for your string literals?

If you use double quotes, then it should work: "\n"

But if you use the other quoting syntaxes, the \n may be treated
literally rather than as an escape sequence, e.g., `\n` is a string of
two characters '\' and 'n'.


T

-- 
Written on the window of a clothing store: No shirt, no shoes, no service.
Aug 16 2013
parent reply "Alan" <alanpotteiger gmail.com> writes:
On Saturday, 17 August 2013 at 05:05:12 UTC, H. S. Teoh wrote:

 On Sat, Aug 17, 2013 at 06:53:16AM +0200, Alan wrote:

 Hello!  The past few hours I've been working on some things 
 and I
 came accross a small bug.
 
 I'm essentially practicing lexing and parsing by implementing a
 (very) simple language.  Everything is going great so far 
 (variable
 declarations, writing to stdout etc...) but I have a small 
 problem
 with my string literals.  I've got them working fine, quotes 
 can be
 escaped etc... But when they include the new line character 
 (\n) for
 example and it's written out it doesn't create a new line but 
 prints
 out those characters raw.  Does anyone have any idea why?
 Any help is very much appreciated!


 Which quotation marks did you use for your string literals?

 If you use double quotes, then it should work: "\n"

 But if you use the other quoting syntaxes, the \n may be treated
 literally rather than as an escape sequence, e.g., `\n` is a 
 string of
 two characters '\' and 'n'.


 T


I don't think it matters how my string literals are recoginized, 
they can be recognized by double or single quotes.
Either way the value is stored in a string then written out, but 
\n is written out raw for some reason.
Aug 16 2013
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, Aug 17, 2013 at 07:19:19AM +0200, Alan wrote:

 On Saturday, 17 August 2013 at 05:05:12 UTC, H. S. Teoh wrote:

On Sat, Aug 17, 2013 at 06:53:16AM +0200, Alan wrote:

Hello!  The past few hours I've been working on some things and I
came accross a small bug.

I'm essentially practicing lexing and parsing by implementing a
(very) simple language.  Everything is going great so far (variable
declarations, writing to stdout etc...) but I have a small problem
with my string literals.  I've got them working fine, quotes can be
escaped etc... But when they include the new line character (\n) for
example and it's written out it doesn't create a new line but prints
out those characters raw.  Does anyone have any idea why?  Any help
is very much appreciated!


Which quotation marks did you use for your string literals?

If you use double quotes, then it should work: "\n"

But if you use the other quoting syntaxes, the \n may be treated
literally rather than as an escape sequence, e.g., `\n` is a string
of two characters '\' and 'n'.


T


 
 I don't think it matters how my string literals are recoginized,
 they can be recognized by double or single quotes.
 Either way the value is stored in a string then written out, but \n
 is written out raw for some reason.


Wait, are you talking about the language that you're parsing having
string literals that contain "\n"? If so, then you need to manually
translate them, since in the input file, they are two literal character
'\' and 'n', and the computer wouldn't know how you want to interpret
them. So you have to do something like this:

	auto parseStringLiteral(R)(R input)
		if (is(ElementType!R : dchar))
	{
		auto value = appender!string();
		while (!input.empty && input.front != '\"')
		{
			// Interpret escape sequences here
			if (input.front == '\\') {
				input.popFront();
				if (input.empty)
					throw new Exception("Unterminated escape sequence");

				switch(input.front) {
				case 'n':
					app.put("\n");
					break;
				// ... put whatever other escapes you
				// want to interpret here
				}
			} else {
				// Not an escape sequence, transcribe to
				// output literally.
				app.put(input.front);
			}
		}
		return value.data;
	}

In other words, none of the escape sequences are implemented for you;
you have to implement them yourself.


T

-- 
"You are a very disagreeable person." "NO."
Aug 16 2013
parent "Alan" <alanpotteiger gmail.com> writes:
On Saturday, 17 August 2013 at 05:37:18 UTC, H. S. Teoh wrote:

 On Sat, Aug 17, 2013 at 07:19:19AM +0200, Alan wrote:

 On Saturday, 17 August 2013 at 05:05:12 UTC, H. S. Teoh wrote:

On Sat, Aug 17, 2013 at 06:53:16AM +0200, Alan wrote:

Hello!  The past few hours I've been working on some things 
and I
came accross a small bug.

I'm essentially practicing lexing and parsing by 
implementing a
(very) simple language.  Everything is going great so far 
(variable
declarations, writing to stdout etc...) but I have a small 
problem
with my string literals.  I've got them working fine, quotes 
can be
escaped etc... But when they include the new line character 
(\n) for
example and it's written out it doesn't create a new line 
but prints
out those characters raw.  Does anyone have any idea why?  
Any help
is very much appreciated!


Which quotation marks did you use for your string literals?

If you use double quotes, then it should work: "\n"

But if you use the other quoting syntaxes, the \n may be 
treated
literally rather than as an escape sequence, e.g., `\n` is a 
string
of two characters '\' and 'n'.


T


 
 I don't think it matters how my string literals are 
 recoginized,
 they can be recognized by double or single quotes.
 Either way the value is stored in a string then written out, 
 but \n
 is written out raw for some reason.


 Wait, are you talking about the language that you're parsing 
 having
 string literals that contain "\n"? If so, then you need to 
 manually
 translate them, since in the input file, they are two literal 
 character
 '\' and 'n', and the computer wouldn't know how you want to 
 interpret
 them. So you have to do something like this:

 	auto parseStringLiteral(R)(R input)
 		if (is(ElementType!R : dchar))
 	{
 		auto value = appender!string();
 		while (!input.empty && input.front != '\"')
 		{
 			// Interpret escape sequences here
 			if (input.front == '\\') {
 				input.popFront();
 				if (input.empty)
 					throw new Exception("Unterminated escape sequence");

 				switch(input.front) {
 				case 'n':
 					app.put("\n");
 					break;
 				// ... put whatever other escapes you
 				// want to interpret here
 				}
 			} else {
 				// Not an escape sequence, transcribe to
 				// output literally.
 				app.put(input.front);
 			}
 		}
 		return value.data;
 	}

 In other words, none of the escape sequences are implemented 
 for you;
 you have to implement them yourself.


 T


OH!  Yes that makes a lot of sense, thanks so much for the help!
Aug 16 2013