www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Re: Adding Unicode operators to D

reply ore-sama <spam here.lot> writes:
Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

Console is a legacy technology (you even still call it "DOS"), why expect features from it?
Oct 24 2008
next sibling parent reply "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

Console is a legacy technology (you even still call it "DOS"), why expect features from it?

So tell me what the alternative is? I had trouble with running D tools from a Cygwin shell. Can't remember if I tried MSYS or not. Anyone using a shell for Windows that works and supports UTF-8 properly? --bb
Oct 24 2008
next sibling parent reply Sergey Gromov <snake.scaly gmail.com> writes:
Sat, 25 Oct 2008 06:43:19 +0900,
Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

Console is a legacy technology (you even still call it "DOS"), why expect features from it?

So tell me what the alternative is? I had trouble with running D tools from a Cygwin shell. Can't remember if I tried MSYS or not. Anyone using a shell for Windows that works and supports UTF-8 properly?

A regular Windows console supports UTF-8 to some extent: * Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.
Oct 24 2008
next sibling parent reply Yigal Chripun <yigal100 gmail.com> writes:
Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 9:15 AM, Sergey Gromov <snake.scaly gmail.com> wrote:
 Sat, 25 Oct 2008 06:43:19 +0900,
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)


tools from a Cygwin shell. Can't remember if I tried MSYS or not. Anyone using a shell for Windows that works and supports UTF-8 properly?

* Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.

I did that but "type <filewith-utf8.txt>" still prints garbage. --bb

so don't use type. use notepad instead... notepad <filewith-utf8.txt> also, MSYS gives you all the linux tools if you really need to be shell only. last resort: nothing stops you from implementing your own "cat" application in D with full Unicode support. most if not all linux shell tools are separate executables anyway and if any still do not support unicode it'll be trivial to roll your own replacements for the bad ones.
Oct 24 2008
next sibling parent reply Benji Smith <dlanguage benjismith.net> writes:
Yigal Chripun wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 9:15 AM, Sergey Gromov <snake.scaly gmail.com> wrote:
 Sat, 25 Oct 2008 06:43:19 +0900,
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)


tools from a Cygwin shell. Can't remember if I tried MSYS or not. Anyone using a shell for Windows that works and supports UTF-8 properly?

* Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.

--bb

so don't use type. use notepad instead... notepad <filewith-utf8.txt> also, MSYS gives you all the linux tools if you really need to be shell only. last resort: nothing stops you from implementing your own "cat" application in D with full Unicode support. most if not all linux shell tools are separate executables anyway and if any still do not support unicode it'll be trivial to roll your own replacements for the bad ones.

Oh, and one of my favorite tricks in Windows is to install cygwin (usually at "C:\cygwin" or whatever their boneheaded installer insists on using) and then add the bin path ("C:\cygwin\bin") to the windows PATH. That way, I can continue using the ordinary windows shell (which I prefer, since it doesn't force me to use the nutty directory names that the cygwin shell uses), but I can still access all the linux commands. Calling grep from a windows shell is the bestest! --benji
Oct 24 2008
parent reply Benji Smith <dlanguage benjismith.net> writes:
Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 10:31 AM, Benji Smith <dlanguage benjismith.net> wrote:
 Yigal Chripun wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 9:15 AM, Sergey Gromov <snake.scaly gmail.com>
 wrote:
 Sat, 25 Oct 2008 06:43:19 +0900,
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

expect features from it?

tools from a Cygwin shell. Can't remember if I tried MSYS or not. Anyone using a shell for Windows that works and supports UTF-8 properly?

* Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.

--bb

notepad <filewith-utf8.txt> also, MSYS gives you all the linux tools if you really need to be shell only. last resort: nothing stops you from implementing your own "cat" application in D with full Unicode support. most if not all linux shell tools are separate executables anyway and if any still do not support unicode it'll be trivial to roll your own replacements for the bad ones.

at "C:\cygwin" or whatever their boneheaded installer insists on using) and then add the bin path ("C:\cygwin\bin") to the windows PATH. That way, I can continue using the ordinary windows shell (which I prefer, since it doesn't force me to use the nutty directory names that the cygwin shell uses), but I can still access all the linux commands. Calling grep from a windows shell is the bestest!

But that has the same problem. Cygtools don't understand windows paths so barf when you say "grep c:\foo.txt" But the Windows shell only will only autocomplete Windows-style paths. I've found the gnuwin32 tools to work a little better on that front. --bb

Wha??? The "grep" tool doesn't read the path. The *shell* interprets the path and passes the text to the program. That's how all the gnu tools are able to pipe their results from one tool to the other. Or at least, that's how I assume it works. Cuz I use grep like every single day. On the "cmd.exe" shell. With windows paths. In fact, just for you, I tested this: grep -i "SHAZZAM" "C:\Documents and Settings\benji\Desktop\my filename with spaces.txt" Worked like a charm. If the path doesn't have spaces, I have no problem with this: grep -i "SHAZZAM" C:\file.txt I tried it in both "command.com" and in "cmd.exe" and didn't experience any problem in either environment. The key is to never never never use the cygwin shell. It's a piece of garbage. But using the executables from the "cygwin\bin" directory within the windows shell... Priceless! --benji
Oct 24 2008
next sibling parent reply Benji Smith <dlanguage benjismith.net> writes:
Bill Baxter wrote:
 Benji Smith wrote:
 The key is to never never never use the cygwin shell. It's a piece of
 garbage. But using the executables from the "cygwin\bin" directory within
 the windows shell... Priceless!

Oh, I didn't realize that. There is one thing that doesn't work, which is probably what gave me the impression it was broken -- Windows paths with wildcards don't work. Like "grep c:\Windows\*.txt". But you're right that it does seem to work for both windows paths, and local wildcards, just not Windows paths with wildcards. But that's great. Thanks for the info. Actually I used to put cygwin\bin on my path years ago, but stopped doing it at some point and switched to gnuwin32. I was under the impression that it worked better then, but actually I've had some trouble with gnuwin32 recently.

Glad I could be of service! --benji
Oct 24 2008
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Benji Smith" wrote
 Bill Baxter wrote:
 Benji Smith wrote:
 The key is to never never never use the cygwin shell. It's a piece of
 garbage. But using the executables from the "cygwin\bin" directory 
 within
 the windows shell... Priceless!

Oh, I didn't realize that. There is one thing that doesn't work, which is probably what gave me the impression it was broken -- Windows paths with wildcards don't work. Like "grep c:\Windows\*.txt". But you're right that it does seem to work for both windows paths, and local wildcards, just not Windows paths with wildcards.


It's not the paths with wildcards that is the problem. In this case, it is the shell. Grep is expecting the shell to expand the wildcards, as it does on unix. For example, you can use this old trick if ls suddenly becomes unavailable to list all files in the current directory: echo * Which is all shell builtin no executables are run. If you ran this from a windows shell you get the same error: grep text /cygdrive/c/Windows/*.txt The windows shell expects the application to handle wildcard expansion, which is why windows command line programs don't always work the same way. Every program has to build in wildcard expansion to support it. -Steve
Oct 24 2008
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Bill Baxter" wrote
 On Sat, Oct 25, 2008 at 1:40 PM, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:
 "Benji Smith" wrote
 Bill Baxter wrote:
 Benji Smith wrote:
 The key is to never never never use the cygwin shell. It's a piece of
 garbage. But using the executables from the "cygwin\bin" directory
 within
 the windows shell... Priceless!

Oh, I didn't realize that. There is one thing that doesn't work, which is probably what gave me the impression it was broken -- Windows paths with wildcards don't work. Like "grep c:\Windows\*.txt". But you're right that it does seem to work for both windows paths, and local wildcards, just not Windows paths with wildcards.


It's not the paths with wildcards that is the problem. In this case, it is the shell. Grep is expecting the shell to expand the wildcards, as it does on unix.

Read again. Particularly this part: "it does seem to work for both windows paths, **and local wildcards**, just not Windows paths with wildcards". (emphasis added) "grep Foo *.txt" works just fine. "grep Foo c:\*.txt" does not.

Then that must be something grep is doing extra. Or perhaps the Windows console selectively expands wildcards? I have no idea. It seems weird that grep would expand only current-directory wildcards (try grep Foo *, and see if it works. Windows normally only expands *.* to mean 'all files'). But in the case of using a cygwin shell, the shell expands all wildcards before passing arguments to grep. That much I do know. I haven't really had a need to use the windows shell in a long time ;) -Steve
Oct 24 2008
parent Benji Smith <dlanguage benjismith.net> writes:
Bill Baxter wrote:
 "it does seem to work for both windows paths, **and local wildcards**,
 just not Windows paths with wildcards".
 (emphasis added)

 "grep Foo *.txt"  works just fine.  "grep Foo c:\*.txt"  does not.


Yep, that was what I said.
 Or perhaps the Windows
 console selectively expands wildcards?  I have no idea.

Don't think so. "echo *" still dutifully prints a "*" to the console. Cygwin grep is doing it, probably in an attempt to be more useful when used from the DOS prompt.
 It seems weird that
 grep would expand only current-directory wildcards (try grep Foo *, and see
 if it works.


Interesting. About 90% of the time, I run grep with the "recursion" flag, so I haven't thought about wildcard expansion in ages. grep -R "some text" . I do know that "wc" does wildcard expansion, even with paths, but you have to use forward slashes. So, to count lines in D programs from the windows shell: wc -l /dev/*.d Unfortunately, there's no "recursion" flag for wc, so I end up doing something dumb like this: wc -l /dev/*.d wc -l /dev/*/*.d wc -l /dev/*/*/*.d Etc. Hmmmmmm. I really should just compile my own wc. After all, Walter's already written the sample code. --benji
Oct 25 2008
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Benji Smith" wrote
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 10:31 AM, Benji Smith <dlanguage benjismith.net> 
 wrote:
 Yigal Chripun wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 9:15 AM, Sergey Gromov <snake.scaly gmail.com>
 wrote:
 Sat, 25 Oct 2008 06:43:19 +0900,
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

expect features from it?

tools from a Cygwin shell. Can't remember if I tried MSYS or not. Anyone using a shell for Windows that works and supports UTF-8 properly?

* Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.

--bb

notepad <filewith-utf8.txt> also, MSYS gives you all the linux tools if you really need to be shell only. last resort: nothing stops you from implementing your own "cat" application in D with full Unicode support. most if not all linux shell tools are separate executables anyway and if any still do not support unicode it'll be trivial to roll your own replacements for the bad ones.

(usually at "C:\cygwin" or whatever their boneheaded installer insists on using) and then add the bin path ("C:\cygwin\bin") to the windows PATH. That way, I can continue using the ordinary windows shell (which I prefer, since it doesn't force me to use the nutty directory names that the cygwin shell uses), but I can still access all the linux commands. Calling grep from a windows shell is the bestest!

But that has the same problem. Cygtools don't understand windows paths so barf when you say "grep c:\foo.txt" But the Windows shell only will only autocomplete Windows-style paths. I've found the gnuwin32 tools to work a little better on that front. --bb

Wha??? The "grep" tool doesn't read the path. The *shell* interprets the path and passes the text to the program. That's how all the gnu tools are able to pipe their results from one tool to the other. Or at least, that's how I assume it works.

No, grep accepts either input. The shell does not change paths to windows style, that is what cygpath is for. But it does interpret backslashes, so you have to double all those. So for instance, in a cygwin shell, this works also: grep -i "SHAZZAM" C:\\Documents\ and\ Settings\\benji\\Desktop\\my\ filename\ with\ spaces.txt The arguments are passed as they are, grep just is smart enough to use either one. Probably many tools are that way, I wouldn't know because I usually do the /cygdrive/c/... form.
 The key is to never never never use the cygwin shell. It's a piece of 
 garbage. But using the executables from the "cygwin\bin" directory within 
 the windows shell... Priceless!

Without the cygwin shell, you lose all bash features, like for, or backticks to execute a command and use it's output. The paths are a minor annoyance IMO. Using the cmd.exe shell is ok for simple tasks, but it pales severely in comparison to the power of bash. So piece of garbage it is not. Something you don't understand how to use properly? definitely ;) -Steve
Oct 24 2008
next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Bill Baxter" wrote
 On Sat, Oct 25, 2008 at 1:33 PM, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:
 "Benji Smith" wrote
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 10:31 AM, Benji Smith 
 <dlanguage benjismith.net>
 wrote:
 Yigal Chripun wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 9:15 AM, Sergey Gromov 
 <snake.scaly gmail.com>
 wrote:
 Sat, 25 Oct 2008 06:43:19 +0900,
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

why expect features from it?

tools from a Cygwin shell. Can't remember if I tried MSYS or not. Anyone using a shell for Windows that works and supports UTF-8 properly?

* Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.

--bb

notepad <filewith-utf8.txt> also, MSYS gives you all the linux tools if you really need to be shell only. last resort: nothing stops you from implementing your own "cat" application in D with full Unicode support. most if not all linux shell tools are separate executables anyway and if any still do not support unicode it'll be trivial to roll your own replacements for the bad ones.

(usually at "C:\cygwin" or whatever their boneheaded installer insists on using) and then add the bin path ("C:\cygwin\bin") to the windows PATH. That way, I can continue using the ordinary windows shell (which I prefer, since it doesn't force me to use the nutty directory names that the cygwin shell uses), but I can still access all the linux commands. Calling grep from a windows shell is the bestest!

But that has the same problem. Cygtools don't understand windows paths so barf when you say "grep c:\foo.txt" But the Windows shell only will only autocomplete Windows-style paths. I've found the gnuwin32 tools to work a little better on that front. --bb

Wha??? The "grep" tool doesn't read the path. The *shell* interprets the path and passes the text to the program. That's how all the gnu tools are able to pipe their results from one tool to the other. Or at least, that's how I assume it works.

No, grep accepts either input. The shell does not change paths to windows style, that is what cygpath is for. But it does interpret backslashes, so you have to double all those. So for instance, in a cygwin shell, this works also: grep -i "SHAZZAM" C:\\Documents\ and\ Settings\\benji\\Desktop\\my\ filename\ with\ spaces.txt The arguments are passed as they are, grep just is smart enough to use either one. Probably many tools are that way, I wouldn't know because I usually do the /cygdrive/c/... form.
 The key is to never never never use the cygwin shell. It's a piece of
 garbage. But using the executables from the "cygwin\bin" directory 
 within
 the windows shell... Priceless!

Without the cygwin shell, you lose all bash features, like for, or backticks to execute a command and use it's output. The paths are a minor annoyance IMO. Using the cmd.exe shell is ok for simple tasks, but it pales severely in comparison to the power of bash. So piece of garbage it is not. Something you don't understand how to use properly? definitely ;)

Yeh, I love the bash shell. Really the only thing keeping me from using it for D work is the fact that it won't auto-complete Windows filenames.

It's ugly, but can be aliased or scripted, look into cygpath: cygpath -w /cygdrive/c/filename.txt outputs: C:\filename.txt so you can use dmd combined with cygpath: dmd `cygpath -w /cygdrive/c/path/to/d/files/*.d` It wouldn't take much to write a bash script to do this for you... -Steve
Oct 24 2008
prev sibling parent Benji Smith <dlanguage benjismith.net> writes:
Steven Schveighoffer wrote:
 So piece of garbage it is not.  Something you don't understand how to use 
 properly? definitely ;)

Definitely! I hope you'll agree that hyperbole is the best thing in the world :) --benji
Oct 25 2008
prev sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Oct 25, 2008 at 2:09 PM, Steven Schveighoffer
<schveiguy yahoo.com> wrote:
 "Bill Baxter" wrote
 On Sat, Oct 25, 2008 at 1:40 PM, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:
 "Benji Smith" wrote
 Bill Baxter wrote:
 Benji Smith wrote:
 The key is to never never never use the cygwin shell. It's a piece of
 garbage. But using the executables from the "cygwin\bin" directory
 within
 the windows shell... Priceless!

Oh, I didn't realize that. There is one thing that doesn't work, which is probably what gave me the impression it was broken -- Windows paths with wildcards don't work. Like "grep c:\Windows\*.txt". But you're right that it does seem to work for both windows paths, and local wildcards, just not Windows paths with wildcards.


It's not the paths with wildcards that is the problem. In this case, it is the shell. Grep is expecting the shell to expand the wildcards, as it does on unix.

Read again. Particularly this part: "it does seem to work for both windows paths, **and local wildcards**, just not Windows paths with wildcards". (emphasis added) "grep Foo *.txt" works just fine. "grep Foo c:\*.txt" does not.

Then that must be something grep is doing extra.

Yep, that was what I said.
 Or perhaps the Windows
 console selectively expands wildcards?  I have no idea.

Don't think so. "echo *" still dutifully prints a "*" to the console. Cygwin grep is doing it, probably in an attempt to be more useful when used from the DOS prompt.
 It seems weird that
 grep would expand only current-directory wildcards (try grep Foo *, and see
 if it works.

Yep that works.
 Windows normally only expands *.* to mean 'all files').

If by that you mean Windows command line programs usually expand *.*, then yeh.
 But in the case of using a cygwin shell, the shell expands all wildcards before
 passing arguments to grep.  That much I do know.  I haven't really had a
 need to use the windows shell in a long time ;)

Yep that's true for Bash. An easy way to tell the Windows shell does nothing is by compiling and running: import std.stdio; void main(string[] args) { writefln("Args: %s", args); } And passing it some wildcards. It never expands anything. Only thing it does do is mess with quotes some. Here's an example: C:\> args.exe * "C:\Program Files" *.* c:\* Args: [args,*,C:\Program Files,*.*,c:\*] --bb
Oct 24 2008
prev sibling parent reply Benji Smith <dlanguage benjismith.net> writes:
Bill Baxter wrote:
 Anyone using a shell for Windows that works and supports UTF-8 properly?

* Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.

I did that but "type <filewith-utf8.txt>" still prints garbage.

That's weird. My machine (WinXp Sp3) has no problem printing UTF-8 to the console. The only special thing I did was changed the font to Lucide Console. --benji
Oct 24 2008
parent reply Benji Smith <dlanguage benjismith.net> writes:
Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 10:23 AM, Benji Smith <dlanguage benjismith.net> wrote:
 Bill Baxter wrote:
 Anyone using a shell for Windows that works and supports UTF-8 properly?

* Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.


console. The only special thing I did was changed the font to Lucide Console.

Ok. Thanks for the info. Knowing that it has actually worked for at least one person gives me motivation to try again. --bb

Write a tiny little D program and see what you get on the console: import tango.io.Stdout; void main() { Stdout("spade, club, heart, diamond: \u2660\u2663\u2665\u2666"); } I don't know anything about the "type" command, and whether it supports UTF-8. But the console itself ought to be able to handle it. Try compiling the above code and see what happens. --benji
Oct 24 2008
next sibling parent reply Yigal Chripun <yigal100 gmail.com> writes:
Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 10:37 AM, Benji Smith <dlanguage benjismith.net> wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 10:23 AM, Benji Smith <dlanguage benjismith.net>
 wrote:
 Bill Baxter wrote:
 Anyone using a shell for Windows that works and supports UTF-8
 properly?

* Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.


console. The only special thing I did was changed the font to Lucide Console.

least one person gives me motivation to try again. --bb

import tango.io.Stdout; void main() { Stdout("spade, club, heart, diamond: \u2660\u2663\u2665\u2666"); } I don't know anything about the "type" command, and whether it supports UTF-8. But the console itself ought to be able to handle it. Try compiling the above code and see what happens. --benji

Ah, I see. I guess more what I want to know is if I had utf-8 source code and the D compiler spit out a message about one of the lines, would that error message come out as garbage? Same for ddbg -- if I'm debugging and say "ps" for "print source" will the result be garbage. I was thinking that "type" would be a simple test if that sort of thing would work. But maybe type is just borked. I did try "cat" and "more" too I think, with same result, though. --bb

Msys does autocomplete. it's not perfect but it works. the path will look unix like though.. i.e. /c/program files/... from what I know (winXP sp 2) - console works for unicode Except for RTL languages like Hebrew. as someone else already noted, this is legacy tech which you shouldn't be using anyway. I don't know if it's fixed in SP3 or not but the new way from MS is their powershell tool based on C#. there are also other 3rd party stuff as well..
Oct 24 2008
parent Robert Fraser <fraserofthenight gmail.com> writes:
Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 11:53 AM, Yigal Chripun <yigal100 gmail.com> wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 10:37 AM, Benji Smith <dlanguage benjismith.net> wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 10:23 AM, Benji Smith <dlanguage benjismith.net>
 wrote:
 Bill Baxter wrote:
 Anyone using a shell for Windows that works and supports UTF-8
 properly?

* Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.


console. The only special thing I did was changed the font to Lucide Console.

least one person gives me motivation to try again. --bb

import tango.io.Stdout; void main() { Stdout("spade, club, heart, diamond: \u2660\u2663\u2665\u2666"); } I don't know anything about the "type" command, and whether it supports UTF-8. But the console itself ought to be able to handle it. Try compiling the above code and see what happens. --benji

code and the D compiler spit out a message about one of the lines, would that error message come out as garbage? Same for ddbg -- if I'm debugging and say "ps" for "print source" will the result be garbage. I was thinking that "type" would be a simple test if that sort of thing would work. But maybe type is just borked. I did try "cat" and "more" too I think, with same result, though. --bb

look unix like though.. i.e. /c/program files/...

Right that's what Cygwin does too, and it's useless if I want to call the DMD compiler. dmd foo.d /c/libs/mydlib.lib "Error: what do you think this is, Linux?"
 from what I know (winXP sp 2) - console works for unicode Except for RTL
 languages like Hebrew. as someone else already noted, this is legacy
 tech which you shouldn't be using anyway. I don't know if it's fixed in
 SP3 or not but the new way from MS is their powershell tool based on C#.
 there are also other 3rd party stuff as well..

Yeh, i've heard of that. Do you (or anyone) have any actual experience with PowerShell? It doesn't seem to be standard equipment on my new Vista box even. Does it require a separate download? Strange if it really is supposed to be "the new way". --bb

PowerShell is MS's concession that there are things better done in a console environment, especially for developers & powerusers. And, yes, it works very well (I'm a fan...). It also contains aliases for all the GNU tools (i.e. ls => dir, etc.). It doesn't come as the default on most OSes simply because Microsoft doesn't expect the average home user to need it. It does come default on Windows Server 2008, because Microsoft expects it to be a useful utility to server admins.
Oct 25 2008
prev sibling parent Sergey Gromov <snake.scaly gmail.com> writes:
Bill Baxter пишет:
 On Sat, Oct 25, 2008 at 10:37 AM, Benji Smith <dlanguage benjismith.net> wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 10:23 AM, Benji Smith <dlanguage benjismith.net>
 wrote:
 Bill Baxter wrote:
 Anyone using a shell for Windows that works and supports UTF-8
 properly?

* Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.


console. The only special thing I did was changed the font to Lucide Console.

least one person gives me motivation to try again. --bb

import tango.io.Stdout; void main() { Stdout("spade, club, heart, diamond: \u2660\u2663\u2665\u2666"); } I don't know anything about the "type" command, and whether it supports UTF-8. But the console itself ought to be able to handle it. Try compiling the above code and see what happens. --benji

Ah, I see. I guess more what I want to know is if I had utf-8 source code and the D compiler spit out a message about one of the lines, would that error message come out as garbage? Same for ddbg -- if I'm debugging and say "ps" for "print source" will the result be garbage. I was thinking that "type" would be a simple test if that sort of thing would work. But maybe type is just borked. I did try "cat" and "more" too I think, with same result, though.

They all work for me: type, cat, less. The file is UTF-8 with BOM. Error messages are printed correctly displaying all the characters in a buggy symbol. But now I remember. It fails to execute any batch files when it's in 65001 codepage. More precisely, it executes exactly one line from a batch file like if there were no more lines. So this pseudo-uniclde mode is useless.
Oct 27 2008
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Bill Baxter" wrote
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

Console is a legacy technology (you even still call it "DOS"), why expect features from it?

So tell me what the alternative is? I had trouble with running D tools from a Cygwin shell. Can't remember if I tried MSYS or not.

Any text-based program uses the same Windows console (unless it's a GUI application, and it uses controls to create a text box, etc). Including cygwin shell. To say it's a legacy technology is like saying Linux is a legacy technology because it's command line based. It's a false experience promoted by Microsoft to try and spread FUD about OSes that mainly support command line tools, like Linux. But command line tools are extremely useful and powerful, much easier to develop, and IMO easier to use. For instance, if you want to find all files that contain a certain text, grep -R text / and you're done. On windows it's 'click the start menu, select search, wait for the search window to pop up, click on the dog, etc'. Freaking annoying if you ask me ;)
 Anyone using a shell for Windows that works and supports UTF-8 properly?

I would guess it should work properly, most everything in windows supports unicode. Perhaps you have some configuration setting not set properly? I'd suggest searching msdn. -Steve
Oct 24 2008
parent reply Yigal Chripun <yigal100 gmail.com> writes:
Steven Schveighoffer wrote:
 "Bill Baxter" wrote
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

features from it?

tools from a Cygwin shell. Can't remember if I tried MSYS or not.

Any text-based program uses the same Windows console (unless it's a GUI application, and it uses controls to create a text box, etc). Including cygwin shell. To say it's a legacy technology is like saying Linux is a legacy technology because it's command line based. It's a false experience promoted by Microsoft to try and spread FUD about OSes that mainly support command line tools, like Linux. But command line tools are extremely useful and powerful, much easier to develop, and IMO easier to use. For instance, if you want to find all files that contain a certain text, grep -R text / and you're done. On windows it's 'click the start menu, select search, wait for the search window to pop up, click on the dog, etc'. Freaking annoying if you ask me ;)
 Anyone using a shell for Windows that works and supports UTF-8 properly?

I would guess it should work properly, most everything in windows supports unicode. Perhaps you have some configuration setting not set properly? I'd suggest searching msdn. -Steve

windows console AKA DOS Box *is* in fact legacy technology. It is replaced by MS Powershell which is based on C#. they actually took many ideas from Linux and incorporated in it. Also, it doesn't have to be either/or situation regarding CLI vs GUI. There's Apple's quicksilver (IIRC the name) which is a gui app with CLI like interface. it has the best from both worlds. PowerShell is GUI based as well. IMO, CLI should be provided as just a widget in the GUI world and not a separate entity.
Oct 25 2008
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Bill Baxter" wrote
 On Sat, Oct 25, 2008 at 8:57 PM, Yigal Chripun <yigal100 gmail.com> wrote:
 Steven Schveighoffer wrote:
 "Bill Baxter" wrote
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

expect features from it?

tools from a Cygwin shell. Can't remember if I tried MSYS or not.

Any text-based program uses the same Windows console (unless it's a GUI application, and it uses controls to create a text box, etc). Including cygwin shell. To say it's a legacy technology is like saying Linux is a legacy technology because it's command line based. It's a false experience promoted by Microsoft to try and spread FUD about OSes that mainly support command line tools, like Linux. But command line tools are extremely useful and powerful, much easier to develop, and IMO easier to use. For instance, if you want to find all files that contain a certain text, grep -R text / and you're done. On windows it's 'click the start menu, select search, wait for the search window to pop up, click on the dog, etc'. Freaking annoying if you ask me ;)
 Anyone using a shell for Windows that works and supports UTF-8 
 properly?

I would guess it should work properly, most everything in windows supports unicode. Perhaps you have some configuration setting not set properly? I'd suggest searching msdn. -Steve


 PowerShell is GUI based as well.

After downloading it and giving it a try, I find this claim somewhat suspect. What makes you say it's GUI based? It has the exact same decorations and goofy menu options as a regular non-GUI Windows console. If it were really a GUI, I doubt they would go through the extra programming effort required to make it look *exactly* like a console app.

I've never used powershell, but most likely you are correct. I think there is a confusion of terms here. Windows Console is the GUI that comes up with the black window, and displays text. It serves as a terminal, not a shell. This is not 'old' technology, it's just an integral piece of the OS. cmd.exe is the command interpreter, which is definitely crappy technology (and somewhat old). The responsible party for displaying UTF properly is the console, not the shell. -Steve
Oct 25 2008
parent ore-sama <spam here.lot> writes:
Steven Schveighoffer Wrote:

 The responsible party for displaying UTF properly is the console, not the 
 shell.
 

Oct 26 2008
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Yigal Chripun wrote:
 Steven Schveighoffer wrote:
 "Bill Baxter" wrote
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

features from it?

tools from a Cygwin shell. Can't remember if I tried MSYS or not.

application, and it uses controls to create a text box, etc). Including cygwin shell. To say it's a legacy technology is like saying Linux is a legacy technology because it's command line based. It's a false experience promoted by Microsoft to try and spread FUD about OSes that mainly support command line tools, like Linux. But command line tools are extremely useful and powerful, much easier to develop, and IMO easier to use. For instance, if you want to find all files that contain a certain text, grep -R text / and you're done. On windows it's 'click the start menu, select search, wait for the search window to pop up, click on the dog, etc'. Freaking annoying if you ask me ;)
 Anyone using a shell for Windows that works and supports UTF-8 properly?

unicode. Perhaps you have some configuration setting not set properly? I'd suggest searching msdn. -Steve

windows console AKA DOS Box *is* in fact legacy technology. It is replaced by MS Powershell which is based on C#. they actually took many ideas from Linux and incorporated in it.

Windows has gotten a lot better in the recent times - ever since it finally started to imitate Unix :o).
 Also, it doesn't have to be either/or situation regarding CLI vs GUI.
 There's Apple's quicksilver (IIRC the name) which is a gui app with CLI
 like interface. it has the best from both worlds. PowerShell is GUI
 based as well. IMO, CLI should be provided as just a widget in the GUI
 world and not a separate entity.

I'm not sure I understand. Widget in the GUI = a window with text in it living side by side, or embedded with, graphical windows? That's been the case for a long time. Andrei
Oct 25 2008
prev sibling next sibling parent reply Robert Fraser <fraserofthenight gmail.com> writes:
Bill Baxter wrote:
 Yigal Chripun wrote:
 PowerShell is GUI based as well.

After downloading it and giving it a try, I find this claim somewhat suspect. What makes you say it's GUI based? It has the exact same decorations and goofy menu options as a regular non-GUI Windows console. If it were really a GUI, I doubt they would go through the extra programming effort required to make it look *exactly* like a console app. --bb

It uses the same console application to do the displaying/execution. And, yes, this application sucks (ever done any serious copy/paste in it?) There's PoshConsole ( http://www.codeplex.com/PoshConsole ), but that TODO list is a bit extensive ;-P. Hopefully by Win7 time, the Windows group gets around to fixing the console, but that's like hoping they'll fix Paint or Notepad ;-P.
Oct 25 2008
next sibling parent KennyTM~ <kennytm gmail.com> writes:
Robert Fraser wrote:
 Bill Baxter wrote:
 Yigal Chripun wrote:
 PowerShell is GUI based as well.

After downloading it and giving it a try, I find this claim somewhat suspect. What makes you say it's GUI based? It has the exact same decorations and goofy menu options as a regular non-GUI Windows console. If it were really a GUI, I doubt they would go through the extra programming effort required to make it look *exactly* like a console app. --bb

It uses the same console application to do the displaying/execution. And, yes, this application sucks (ever done any serious copy/paste in it?) There's PoshConsole ( http://www.codeplex.com/PoshConsole ), but that TODO list is a bit extensive ;-P. Hopefully by Win7 time, the Windows group gets around to fixing the console, but that's like hoping they'll fix Paint or Notepad ;-P.

Hey, they do have fixed MSPaint and WordPad! :)
Oct 25 2008
prev sibling parent reply torhu <no spam.invalid> writes:
Robert Fraser wrote:
 It uses the same console application to do the displaying/execution. 
 And, yes, this application sucks (ever done any serious copy/paste in it?)

That works fine for me if I enable Quick edit mode in the options. Then the right mouse button will do both copy and paste.
Oct 26 2008
parent reply torhu <no spam.invalid> writes:
Bill Baxter wrote:
 On Mon, Oct 27, 2008 at 1:51 AM, torhu <no spam.invalid> wrote:
 Robert Fraser wrote:
 It uses the same console application to do the displaying/execution. And,
 yes, this application sucks (ever done any serious copy/paste in it?)

That works fine for me if I enable Quick edit mode in the options. Then the right mouse button will do both copy and paste.

Except it only does block-oriented rectangular selection, which is odd for something that is primarily line-oriented.

Yeah, that's true. Pretty stupid.
Oct 26 2008
parent Robert Fraser <fraserofthenight gmail.com> writes:
torhu wrote:
 Bill Baxter wrote:
 On Mon, Oct 27, 2008 at 1:51 AM, torhu <no spam.invalid> wrote:
 Robert Fraser wrote:
 It uses the same console application to do the displaying/execution. 
 And,
 yes, this application sucks (ever done any serious copy/paste in it?)

That works fine for me if I enable Quick edit mode in the options. Then the right mouse button will do both copy and paste.

Except it only does block-oriented rectangular selection, which is odd for something that is primarily line-oriented.

Yeah, that's true. Pretty stupid.

My main problem is that you can't do it just with the keyboard, which is my standard method. I also take issue with the fact you can't copy more than is visible on a single screen, which goes along with the block selection mode.
Oct 26 2008
prev sibling parent Yigal Chripun <yigal100 gmail.com> writes:
Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 8:57 PM, Yigal Chripun <yigal100 gmail.com> wrote:
 Steven Schveighoffer wrote:
 "Bill Baxter" wrote
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

features from it?

tools from a Cygwin shell. Can't remember if I tried MSYS or not.

application, and it uses controls to create a text box, etc). Including cygwin shell. To say it's a legacy technology is like saying Linux is a legacy technology because it's command line based. It's a false experience promoted by Microsoft to try and spread FUD about OSes that mainly support command line tools, like Linux. But command line tools are extremely useful and powerful, much easier to develop, and IMO easier to use. For instance, if you want to find all files that contain a certain text, grep -R text / and you're done. On windows it's 'click the start menu, select search, wait for the search window to pop up, click on the dog, etc'. Freaking annoying if you ask me ;)
 Anyone using a shell for Windows that works and supports UTF-8 properly?

unicode. Perhaps you have some configuration setting not set properly? I'd suggest searching msdn. -Steve


 PowerShell is GUI based as well.

After downloading it and giving it a try, I find this claim somewhat suspect. What makes you say it's GUI based? It has the exact same decorations and goofy menu options as a regular non-GUI Windows console. If it were really a GUI, I doubt they would go through the extra programming effort required to make it look *exactly* like a console app. --bb

I've just checked (it's been a long time since I used it) and you're correct. I don't know Why I remembered it as being GUI based, maybe the blue color threw me off..sorry for the confusion. but I'm sure that there are 3rd party GUI based shells for Windows.
Oct 27 2008
prev sibling parent ore-sama <spam here.lot> writes:
Bill Baxter Wrote:

 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

Console is a legacy technology (you even still call it "DOS"), why expect features from it?

So tell me what the alternative is? I had trouble with running D tools from a Cygwin shell. Can't remember if I tried MSYS or not.

gui of course. MSYS's console is gui in fact.
Oct 25 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Oct 25, 2008 at 9:15 AM, Sergey Gromov <snake.scaly gmail.com> wrote:
 Sat, 25 Oct 2008 06:43:19 +0900,
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

Console is a legacy technology (you even still call it "DOS"), why expect features from it?

So tell me what the alternative is? I had trouble with running D tools from a Cygwin shell. Can't remember if I tried MSYS or not. Anyone using a shell for Windows that works and supports UTF-8 properly?

A regular Windows console supports UTF-8 to some extent: * Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.

I did that but "type <filewith-utf8.txt>" still prints garbage. --bb
Oct 24 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Oct 25, 2008 at 10:23 AM, Benji Smith <dlanguage benjismith.net> wrote:
 Bill Baxter wrote:
 Anyone using a shell for Windows that works and supports UTF-8 properly?

A regular Windows console supports UTF-8 to some extent: * Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.

I did that but "type <filewith-utf8.txt>" still prints garbage.

That's weird. My machine (WinXp Sp3) has no problem printing UTF-8 to the console. The only special thing I did was changed the font to Lucide Console.

Ok. Thanks for the info. Knowing that it has actually worked for at least one person gives me motivation to try again. --bb
Oct 24 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Oct 25, 2008 at 10:23 AM, Yigal Chripun <yigal100 gmail.com> wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 9:15 AM, Sergey Gromov <snake.scaly gmail.com> wrote:
 Sat, 25 Oct 2008 06:43:19 +0900,
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)


tools from a Cygwin shell. Can't remember if I tried MSYS or not. Anyone using a shell for Windows that works and supports UTF-8 properly?

* Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.

I did that but "type <filewith-utf8.txt>" still prints garbage. --bb

so don't use type. use notepad instead... notepad <filewith-utf8.txt>

Ok what about grep and sort and uniq then? Can notepad do that? I have all these tools that work fine in my DOS shell. I never use "type". It was simply meant as the most basic possible tool -- as in if "type" doesn't work nothing will.
 also, MSYS gives you all the linux tools if you really need to be shell
 only.

I think part of the problem I had with Cygwin shell was that it can't auto-complete dos filenames, but D programs on Windows can't accept Cygwin paths. So it was a pain to work with command-line tools (like DMD itself) that take filenames. So I don't think MSYS helps there either.
 last resort: nothing stops you from implementing your own "cat"
 application in D with full Unicode support.

 most if not all linux shell tools are separate executables anyway and if
 any still do not support unicode it'll be trivial to roll your own
 replacements for the bad ones.

Oct 24 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Oct 25, 2008 at 10:31 AM, Benji Smith <dlanguage benjismith.net> wrote:
 Yigal Chripun wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 9:15 AM, Sergey Gromov <snake.scaly gmail.com>
 wrote:
 Sat, 25 Oct 2008 06:43:19 +0900,
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

Console is a legacy technology (you even still call it "DOS"), why expect features from it?

So tell me what the alternative is? I had trouble with running D tools from a Cygwin shell. Can't remember if I tried MSYS or not. Anyone using a shell for Windows that works and supports UTF-8 properly?

A regular Windows console supports UTF-8 to some extent: * Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.

I did that but "type <filewith-utf8.txt>" still prints garbage. --bb

so don't use type. use notepad instead... notepad <filewith-utf8.txt> also, MSYS gives you all the linux tools if you really need to be shell only. last resort: nothing stops you from implementing your own "cat" application in D with full Unicode support. most if not all linux shell tools are separate executables anyway and if any still do not support unicode it'll be trivial to roll your own replacements for the bad ones.

Oh, and one of my favorite tricks in Windows is to install cygwin (usually at "C:\cygwin" or whatever their boneheaded installer insists on using) and then add the bin path ("C:\cygwin\bin") to the windows PATH. That way, I can continue using the ordinary windows shell (which I prefer, since it doesn't force me to use the nutty directory names that the cygwin shell uses), but I can still access all the linux commands. Calling grep from a windows shell is the bestest!

But that has the same problem. Cygtools don't understand windows paths so barf when you say "grep c:\foo.txt" But the Windows shell only will only autocomplete Windows-style paths. I've found the gnuwin32 tools to work a little better on that front. --bb
Oct 24 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Oct 25, 2008 at 10:37 AM, Benji Smith <dlanguage benjismith.net> wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 10:23 AM, Benji Smith <dlanguage benjismith.net>
 wrote:
 Bill Baxter wrote:
 Anyone using a shell for Windows that works and supports UTF-8
 properly?

A regular Windows console supports UTF-8 to some extent: * Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.

I did that but "type <filewith-utf8.txt>" still prints garbage.

That's weird. My machine (WinXp Sp3) has no problem printing UTF-8 to the console. The only special thing I did was changed the font to Lucide Console.

Ok. Thanks for the info. Knowing that it has actually worked for at least one person gives me motivation to try again. --bb

Write a tiny little D program and see what you get on the console: import tango.io.Stdout; void main() { Stdout("spade, club, heart, diamond: \u2660\u2663\u2665\u2666"); } I don't know anything about the "type" command, and whether it supports UTF-8. But the console itself ought to be able to handle it. Try compiling the above code and see what happens. --benji

Ah, I see. I guess more what I want to know is if I had utf-8 source code and the D compiler spit out a message about one of the lines, would that error message come out as garbage? Same for ddbg -- if I'm debugging and say "ps" for "print source" will the result be garbage. I was thinking that "type" would be a simple test if that sort of thing would work. But maybe type is just borked. I did try "cat" and "more" too I think, with same result, though. --bb
Oct 24 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Oct 25, 2008 at 11:39 AM, Benji Smith <dlanguage benjismith.net> wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 10:31 AM, Benji Smith <dlanguage benjismith.net>
 wrote:
 Yigal Chripun wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 9:15 AM, Sergey Gromov <snake.scaly gmail.com>
 wrote:
 Sat, 25 Oct 2008 06:43:19 +0900,
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

Console is a legacy technology (you even still call it "DOS"), why expect features from it?

So tell me what the alternative is? I had trouble with running D tools from a Cygwin shell. Can't remember if I tried MSYS or not. Anyone using a shell for Windows that works and supports UTF-8 properly?

A regular Windows console supports UTF-8 to some extent: * Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.

I did that but "type <filewith-utf8.txt>" still prints garbage. --bb

so don't use type. use notepad instead... notepad <filewith-utf8.txt> also, MSYS gives you all the linux tools if you really need to be shell only. last resort: nothing stops you from implementing your own "cat" application in D with full Unicode support. most if not all linux shell tools are separate executables anyway and if any still do not support unicode it'll be trivial to roll your own replacements for the bad ones.

Oh, and one of my favorite tricks in Windows is to install cygwin (usually at "C:\cygwin" or whatever their boneheaded installer insists on using) and then add the bin path ("C:\cygwin\bin") to the windows PATH. That way, I can continue using the ordinary windows shell (which I prefer, since it doesn't force me to use the nutty directory names that the cygwin shell uses), but I can still access all the linux commands. Calling grep from a windows shell is the bestest!

But that has the same problem. Cygtools don't understand windows paths so barf when you say "grep c:\foo.txt" But the Windows shell only will only autocomplete Windows-style paths. I've found the gnuwin32 tools to work a little better on that front. --bb

Wha??? The "grep" tool doesn't read the path. The *shell* interprets the path and passes the text to the program. That's how all the gnu tools are able to pipe their results from one tool to the other. Or at least, that's how I assume it works. Cuz I use grep like every single day. On the "cmd.exe" shell. With windows paths. In fact, just for you, I tested this: grep -i "SHAZZAM" "C:\Documents and Settings\benji\Desktop\my filename with spaces.txt" Worked like a charm. If the path doesn't have spaces, I have no problem with this: grep -i "SHAZZAM" C:\file.txt I tried it in both "command.com" and in "cmd.exe" and didn't experience any problem in either environment. The key is to never never never use the cygwin shell. It's a piece of garbage. But using the executables from the "cygwin\bin" directory within the windows shell... Priceless!

Oh, I didn't realize that. There is one thing that doesn't work, which is probably what gave me the impression it was broken -- Windows paths with wildcards don't work. Like "grep c:\Windows\*.txt". But you're right that it does seem to work for both windows paths, and local wildcards, just not Windows paths with wildcards. But that's great. Thanks for the info. Actually I used to put cygwin\bin on my path years ago, but stopped doing it at some point and switched to gnuwin32. I was under the impression that it worked better then, but actually I've had some trouble with gnuwin32 recently. --bb
Oct 24 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Oct 25, 2008 at 11:53 AM, Yigal Chripun <yigal100 gmail.com> wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 10:37 AM, Benji Smith <dlanguage benjismith.net> wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 10:23 AM, Benji Smith <dlanguage benjismith.net>
 wrote:
 Bill Baxter wrote:
 Anyone using a shell for Windows that works and supports UTF-8
 properly?

* Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.


console. The only special thing I did was changed the font to Lucide Console.

least one person gives me motivation to try again. --bb

import tango.io.Stdout; void main() { Stdout("spade, club, heart, diamond: \u2660\u2663\u2665\u2666"); } I don't know anything about the "type" command, and whether it supports UTF-8. But the console itself ought to be able to handle it. Try compiling the above code and see what happens. --benji

Ah, I see. I guess more what I want to know is if I had utf-8 source code and the D compiler spit out a message about one of the lines, would that error message come out as garbage? Same for ddbg -- if I'm debugging and say "ps" for "print source" will the result be garbage. I was thinking that "type" would be a simple test if that sort of thing would work. But maybe type is just borked. I did try "cat" and "more" too I think, with same result, though. --bb

Msys does autocomplete. it's not perfect but it works. the path will look unix like though.. i.e. /c/program files/...

Right that's what Cygwin does too, and it's useless if I want to call the DMD compiler. dmd foo.d /c/libs/mydlib.lib "Error: what do you think this is, Linux?"
 from what I know (winXP sp 2) - console works for unicode Except for RTL
 languages like Hebrew. as someone else already noted, this is legacy
 tech which you shouldn't be using anyway. I don't know if it's fixed in
 SP3 or not but the new way from MS is their powershell tool based on C#.
 there are also other 3rd party stuff as well..

Yeh, i've heard of that. Do you (or anyone) have any actual experience with PowerShell? It doesn't seem to be standard equipment on my new Vista box even. Does it require a separate download? Strange if it really is supposed to be "the new way". --bb
Oct 24 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
 But that has the same problem.  Cygtools don't understand windows
 paths so barf when you say "grep c:\foo.txt"  But the Windows shell
 only will only autocomplete Windows-style paths.

 I've found the gnuwin32 tools to work a little better on that front.

The "grep" tool doesn't read the path. The *shell* interprets the path and passes the text to the program. That's how all the gnu tools are able to pipe their results from one tool to the other. Or at least, that's how I assume it works.

No, that's how it works with the Bash shell and most Unix shells, but the Windows console doesn't do that stuff. It's up to each app to interpret and expand wildcards like *.txt. So the cygwin progs must be explicitly checking to see if they got a * from a stupid DOS console and doing the glob themselves. But the implementation is apparently imperfect since it doesn't work on full DOS paths with wildcards. --bb
Oct 24 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Oct 25, 2008 at 1:33 PM, Steven Schveighoffer
<schveiguy yahoo.com> wrote:
 "Benji Smith" wrote
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 10:31 AM, Benji Smith <dlanguage benjismith.net>
 wrote:
 Yigal Chripun wrote:
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 9:15 AM, Sergey Gromov <snake.scaly gmail.com>
 wrote:
 Sat, 25 Oct 2008 06:43:19 +0900,
 Bill Baxter wrote:
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

expect features from it?

tools from a Cygwin shell. Can't remember if I tried MSYS or not. Anyone using a shell for Windows that works and supports UTF-8 properly?

* Change console font to Lucida Console * issue "chcp 65001" You can even get more fonts into there with a bit of hackery.

--bb

notepad <filewith-utf8.txt> also, MSYS gives you all the linux tools if you really need to be shell only. last resort: nothing stops you from implementing your own "cat" application in D with full Unicode support. most if not all linux shell tools are separate executables anyway and if any still do not support unicode it'll be trivial to roll your own replacements for the bad ones.

(usually at "C:\cygwin" or whatever their boneheaded installer insists on using) and then add the bin path ("C:\cygwin\bin") to the windows PATH. That way, I can continue using the ordinary windows shell (which I prefer, since it doesn't force me to use the nutty directory names that the cygwin shell uses), but I can still access all the linux commands. Calling grep from a windows shell is the bestest!

But that has the same problem. Cygtools don't understand windows paths so barf when you say "grep c:\foo.txt" But the Windows shell only will only autocomplete Windows-style paths. I've found the gnuwin32 tools to work a little better on that front. --bb

Wha??? The "grep" tool doesn't read the path. The *shell* interprets the path and passes the text to the program. That's how all the gnu tools are able to pipe their results from one tool to the other. Or at least, that's how I assume it works.

No, grep accepts either input. The shell does not change paths to windows style, that is what cygpath is for. But it does interpret backslashes, so you have to double all those. So for instance, in a cygwin shell, this works also: grep -i "SHAZZAM" C:\\Documents\ and\ Settings\\benji\\Desktop\\my\ filename\ with\ spaces.txt The arguments are passed as they are, grep just is smart enough to use either one. Probably many tools are that way, I wouldn't know because I usually do the /cygdrive/c/... form.
 The key is to never never never use the cygwin shell. It's a piece of
 garbage. But using the executables from the "cygwin\bin" directory within
 the windows shell... Priceless!

Without the cygwin shell, you lose all bash features, like for, or backticks to execute a command and use it's output. The paths are a minor annoyance IMO. Using the cmd.exe shell is ok for simple tasks, but it pales severely in comparison to the power of bash. So piece of garbage it is not. Something you don't understand how to use properly? definitely ;)

Yeh, I love the bash shell. Really the only thing keeping me from using it for D work is the fact that it won't auto-complete Windows filenames. --bb
Oct 24 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Oct 25, 2008 at 1:40 PM, Steven Schveighoffer
<schveiguy yahoo.com> wrote:
 "Benji Smith" wrote
 Bill Baxter wrote:
 Benji Smith wrote:
 The key is to never never never use the cygwin shell. It's a piece of
 garbage. But using the executables from the "cygwin\bin" directory
 within
 the windows shell... Priceless!

Oh, I didn't realize that. There is one thing that doesn't work, which is probably what gave me the impression it was broken -- Windows paths with wildcards don't work. Like "grep c:\Windows\*.txt". But you're right that it does seem to work for both windows paths, and local wildcards, just not Windows paths with wildcards.


It's not the paths with wildcards that is the problem. In this case, it is the shell. Grep is expecting the shell to expand the wildcards, as it does on unix.

Read again. Particularly this part: "it does seem to work for both windows paths, **and local wildcards**, just not Windows paths with wildcards". (emphasis added) "grep Foo *.txt" works just fine. "grep Foo c:\*.txt" does not. --bb
Oct 24 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Sat, Oct 25, 2008 at 8:57 PM, Yigal Chripun <yigal100 gmail.com> wrote:
 Steven Schveighoffer wrote:
 "Bill Baxter" wrote
 On Sat, Oct 25, 2008 at 6:37 AM, ore-sama <spam here.lot> wrote:
 Bill Baxter Wrote:

 (like I haven't been able to figure out how to get the
 DOS console in Windows to display UTF-8)

features from it?

tools from a Cygwin shell. Can't remember if I tried MSYS or not.

Any text-based program uses the same Windows console (unless it's a GUI application, and it uses controls to create a text box, etc). Including cygwin shell. To say it's a legacy technology is like saying Linux is a legacy technology because it's command line based. It's a false experience promoted by Microsoft to try and spread FUD about OSes that mainly support command line tools, like Linux. But command line tools are extremely useful and powerful, much easier to develop, and IMO easier to use. For instance, if you want to find all files that contain a certain text, grep -R text / and you're done. On windows it's 'click the start menu, select search, wait for the search window to pop up, click on the dog, etc'. Freaking annoying if you ask me ;)
 Anyone using a shell for Windows that works and supports UTF-8 properly?

I would guess it should work properly, most everything in windows supports unicode. Perhaps you have some configuration setting not set properly? I'd suggest searching msdn. -Steve


 PowerShell is GUI based as well.

After downloading it and giving it a try, I find this claim somewhat suspect. What makes you say it's GUI based? It has the exact same decorations and goofy menu options as a regular non-GUI Windows console. If it were really a GUI, I doubt they would go through the extra programming effort required to make it look *exactly* like a console app. --bb
Oct 25 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Sun, Oct 26, 2008 at 9:18 AM, Robert Fraser
<fraserofthenight gmail.com> wrote:
 Bill Baxter wrote:
 Yigal Chripun wrote:
 PowerShell is GUI based as well.

After downloading it and giving it a try, I find this claim somewhat suspect. What makes you say it's GUI based? It has the exact same decorations and goofy menu options as a regular non-GUI Windows console. If it were really a GUI, I doubt they would go through the extra programming effort required to make it look *exactly* like a console app. --bb

It uses the same console application to do the displaying/execution. And, yes, this application sucks (ever done any serious copy/paste in it?) There's PoshConsole ( http://www.codeplex.com/PoshConsole ), but that TODO list is a bit extensive ;-P. Hopefully by Win7 time, the Windows group gets around to fixing the console, but that's like hoping they'll fix Paint or Notepad ;-P.

I'm using "Console2" as my facade on the console window. Works pretty nicely. http://sourceforge.net/projects/console/ --bb
Oct 25 2008
prev sibling next sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Mon, Oct 27, 2008 at 1:51 AM, torhu <no spam.invalid> wrote:
 Robert Fraser wrote:
 It uses the same console application to do the displaying/execution. And,
 yes, this application sucks (ever done any serious copy/paste in it?)

That works fine for me if I enable Quick edit mode in the options. Then the right mouse button will do both copy and paste.

Except it only does block-oriented rectangular selection, which is odd for something that is primarily line-oriented. --bb
Oct 26 2008
prev sibling parent "Bill Baxter" <wbaxter gmail.com> writes:
On Mon, Oct 27, 2008 at 1:52 PM, Robert Fraser
<fraserofthenight gmail.com> wrote:
 torhu wrote:
 Bill Baxter wrote:
 On Mon, Oct 27, 2008 at 1:51 AM, torhu <no spam.invalid> wrote:
 Robert Fraser wrote:
 It uses the same console application to do the displaying/execution.
 And,
 yes, this application sucks (ever done any serious copy/paste in it?)

That works fine for me if I enable Quick edit mode in the options. Then the right mouse button will do both copy and paste.

Except it only does block-oriented rectangular selection, which is odd for something that is primarily line-oriented.

Yeah, that's true. Pretty stupid.

My main problem is that you can't do it just with the keyboard, which is my standard method. I also take issue with the fact you can't copy more than is visible on a single screen, which goes along with the block selection mode.

By the way I tried running powershell as a tab inside the Console2 prog I mentioned before and it does work fine. --bb
Oct 26 2008