www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.dwt - StyledText not accept Multi-bytes characters

reply yidabu <yidabu.spam gmail.com> writes:
program crashed when StyledText.text encounter multi-bytes characters, e.g.
Chinese.

code below causes EXCEPTION_ACCESS_VIOLATION :
Unhandled Exception: EXCEPTION_ACCESS_VIOLATION(0xc0000005) at ntdll.dll
(0x7c90316c) thread(1260)
->us
#0 ?? () from ntdll.dll

Code:

import dwt.DWT;
import dwt.custom.StyledText;
import dwt.custom.StyleRange;
import dwt.layout.FillLayout;
import dwt.widgets.Display;
import dwt.widgets.Shell;
import dwt.widgets.Listener;
import dwt.widgets.Event;
import dwt.graphics.Color;
import dwt.graphics.Point;

import dwt.dwthelper.utils;

void main() {
    static String SEARCH_STRING = "box";
    Display display = new Display();
    Color RED = display.getSystemColor(DWT.COLOR_RED);
    Shell shell = new Shell(display);
    shell.setBounds(10,10,250,250);
    StyledText text = new StyledText(shell, DWT.NONE);
    text.setBounds(10,10,200,200);
    text.setText("StyledText not accept Chinese 中国? ");
    shell.open();
    while (!shell.isDisposed()) {
        if (!display.readAndDispatch()) display.sleep();
    }
    display.dispose();
}




-- 
yidabu <yidabu.spam gmail.com>
http://www.dsource.org/projects/dwin

D 语言-中文(D Chinese):
http://www.d-programming-language-china.org/
http://bbs.d-programming-language-china.org/
http://dwin.d-programming-language-china.org/
http://scite4d.d-programming-language-china.org/
Aug 18 2008
next sibling parent reply Frank Benoit <keinfarbton googlemail.com> writes:
yidabu schrieb:
 program crashed when StyledText.text encounter multi-bytes characters, e.g.
Chinese.
 
 code below causes EXCEPTION_ACCESS_VIOLATION :
 Unhandled Exception: EXCEPTION_ACCESS_VIOLATION(0xc0000005) at ntdll.dll
(0x7c90316c) thread(1260)
 ->us
 #0 ?? () from ntdll.dll
 
 Code:
 
 import dwt.DWT;
 import dwt.custom.StyledText;
 import dwt.custom.StyleRange;
 import dwt.layout.FillLayout;
 import dwt.widgets.Display;
 import dwt.widgets.Shell;
 import dwt.widgets.Listener;
 import dwt.widgets.Event;
 import dwt.graphics.Color;
 import dwt.graphics.Point;
 
 import dwt.dwthelper.utils;
 
 void main() {
     static String SEARCH_STRING = "box";
     Display display = new Display();
     Color RED = display.getSystemColor(DWT.COLOR_RED);
     Shell shell = new Shell(display);
     shell.setBounds(10,10,250,250);
     StyledText text = new StyledText(shell, DWT.NONE);
     text.setBounds(10,10,200,200);
     text.setText("StyledText not accept Chinese 涓浗? ");
     shell.open();
     while (!shell.isDisposed()) {
         if (!display.readAndDispatch()) display.sleep();
     }
     display.dispose();
 }
 
 
 
 

Yes, there is indeed trouble with it. StyledText uses dwt.graphics.TextLayout which implements all this and is the troublemaker. Especially on windows the ported code is not working correctly. This is because of dwt uses utf8 storage but the windows api uses utf16. This make the calculation of indices very hard. It is on my TODO list since a while now :/
Aug 18 2008
parent reply John Reimer <terminal.node gmail.com> writes:
Frank Benoit wrote:

 yidabu schrieb:
 program crashed when StyledText.text encounter multi-bytes characters,
 e.g. Chinese.
 
 code below causes EXCEPTION_ACCESS_VIOLATION :
 Unhandled Exception: EXCEPTION_ACCESS_VIOLATION(0xc0000005) at ntdll.dll
 (0x7c90316c) thread(1260) ->us
 #0 ?? () from ntdll.dll
 
 Code:
 
 import dwt.DWT;
 import dwt.custom.StyledText;
 import dwt.custom.StyleRange;
 import dwt.layout.FillLayout;
 import dwt.widgets.Display;
 import dwt.widgets.Shell;
 import dwt.widgets.Listener;
 import dwt.widgets.Event;
 import dwt.graphics.Color;
 import dwt.graphics.Point;
 
 import dwt.dwthelper.utils;
 
 void main() {
     static String SEARCH_STRING = "box";
     Display display = new Display();
     Color RED = display.getSystemColor(DWT.COLOR_RED);
     Shell shell = new Shell(display);
     shell.setBounds(10,10,250,250);
     StyledText text = new StyledText(shell, DWT.NONE);
     text.setBounds(10,10,200,200);
     text.setText("StyledText not accept Chinese 涓浗? ");
     shell.open();
     while (!shell.isDisposed()) {
         if (!display.readAndDispatch()) display.sleep();
     }
     display.dispose();
 }
 
 
 
 

Yes, there is indeed trouble with it. StyledText uses dwt.graphics.TextLayout which implements all this and is the troublemaker. Especially on windows the ported code is not working correctly. This is because of dwt uses utf8 storage but the windows api uses utf16. This make the calculation of indices very hard. It is on my TODO list since a while now :/

So what does this mean? Can't the utf8 text just be upconverted to utf16 using something like tango.text.convert.Utf.toString16 before the system function is called? I'd be interested to see the problem fully described so that I understand what to look out for while porting code. Is the information somewhere on our site? I recall you mentioning the issue in IRC recently. Thanks. -JJR
Aug 18 2008
parent Frank Benoit <keinfarbton googlemail.com> writes:
John Reimer schrieb:

 So what does this mean?  Can't the utf8 text just be upconverted to utf16
 using something like tango.text.convert.Utf.toString16 before the system
 function is called? 
 
 I'd be interested to see the problem fully described so that I understand
 what to look out for while porting code.  Is the information somewhere on
 our site?  I recall you mentioning the issue in IRC recently.
 
 Thanks.
 
 -JJR

Both is needed, the utf8 and utf16. I think the best is to have both versions in parallel and also maintain the indices for both.
Aug 19 2008
prev sibling next sibling parent reply Frank Benoit <keinfarbton googlemail.com> writes:
yidabu schrieb:
 program crashed when StyledText.text encounter multi-bytes characters, e.g.
Chinese.
 
 code below causes EXCEPTION_ACCESS_VIOLATION :
 Unhandled Exception: EXCEPTION_ACCESS_VIOLATION(0xc0000005) at ntdll.dll
(0x7c90316c) thread(1260)
 ->us
 #0 ?? () from ntdll.dll
 
 Code:
 
 import dwt.DWT;
 import dwt.custom.StyledText;
 import dwt.custom.StyleRange;
 import dwt.layout.FillLayout;
 import dwt.widgets.Display;
 import dwt.widgets.Shell;
 import dwt.widgets.Listener;
 import dwt.widgets.Event;
 import dwt.graphics.Color;
 import dwt.graphics.Point;
 
 import dwt.dwthelper.utils;
 
 void main() {
     static String SEARCH_STRING = "box";
     Display display = new Display();
     Color RED = display.getSystemColor(DWT.COLOR_RED);
     Shell shell = new Shell(display);
     shell.setBounds(10,10,250,250);
     StyledText text = new StyledText(shell, DWT.NONE);
     text.setBounds(10,10,200,200);
     text.setText("StyledText not accept Chinese 涓浗? ");
     shell.open();
     while (!shell.isDisposed()) {
         if (!display.readAndDispatch()) display.sleep();
     }
     display.dispose();
 }
 
 
 
 

I did some changes to TextLayout. Your example works now. More testing welcome.
Aug 21 2008
parent Frank Benoit <keinfarbton googlemail.com> writes:
yidabu schrieb:
 On Thu, 21 Aug 2008 09:59:31 +0200
 Frank Benoit <keinfarbton googlemail.com> wrote:
 
 yidabu schrieb:
 program crashed when StyledText.text encounter multi-bytes characters, e.g.
Chinese.

 code below causes EXCEPTION_ACCESS_VIOLATION :
 Unhandled Exception: EXCEPTION_ACCESS_VIOLATION(0xc0000005) at ntdll.dll
(0x7c90316c) thread(1260)
 ->us
 #0 ?? () from ntdll.dll

 Code:

 import dwt.DWT;
 import dwt.custom.StyledText;
 import dwt.custom.StyleRange;
 import dwt.layout.FillLayout;
 import dwt.widgets.Display;
 import dwt.widgets.Shell;
 import dwt.widgets.Listener;
 import dwt.widgets.Event;
 import dwt.graphics.Color;
 import dwt.graphics.Point;

 import dwt.dwthelper.utils;

 void main() {
     static String SEARCH_STRING = "box";
     Display display = new Display();
     Color RED = display.getSystemColor(DWT.COLOR_RED);
     Shell shell = new Shell(display);
     shell.setBounds(10,10,250,250);
     StyledText text = new StyledText(shell, DWT.NONE);
     text.setBounds(10,10,200,200);
     text.setText("StyledText not accept Chinese 涓浗? ");
     shell.open();
     while (!shell.isDisposed()) {
         if (!display.readAndDispatch()) display.sleep();
     }
     display.dispose();
 }

Your example works now. More testing welcome.

dwtx.lib building error: dwt\ole\win32\OleControlSite.d(621): Error: e2ir: cannot cast from __GUID** to _ _GUID

Should be fixed now.
Aug 21 2008
prev sibling parent yidabu <yidabu.spam gmail.com> writes:
On Thu, 21 Aug 2008 09:59:31 +0200
Frank Benoit <keinfarbton googlemail.com> wrote:

 yidabu schrieb:
 program crashed when StyledText.text encounter multi-bytes characters, e.g.
Chinese.
 
 code below causes EXCEPTION_ACCESS_VIOLATION :
 Unhandled Exception: EXCEPTION_ACCESS_VIOLATION(0xc0000005) at ntdll.dll
(0x7c90316c) thread(1260)
 ->us
 #0 ?? () from ntdll.dll
 
 Code:
 
 import dwt.DWT;
 import dwt.custom.StyledText;
 import dwt.custom.StyleRange;
 import dwt.layout.FillLayout;
 import dwt.widgets.Display;
 import dwt.widgets.Shell;
 import dwt.widgets.Listener;
 import dwt.widgets.Event;
 import dwt.graphics.Color;
 import dwt.graphics.Point;
 
 import dwt.dwthelper.utils;
 
 void main() {
     static String SEARCH_STRING = "box";
     Display display = new Display();
     Color RED = display.getSystemColor(DWT.COLOR_RED);
     Shell shell = new Shell(display);
     shell.setBounds(10,10,250,250);
     StyledText text = new StyledText(shell, DWT.NONE);
     text.setBounds(10,10,200,200);
     text.setText("StyledText not accept Chinese 中国? ");
     shell.open();
     while (!shell.isDisposed()) {
         if (!display.readAndDispatch()) display.sleep();
     }
     display.dispose();
 }
 
 
 
 

I did some changes to TextLayout. Your example works now. More testing welcome.

dwtx.lib building error: dwt\ole\win32\OleControlSite.d(621): Error: e2ir: cannot cast from __GUID** to _ _GUID -- yidabu <yidabu.spam gmail.com> http://www.dsource.org/projects/dwin D 语言-中文(D Chinese): http://www.d-programming-language-china.org/ http://bbs.d-programming-language-china.org/ http://dwin.d-programming-language-china.org/ http://scite4d.d-programming-language-china.org/
Aug 21 2008