www.digitalmars.com         C & C++   DMDScript  

D - D--

reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
I'm forking from the "Compiling Phobos" thread to discus D-- here.

Basically, D-- would be a subset of the D language that would allow
usable as a JIT compiled scripting language for cases where you didn't
trust the origin of the code.  D-- could be used for mods in games, for
small addons to run in office applications, or even (someday) to write
applets.

What I want to address in this thread is:
1) What features of D need to be removed to allow safe JIT compilation
of untrusted code?
    * No pointers?  Maybe pointers are ok if pointer arithmetic is not
allowed?  If you do this, it's not much different than an array with a
fixed length of 1...
    * No casts (use unions instead, if you must)
    * No asm
    * All arrays must be bounds checked
2) What features need to be limited to allow safe JIT compilation of
untrusted code?
    * imports must be limited to safe libraries only

NOTE: I'm assuming here that the JIT compiled code would be linking to
precompiled dynamic libraries; these libraries could be written in full
D (not necessarily D--) and could have access to all of D's features and
functions.  However, they would only export a small subset of "safe"
functions.

--
The Villagers are Online! villagersonline.com

.[ (the fox.(quick,brown)) jumped.over(the dog.lazy) ]
.[ (a version.of(English).(precise.more)) is(possible) ]
?[ you want.to(help(develop(it))) ]
Apr 29 2002
next sibling parent reply "Walter" <walter digitalmars.com> writes:
"Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
news:3CCDB2E5.315897B6 deming-os.org...
 I'm forking from the "Compiling Phobos" thread to discus D-- here.

 Basically, D-- would be a subset of the D language that would allow
 usable as a JIT compiled scripting language for cases where you didn't
 trust the origin of the code.  D-- could be used for mods in games, for
 small addons to run in office applications, or even (someday) to write
 applets.

 What I want to address in this thread is:
 1) What features of D need to be removed to allow safe JIT compilation
 of untrusted code?
     * No pointers?  Maybe pointers are ok if pointer arithmetic is not
 allowed?  If you do this, it's not much different than an array with a
 fixed length of 1...
     * No casts (use unions instead, if you must)
     * No asm
     * All arrays must be bounds checked
 2) What features need to be limited to allow safe JIT compilation of
 untrusted code?
     * imports must be limited to safe libraries only

You're right, but you could not allow unions.
Apr 29 2002
parent reply Patrick Down <pat codemoon.com> writes:
"Walter" <walter digitalmars.com> wrote in news:aakeis$1ifn$1
 digitaldaemon.com:

 
 "Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
 news:3CCDB2E5.315897B6 deming-os.org...
 I'm forking from the "Compiling Phobos" thread to discus D-- here.

     * No casts (use unions instead, if you must)

You're right, but you could not allow unions.

Yes unions are bad. However casting between object types in the inheritance hierarchy is needed. I think what you want is no casts that are not typechecked at runtime.
Apr 29 2002
parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Patrick Down wrote:

 "Walter" <walter digitalmars.com> wrote in news:aakeis$1ifn$1
  digitaldaemon.com:

 "Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
 news:3CCDB2E5.315897B6 deming-os.org...
 I'm forking from the "Compiling Phobos" thread to discus D-- here.

     * No casts (use unions instead, if you must)

You're right, but you could not allow unions.

Yes unions are bad. However casting between object types in the inheritance hierarchy is needed. I think what you want is no casts that are not typechecked at runtime.

I see what you guys mean about unions; certainly they are dangerous if one of the members is a class reference, an array object, or a pointer. I'm not against removing unions altogether...but they are useful from time to time. Is there a consensus to remove them altogether? Also, I forgot that casting between classes is pretty much mandatory...but yes, things need to be typechecked at runtime if you can't guarantee safety at compile time. Finally, I realized that if you're running with possibly malicious code, you must check all of the in requirements for every function call, at least for the functions in the libraries that get called by the untrusted code. Likewise, if the trusted code calls into the untrusted code, the "out" condition must be checked. 1) Restrictions * No pointer arithmetic, possibly no pointers * No casts except from one class to another, and these must be typechecked. If the cast is guaranteed to be safe, such as casting an object to a base class, this check can be at compile time. Otherwise, it must be checked at runtime. * No asm * No unions that include arrays, pointers, or class references. Arrays made up entire of nonpointer types, including structs made up entirely of nonpointer types. If there's general consensus, we could get rid of unions altogether... 2) Requirements * All arrays must be bounds checked * Imports limited to safe libraries only * All casts must be typechecked (only casts from one class to another are allowed) * When untrusted code calls a function in trusted code, all of the "in" conditions of the function must always be checked. Checking out conditions for those calls is recommended but not required. Likewise, if a function in trusted code calls a function in untrusted code, the "out" condition must be checked, and it is recommended but not required that the "in" condition be checked as well. -- The Villagers are Online! http://villagersonline.com .[ (the fox.(quick,brown)) jumped.over(the dog.lazy) ] .[ (a version.of(English).(precise.more)) is(possible) ] ?[ you want.to(help(develop(it))) ]
Apr 29 2002
parent reply "Walter" <walter digitalmars.com> writes:
"Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
news:3CCDD7C5.7923CF36 deming-os.org...
 I see what you guys mean about unions; certainly they are dangerous if
 one of the members is a class reference, an array object, or a pointer.
 I'm not against removing unions altogether...but they are useful from
 time to time.  Is there a consensus to remove them altogether?

If you're going to have a "safe" language, you cannot allow conversions of random bit patterns into arbitrary types. Hence, unions cannot be allowed.
Apr 29 2002
next sibling parent Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Walter wrote:

 "Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
 news:3CCDD7C5.7923CF36 deming-os.org...
 I see what you guys mean about unions; certainly they are dangerous if
 one of the members is a class reference, an array object, or a pointer.
 I'm not against removing unions altogether...but they are useful from
 time to time.  Is there a consensus to remove them altogether?

If you're going to have a "safe" language, you cannot allow conversions of random bit patterns into arbitrary types. Hence, unions cannot be allowed.

But if we're talking about only "nonpointer" types...that means only integers and floats are allowed. IMHO, it's fine to have bit conversions rom integer to float in an arbitrary language...you just can't have conversions from integer to pointer and stuff like that. Or am I missing something important here? -- The Villagers are Online! villagersonline.com .[ (the fox.(quick,brown)) jumped.over(the dog.lazy) ] .[ (a version.of(English).(precise.more)) is(possible) ] ?[ you want.to(help(develop(it))) ]
Apr 30 2002
prev sibling parent reply "Martin M. Pedersen" <mmp www.moeller-pedersen.dk> writes:
Hi,

"Walter" <walter digitalmars.com> wrote in message
news:aaklkn$1pc9$1 digitaldaemon.com...
 If you're going to have a "safe" language, you cannot allow conversions of
 random bit patterns into arbitrary types. Hence, unions cannot be allowed.

Obviously, you cannot allow unions in the form they appear in C/C++. However, when I think of unions and how I use them, they might serve a purpose in D and might be supported anyway in a restricted form. I use them rarely - the most common example I can think of is YACC parsers. As far as I can see, the compiler could generate a hidden "selector" member, that could be set on assignment, validated on reference, and used by the garbage collector. This would cause a minor run-time penalty, but would ease converting existing C code to D and for example making YACC support D. Regards, Martin M. Pedersen
Apr 30 2002
parent reply Keith Ray <k1e2i3t4h5r6a7y 1m2a3c4.5c6o7m> writes:
In article <aamedh$2lp6$1 digitaldaemon.com>,
 "Martin M. Pedersen" <mmp www.moeller-pedersen.dk> wrote:

 Hi,
 
 "Walter" <walter digitalmars.com> wrote in message
 news:aaklkn$1pc9$1 digitaldaemon.com...
 If you're going to have a "safe" language, you cannot allow conversions of
 random bit patterns into arbitrary types. Hence, unions cannot be allowed.

Obviously, you cannot allow unions in the form they appear in C/C++. However, when I think of unions and how I use them, they might serve a purpose in D and might be supported anyway in a restricted form. I use them rarely - the most common example I can think of is YACC parsers. As far as I can see, the compiler could generate a hidden "selector" member, that could be set on assignment, validated on reference, and used by the garbage collector. This would cause a minor run-time penalty, but would ease converting existing C code to D and for example making YACC support D.

Sounds a lot like polymorphism... with virtual get and set functions. -- C. Keith Ray <http://homepage.mac.com/keithray/xpminifaq.html>
Apr 30 2002
parent reply "Martin M. Pedersen" <mmp www.moeller-pedersen.dk> writes:
Hi,

"Keith Ray" <k1e2i3t4h5r6a7y 1m2a3c4.5c6o7m> wrote in message
news:k1e2i3t4h5r6a7y-C0A6F7.09091330042002 digitalmars.com...
 In article <aamedh$2lp6$1 digitaldaemon.com>,

 As far as I can see, the compiler could generate a hidden "selector"


 that could be set on assignment, validated on reference, and used by the
 garbage collector.


 Sounds a lot like polymorphism... with virtual get and set functions.

Yes, it would be a lot polymorphism. But it saves you for defining a class hierachy. I don't know if unions should be built into the language. Other new features might be more important to be supported. My point was simply that unions have other purposes than simply bypassing the type system, it would be possible for D to support type safe uses of unions, and so, they shouldn't be completely disregarded simply because of their unsafe nature in C/C++. Regards, Martin M Pedersen
Apr 30 2002
parent reply "Sean L. Palmer" <spalmer iname.com> writes:
Unions could be supported (semantically), but implemented the same as
structs.  Stuff should still compile and run unless they do crap like
assert(sizeof(union_type) < sizeof(struct_type)) or rely on changing one
union member causing another union member to change value in a predictable
way.

Sean

"Martin M. Pedersen" <mmp www.moeller-pedersen.dk> wrote in message
news:aaml7l$a0o$1 digitaldaemon.com...
 Hi,

 "Keith Ray" <k1e2i3t4h5r6a7y 1m2a3c4.5c6o7m> wrote in message
 news:k1e2i3t4h5r6a7y-C0A6F7.09091330042002 digitalmars.com...
 In article <aamedh$2lp6$1 digitaldaemon.com>,

 As far as I can see, the compiler could generate a hidden "selector"


 that could be set on assignment, validated on reference, and used by



 garbage collector.


 Sounds a lot like polymorphism... with virtual get and set functions.

Yes, it would be a lot polymorphism. But it saves you for defining a class hierachy. I don't know if unions should be built into the language. Other new

 might be more important to be supported. My point was simply that unions
 have other purposes than simply bypassing the type system, it would be
 possible for D to support type safe uses of unions, and so, they shouldn't
 be completely disregarded simply because of their unsafe nature in C/C++.

 Regards,
 Martin M Pedersen

Apr 30 2002
parent Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
"Sean L. Palmer" wrote:

 Unions could be supported (semantically), but implemented the same as
 structs.  Stuff should still compile and run unless they do crap like
 assert(sizeof(union_type) < sizeof(struct_type)) or rely on changing one
 union member causing another union member to change value in a predictable
 way.

The latter case is why I wouldn't want unions-as-struct. Mapping the binary implementation of one type to another is one of the well-known uses of unions. If we changed them to structs, it would cause subtle errors. We could, however, add a syntax error such as: "Unions not supported in this language. Please use a struct instead." -- The Villagers are Online! villagersonline.com .[ (the fox.(quick,brown)) jumped.over(the dog.lazy) ] .[ (a version.of(English).(precise.more)) is(possible) ] ?[ you want.to(help(develop(it))) ]
May 01 2002
prev sibling next sibling parent reply C.R.Chafer (DrWhat?) <blackmarlin nospam.asean-mail.com> writes:
     * No casts (use unions instead, if you must)

I disagree with this one - unions can be far worse than casts and should themselves be disallowed - imagine union { int niceInteger; int*nastyPointer; } ...that could really make a mess of things. I find unions of little use outside systems code anyway - they are only a means of reducing memory usage, and could be dropped altogether. Maybe casts should be allowed - but only a limited subset. There is no harm in something like ... long myLong = 7; int myInt; myInt = (int)myLong; (though in this case it would be auto converted). Perhaps reducing the stong typing or inbuilt data types could solve this problem - ie why not use something like javascripts 'var'? Or just dissallow casts and import a library to do common safe castings as (inlined) procedures. C 2002/4/29
Apr 29 2002
parent C.R.Chafer (DrWhat?) <blackmarlin nospam.asean-mail.com> writes:
Or allow only allow casts which may be checked at compile time or via RTTI 
provided that is object either is the type casted to or inherits from the 
cast type.

C 2002/4/29
Apr 29 2002
prev sibling next sibling parent reply "Guy Pascarella" <Guy mail.rit.edu> writes:
I think that everyone is on the right track for a stripped down "safe"
version of D, but you might want to call it something other than D--.  Maybe
D'Light (aka delight) or something.

-Guy

"Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
news:3CCDB2E5.315897B6 deming-os.org...
 I'm forking from the "Compiling Phobos" thread to discus D-- here.

 Basically, D-- would be a subset of the D language that would allow
 usable as a JIT compiled scripting language for cases where you didn't
 trust the origin of the code.  D-- could be used for mods in games, for
 small addons to run in office applications, or even (someday) to write
 applets.

 What I want to address in this thread is:
 1) What features of D need to be removed to allow safe JIT compilation
 of untrusted code?
     * No pointers?  Maybe pointers are ok if pointer arithmetic is not
 allowed?  If you do this, it's not much different than an array with a
 fixed length of 1...
     * No casts (use unions instead, if you must)
     * No asm
     * All arrays must be bounds checked
 2) What features need to be limited to allow safe JIT compilation of
 untrusted code?
     * imports must be limited to safe libraries only

 NOTE: I'm assuming here that the JIT compiled code would be linking to
 precompiled dynamic libraries; these libraries could be written in full
 D (not necessarily D--) and could have access to all of D's features and
 functions.  However, they would only export a small subset of "safe"
 functions.

 --
 The Villagers are Online! villagersonline.com

 .[ (the fox.(quick,brown)) jumped.over(the dog.lazy) ]
 .[ (a version.of(English).(precise.more)) is(possible) ]
 ?[ you want.to(help(develop(it))) ]

Apr 29 2002
parent reply C.R.Chafer <blackmarlin nospam.asean-mail.com> writes:
Guy Pascarella wrote:

 I think that everyone is on the right track for a stripped down "safe"
 version of D, but you might want to call it something other than D--. 
 Maybe D'Light (aka delight) or something.
 
 -Guy

DeLight : great idea :-) C 2002/4/30
Apr 29 2002
parent Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
"C.R.Chafer" wrote:

 Guy Pascarella wrote:

 I think that everyone is on the right track for a stripped down "safe"
 version of D, but you might want to call it something other than D--.
 Maybe D'Light (aka delight) or something.

 -Guy

DeLight : great idea :-) C 2002/4/30

I'm not opposed to "Delight"...though I prefer D-- because of its reference to C++. However, the decision doesn't have to be made yet. Let's leave that to later and focus on discussing the technical requirements. -- The Villagers are Online! villagersonline.com .[ (the fox.(quick,brown)) jumped.over(the dog.lazy) ] .[ (a version.of(English).(precise.more)) is(possible) ] ?[ you want.to(help(develop(it))) ]
May 01 2002
prev sibling parent reply Alexander Lahmann <info elco-pro.de> writes:
Russ Lewis wrote:

 I'm forking from the "Compiling Phobos" thread to discus D-- here.

A very important feature should be implemented: You should be able to make an object visible to the script languages (D-- or ECMAscript). All objects should be hidden to the script language by default. Best regards, Mark Junker
May 12 2002
parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Alexander Lahmann wrote:

 Russ Lewis wrote:

 I'm forking from the "Compiling Phobos" thread to discus D-- here.

A very important feature should be implemented: You should be able to make an object visible to the script languages (D-- or ECMAscript). All objects should be hidden to the script language by default. Best regards, Mark Junker

If I understand what you want, D already has this feature. Simply mark objects as "private" in the D file that declares them, and they should not be visible from other D files that import that file. -- The Villagers are Online! http://villagersonline.com .[ (the fox.(quick,brown)) jumped.over(the dog.lazy) ] .[ (a version.of(English).(precise.more)) is(possible) ] ?[ you want.to(help(develop(it))) ]
May 12 2002
parent Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Russ Lewis wrote:

 Alexander Lahmann wrote:

 Russ Lewis wrote:

 I'm forking from the "Compiling Phobos" thread to discus D-- here.

A very important feature should be implemented: You should be able to make an object visible to the script languages (D-- or ECMAscript). All objects should be hidden to the script language by default. Best regards, Mark Junker

If I understand what you want, D already has this feature. Simply mark objects as "private" in the D file that declares them, and they should not be visible from other D files that import that file.

But, this is a very good note...any "safe D library" will have to tightly control which objects that "untrusted" code can access. -- The Villagers are Online! http://villagersonline.com .[ (the fox.(quick,brown)) jumped.over(the dog.lazy) ] .[ (a version.of(English).(precise.more)) is(possible) ] ?[ you want.to(help(develop(it))) ]
May 12 2002