www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.internals - A Proof of Concept for the baremetal D runtime plans

reply mbalantz <arbofo hotmail.com> writes:
Hello, my name is Mark Balantzyan, an aspiring developer and Google 
Summer of Code 2019 applicant. It took much varied realizations of 
discernment between what is permissible in C and C++ but not in D during 
my development of this proof of concept. I would hope to say I have 
ported it, however I am not entirely sure as the output complains of 
linker errors though producing an executable.

I attach a link to an archive containing the source with executable, 
just sharing here in the newsgroup. It is a standard pong game with AI 
that appears to need an Intel 80836 PE32 environment to run (16-bit/DOS 
maybe?), and I was not able to test it on my machines. Just sharing it 
here for ideas.

The original is at https://github.com/flightcrank/pong.git .

N.B. Further of my research showed to me that such exe's need managed 
headers (mentioning dot net version preceded by BSJB, or something to 
that order).

Regards,
Mark

Link to D-ported pong (pongd):

https://drive.google.com/file/d/1kWltW8k6J6v_XTXwLBJNLzChox72Ko3x/view?usp=sharing
Apr 25 2019
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Friday, 26 April 2019 at 01:24:21 UTC, mbalantz wrote:

 Hello, my name is Mark Balantzyan, an aspiring developer and 
 Google Summer of Code 2019 applicant. It took much varied 
 realizations of discernment between what is permissible in C 
 and C++ but not in D during my development of this proof of 
 concept. I would hope to say I have ported it, however I am not 
 entirely sure as the output complains of linker errors though 
 producing an executable.


A few questions:
* What are the linker errors?
* What hardware architecture are you running on?
* Is it truly bare metal, are is there an underlying OS?
* Looking at the source code, it appears SDL is doing the heavy 
lifting.  Perhaps it is SDL that needs porting?

Mike
Apr 28 2019
parent reply mbalantz <arbofo hotmail.com> writes:
On 2019-04-28 5:43 p.m., Mike Franklin wrote:

 On Friday, 26 April 2019 at 01:24:21 UTC, mbalantz wrote:

 Hello, my name is Mark Balantzyan, an aspiring developer and Google 
 Summer of Code 2019 applicant. It took much varied realizations of 
 discernment between what is permissible in C and C++ but not in D 
 during my development of this proof of concept. I would hope to say I 
 have ported it, however I am not entirely sure as the output complains 
 of linker errors though producing an executable.


 
 A few questions:
 * What are the linker errors?
 * What hardware architecture are you running on?
 * Is it truly bare metal, are is there an underlying OS?
 * Looking at the source code, it appears SDL is doing the heavy 
 lifting.  Perhaps it is SDL that needs porting?
 
 Mike
 
 


* The linker errors complain about undefined symbols and mention 
alphanumeric strings as reference, containing the SDL binding for D 
keywords and SDL keywords as per the functions that used them in the 
pongd.d file.

* I downloaded and installed the dmd compiler on x86_64 Windows 10 
Education on an AMD Ryzen 1950X. Therefore:

* ..it is not truly baremetal. I intended to share it to show a demo, 
and then maybe we could look into reverse engineering it to be 
baremetal. Though I don't know if I picked a good demo for running 
baremetal (i.e. a pong game).

* Yes, it is quite dependent on SDL. Porting SDL to D would be no small 
task, I would think? I tried using the bindbc bindings provided in D's 
library to provide the SDL. Though that is not sufficient? The linker 
errors complain of undefined symbols for a lot of SDL functions.

Thanks for you reply,
Mark
May 03 2019
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Friday, 3 May 2019 at 23:20:13 UTC, mbalantz wrote:


 * The linker errors complain about undefined symbols and 
 mention alphanumeric strings as reference, containing the SDL 
 binding for D keywords and SDL keywords as per the functions 
 that used them in the pongd.d file.


I'm sorry, but I don't fully understand.  Please copy and paste a 
few of the errors here.


 * I downloaded and installed the dmd compiler on x86_64 Windows 
 10 Education on an AMD Ryzen 1950X. Therefore:


I think DMD is fine if you want to do bare-metal programming on 
x86_64.


 * ..it is not truly baremetal. I intended to share it to show a 
 demo, and then maybe we could look into reverse engineering it 
 to be baremetal. Though I don't know if I picked a good demo 
 for running baremetal (i.e. a pong game).


What I'm able to ascertain from this is that you've ported a pong 
game to SDL, not to bare metal.  If SDL were ported to bare 
metal, then it would probably work because SDL abstracts the 
architecture and platform for you.


 * Yes, it is quite dependent on SDL. Porting SDL to D would be 
 no small task, I would think? I tried using the bindbc bindings 
 provided in D's library to provide the SDL. Though that is not 
 sufficient? The linker errors complain of undefined symbols for 
 a lot of SDL functions.


Yes, porting SDL to D would be a big task.  I don't think it's 
necessary though because D has good support for binding to C 
APIs.  You can try compiling with -betterC to see if that gets 
rid of some of the linker errors you're encountering.  Otherwise, 
I'd need to see the linker output to understand what's going on.

Mike
May 03 2019
parent reply mbalantz <arbofo hotmail.com> writes:
On 2019-05-03 4:31 p.m., Mike Franklin wrote:

 On Friday, 3 May 2019 at 23:20:13 UTC, mbalantz wrote:
 

 * The linker errors complain about undefined symbols and mention 
 alphanumeric strings as reference, containing the SDL binding for D 
 keywords and SDL keywords as per the functions that used them in the 
 pongd.d file.


 
 I'm sorry, but I don't fully understand.  Please copy and paste a few of 
 the errors here.
 

 * I downloaded and installed the dmd compiler on x86_64 Windows 10 
 Education on an AMD Ryzen 1950X. Therefore:


 
 I think DMD is fine if you want to do bare-metal programming on x86_64.
 

 * ..it is not truly baremetal. I intended to share it to show a demo, 
 and then maybe we could look into reverse engineering it to be 
 baremetal. Though I don't know if I picked a good demo for running 
 baremetal (i.e. a pong game).


 
 What I'm able to ascertain from this is that you've ported a pong game 
 to SDL, not to bare metal.  If SDL were ported to bare metal, then it 
 would probably work because SDL abstracts the architecture and platform 
 for you.
 

 * Yes, it is quite dependent on SDL. Porting SDL to D would be no 
 small task, I would think? I tried using the bindbc bindings provided 
 in D's library to provide the SDL. Though that is not sufficient? The 
 linker errors complain of undefined symbols for a lot of SDL functions.


 
 Yes, porting SDL to D would be a big task.  I don't think it's necessary 
 though because D has good support for binding to C APIs.  You can try 
 compiling with -betterC to see if that gets rid of some of the linker 
 errors you're encountering.  Otherwise, I'd need to see the linker 
 output to understand what's going on.
 
 Mike
 
 


Thanks again for getting back to me. Sorry, forgot to include linker 
output. The errors are Error 42: symbol undefined and "linker exited 
status 21"

As follows:

PS C:\Users\mbala\workspace\D\pongd_actual\source> dmd pongd.d 
-odC:\Users\mbala\workspace\D\pongd\out\production\pongd\obj -ofpongd 
-IC:\Users\mbala\Download
s\SDL2-2.0.9\x86_64-w64-mingw32\include\SDL2 
-LC:\Users\mbala\Downloads\SDL2-2.0.9-win32-x64\SDL2.dll
pongd.d(399): Deprecation: switch case fallthrough - use 'goto case;' if 
intended
pongd.d(401): Deprecation: switch case fallthrough - use 'goto case;' if 
intended
pongd.d(403): Deprecation: switch case fallthrough - use 'goto default;' 
if intended
OPTLINK (R) for Win32  Release 8.00.17
Copyright (C) Digital Mars 1989-2013  All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
OPTLINK : Warning 9: Unknown Option : NOIC
:\Users\mbala\Downloads\SDL2-2.0.9-win32-x64\SDL2.dll
  Warning 2: File Not Found 
:\Users\mbala\Downloads\SDL2-2.0.9-win32-x64\SDL2.dll
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined __D6bindbc3sdl4bindQj8SDL_QuitPUNbNiZv
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind8sdlvideo17SDL_DestroyWindowPUNbNiPSQCaQBwQBvQBt10SDL_WindowZv
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind9sdlrender19SDL_DestroyRendererPUNbNiPSQCdQBzQByQBw12SDL_RendererZv
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind10sdlsurface15SDL_FreeSurfacePUNbNiPSQCbQBxQBwQBu11SDL_SurfaceZv
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind9sdlrender17SDL_RenderPresentPUNbNiPSQCbQBxQBwQBu12SDL_RendererZv
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4Çäï9Çâëre¨er14SDL_RÇàîCopyPUNbNiPSQByÐuÐtÐr12Çèñ╔Çâ£DaQCwÐvÐt11Çä£TexturePxSQEcQDyÐx7ÇàÒct8Çå╝ctxPQBdZi
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind9sdlrender17SDL_UpdateTexturePUNbNiPSQCbQBxQBwQBu11SDL_TexturePxSQDdQCzQCy7sdlrect8SDL_RectPxviZi
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined __D6bindbc3sdl4bindQj8SDL_InitPUNbNikZi
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind10sdlsurface12SDL_FillRectPUNbNiPSQByQBuQBtQBr11SDL_SurfacePxSQDaQCwQCv7sdlrect8SDL_RectkZi
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind10sdlsurface15SDL_SetColorKeyPUNbNiPSQCbQBxQBwQBu11SDL_SurfaceikZi
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind9sdlpixels10SDL_MapRGBPUNbNiPxSQBvQBrQBqQBo15SDL_PixelFormathhhZk
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind8sdlerror12SDL_GetErrorPUNbNiZPxa
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind10sdlsurface11SDL_LoadBMPFNbNiPxaZPSQCaQBwQBvQBt11SDL_Surface
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4Çäï9Çâëre¨er28SDL_CreateTextureFromS¨facePUNbNiPSQCmÐiÐhÐf12Çä▓RÇà¥╔Çâ£DoÐkÐj10ÇâÍsÇå╣11ÇäÑÇçãZÇâÑEzÐvÐuÐsÇå£ÇçÝ
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4Çäï9Çâëre¨er27SDL_CreateWÇâÜowAßRÇàø╔PUNbNiiikPPSQCpÐlÐk8Çâ╝video10Çä╗ÇåÁÇäíDwÐsÐrÐp12ÇäøÇêÃZi
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined __D6bindbc3sdl4bind8sdltimer9SDL_DelayPUNbNikZv
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind9sdlrender15SDL_RenderClearPUNbNiPSQBzQBvQBuQBs12SDL_RendererZi
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind11sdlkeyboard20SDL_GetKeyboardStatePUNbNiPiZPh
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind9sdlevents14SDL_PumpEventsPUNbNiZv
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind8sdltimer12SDL_GetTicksPUNbNiZk
C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
__D6bindbc3sdl4bind8sdlvideo17SDL_GetWindowSizePUNbNiPSQCaQBwQBvQBt10SDL_WindowPiQcZv
Error: linker exited with status 21
PS C:\Users\mbala\workspace\D\pongd_actual\source>
May 03 2019
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Friday, 3 May 2019 at 23:49:23 UTC, mbalantz wrote:


 Thanks again for getting back to me. Sorry, forgot to include 
 linker output. The errors are Error 42: symbol undefined and 
 "linker exited status 21"

 As follows:

 PS C:\Users\mbala\workspace\D\pongd_actual\source> dmd pongd.d 
 -odC:\Users\mbala\workspace\D\pongd\out\production\pongd\obj 
 -ofpongd -IC:\Users\mbala\Download
 s\SDL2-2.0.9\x86_64-w64-mingw32\include\SDL2 
 -LC:\Users\mbala\Downloads\SDL2-2.0.9-win32-x64\SDL2.dll
 pongd.d(399): Deprecation: switch case fallthrough - use 'goto 
 case;' if intended
 pongd.d(401): Deprecation: switch case fallthrough - use 'goto 
 case;' if intended
 pongd.d(403): Deprecation: switch case fallthrough - use 'goto 
 default;' if intended
 OPTLINK (R) for Win32  Release 8.00.17
 Copyright (C) Digital Mars 1989-2013  All rights reserved.
 http://www.digitalmars.com/ctg/optlink.html
 OPTLINK : Warning 9: Unknown Option : NOIC
 :\Users\mbala\Downloads\SDL2-2.0.9-win32-x64\SDL2.dll
  Warning 2: File Not Found 
 :\Users\mbala\Downloads\SDL2-2.0.9-win32-x64\SDL2.dll
 C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
 __D6bindbc3sdl4bindQj8SDL_QuitPUNbNiZv
 C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
 __D6bindbc3sdl4bind8sdlvideo17SDL_DestroyWindowPUNbNiPSQCaQBwQBvQBt10SDL_WindowZv
 C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined 
 __D6bindbc3sdl4bind9sdlrender19SDL_DestroyRendererPUNbNiPSQCdQBzQByQBw12SDL_RendererZv
 C:\Users\mbala\workspace\D\pongd\out\production\pongd\obj\pongd.obj(pongd)
  Error 42: Symbol Undefined


[..]

I donwloaded your code again, and the pongd.exe file contains a 
trojan:Win32/Fuery.!cl.  Not sure if that's a false positive, but 
it certainly wasn't encouraging.

What bothers me is why do you have a pongd.exe file if you're 
having trouble linking.  Seems very suspcious.  Where did that 
.exe come from if you're having trouble getting a build?

I see the message from optlink.  This is a linker (created by 
Walter, I think) that is used to link 32-bit code on Windows.  
Try compiling with -w64 instead.  -w64 will create a 64-bit 
binary and will use either LLVM's linker or Visual Studio's 
linker depending on which one it finds first.

Next, I see you are compiling pongd.d, but you're not compiling 
the code in the bindbc directory and I think that is the source 
of your linker errors.  You'll need to compile the bindbc 
directory to object code and link that in to you final executable.

Mike
May 03 2019
parent reply mbalantz <arbofo hotmail.com> writes:
On 2019-05-03 5:30 p.m., Mike Franklin wrote:


 I donwloaded your code again, and the pongd.exe file contains a 
 trojan:Win32/Fuery.!cl.  Not sure if that's a false positive, but it 
 certainly wasn't encouraging.
 


Yes, I realize there's something funny going on every time I compile the 
pongd.d file on my Windows box. I once took a look at the assembly of 
one of the pongd.exe's I compiled and saw that control flow quickly 
proceeded to a buffer overflow. That occurred during one of my builds, 
wasn't sure if it was going to happen again or for anyone else. Sorry 
for that trouble.



 I see the message from optlink.  This is a linker (created by Walter, I 
 think) that is used to link 32-bit code on Windows. Try compiling with 
 -w64 instead.  -w64 will create a 64-bit binary and will use either 
 LLVM's linker or Visual Studio's linker depending on which one it finds 
 first.
 
 Next, I see you are compiling pongd.d, but you're not compiling the code 
 in the bindbc directory and I think that is the source of your linker 
 errors.  You'll need to compile the bindbc directory to object code and 
 link that in to you final executable.
 
 Mike


Going to work on it. Again, sorry for that possible virus. I have to 
clean up my computer, not trying to spread viruses around.

Thank you,
Mark
May 03 2019
parent reply mbalantz <arbofo hotmail.com> writes:
On 2019-05-03 6:15 p.m., mbalantz wrote:

 On 2019-05-03 5:30 p.m., Mike Franklin wrote:
 

 I donwloaded your code again, and the pongd.exe file contains a 
 trojan:Win32/Fuery.!cl.  Not sure if that's a false positive, but it 
 certainly wasn't encouraging.


 
 Yes, I realize there's something funny going on every time I compile the 
 pongd.d file on my Windows box. I once took a look at the assembly of 
 one of the pongd.exe's I compiled and saw that control flow quickly 
 proceeded to a buffer overflow. That occurred during one of my builds, 
 wasn't sure if it was going to happen again or for anyone else. Sorry 
 for that trouble.
 
 

 I see the message from optlink.  This is a linker (created by Walter, 
 I think) that is used to link 32-bit code on Windows. Try compiling 
 with -w64 instead.  -w64 will create a 64-bit binary and will use 
 either LLVM's linker or Visual Studio's linker depending on which one 
 it finds first.

 Next, I see you are compiling pongd.d, but you're not compiling the 
 code in the bindbc directory and I think that is the source of your 
 linker errors.  You'll need to compile the bindbc directory to object 
 code and link that in to you final executable.

 Mike


 
 Going to work on it. Again, sorry for that possible virus. I have to 
 clean up my computer, not trying to spread viruses around.
 
 Thank you,
 Mark
 


Hello Mike and D community,

-w64 didn't work in my installation, perhaps if you please provide me a 
list of steps with -w64 in it, it may succeed? Also, my 
C:\D\dmd2\windows\bin64 directory only contains one file, libcurl.dll. 
Therefore maybe this is why -w64 is not found.

I share two files here (I got rid of the one flagged likely erroneously 
as a trojan): one is a text file showing output of steps I performed 
after trying to link all the object files in my bindbc directory along 
with the pongd.obj; and I put here a link to a zip archive of the build 
directory WITHOUT the .exe, which does in fact still get created after 
optlink, for security reasons.

https://drive.google.com/file/d/1HD6tbm0xUJBjYON6y6JSTqZl3JDSz6f3/view?usp=sharing

^^(linkall-pongd-output.txt)

https://drive.google.com/file/d/1eHMtxVKmxwgYzqeOwgcNZWdnltqZPiHh/view?usp=sharing

^^ pong-in-d.zip

I might have to start a new thread as this is getting less and less 
about trying to demo against baremetal and more about getting my pong 
file to compile/be ported into D, of which I'd still have some interest.

Thank you,
Mark
May 03 2019
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Saturday, 4 May 2019 at 03:06:40 UTC, mbalantz wrote:


 Hello Mike and D community,

 -w64 didn't work in my installation, perhaps if you please 
 provide me a list of steps with -w64 in it, it may succeed?


Sorry, I meant -m64, not -w64.  Type `dmd --help` to see a list 
of flags you can pass to the compiler


 Also, my C:\D\dmd2\windows\bin64 directory only contains one 
 file, libcurl.dll.


Your 64-bit libraries will be found in C:\D\dmd2\windows\lib64

phobos64.lib is the D standard library (Phobos) + the D runtime 
in one file.


In general, for each source file you'll do

dmd -c -m64 file1.d
dmd -c -m64 file2.d
dmd -c -m64 file3.d
etc.

Of course you'll also have to pass the appropriate -I flags for 
your imports.

That will create a file1.obj, file2.obj, and file3.obj object 
files.  When you have all your source files compiled to object 
files then you wan to link the object files to create the .exe.

dmd -m64 file1.obj file2.obj file3.obj -o=pongd.exe

This last invocation of `dmd` will not compile any code because 
there aren't any .d files.  Instead `dmd` will pass all of the 
.obj files to the linker on your system to generate an .exe file.

What are you using for your development environment?  You can 
download Visual D (https://github.com/dlang/visuald/releases) and 
use Visual Studio if that's the development environment you're 
most comfortable with.


 I might have to start a new thread as this is getting less and 
 less about trying to demo against baremetal and more about 
 getting my pong file to compile/be ported into D, of which I'd 
 still have some interest.


Post to the "Learn" forum for these kinds of questions.

Mike
May 03 2019
parent Mike Franklin <slavo5150 yahoo.com> writes:
On Saturday, 4 May 2019 at 03:56:42 UTC, Mike Franklin wrote:


 dmd -m64 file1.obj file2.obj file3.obj -o=pongd.exe


Sorry, that should be `-of=pongd.exe`.  See the output of `dmd 
--help`

Mike
May 03 2019