www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D for Android

reply "Joakim" <dlang joakim.airpost.net> writes:
Well, Android/x86 for now.  I've been plugging away at getting D 
running on Android/x86 and got all of the druntime modules' unit 
tests and 37 of 50 phobos modules' unit tests to pass. I had to 
hack dmd into producing something like packed TLS for ELF, my 
patch is online here:

http://164.138.25.188/dmd/packed_tls_for_elf.patch

I simply turned off all TLS flags for ELF and spliced in the 
el_picvar patch from OS X to call ___tls_get_addr.  Somebody who 
knows dmd better than me should verify to make sure this is right.

I've also put online preliminary pulls for druntime and phobos:

https://github.com/D-Programming-Language/druntime/pull/784
https://github.com/D-Programming-Language/phobos/pull/2150

Now that a significant chunk of D is working on Android/x86, I'm 
looking for others to pitch in.  We really need to get D on 
mobile, and Android/x86 is an ideal place to start.  Dan Olson 
has done some nice work getting D on iOS using ldc, I'm sure he 
could use help too:

http://forum.dlang.org/thread/m2txc2kqxv.fsf comcast.net
http://forum.dlang.org/thread/m2d2h15ao3.fsf comcast.net

Stuff remaining to be done:

1. Fix all phobos unit tests.  Those who know the failing modules 
better would be best equipped to get them to work.

2. I tried creating an Android app, ie an apk, which is really 
just a shared library called from the Dalvik JVM, as opposed to 
the standalone executables I've been running from the Android 
command line so far.  The apk enters the D code and then 
segfaults in the new TLS support, I'll debug that next.

3. Use ldc/gdc to build for Android/ARM.

4. Start translating various headers on Android so they can be 
called from D, ie EGL, OpenGL ES, sensors, etc.

5. Integrate the D compilers into the existing Makefile-based 
build system of the Android NDK.  Right now, I extract the 
necessary compiler and linker commands and run them by hand when 
necessary.

All you need to get going is to download the latest Android NDK 
(http://developer.android.com/tools/sdk/ndk/index.html) and run 
Android/x86 (http://www.android-x86.org/, I recommend the 4.3 
build) in a VM.  I'll put up some basic setup and build 
instructions if someone is interested.
May 08 2014
next sibling parent "FrankLike" <1150015857 qq.com> writes:
On Thursday, 8 May 2014 at 16:16:22 UTC, Joakim wrote:
 Well, Android/x86 for now.  I've been plugging away at getting 
 D running on Android/x86 and got all of the druntime modules' 
 unit tests and 37 of 50 phobos modules' unit tests to pass. I 
 had to hack dmd into producing something like packed TLS for 
 ELF, my patch is online here:

 http://164.138.25.188/dmd/packed_tls_for_elf.patch

 I simply turned off all TLS flags for ELF and spliced in the 
 el_picvar patch from OS X to call ___tls_get_addr.  Somebody 
 who knows dmd better than me should verify to make sure this is 
 right.

 I've also put online preliminary pulls for druntime and phobos:

 https://github.com/D-Programming-Language/druntime/pull/784
 https://github.com/D-Programming-Language/phobos/pull/2150

 Now that a significant chunk of D is working on Android/x86, 
 I'm looking for others to pitch in.  We really need to get D on 
 mobile, and Android/x86 is an ideal place to start.  Dan Olson 
 has done some nice work getting D on iOS using ldc, I'm sure he 
 could use help too:

 http://forum.dlang.org/thread/m2txc2kqxv.fsf comcast.net
 http://forum.dlang.org/thread/m2d2h15ao3.fsf comcast.net

 Stuff remaining to be done:

 1. Fix all phobos unit tests.  Those who know the failing 
 modules better would be best equipped to get them to work.

 2. I tried creating an Android app, ie an apk, which is really 
 just a shared library called from the Dalvik JVM, as opposed to 
 the standalone executables I've been running from the Android 
 command line so far.  The apk enters the D code and then 
 segfaults in the new TLS support, I'll debug that next.

 3. Use ldc/gdc to build for Android/ARM.

 4. Start translating various headers on Android so they can be 
 called from D, ie EGL, OpenGL ES, sensors, etc.

 5. Integrate the D compilers into the existing Makefile-based 
 build system of the Android NDK.  Right now, I extract the 
 necessary compiler and linker commands and run them by hand 
 when necessary.

 All you need to get going is to download the latest Android NDK 
 (http://developer.android.com/tools/sdk/ndk/index.html) and run 
 Android/x86 (http://www.android-x86.org/, I recommend the 4.3 
 build) in a VM.  I'll put up some basic setup and build 
 instructions if someone is interested.

interesting.
May 08 2014
prev sibling next sibling parent "w0rp" <devw0rp gmail.com> writes:
Awesome. I appreciate this work.
May 08 2014
prev sibling next sibling parent Etienne <etcimon gmail.com> writes:
On 2014-05-08 12:16 PM, Joakim wrote:
 All you need to get going is to download the latest Android NDK
 (http://developer.android.com/tools/sdk/ndk/index.html) and run
 Android/x86 (http://www.android-x86.org/, I recommend the 4.3 build) in
 a VM.  I'll put up some basic setup and build instructions if someone is
 interested.

Thanks for this, it's more than necessary and I believe no time will have been wasted.
May 09 2014
prev sibling next sibling parent reply "Chris" <wendlec tcd.ie> writes:
On Thursday, 8 May 2014 at 16:16:22 UTC, Joakim wrote:
 Well, Android/x86 for now.  I've been plugging away at getting 
 D running on Android/x86 and got all of the druntime modules' 
 unit tests and 37 of 50 phobos modules' unit tests to pass. I 
 had to hack dmd into producing something like packed TLS for 
 ELF, my patch is online here:

 http://164.138.25.188/dmd/packed_tls_for_elf.patch

 I simply turned off all TLS flags for ELF and spliced in the 
 el_picvar patch from OS X to call ___tls_get_addr.  Somebody 
 who knows dmd better than me should verify to make sure this is 
 right.

 I've also put online preliminary pulls for druntime and phobos:

 https://github.com/D-Programming-Language/druntime/pull/784
 https://github.com/D-Programming-Language/phobos/pull/2150

 Now that a significant chunk of D is working on Android/x86, 
 I'm looking for others to pitch in.  We really need to get D on 
 mobile, and Android/x86 is an ideal place to start.  Dan Olson 
 has done some nice work getting D on iOS using ldc, I'm sure he 
 could use help too:

 http://forum.dlang.org/thread/m2txc2kqxv.fsf comcast.net
 http://forum.dlang.org/thread/m2d2h15ao3.fsf comcast.net

 Stuff remaining to be done:

 1. Fix all phobos unit tests.  Those who know the failing 
 modules better would be best equipped to get them to work.

 2. I tried creating an Android app, ie an apk, which is really 
 just a shared library called from the Dalvik JVM, as opposed to 
 the standalone executables I've been running from the Android 
 command line so far.  The apk enters the D code and then 
 segfaults in the new TLS support, I'll debug that next.

 3. Use ldc/gdc to build for Android/ARM.

 4. Start translating various headers on Android so they can be 
 called from D, ie EGL, OpenGL ES, sensors, etc.

 5. Integrate the D compilers into the existing Makefile-based 
 build system of the Android NDK.  Right now, I extract the 
 necessary compiler and linker commands and run them by hand 
 when necessary.

 All you need to get going is to download the latest Android NDK 
 (http://developer.android.com/tools/sdk/ndk/index.html) and run 
 Android/x86 (http://www.android-x86.org/, I recommend the 4.3 
 build) in a VM.  I'll put up some basic setup and build 
 instructions if someone is interested.

I can't tell you how much I appreciate this! It's wonderful. All the stuff I'm working on will have to go on smart phones and tablets sooner or later. People do ask for it, because everything is an "app" these days. Much as I appreciate all the efforts to improve D as a language (GC, library etc.), if we can't get into the mobile market, D won't take off. People think "app". "What do you think?" - "Think? Is there an app for that?"
May 09 2014
parent Xavier Bigand <flamaros.xavier gmail.com> writes:
Le 09/05/2014 15:22, Chris a écrit :
 On Thursday, 8 May 2014 at 16:16:22 UTC, Joakim wrote:
 Well, Android/x86 for now.  I've been plugging away at getting D
 running on Android/x86 and got all of the druntime modules' unit tests
 and 37 of 50 phobos modules' unit tests to pass. I had to hack dmd
 into producing something like packed TLS for ELF, my patch is online
 here:

 http://164.138.25.188/dmd/packed_tls_for_elf.patch

 I simply turned off all TLS flags for ELF and spliced in the el_picvar
 patch from OS X to call ___tls_get_addr.  Somebody who knows dmd
 better than me should verify to make sure this is right.

 I've also put online preliminary pulls for druntime and phobos:

 https://github.com/D-Programming-Language/druntime/pull/784
 https://github.com/D-Programming-Language/phobos/pull/2150

 Now that a significant chunk of D is working on Android/x86, I'm
 looking for others to pitch in.  We really need to get D on mobile,
 and Android/x86 is an ideal place to start.  Dan Olson has done some
 nice work getting D on iOS using ldc, I'm sure he could use help too:

 http://forum.dlang.org/thread/m2txc2kqxv.fsf comcast.net
 http://forum.dlang.org/thread/m2d2h15ao3.fsf comcast.net

 Stuff remaining to be done:

 1. Fix all phobos unit tests.  Those who know the failing modules
 better would be best equipped to get them to work.

 2. I tried creating an Android app, ie an apk, which is really just a
 shared library called from the Dalvik JVM, as opposed to the
 standalone executables I've been running from the Android command line
 so far.  The apk enters the D code and then segfaults in the new TLS
 support, I'll debug that next.

 3. Use ldc/gdc to build for Android/ARM.

 4. Start translating various headers on Android so they can be called
 from D, ie EGL, OpenGL ES, sensors, etc.

 5. Integrate the D compilers into the existing Makefile-based build
 system of the Android NDK.  Right now, I extract the necessary
 compiler and linker commands and run them by hand when necessary.

 All you need to get going is to download the latest Android NDK
 (http://developer.android.com/tools/sdk/ndk/index.html) and run
 Android/x86 (http://www.android-x86.org/, I recommend the 4.3 build)
 in a VM.  I'll put up some basic setup and build instructions if
 someone is interested.

I can't tell you how much I appreciate this! It's wonderful. All the stuff I'm working on will have to go on smart phones and tablets sooner or later. People do ask for it, because everything is an "app" these days. Much as I appreciate all the efforts to improve D as a language (GC, library etc.), if we can't get into the mobile market, D won't take off. People think "app". "What do you think?" - "Think? Is there an app for that?"

+1
May 09 2014
prev sibling next sibling parent "Szymon Gatner" <noemail gmail.com> writes:
On Thursday, 8 May 2014 at 16:16:22 UTC, Joakim wrote:

Great to hear! Much appreciated.
May 09 2014
prev sibling next sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Thursday, 8 May 2014 at 16:16:22 UTC, Joakim wrote:
 All you need to get going is to download the latest Android NDK 
 (http://developer.android.com/tools/sdk/ndk/index.html) and run 
 Android/x86 (http://www.android-x86.org/, I recommend the 4.3 
 build) in a VM.  I'll put up some basic setup and build 
 instructions if someone is interested.

Please do. I'd recommend adding it to the Core Development section on the wiki so that it doesn't get lost. http://wiki.dlang.org/ Thanks!
May 10 2014
parent reply Etienne <etcimon gmail.com> writes:
I've been reading on Emscripten and LDC and how they would be nice 
together, and came across this nice little library:
http://www.leaningtech.com/duetto/examples/

It's a C++ server/client framework that compiles to JS through clang => 
LLVM bytecode => ASM.js.

I'd like to point out that asm.js is a very fast subset of the 
javascript language that allows almost native speeds (3x slowdown vs C 
only) which enables games to be run in the browser without external 
dependencies.

This had me thinking of forking the Emscripten project in order to 
support the druntime / phobos library & give it access to the browser 
DOM interface, as to allow javascript web applications to be written 
fully in D (to produce interfaces like openUI5 and more).

This would have multiple advantages:
- Suitable for huge front-end development frameworks using safe typing 
and contractual programming
- Opens the door to petaFLOPS of distributed CPU power based simply on 
website visitation without affecting user experience
- Allows Javascript proprietary code to be harder to de-obfuscate or 
reverse engineer
- FAST Browser-based database engines and full runtime capabilities can 
be developed (and added as browser extensions for persistence) to be 
actively used by the server with a simple interface

And most of all, it simplifies web development.

I'm not going to instantly call for a project to be started, but I'd 
love to hear some why do's and why don'ts b/c I don't like to rely on my 
point of view alone.
May 13 2014
parent Etienne <etcimon gmail.com> writes:
wrong button ;)
May 13 2014
prev sibling parent reply "Joakim" <dlang joakim.airpost.net> writes:
On Thursday, 8 May 2014 at 16:16:22 UTC, Joakim wrote:
 Well, Android/x86 for now.  I've been plugging away at getting 
 D running on Android/x86 and got all of the druntime modules' 
 unit tests and 37 of 50 phobos modules' unit tests to pass. I 
 had to hack dmd into producing something like packed TLS for 
 ELF, my patch is online here:

 http://164.138.25.188/dmd/packed_tls_for_elf.patch

 I simply turned off all TLS flags for ELF and spliced in the 
 el_picvar patch from OS X to call ___tls_get_addr.  Somebody 
 who knows dmd better than me should verify to make sure this is 
 right.

 I've also put online preliminary pulls for druntime and phobos:

 https://github.com/D-Programming-Language/druntime/pull/784
 https://github.com/D-Programming-Language/phobos/pull/2150

 Now that a significant chunk of D is working on Android/x86, 
 I'm looking for others to pitch in.  We really need to get D on 
 mobile, and Android/x86 is an ideal place to start.  Dan Olson 
 has done some nice work getting D on iOS using ldc, I'm sure he 
 could use help too:

 http://forum.dlang.org/thread/m2txc2kqxv.fsf comcast.net
 http://forum.dlang.org/thread/m2d2h15ao3.fsf comcast.net

 Stuff remaining to be done:

 1. Fix all phobos unit tests.  Those who know the failing 
 modules better would be best equipped to get them to work.

 2. I tried creating an Android app, ie an apk, which is really 
 just a shared library called from the Dalvik JVM, as opposed to 
 the standalone executables I've been running from the Android 
 command line so far.  The apk enters the D code and then 
 segfaults in the new TLS support, I'll debug that next.

 3. Use ldc/gdc to build for Android/ARM.

 4. Start translating various headers on Android so they can be 
 called from D, ie EGL, OpenGL ES, sensors, etc.

 5. Integrate the D compilers into the existing Makefile-based 
 build system of the Android NDK.  Right now, I extract the 
 necessary compiler and linker commands and run them by hand 
 when necessary.

 All you need to get going is to download the latest Android NDK 
 (http://developer.android.com/tools/sdk/ndk/index.html) and run 
 Android/x86 (http://www.android-x86.org/, I recommend the 4.3 
 build) in a VM.  I'll put up some basic setup and build 
 instructions if someone is interested.

An update on the porting effort: I just got most modules' unit tests to pass as part of an Android/x86 app, a native "apk," ie a D shared library called from a small C wrapper, no Java required. :) All druntime/phobos modules but one, std.datetime, passed their tests on the Android/x86 command-line a couple months ago. Three of those modules segfault when run as part of the tests in an apk- core.thread, std.parallelism, and std.socket- going to look at those next. I'm guessing the first two might be related to the C wrapper also calling pthreads, as they passed on the command-line. This means most of 1. and 2. above can be crossed off the list. I'll start cleaning up my fairly simple build process and document it on the wiki, so that others can easily play with D on Android/x86. Most of the patches so far have been merged into git master, with the dmd patch above now a PR: https://github.com/D-Programming-Language/dmd/pull/3643 Only small tweaks were needed beyond that, which I'll submit as PRs in the coming days. Looking forward to help on 3., 4., and 5. above, and to seeing what others do with this new platform.
Jul 23 2014
next sibling parent "Elvis Zhou" <elvis.x.zhou gmail.com> writes:
On Thursday, 24 July 2014 at 01:17:44 UTC, Joakim wrote:
 On Thursday, 8 May 2014 at 16:16:22 UTC, Joakim wrote:
 Well, Android/x86 for now.  I've been plugging away at getting 
 D running on Android/x86 and got all of the druntime modules' 
 unit tests and 37 of 50 phobos modules' unit tests to pass. I 
 had to hack dmd into producing something like packed TLS for 
 ELF, my patch is online here:

 http://164.138.25.188/dmd/packed_tls_for_elf.patch

 I simply turned off all TLS flags for ELF and spliced in the 
 el_picvar patch from OS X to call ___tls_get_addr.  Somebody 
 who knows dmd better than me should verify to make sure this 
 is right.

 I've also put online preliminary pulls for druntime and phobos:

 https://github.com/D-Programming-Language/druntime/pull/784
 https://github.com/D-Programming-Language/phobos/pull/2150

 Now that a significant chunk of D is working on Android/x86, 
 I'm looking for others to pitch in.  We really need to get D 
 on mobile, and Android/x86 is an ideal place to start.  Dan 
 Olson has done some nice work getting D on iOS using ldc, I'm 
 sure he could use help too:

 http://forum.dlang.org/thread/m2txc2kqxv.fsf comcast.net
 http://forum.dlang.org/thread/m2d2h15ao3.fsf comcast.net

 Stuff remaining to be done:

 1. Fix all phobos unit tests.  Those who know the failing 
 modules better would be best equipped to get them to work.

 2. I tried creating an Android app, ie an apk, which is really 
 just a shared library called from the Dalvik JVM, as opposed 
 to the standalone executables I've been running from the 
 Android command line so far.  The apk enters the D code and 
 then segfaults in the new TLS support, I'll debug that next.

 3. Use ldc/gdc to build for Android/ARM.

 4. Start translating various headers on Android so they can be 
 called from D, ie EGL, OpenGL ES, sensors, etc.

 5. Integrate the D compilers into the existing Makefile-based 
 build system of the Android NDK.  Right now, I extract the 
 necessary compiler and linker commands and run them by hand 
 when necessary.

 All you need to get going is to download the latest Android 
 NDK (http://developer.android.com/tools/sdk/ndk/index.html) 
 and run Android/x86 (http://www.android-x86.org/, I recommend 
 the 4.3 build) in a VM.  I'll put up some basic setup and 
 build instructions if someone is interested.

An update on the porting effort: I just got most modules' unit tests to pass as part of an Android/x86 app, a native "apk," ie a D shared library called from a small C wrapper, no Java required. :) All druntime/phobos modules but one, std.datetime, passed their tests on the Android/x86 command-line a couple months ago. Three of those modules segfault when run as part of the tests in an apk- core.thread, std.parallelism, and std.socket- going to look at those next. I'm guessing the first two might be related to the C wrapper also calling pthreads, as they passed on the command-line. This means most of 1. and 2. above can be crossed off the list. I'll start cleaning up my fairly simple build process and document it on the wiki, so that others can easily play with D on Android/x86. Most of the patches so far have been merged into git master, with the dmd patch above now a PR: https://github.com/D-Programming-Language/dmd/pull/3643 Only small tweaks were needed beyond that, which I'll submit as PRs in the coming days. Looking forward to help on 3., 4., and 5. above, and to seeing what others do with this new platform.

Congratulations! I'll definitely give it a try when Android/Arm get ready and I think your great efforts will give D a new life as the best programming language for mobile platform.D may be another ObjC once it runs on Android/iOS stably.
Jul 23 2014
prev sibling next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Thursday, 24 July 2014 at 01:17:44 UTC, Joakim wrote:
 This means most of 1. and 2. above can be crossed off the list.
  I'll start cleaning up my fairly simple build process and 
 document it on the wiki, so that others can easily play with D 
 on Android/x86.  Most of the patches so far have been merged 
 into git master, with the dmd patch above now a PR:

 https://github.com/D-Programming-Language/dmd/pull/3643

 Only small tweaks were needed beyond that, which I'll submit as 
 PRs in the coming days.  Looking forward to help on 3., 4., and 
 5. above, and to seeing what others do with this new platform.

Very awesome news. Great work!
Jul 23 2014
prev sibling parent reply "Joakim" <dlang joakim.airpost.net> writes:
On Thursday, 24 July 2014 at 01:17:44 UTC, Joakim wrote:
  I'll start cleaning up my fairly simple build process and 
 document it on the wiki, so that others can easily play with D 
 on Android/x86.

Finally cleaned up my build process a bit and wrote it up on the wiki: http://wiki.dlang.org/Build_DMD_for_Android It got a little long, as I don't have everything scripted yet, tried to explain all my steps, and attempted to put everything in one place, including listing the necessary android adb commands. Hopefully, some of you can help script more of it and eventually integrate D into the Android build scripts. Let me know if you have any problems with the process.
 Looking forward to help on 3., 4., and 5. above, and to seeing 
 what others do with this new platform.

I finally built a native Android/x86 GUI app in D, translated from a sample C app in the Android NDK, after translating a bunch of android headers and putting them up on github: https://github.com/joakim-noah/android Thanks to Jacob Carlborg and the excellent work he's done on dstep, for allowing me to automate translation of most of those headers and save a bunch of time. :) I'll submit the repo to deimos or dub, whichever is the better place to put it. I'll keep updating the wiki with more info, how to build for the command-line and run the druntime/phobos unit tests, along with directions for future work.
Aug 17 2014
next sibling parent Jacob Carlborg <doob me.com> writes:
On 17/08/14 20:19, Joakim wrote:

 Thanks to Jacob Carlborg and the excellent work he's done on dstep, for
 allowing me to automate translation of most of those headers and save a
 bunch of time. :) I'll submit the repo to deimos or dub, whichever is
 the better place to put it.

Cool :) I recommend dub. -- /Jacob Carlborg
Aug 18 2014
prev sibling next sibling parent Xavier Bigand <flamaros.xavier gmail.com> writes:
Le 17/08/2014 20:19, Joakim a écrit :
 On Thursday, 24 July 2014 at 01:17:44 UTC, Joakim wrote:
  I'll start cleaning up my fairly simple build process and document it
 on the wiki, so that others can easily play with D on Android/x86.

Finally cleaned up my build process a bit and wrote it up on the wiki: http://wiki.dlang.org/Build_DMD_for_Android It got a little long, as I don't have everything scripted yet, tried to explain all my steps, and attempted to put everything in one place, including listing the necessary android adb commands. Hopefully, some of you can help script more of it and eventually integrate D into the Android build scripts. Let me know if you have any problems with the process.
 Looking forward to help on 3., 4., and 5. above, and to seeing what
 others do with this new platform.

I finally built a native Android/x86 GUI app in D, translated from a sample C app in the Android NDK, after translating a bunch of android headers and putting them up on github: https://github.com/joakim-noah/android Thanks to Jacob Carlborg and the excellent work he's done on dstep, for allowing me to automate translation of most of those headers and save a bunch of time. :) I'll submit the repo to deimos or dub, whichever is the better place to put it. I'll keep updating the wiki with more info, how to build for the command-line and run the druntime/phobos unit tests, along with directions for future work.

It's maybe more than we need to port DQuick to Android. Really nice.
Sep 03 2014
prev sibling parent reply "Joakim" <dlang joakim.fea.st> writes:
On Sunday, 17 August 2014 at 18:19:29 UTC, Joakim wrote:
 I'll keep updating the wiki with more info, how to build for 
 the command-line and run the druntime/phobos unit tests, along 
 with directions for future work.

An update: I tinkered with ldc and Android/ARM for the hackathon. Didn't get that working yet, but I took Dan Olson's advice to reuse the existing llvm TLS function calls and got ldc to work with Android/x86 a couple days ago. 41 of 42 druntime modules' tests pass: only core.internal convert doesn't, but that module's tests aren't normally run by ldc anyway. Around 15 of 71 phobos modules' tests fail somewhere, most likely related to "long double" issues that either need to be backported, as git master for ldc is still using a year-old druntime/phobos that doesn't have some subsequent Android PRs, or fixed, but I haven't looked into those much yet. Obviously, I can reuse some of this work for Android/ARM, so it should help. I'm putting these in-progress patches online, in case my ultrabook gets run over by a bus or someone else would like to tinker with them. All it required was a small patch to llvm: https://gist.github.com/joakim-noah/1fb23fba1ba5b7e87e1a and these patches for ldc, druntime, and phobos: https://gist.github.com/joakim-noah/bb0b04965470332d10e4 https://gist.github.com/joakim-noah/c27a0c7f2b5b65cdc268 https://gist.github.com/joakim-noah/07ed7ab1e5732ab91126 Most of the druntime and phobos patches were backported from PRs already committed in their respective master branches. The only real new work is in druntime's rt.sections_ldc. It's all a bit hacky right now, but I'll clean it up and submit PRs eventually. I tried the same approach that worked on x86 on ARM, with the similar ARM GOT relocation as can be seen in the llvm patch above, but it didn't work. I'm reading up on the relevant ELF relocation info and looking at the ARM assembly. If anybody better versed with those low-level linker details would like to chip in, that should be the last remaining piece to get much of Android/ARM working.
May 07
next sibling parent reply "Vadim Lopatin" <coolreader.org gmail.com> writes:
On Thursday, 7 May 2015 at 15:58:01 UTC, Joakim wrote:
 On Sunday, 17 August 2014 at 18:19:29 UTC, Joakim wrote:
 I'll keep updating the wiki with more info, how to build for 
 the command-line and run the druntime/phobos unit tests, along 
 with directions for future work.

An update: I tinkered with ldc and Android/ARM for the hackathon. Didn't get that working yet, but I took Dan Olson's advice to reuse the existing llvm TLS function calls and got ldc to work with Android/x86 a couple days ago. 41 of 42 druntime modules' tests pass: only core.internal convert doesn't, but that module's tests aren't normally run by ldc anyway. Around 15 of 71 phobos modules' tests fail somewhere, most likely related to "long double" issues that either need to be backported, as git master for ldc is still using a year-old druntime/phobos that doesn't have some subsequent Android PRs, or fixed, but I haven't looked into those much yet. Obviously, I can reuse some of this work for Android/ARM, so it should help. I'm putting these in-progress patches online, in case my ultrabook gets run over by a bus or someone else would like to tinker with them. All it required was a small patch to llvm: https://gist.github.com/joakim-noah/1fb23fba1ba5b7e87e1a and these patches for ldc, druntime, and phobos: https://gist.github.com/joakim-noah/bb0b04965470332d10e4 https://gist.github.com/joakim-noah/c27a0c7f2b5b65cdc268 https://gist.github.com/joakim-noah/07ed7ab1e5732ab91126 Most of the druntime and phobos patches were backported from PRs already committed in their respective master branches. The only real new work is in druntime's rt.sections_ldc. It's all a bit hacky right now, but I'll clean it up and submit PRs eventually. I tried the same approach that worked on x86 on ARM, with the similar ARM GOT relocation as can be seen in the llvm patch above, but it didn't work. I'm reading up on the relevant ELF relocation info and looking at the ARM assembly. If anybody better versed with those low-level linker details would like to chip in, that should be the last remaining piece to get much of Android/ARM working.

Good news! Thank you for your work! Once Android support become working, I'm planning to add Android support to DlangUI library. It's actually a port of C++ UI library I'm using on Android for Cool Reader GL app. So it should not be hard to port it back to Android. Best regards, Vadim
May 07
parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 8/05/2015 6:33 p.m., Vadim Lopatin wrote:
 On Thursday, 7 May 2015 at 15:58:01 UTC, Joakim wrote:
 On Sunday, 17 August 2014 at 18:19:29 UTC, Joakim wrote:
 I'll keep updating the wiki with more info, how to build for the
 command-line and run the druntime/phobos unit tests, along with
 directions for future work.

An update: I tinkered with ldc and Android/ARM for the hackathon. Didn't get that working yet, but I took Dan Olson's advice to reuse the existing llvm TLS function calls and got ldc to work with Android/x86 a couple days ago. 41 of 42 druntime modules' tests pass: only core.internal convert doesn't, but that module's tests aren't normally run by ldc anyway. Around 15 of 71 phobos modules' tests fail somewhere, most likely related to "long double" issues that either need to be backported, as git master for ldc is still using a year-old druntime/phobos that doesn't have some subsequent Android PRs, or fixed, but I haven't looked into those much yet. Obviously, I can reuse some of this work for Android/ARM, so it should help. I'm putting these in-progress patches online, in case my ultrabook gets run over by a bus or someone else would like to tinker with them. All it required was a small patch to llvm: https://gist.github.com/joakim-noah/1fb23fba1ba5b7e87e1a and these patches for ldc, druntime, and phobos: https://gist.github.com/joakim-noah/bb0b04965470332d10e4 https://gist.github.com/joakim-noah/c27a0c7f2b5b65cdc268 https://gist.github.com/joakim-noah/07ed7ab1e5732ab91126 Most of the druntime and phobos patches were backported from PRs already committed in their respective master branches. The only real new work is in druntime's rt.sections_ldc. It's all a bit hacky right now, but I'll clean it up and submit PRs eventually. I tried the same approach that worked on x86 on ARM, with the similar ARM GOT relocation as can be seen in the llvm patch above, but it didn't work. I'm reading up on the relevant ELF relocation info and looking at the ARM assembly. If anybody better versed with those low-level linker details would like to chip in, that should be the last remaining piece to get much of Android/ARM working.

Good news! Thank you for your work! Once Android support become working, I'm planning to add Android support to DlangUI library. It's actually a port of C++ UI library I'm using on Android for Cool Reader GL app. So it should not be hard to port it back to Android. Best regards, Vadim

I'm also looking forward to it. Assuming a getting started tutorial is made, I'll probably get Devisualization.Window support for it as well within a month.
May 07
prev sibling next sibling parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
"Joakim" <dlang joakim.fea.st> writes:

 Around 15 of 71 phobos modules' tests fail somewhere, most likely
 related to "long double" issues that either need to be backported, as
 git master for ldc is still using a year-old druntime/phobos that
 doesn't have some subsequent Android PRs, or fixed, but I haven't
 looked into those much yet.  Obviously, I can reuse some of this work
 for Android/ARM, so it should help.

Joakim, the ldc merge-2.067 branch works well and has updated druntime/phobos. I know it has some of your Android updates. You may have more fun working with it.
 I tried the same approach that worked on x86 on ARM, with the similar
 ARM GOT relocation as can be seen in the llvm patch above, but it
 didn't work.  I'm reading up on the relevant ELF relocation info and
 looking at the ARM assembly.  If anybody better versed with those
 low-level linker details would like to chip in, that should be the
 last remaining piece to get much of Android/ARM working.

I am not better versed, but I do like to tinker, so maybe I'll see something this weekend. I wonder if we should keep a common LLVM fork with changes to support D on targets without builtin TLS? I did another LLVM hack in x86 backend to support TLS on iPhone sim, and have been thinking about adding a fallback TLS, like emulated tls that gcc and gdc has, to support TLS using a target plugin to get lookup address. That way it could be wired into any old embedded multi thread RTOS with some sort of thread-local support. Maybe someday could even get LLVM to accept some patches.
May 08
parent reply "Joakim" <dlang joakim.fea.st> writes:
On Friday, 8 May 2015 at 16:42:21 UTC, Dan Olson wrote:
 Joakim, the ldc merge-2.067 branch works well and has updated
 druntime/phobos.  I know it has some of your Android updates.  
 You may
 have more fun working with it.

OK, I'll try it. I didn't want to run into bugs unrelated to cross-compiling, so I hadn't tried it yet.
 I wonder if we should keep a common LLVM fork with changes to 
 support D
 on targets without builtin TLS?  I did another LLVM hack in x86 
 backend
 to support TLS on iPhone sim, and have been thinking about 
 adding a
 fallback TLS, like emulated tls that gcc and gdc has, to 
 support TLS
 using a target plugin to get lookup address.  That way it could 
 be wired
 into any old embedded multi thread RTOS with some sort of 
 thread-local
 support.

 Maybe someday could even get LLVM to accept some patches.

Sounds like a worthwhile idea, though my patches to llvm are minimal. I'm sure they'd seriously consider accepting such patches, since the gcc backend already has this built in. I just checked last week and the recent Android NDK's gcc has emulated TLS enabled by default, while the same C/C++ source employing TLS __thread and compiled with the NDK's clang will segfault because it doesn't have emulated TLS. Emulated TLS wasn't added to the NDK's gcc by google either: it appears to have just slipped in from upstream, which is perhaps why they seem to be unaware that it works for one compiler in the NDK and not the other. Let the llvm guys know of this disparity, and I'm sure they'll be up for remedying it.
May 09
parent reply Dan Olson <zans.is.for.cans yahoo.com> writes:
"Joakim" <dlang joakim.fea.st> writes:

 I just checked last week and the recent Android NDK's gcc has emulated
 TLS enabled by default, while the same C/C++ source employing TLS
 __thread and compiled with the NDK's clang will segfault because it
 doesn't have emulated TLS.  Emulated TLS wasn't added to the NDK's gcc
 by google either: it appears to have just slipped in from upstream,
 which is perhaps why they seem to be unaware that it works for one
 compiler in the NDK and not the other.

 Let the llvm guys know of this disparity, and I'm sure they'll be up
 for remedying it.

Just out of curiosity, shouldn't GDC then be better suited for Android than LDC? LDC works well with iOS only because Apple contributes to LLVM. Later, mid-summer perhaps, I plan to get both iOS and Android arm64 devicse so will be joining in the effort. I have too much other stuff going on now though outside of D-Land and have to stick to little D projects that won't vacuum away an afternoon. -- Dan
May 18
parent reply "Joakim" <dlang joakim.fea.st> writes:
On Monday, 18 May 2015 at 14:30:54 UTC, Dan Olson wrote:
 "Joakim" <dlang joakim.fea.st> writes:

 I just checked last week and the recent Android NDK's gcc has 
 emulated
 TLS enabled by default, while the same C/C++ source employing 
 TLS
 __thread and compiled with the NDK's clang will segfault 
 because it
 doesn't have emulated TLS.  Emulated TLS wasn't added to the 
 NDK's gcc
 by google either: it appears to have just slipped in from 
 upstream,
 which is perhaps why they seem to be unaware that it works for 
 one
 compiler in the NDK and not the other.

 Let the llvm guys know of this disparity, and I'm sure they'll 
 be up
 for remedying it.

Just out of curiosity, shouldn't GDC then be better suited for Android than LDC? LDC works well with iOS only because Apple contributes to LLVM.

I don't think it matters, as llvm works very well on linux and the changes google makes for Android to the NDK compilers are minimal: https://android.googlesource.com/toolchain/llvm/+log/release_36 https://android.googlesource.com/toolchain/clang/+log/release_36 https://android.googlesource.com/toolchain/gcc/+log/master/gcc-4.9 The one difference is TLS using the __thread annotation, which was documented as unsupported by the Android NDK for a long time but now happens to work with gcc and not clang/llvm. However, the gdc guys trying to use gcc's emulated TLS have reported problems getting it working with the GC. Maybe that's since been fixed, but that's the last I heard.
 Later, mid-summer perhaps, I plan to get both iOS and Android 
 arm64
 devicse so will be joining in the effort. I have too much other 
 stuff
 going on now though outside of D-Land and have to stick to 
 little D
 projects that won't vacuum away an afternoon.

Sure, have fun with your new devices. :) Hopefully, I'll get Android/ARM working before then, but I don't and won't have any AArch64 devices to test. Not that it matters, as 64-bit ARM has even less share than x86 right now.
May 18
next sibling parent Johannes Pfau <nospam example.com> writes:
Am Mon, 18 May 2015 15:47:06 +0000
schrieb "Joakim" <dlang joakim.fea.st>:

  However, 
 the gdc guys trying to use gcc's emulated TLS have reported 
 problems getting it working with the GC.  Maybe that's since been 
 fixed, but that's the last I heard.

We've fixed that in the meantime, emulated TLS is working fine. The main problem is making the GC fast with emulated TLS. For now we've implemented a slower but 100% correct solution which scans every TLS variable in every thread separately. (GCCs emutls doesn't use contiguous memory). But IIRC to get a 2x collection slowdown compared to native TLS you need 10_000s of TLS variables and many threads, so it probably doesn't matter.
May 18
prev sibling parent reply "Joakim" <dlang joakim.fea.st> writes:
On Monday, 18 May 2015 at 15:47:07 UTC, Joakim wrote:
 Sure, have fun with your new devices. :) Hopefully, I'll get 
 Android/ARM working before then, but I don't and won't have any 
 AArch64 devices to test.  Not that it matters, as 64-bit ARM 
 has even less share than x86 right now.

Earlier this week, I stumbled across a way to get TLS working with ldc for Android/ARM, similar to the approach used for Android/x86 so far. Exception-handling on ARM for ldc is currently unfinished (https://github.com/ldc-developers/ldc/issues/489), so if I disable a handful of tests related to that, I get 36 of 42 druntime modules' unit tests and around 31 of 70 phobos modules' unit tests to pass. All tests were run from the command line on my Android tablet. It appears there are issues related to unicode and the GC causing many of the remaining failures.
May 25
parent reply "Joakim" <dlang joakim.fea.st> writes:
On Monday, 25 May 2015 at 20:08:48 UTC, Joakim wrote:
 On Monday, 18 May 2015 at 15:47:07 UTC, Joakim wrote:
 Sure, have fun with your new devices. :) Hopefully, I'll get 
 Android/ARM working before then, but I don't and won't have 
 any AArch64 devices to test.  Not that it matters, as 64-bit 
 ARM has even less share than x86 right now.

Earlier this week, I stumbled across a way to get TLS working with ldc for Android/ARM, similar to the approach used for Android/x86 so far. Exception-handling on ARM for ldc is currently unfinished (https://github.com/ldc-developers/ldc/issues/489), so if I disable a handful of tests related to that, I get 36 of 42 druntime modules' unit tests and around 31 of 70 phobos modules' unit tests to pass. All tests were run from the command line on my Android tablet. It appears there are issues related to unicode and the GC causing many of the remaining failures.

Some good news, I've made progress on the port to Android/ARM, using ldc's 2.067 branch. Currently, all 46 modules in druntime and 85 of 88 modules in phobos pass their tests (I had to comment out a few tests across four modules) when run on the command-line. There is a GC issue that causes 2-3 other modules to hang only when the tests are run as part of an Android app/apk, ie a D shared library that's invoked by the Java runtime. I've compiled an Android/ARM app that will run the remaining majority of tests on Android 5 Lollipop or newer, which you can download and try out on your Android 5 devices: https://github.com/joakim-noah/android/releases/tag/apk All tests run on my Android 5.1 device, while the last two modules tested by this app hang on an Android 5.0 device I tested. All patches used are linked from the above release.
Jul 30
next sibling parent "Etienne Cimon" <etcimon gmail.com> writes:
On Thursday, 30 July 2015 at 19:38:12 UTC, Joakim wrote:
 On Monday, 25 May 2015 at 20:08:48 UTC, Joakim wrote:
 On Monday, 18 May 2015 at 15:47:07 UTC, Joakim wrote:
 Sure, have fun with your new devices. :) Hopefully, I'll get 
 Android/ARM working before then, but I don't and won't have 
 any AArch64 devices to test.  Not that it matters, as 64-bit 
 ARM has even less share than x86 right now.

Earlier this week, I stumbled across a way to get TLS working with ldc for Android/ARM, similar to the approach used for Android/x86 so far. Exception-handling on ARM for ldc is currently unfinished (https://github.com/ldc-developers/ldc/issues/489), so if I disable a handful of tests related to that, I get 36 of 42 druntime modules' unit tests and around 31 of 70 phobos modules' unit tests to pass. All tests were run from the command line on my Android tablet. It appears there are issues related to unicode and the GC causing many of the remaining failures.

Some good news, I've made progress on the port to Android/ARM, using ldc's 2.067 branch. Currently, all 46 modules in druntime and 85 of 88 modules in phobos pass their tests (I had to comment out a few tests across four modules) when run on the command-line. There is a GC issue that causes 2-3 other modules to hang only when the tests are run as part of an Android app/apk, ie a D shared library that's invoked by the Java runtime. I've compiled an Android/ARM app that will run the remaining majority of tests on Android 5 Lollipop or newer, which you can download and try out on your Android 5 devices: https://github.com/joakim-noah/android/releases/tag/apk All tests run on my Android 5.1 device, while the last two modules tested by this app hang on an Android 5.0 device I tested. All patches used are linked from the above release.

Thanks, I didn't remember you were the one working on this. I've been following this and I'm just as eager to start testing my libraries with it. I think Android could also use a cross-platform web plugin framework. I've started to refactor the idea, and just being able to enhance a website with native code on any platform would be great, it would really make up for being forced into doing all-javascript when writing the UI in HTML5/CSS right now.
Jul 30
prev sibling next sibling parent "Elvis Zhou" <elvis.x.zhou gmail.com> writes:
On Thursday, 30 July 2015 at 19:38:12 UTC, Joakim wrote:
 On Monday, 25 May 2015 at 20:08:48 UTC, Joakim wrote:
 [...]

Some good news, I've made progress on the port to Android/ARM, using ldc's 2.067 branch. Currently, all 46 modules in druntime and 85 of 88 modules in phobos pass their tests (I had to comment out a few tests across four modules) when run on the command-line. There is a GC issue that causes 2-3 other modules to hang only when the tests are run as part of an Android app/apk, ie a D shared library that's invoked by the Java runtime. [...]

Really exciting news, I'll give it a try today, your great effort is much appreciated!
Jul 30
prev sibling next sibling parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 31/07/2015 7:38 a.m., Joakim wrote:
 On Monday, 25 May 2015 at 20:08:48 UTC, Joakim wrote:
 On Monday, 18 May 2015 at 15:47:07 UTC, Joakim wrote:
 Sure, have fun with your new devices. :) Hopefully, I'll get
 Android/ARM working before then, but I don't and won't have any
 AArch64 devices to test.  Not that it matters, as 64-bit ARM has even
 less share than x86 right now.

Earlier this week, I stumbled across a way to get TLS working with ldc for Android/ARM, similar to the approach used for Android/x86 so far. Exception-handling on ARM for ldc is currently unfinished (https://github.com/ldc-developers/ldc/issues/489), so if I disable a handful of tests related to that, I get 36 of 42 druntime modules' unit tests and around 31 of 70 phobos modules' unit tests to pass. All tests were run from the command line on my Android tablet. It appears there are issues related to unicode and the GC causing many of the remaining failures.

Some good news, I've made progress on the port to Android/ARM, using ldc's 2.067 branch. Currently, all 46 modules in druntime and 85 of 88 modules in phobos pass their tests (I had to comment out a few tests across four modules) when run on the command-line. There is a GC issue that causes 2-3 other modules to hang only when the tests are run as part of an Android app/apk, ie a D shared library that's invoked by the Java runtime. I've compiled an Android/ARM app that will run the remaining majority of tests on Android 5 Lollipop or newer, which you can download and try out on your Android 5 devices: https://github.com/joakim-noah/android/releases/tag/apk All tests run on my Android 5.1 device, while the last two modules tested by this app hang on an Android 5.0 device I tested. All patches used are linked from the above release.

So how far off are we in having a D Android stack? With full redistribution support on dlang.org.
Jul 30
parent reply "Joakim" <dlang joakim.fea.st> writes:
On Friday, 31 July 2015 at 04:45:30 UTC, Rikki Cattermole wrote:
 So how far off are we in having a D Android stack?

Depends what you mean, native apps do not have access to the Android GUI and some other APIs, which are only available through Java, ie JNI. So if you simply mean running all of D on Android, not far, a couple more modules to fix and one important GC issue, as I mentioned. If you mean GUI and those other Java APIs, OpenGL is available now but I haven't messed with JNI yet, doubt it'll be a problem though.
 With full redistribution support on dlang.org.

Not sure what you mean by redistribution, the Android NDK? If you just mean distributing a D cross-compiler for Android/ARM, shouldn't take long to fix these remaining issues, but it all depends on how many people pitch in.
Jul 31
parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 1/08/2015 4:43 a.m., Joakim wrote:
 On Friday, 31 July 2015 at 04:45:30 UTC, Rikki Cattermole wrote:
 So how far off are we in having a D Android stack?

Depends what you mean, native apps do not have access to the Android GUI and some other APIs, which are only available through Java, ie JNI. So if you simply mean running all of D on Android, not far, a couple more modules to fix and one important GC issue, as I mentioned. If you mean GUI and those other Java APIs, OpenGL is available now but I haven't messed with JNI yet, doubt it'll be a problem though.

I'm aware of these issues, it's one of the reasons I forked Djvm. Unfortunately I haven't yet worked more on my Java parser which would make making bindings pretty easy.
 With full redistribution support on dlang.org.

Not sure what you mean by redistribution, the Android NDK? If you just mean distributing a D cross-compiler for Android/ARM, shouldn't take long to fix these remaining issues, but it all depends on how many people pitch in.

More or less neither. Basically D cross compilers on all the main platforms that really only require, here is my NDK install now lets make .adk's. Distributed over dlang.org downloads page. But what you are saying is very appealing ;) I'm definitely looking forward to the point where I have no choice but to continue working on my fork of Djvm to make it worth while to use Android with D :)
Jul 31
prev sibling next sibling parent Dan Olson <gorox comcast.net> writes:
"Joakim" <dlang joakim.fea.st> writes:

 Some good news, I've made progress on the port to Android/ARM, using
 ldc's 2.067 branch.  Currently, all 46 modules in druntime and 85 of
 88 modules in phobos pass their tests (I had to comment out a few
 tests across four modules) when run on the command-line.

:-)
Aug 01
prev sibling parent reply "Elvis Zhou" <elvis.x.zhou gmail.com> writes:
On Thursday, 30 July 2015 at 19:38:12 UTC, Joakim wrote:
 On Monday, 25 May 2015 at 20:08:48 UTC, Joakim wrote:
 [...]

Some good news, I've made progress on the port to Android/ARM, using ldc's 2.067 branch. Currently, all 46 modules in druntime and 85 of 88 modules in phobos pass their tests (I had to comment out a few tests across four modules) when run on the command-line. There is a GC issue that causes 2-3 other modules to hang only when the tests are run as part of an Android app/apk, ie a D shared library that's invoked by the Java runtime. [...]

Would those patches for ldc/druntime/phobos be applied & merged into LDC eventually?
Aug 03
parent "Joakim" <dlang joakim.fea.st> writes:
On Monday, 3 August 2015 at 12:46:51 UTC, Elvis Zhou wrote:
 On Thursday, 30 July 2015 at 19:38:12 UTC, Joakim wrote:
 On Monday, 25 May 2015 at 20:08:48 UTC, Joakim wrote:
 [...]

Some good news, I've made progress on the port to Android/ARM, using ldc's 2.067 branch. Currently, all 46 modules in druntime and 85 of 88 modules in phobos pass their tests (I had to comment out a few tests across four modules) when run on the command-line. There is a GC issue that causes 2-3 other modules to hang only when the tests are run as part of an Android app/apk, ie a D shared library that's invoked by the Java runtime. [...]

Would those patches for ldc/druntime/phobos be applied & merged into LDC eventually?

For the ones I wrote which have not been upstreamed already, yes, I'll submit PRs once I get them cleaned up. For example, I'd like to devise a way not to use dl_iterate_phdr to load pre-initialized data, so that Android versions older than 5.0 can run D too. I need to look into employing the same bracketed sections approach that dmd uses.
Aug 03
prev sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 7 May 2015 at 15:58:01 UTC, Joakim wrote:
 It's all a bit hacky right now, but I'll clean it up and submit 
 PRs eventually.

Yes, please do. ;) — David
May 18