www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - MatD - Matlab external interface bindings for D

reply Trass3r <un known.com> writes:
I've created some bindings to access Matlab's external interface from D,
i.e. manipulating .mat files, accessing the current workspace and creating  
.mex files.

Thought I'd share it:
http://bitbucket.org/trass3r/matd/wiki/Home
Aug 16 2010
next sibling parent BCS <none anon.com> writes:
Hello Trass3r,

 I've created some bindings to access Matlab's external interface from
 D, i.e. manipulating .mat files, accessing the current workspace and
 creating  .mex files.
 
 Thought I'd share it:
 http://bitbucket.org/trass3r/matd/wiki/Home

If you are looking for a place to host that I can get you access to: http://dsource.org/projects/scrapple -- ... <IXOYE><
Aug 16 2010
prev sibling next sibling parent reply eles <eles eles.com> writes:
Hi,

 I am interested in using the MatD and mex files in D. Also, if
possible, to contribute.

 For the time being, I see that you are distributing a set of
win32 .lib with MatD. I tried to compile the mex examples you
provided in Linux 64 bit, but dmd failed to link, I think it is
because there is a 32 bit compiler (and the Matlab libmex.so
librariest ec. are 64 bit).

 Now, coming back to the Win32 environment:

 I encounter the following problem.

 On the one hand, when compiling/linking against the libmex.lib and
libmx.lib files that you provided, a (correct) .dll file (or
a .mexw32 file) is generated, along with two other files called
arrayProduct.map and arrayProduct.obj. This is just perfect, the .DLL
works and is recognized by Matlab (win32 bit) as a MEX file.

 On the other hand, when compiling/linking against the .lib files
that come with Matlab (in C:\Matlab\extern\lib\win32\lcc or C:\Matlab
\extern\lib\win32\microsoft folders), only the arrayProduct.map and
arrayProduct.obj are created. Where is the .DLL file? Why dmd (2.048)
does not generate a .DLL file in these cases?

 Finally, when compiling/linking against the 3rd set of .lib files
that come with Matlab, namely those in C:\Matlab\extern\lib
\win32\watcom, a strange thing happens: all three files
arrayProduct.obj, arrayProduct.map and arrayProduct.DLL are created,
but the compilation presumably fails since it displays:

D:\testare_mex_d>dmd arrayProduct.d dllmain.d dlldef.def c:\matlab
\extern\lib\wi
n32\watcom\libmex.lib c:\matlab\extern\lib\win32\watcom\libmx.lib
OPTLINK (R) for Win32  Release 8.00.2
Copyright (C) Digital Mars 1989-2009  All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
arrayProduct.obj(arrayProduct)
 Error 42: Symbol Undefined _mxGetPr
arrayProduct.obj(arrayProduct)
 Error 42: Symbol Undefined _mxCreateDoubleMatrix
arrayProduct.obj(arrayProduct)
 Error 42: Symbol Undefined _mxGetN
arrayProduct.obj(arrayProduct)
 Error 42: Symbol Undefined _mxGetScalar
arrayProduct.obj(arrayProduct)
 Error 42: Symbol Undefined _mexErrMsgIdAndTxt
arrayProduct.obj(arrayProduct)
 Error 42: Symbol Undefined _mxGetM
arrayProduct.obj(arrayProduct)
 Error 42: Symbol Undefined _mxGetNumberOfElements
arrayProduct.obj(arrayProduct)
 Error 42: Symbol Undefined _mxIsComplex
arrayProduct.obj(arrayProduct)
 Error 42: Symbol Undefined _mxIsDouble
--- errorlevel 9


 Why this? Are the .lib files that you are distributing along with
MatD special in any way?

 Thanks,

 eles
Aug 21 2010
next sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
Hi,

I have not tried it with the MatLab libraries, but optlink can only 
handle libraries in the omf-format. Libraries in other formats are 
either ignored (in the best case) or cause optlink to exit without 
error, but also without a generated executable.

Trass3r probably used coffimplib 
(ftp://ftp.digitalmars.com/coffimplib.zip) to convert the microsoft 
compatible import libraries to omf format to help optlink.

Rainer


eles wrote:
 Hi,
 
  I am interested in using the MatD and mex files in D. Also, if
 possible, to contribute.
 
  For the time being, I see that you are distributing a set of
 win32 .lib with MatD. I tried to compile the mex examples you
 provided in Linux 64 bit, but dmd failed to link, I think it is
 because there is a 32 bit compiler (and the Matlab libmex.so
 librariest ec. are 64 bit).
 
  Now, coming back to the Win32 environment:
 
  I encounter the following problem.
 
  On the one hand, when compiling/linking against the libmex.lib and
 libmx.lib files that you provided, a (correct) .dll file (or
 a .mexw32 file) is generated, along with two other files called
 arrayProduct.map and arrayProduct.obj. This is just perfect, the .DLL
 works and is recognized by Matlab (win32 bit) as a MEX file.
 
  On the other hand, when compiling/linking against the .lib files
 that come with Matlab (in C:\Matlab\extern\lib\win32\lcc or C:\Matlab
 \extern\lib\win32\microsoft folders), only the arrayProduct.map and
 arrayProduct.obj are created. Where is the .DLL file? Why dmd (2.048)
 does not generate a .DLL file in these cases?
 
  Finally, when compiling/linking against the 3rd set of .lib files
 that come with Matlab, namely those in C:\Matlab\extern\lib
 \win32\watcom, a strange thing happens: all three files
 arrayProduct.obj, arrayProduct.map and arrayProduct.DLL are created,
 but the compilation presumably fails since it displays:
 
 D:\testare_mex_d>dmd arrayProduct.d dllmain.d dlldef.def c:\matlab
 \extern\lib\wi
 n32\watcom\libmex.lib c:\matlab\extern\lib\win32\watcom\libmx.lib
 OPTLINK (R) for Win32  Release 8.00.2
 Copyright (C) Digital Mars 1989-2009  All rights reserved.
 http://www.digitalmars.com/ctg/optlink.html
 arrayProduct.obj(arrayProduct)
  Error 42: Symbol Undefined _mxGetPr
 arrayProduct.obj(arrayProduct)
  Error 42: Symbol Undefined _mxCreateDoubleMatrix
 arrayProduct.obj(arrayProduct)
  Error 42: Symbol Undefined _mxGetN
 arrayProduct.obj(arrayProduct)
  Error 42: Symbol Undefined _mxGetScalar
 arrayProduct.obj(arrayProduct)
  Error 42: Symbol Undefined _mexErrMsgIdAndTxt
 arrayProduct.obj(arrayProduct)
  Error 42: Symbol Undefined _mxGetM
 arrayProduct.obj(arrayProduct)
  Error 42: Symbol Undefined _mxGetNumberOfElements
 arrayProduct.obj(arrayProduct)
  Error 42: Symbol Undefined _mxIsComplex
 arrayProduct.obj(arrayProduct)
  Error 42: Symbol Undefined _mxIsDouble
 --- errorlevel 9
 
 
  Why this? Are the .lib files that you are distributing along with
 MatD special in any way?
 
  Thanks,
 
  eles

Aug 22 2010
next sibling parent eles <eles eles.com> writes:
== Quote from Rainer Schuetze (r.sagitario gmx.de)'s article
 Hi,
 I have not tried it with the MatLab libraries, but optlink can only
 handle libraries in the omf-format. Libraries in other formats are
 either ignored (in the best case) or cause optlink to exit without
 error, but also without a generated executable.

I find that a strange behavior. A warning should be issued.
Aug 22 2010
prev sibling parent eles <eles eles.com> writes:
== Quote from Rainer Schuetze (r.sagitario gmx.de)'s article
 Hi,
 I have not tried it with the MatLab libraries, but optlink can only
 handle libraries in the omf-format. Libraries in other formats are
 either ignored (in the best case) or cause optlink to exit without
 error, but also without a generated executable.
 Trass3r probably used coffimplib
 (ftp://ftp.digitalmars.com/coffimplib.zip) to convert the microsoft
 compatible import libraries to omf format to help optlink.
 Rainer

I just checked and you are right. I used coffimplib.exe to convert the Matlab .lib files in the "microsoft" subfolder (presumably in coff format) to new .lib files in omf format. It works as expected, the compiler links them in and the .DLL file is correctly generated. Thank you.
Aug 22 2010
prev sibling parent reply eles <eles eles.com> writes:
Hi,

 Here are two .bat files that help converting an entire folder
of .lib files in omf format. A subfolder "omf" will be created in the
current folder.

 coffimplib needs to be in the current folder, too.

 The batch file "convert_all.bat" is destined to be run in the C:
\Matlab\extern\lib\win32\microsoft folder.

 ====file: convert_all.bat========

 echo off
echo [convert_all.bat]
md omf
for /f %%a IN ('dir /b *.lib') do call convert_one.bat %%a

 ====file: convert_one.bat========

 echo off
echo [convert_one.bat] converting %1
cd omf
..\coffimplib ..\%1 .\%1
cd ..

===================================

eles
Aug 23 2010
parent eles <eles eles.com> writes:
Hello,

 Here is (attachement) a translation of file mwdebug.h
(matd.c.mwdebug).

 There is a line commented out, specifically:

 //extern FILE* matGetFp_d(MATFile* ph, const(char)* file, int
linenum);

since I don't know how to replace the C FILE structure.

 In principle, module matd.c.mwdebug could be added to the MatD
project.

 eles
Aug 23 2010
prev sibling next sibling parent Trass3r <un known.com> writes:
  I am interested in using the MatD and mex files in D. Also, if
 possible, to contribute.

Cool :)
  For the time being, I see that you are distributing a set of
 win32 .lib with MatD. I tried to compile the mex examples you
 provided in Linux 64 bit, but dmd failed to link, I think it is
 because there is a 32 bit compiler (and the Matlab libmex.so
 librariest ec. are 64 bit).

Yeah, you need to create mex64 files for use with Matlab x64. That's why I haven't tested it on Win64 yet.
  On the one hand, when compiling/linking against the libmex.lib and
 libmx.lib files that you provided, a (correct) .dll file (or
 a .mexw32 file) is generated, along with two other files called
 arrayProduct.map and arrayProduct.obj. This is just perfect, the .DLL
 works and is recognized by Matlab (win32 bit) as a MEX file.

As Rainer already pointed out optlink only supports omf import libraries. I created them using implib.
Aug 22 2010
prev sibling next sibling parent Trass3r <un known.com> writes:
  Here is (attachement) a translation of file mwdebug.h
 (matd.c.mwdebug).

Thanks for your work but unfortunately I already converted mwdebug.h in the first place. The reason why it isn't committed yet is these functions aren't provided by Matlab's dlls. mwdebug.cpp contains the implementation. I want to handle this in a clever way together with solving another big problem: Detouring of function calls is done via the C preprocessor. In fact the current code is most probably invalid in 64Bit environments since MX_COMPAT_32 isn't handled correctly yet. I don't know if there is a way to handle all this without string mixins. Some comments on your code: - You need to use "extern(C):", extern itself is equivalent to extern(D) - Arrays are different in D, so mxArray*[] plhs becomes mxArray** plhs
 since I don't know how to replace the C FILE structure.

import core.stdc.stdio;
Aug 23 2010
prev sibling next sibling parent Trass3r <un known.com> writes:
 Detouring of function calls is done via the C preprocessor. In fact the  
 current code is most probably invalid in 64Bit environments since  
 MX_COMPAT_32 isn't handled correctly yet.

I forgot to mention that any combination of ARRAY_ACCESS_INLINING, MX_COMPAT_32, ARGCHECK needs to be handled. Even the Matlab devs seem to be unsure about that: " #if 0 /* * Conflicts with 32-bit compatibility (-> MX_COMPAT_32) * Needs revisiting. Is this layer still worthwhile? * XXX */ #ifdef ARGCHECK "
Aug 23 2010
prev sibling parent Trass3r <un known.com> writes:
 I don't know if there is a way to handle all this without string mixins.

i.e. something along the lines of mixin(foo(" ... /** * Get pointer to dimension array */ const(mwSize)* mxGetDimensions(const(mxArray)* pa); ... ")); and foo replaces the function defs with func(...){return func_730(...)} via RegEx replace if MX_COMPAT_32 is not defined and so on. (IF RegEx works at compile-time)
Aug 23 2010