www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Commercial video processing app in D (experience report)

reply thedeemon <dlang thedeemon.com> writes:
Hi,
I just wanted to share some experience of using D in industry.
Recently my little company released version 2.0 of our flagship 
product Video Enhancer, a video processing application for 
Windows, and this time it's written in D.
http://www.infognition.com/VideoEnhancer/

Couple of screenshots:
http://data.infognition.com/VideoEnhancer/ve2d-filters.jpg
http://data.infognition.com/VideoEnhancer/ve2e-save.jpg

Version 1 was born like 10 years ago and was of course written in 
C++. It consisted of main GUI executable and 5 dynamically loaded 
DirectShow filters. For GUI version 1 used MFC and a third-party 
skinning engine. This skinning engine had its own problems but 
since we didn't have its sources we couldn't fix them, meanwhile 
its author disappeared in sands of time. So when time has come to 
create version 2 I chose the best available language and an open 
source GUI library with 100% control and customizability - 
DLangUI. Overall, I'm pretty happy with this choice.

Version 2 is quite different from v1 in feature set and internal 
structure, it's not a direct translation. It consists of two 
executables running in tandem (one does GUI, the other deals with 
video) and 2 dynamically loaded DirectShow filters. Both main 
executables are purely in D, while the DirectShow filters are 
still in C++. Heavy number crunching, including our main feature 
- motion-based video upscaler - is still in C++, because of heavy 
SIMD usage and Intel compiler.

Main executable of version 1 was ~34K lines of C++ (half of which 
were libraries like pugixml) and full build took ~90 seconds.
Main executables of version 2 are in total ~7.5K lines of D (of 
which 2K are auto generated by IDL2D tool) and full build of GUI 
app takes 7 seconds (and the worker app builds in 3-4 seconds), 
so that's a really nice improvement.
Thanks to Phobos we don't need many libraries: things like XML 
parsing, ZIP unpacking and many others are all covered by the 
standard library. Only two additional libraries were used: 
Cerealed for serialization of messages the two processes 
exchange, and DLangUI.

Some things to reflect on:

Compiler
Compiler used is DMD 2.070, 32-bit target. Video Enhancer 
supports 200+ plugins from VirtualDub and they happen to be 
32-bit, so our app has to be 32-bit too. Speed of code generated 
by DMD is more than enough, even debug builds were fast enough. 
The default linker is used (not the MS one), and I was worried 
there might be some troubles with antivirus false positives (that 
happened before when using optlink) but no, everything went 
smooth and no problems with optlink arose whatsoever.

IDE
Visual Studio 2010 with VisualD. I've used this combo for many 
years, generally quite successively. Last year its D parser had 
some problems that made it crash on code that used DLangUI, and 
that was painful. I even made a patch that made the crash silent, 
so VisualD would silently reload the parser and continue working. 
It worked, but luckily authors of D parser and VisualD quickly 
found the crash cause and fixed it, since then everything works 
smoothly out of the box. I like how well DML works there, with 
syntax highlighting and autocompletion:
http://data.infognition.com/VideoEnhancer/dml.png

Builds
We're not using Dub to build the app, it tends to be slow and 
rebuild dependencies too often (or maybe I just haven't learnt to 
use it properly). Instead we use Dub to build the libraries and 
produce .lib files, then reference libraries sources and lib 
files in VisualD project of the main apps and then use VisualD's 
simple building process that just invokes DMD.

Cerealed
This compile-time-introspection-based serializaition lib is 
really great: powerful and easy to use. We're probably using an 
old version, haven't updated for some time, and the version we 
use sometimes had problems serializing certain types (like 
bool[], IIRC), so sometimes we had to tweak our message types to 
make it compile, but most of the time it just works.

DLangUI
Very nice library. Documentation is very sparse though, so 
learning to use DLangUI often means reading source code of 
examples and the lib itself, and sometimes even that's not enough 
and you need to learn some Android basics, since it originates 
from Android world. But once you learn how to use it, how to 
encode what you need in DML (a QML counterpart) or add required 
functionality by overriding some method of its class, it's really 
great and pleasant to use. Many times I was so happy the source 
code is available, first for learning, then for tweaking and 
fixing bugs. I've found a few minor bugs and sent a few trivial 
fixes that were merged quickly. DLangUI is cross-platform and has 
several backends for drawing and font rendering. We're using its 
minimal build targeted to use Win32 API (had to tweak dub.json a 
bit). We don't use OpenGL, as it's not really guaranteed to work 
well on any Windows box. Using just WinAPI makes our app smaller, 
more stable and avoids dependencies.

Garbage Collection
Totally fine if you know what you're doing. Some people say you 
can't make responsive GUI apps in GC-ed languages. That's a myth. 
In this application we rely on GC almost as freely as one would 
in C#, allocate strings, messages, widgets and other small litter 
without having to worry about deallocation, and it works well, 
I've never seen any performance problems related to GC in this 
app, and no GC pauses were visible at all. However this is a 
32-bit app and some care must be taken to avoid excessive memory 
use. Firstly and mainly, where bitmaps for GUI elements and video 
frames are allocated. DLangUI tends to allocate everything in GC 
heap without second thought and has been allocating some arrays 
even just to draw some pictures rescaled. If you're resizing a 
window and it has some big bitmap that gets resized too, by 
default DLangUI tends to just allocate a new buffer for each new 
size, this does not look good in memory section of Task Manager. 
However if instead of allocating a new ColorDrawBuf, for example, 
you use its resize() method and don't forget to call 
assumeSafeAppend() for its buffer, then it stops eating memory 
and behaves well, even though it's still "managed", no manual 
management required. In couple of places I did change ordinary 
arrays to std.container.array to reduce allocations, the code did 
not change much but there was much less work for GC. One such 
change where it was important (allocation during bitmap drawing) 
was merged back to DLangUI, so don't worry about that one.

COM
To deal with video (reading, parsing, decoding, encoding..) we 
use DirectShow which is COM-based. I've found a lot of good stuff 
for working with COM in VisualD sources. There is IDL2D utility 
that converts IDL files from Windows SDK to D source files. We 
used it actively (~2K lines generated). There is also TLB to IDL 
converter if you need it (I haven't tried to use it myself). The 
way it converts COM interfaces, it adds their IIDs (GUIDs) as a 
static member of the interface, so when you need to 
QueryInterface() you don't need to provide IID for the interface, 
it's already there, and this is used by the ComPtr smart pointer 
that deals with most COM stuff like reference counting too. I 
borrowed initial ComPtr implementation from VisualD and then 
changed it a lot, most importantly, my version also checks for 
errors automatically, now I have Error Monad for free:
http://www.infognition.com/blog/2016/error_checking_smart_pointer.html

D features used.
Apart from the smart pointer above, we used compile-time 
introspection to pass and automatically dispatch messages between 
two processes, there is a blog post about that too. We used User 
Defined Attributes to describe what to do with errors if they 
arise in message handlers
http://stuff.thedeemon.com/lj/onerror.png
and then proper error handling was performed automatically, no 
need to repeat it in each method, just add one word of annotation 
and it works.
We used std.concurrency message passing to talk between GUI and 
other threads.
We enjoyed the ability to call C++ interfaces directly from D and 
D methods from C++, no FFI required, that was really nice and 
helped a lot.
Ability to include binary data as simple as import("file.dat") 
was also very nice. This way we include some images and also 
bytecode for a little VM, generated by a compiler written in 
OCaml, but that is another story.
That's what comes to mind now, the rest of the code is quite 
boring I guess.

TL;DR: at least in some areas D is a fine successor to C++ and 
can be used to make real world video processing apps that people 
around the world use and pay for.
Apr 27 2016
next sibling parent reply FreeSlave <freeslave93 gmail.com> writes:
That's great. I'm surprised someone used DlangUI for commercial 
app. I would not say it's quite ready even for free ones. It's 
cool you merged your changes.

On Wednesday, 27 April 2016 at 12:42:05 UTC, thedeemon wrote:
 Couple of screenshots:
 http://data.infognition.com/VideoEnhancer/ve2d-filters.jpg
 http://data.infognition.com/VideoEnhancer/ve2e-save.jpg
Screenshots are so blurred. Do you encourage us to use video enhancer to get better image quality? :)
Apr 27 2016
parent reply thedeemon <dlang thedeemon.com> writes:
On Wednesday, 27 April 2016 at 13:04:27 UTC, FreeSlave wrote:

 Screenshots are so blurred.
They are not. Just click to enlarge, your browser blurred them while resizing.
Apr 27 2016
parent reply FreeSlave <freeslave93 gmail.com> writes:
On Wednesday, 27 April 2016 at 13:58:13 UTC, thedeemon wrote:
 On Wednesday, 27 April 2016 at 13:04:27 UTC, FreeSlave wrote:

 Screenshots are so blurred.
They are not. Just click to enlarge, your browser blurred them while resizing.
They are. Well, maybe blurred is not the right word, but screenshots have obvious jpeg artifacts. Even those that have png extension. Just look at window title in window header.
Apr 27 2016
parent reply FreeSlave <freeslave93 gmail.com> writes:
On Wednesday, 27 April 2016 at 14:07:18 UTC, FreeSlave wrote:
 On Wednesday, 27 April 2016 at 13:58:13 UTC, thedeemon wrote:
 On Wednesday, 27 April 2016 at 13:04:27 UTC, FreeSlave wrote:

 Screenshots are so blurred.
They are not. Just click to enlarge, your browser blurred them while resizing.
They are. Well, maybe blurred is not the right word, but screenshots have obvious jpeg artifacts. Even those that have png extension. Just look at window title in window header.
Ok, I'm not right about png ones. They look ok if I download them to computer and open in photo viewer. Weird Google Chrome renders them worse then they are (in 100% scale).
Apr 27 2016
parent FreeSlave <freeslave93 gmail.com> writes:
On Wednesday, 27 April 2016 at 14:14:02 UTC, FreeSlave wrote:
 On Wednesday, 27 April 2016 at 14:07:18 UTC, FreeSlave wrote:
 On Wednesday, 27 April 2016 at 13:58:13 UTC, thedeemon wrote:
 On Wednesday, 27 April 2016 at 13:04:27 UTC, FreeSlave wrote:

 Screenshots are so blurred.
They are not. Just click to enlarge, your browser blurred them while resizing.
They are. Well, maybe blurred is not the right word, but screenshots have obvious jpeg artifacts. Even those that have png extension. Just look at window title in window header.
Ok, I'm not right about png ones. They look ok if I download them to computer and open in photo viewer. Weird Google Chrome renders them worse then they are (in 100% scale).
Images look good (even jpeg ones) if I use open original image in new tab. Though both have the same address. Can't explain what the issue here.
Apr 27 2016
prev sibling next sibling parent Guillaume Piolat <contact gam3sfrommars.fr> writes:
On Wednesday, 27 April 2016 at 12:42:05 UTC, thedeemon wrote:
 Compiler
 Compiler used is DMD 2.070, 32-bit target. Video Enhancer 
 supports 200+ plugins from VirtualDub and they happen to be 
 32-bit, so our app has to be 32-bit too. Speed of code 
 generated by DMD is more than enough, even debug builds were 
 fast enough. The default linker is used (not the MS one), and I 
 was worried there might be some troubles with antivirus false 
 positives (that happened before when using optlink) but no, 
 everything went smooth and no problems with optlink arose 
 whatsoever.
Note that starting with the newest LDC releases you can have Win32 builds. The parameters to pass to the linker to avoid a VS runtime dependency are: link.exe [...stuff...] libcmt.lib /nodefaultlib:msvcrt.lib /nodefaultlib:vcruntime.lib Users report such builds working on XP, Vista and later of course. The advantages are faster binaries (typically 2x faster) and importantly lack of backend regressions.
Apr 27 2016
prev sibling next sibling parent Joakim <dlang joakim.fea.st> writes:
On Wednesday, 27 April 2016 at 12:42:05 UTC, thedeemon wrote:
 Hi,
 I just wanted to share some experience of using D in industry.
 Recently my little company released version 2.0 of our flagship 
 product Video Enhancer, a video processing application for 
 Windows, and this time it's written in D.
 http://www.infognition.com/VideoEnhancer/

 [...]
Thanks for the great writeup! :)
Apr 27 2016
prev sibling next sibling parent reply Christof Schardt <christof schardt.info> writes:
On Wednesday, 27 April 2016 at 12:42:05 UTC, thedeemon wrote:
 IDE
 Visual Studio 2010 with VisualD. I've used this combo for many 
 years, generally quite successively. Last year its D parser had
Thanks for this excellent contribution. It gives a lot of insights. Just a question: When working with C++, did you use VisualAssist? I find this VisualC++-AddOn so incredibly useful, that the lack of something comparable for D is one of the biggest obstacles for switching to D (another obstacle is the number of ~500kloc of my project, music notation).
Apr 27 2016
parent thedeemon <dlang thedeemon.com> writes:
On Wednesday, 27 April 2016 at 15:57:19 UTC, Christof Schardt 
wrote:

 Just a question: When working with C++, did you use 
 VisualAssist?
I've used it previously in earlier VS versions but not in VS2010. VisualAssist is really great, I agree. VisualD is far from it but at least it's better than C++ support in plain VS2010 without VA.
Apr 27 2016
prev sibling next sibling parent reply thedeemon <dlang thedeemon.com> writes:
On Wednesday, 27 April 2016 at 12:42:05 UTC, thedeemon wrote:

  full build of GUI app takes 7 seconds
Forgot to mention one anecdote: the build time increases by another 7 seconds if I use std.net.curl.get() function instead of std.net.curl.HTTP struct for doing a simple GET from our site.
Apr 27 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 04/27/2016 01:17 PM, thedeemon wrote:
 On Wednesday, 27 April 2016 at 12:42:05 UTC, thedeemon wrote:

  full build of GUI app takes 7 seconds
Forgot to mention one anecdote: the build time increases by another 7 seconds if I use std.net.curl.get() function instead of std.net.curl.HTTP struct for doing a simple GET from our site.
What is the reason for this, could you please investigate a bit? -- Andrei
Apr 27 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/27/2016 5:42 AM, thedeemon wrote:
 I just wanted to share some experience of using D in industry.
Wonderful, thanks for taking the time to write this up. I'm especially pleased that you found great uses for a couple features that were a bit speculative because they are unusual - the user defined attributes, and the file binary data imports.
Apr 27 2016
parent Jacob Carlborg <doob me.com> writes:
On 2016-04-28 01:53, Walter Bright wrote:

 Wonderful, thanks for taking the time to write this up. I'm especially
 pleased that you found great uses for a couple features that were a bit
 speculative because they are unusual - the user defined attributes, and
 the file binary data imports.
I'm using the string import feature in DStep to bundle Clang internal header files. It's a great feature that makes distribution a lot easier since only a single executable is everything that is needed. -- /Jacob Carlborg
Apr 28 2016
prev sibling next sibling parent reply Relja Ljubobratovic <ljubobratovic.relja gmail.com> writes:
On Wednesday, 27 April 2016 at 12:42:05 UTC, thedeemon wrote:
 Hi,
 I just wanted to share some experience of using D in industry.
 Recently my little company released version 2.0 of our flagship 
 product Video Enhancer, a video processing application for 
 Windows, and this time it's written in D.
 http://www.infognition.com/VideoEnhancer/
Awesome work, congratulations! Can you share with us some of your experience working on image and video processing modules in the app, such as are filters here: http://www.infognition.com/VideoEnhancer/filters.html If I may ask, was that part implemented in D, C++, or was some 3rd party library used? Thanks, and again - big congrats! Relja
Apr 27 2016
parent reply thedeemon <dlang thedeemon.com> writes:
On Thursday, 28 April 2016 at 06:22:18 UTC, Relja Ljubobratovic 
wrote:

 Can you share with us some of your experience working on image 
 and video processing modules in the app, such as are filters 
 here:
 http://www.infognition.com/VideoEnhancer/filters.html

 If I may ask, was that part implemented in D, C++, or was some 
 3rd party library used?
Thanks! The filters listed there are third-party plugins originally created for VirtualDub ( http://virtualdub.org/ ) by different people, in C++. We made just 2-3 of them, like motion-based temporal denoiser (Film Dirt Cleaner) and Intelligent Brightness filter for automatic brightness/contrast correction. Our most interesting and distinctive piece of tech is our Super Resolution engine for video upsizing and it's not in that list, it's built-in in the app (and also available separately as plugins for some other hosts). All this image processing stuff is written in C++ and works directly with raw image bytes, no special libraries involved. When video processing starts our filters usually launch a bunch of worker threads and these threads work in parallel each on its part of video frame (divided into horizontal stripes usually). Inside they often work block-wise and we have a bunch of template classes for different blocks (RGB or monochrome) parameterized by pixel data type and often block size, so the size is often is known at compile-time and compiler can unroll the loops properly. When doing motion search we're using our vector class parameterized by precision, so we have vectors of different precision (low-res pixel, high-res pixel, half-pixel, quarter-pixel etc.) and type system makes sure I don't add or mix vectors of different precision and don't pass a half-pixel-precise vector to a block reading routine that expects quarter-pixel precise coordinates. Where it makes sense and possible we use SIMD classes like F32vec4 and/or SIMD intrinsics for pixel operations. Video Enhancer allows chaining several VD filters and our SR rescaler instances to a pipeline and it's also parallelized, so when first filter finishes with frame X it can immediately start working on frame X+1 while the next filter is still working on frame X. Previously it was organized as a chain of DirectShow filters with a special Parallelizer filter inserted between video processing ones, this Parallelizer had some frame queue inside and separated receiving and sending threads, allowing the connected filters to work in parallel. In version 2 it's trickier, since we need to be able to seek to different positions in the video and some filters may request a few frames before and after the current, so sequential pipeline doesn't suffice anymore, now we build a virtual chain inside one big DirectShow filter, and each node in that chain has its worker thread and they do message passing to communicate. After all, we now have a big DirectShow filter in 11K lines of C++ that does both Super Resolution resizing and invoking VirtualDub plugins (imitating VirtualDub for them) and doing colorspace conversions where necessary and organizing them all into a pipeline that is pull-based inside but behaves as push-based DirectShow filter outside. So the D part is using COM to build and run a DirectShow graph with all the readers, splitters, codecs and of course our big video processing DirectShow filter, it talks to it via COM and some callbacks but doesn't do much with video frames apart from copying. Btw, if you're interested in an image processing app in pure D, I've got one too: http://www.infognition.com/blogsort/ (sources: https://bitbucket.org/infognition/bsort )
Apr 28 2016
parent Relja Ljubobratovic <ljubobratovic.relja gmail.com> writes:
Awesome! Thanks so much for such detailed explanation!

 Btw, if you're interested in an image processing app in pure D, 
 I've got one too:
 http://www.infognition.com/blogsort/
 (sources: https://bitbucket.org/infognition/bsort )
Great, I'll check it out - Thanks!
Apr 28 2016
prev sibling next sibling parent Chris <wendlec tcd.ie> writes:
On Wednesday, 27 April 2016 at 12:42:05 UTC, thedeemon wrote:
 Hi,
 I just wanted to share some experience of using D in industry.
 Recently my little company released version 2.0 of our flagship 
 product Video Enhancer, a video processing application for 
 Windows, and this time it's written in D.
 http://www.infognition.com/VideoEnhancer/
[snip]
 DLangUI
 Very nice library. Documentation is very sparse though, so 
 learning to use DLangUI often means reading source code of 
 examples and the lib itself, and sometimes even that's not 
 enough and you need to learn some Android basics, since it 
 originates from Android world. But once you learn how to use 
 it, how to encode what you need in DML (a QML counterpart) or 
 add required functionality by overriding some method of its 
 class, it's really great and pleasant to use. Many times I was 
 so happy the source code is available, first for learning, then 
 for tweaking and fixing bugs. I've found a few minor bugs and 
 sent a few trivial fixes that were merged quickly. DLangUI is 
 cross-platform and has several backends for drawing and font 
 rendering. We're using its minimal build targeted to use Win32 
 API (had to tweak dub.json a bit). We don't use OpenGL, as it's 
 not really guaranteed to work well on any Windows box. Using 
 just WinAPI makes our app smaller, more stable and avoids 
 dependencies.
[snip] Another reason to embrace DLangUI. One starting point would be to improve the documentation and write a few tutorials (including DML, themes etc.)
Apr 28 2016
prev sibling next sibling parent Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 27 April 2016 at 12:42:05 UTC, thedeemon wrote:
 Cerealed
 This compile-time-introspection-based serializaition lib is 
 really great: powerful and easy to use. We're probably using an 
 old version, haven't updated for some time, and the version we 
 use sometimes had problems serializing certain types (like 
 bool[], IIRC), so sometimes we had to tweak our message types 
 to make it compile, but most of the time it just works.
Thanks for the kind words! Can you let me know what was wrong with serialising bool[] and whatever other types you had problems with please? I'd like to fix them. Thanks! Atila
Apr 28 2016
prev sibling parent reply TheGag96 <thegag96 gmail.com> writes:
On Wednesday, 27 April 2016 at 12:42:05 UTC, thedeemon wrote:
 (snip)
Sorry to bump this thread, but how did you handle multiple windows using DlangUI? (As in, were you able to prevent input on the main window while another one was open, etc.) I know Vadim/buggins is working on improving that right now but I'd like to know if there's a clean way to handle more than one window already.
May 05 2016
parent thedeemon <dlang thedeemon.com> writes:
On Thursday, 5 May 2016 at 18:33:44 UTC, TheGag96 wrote:
 On Wednesday, 27 April 2016 at 12:42:05 UTC, thedeemon wrote:
 (snip)
Sorry to bump this thread, but how did you handle multiple windows using DlangUI? (As in, were you able to prevent input on the main window while another one was open, etc.) I know Vadim/buggins is working on improving that right now but I'd like to know if there's a clean way to handle more than one window already.
In our case there is one main window and a few other windows get opened one at a time (Platform.instance.createWindow with WindowFlag.Modal), increasing number of open windows to 2, and sometimes native dialogs are invoked in addition to the 2 DLangUI windows. I remember there was some glitches like if you press Enter in OS-native file dialog and it gets closed then an odd Enter key press message may be received by DLangUI window, but we used one simple trick: before opening some modal window change focus in current window to something harmless. After that everything worked quite well. But we're not using more than one active (i.e. not shadowed by a modal one) window at a time, so don't meet with issues you might meet there.
May 05 2016