digitalmars.D.learn - Re: object splitting in multiple threads
- yes <yes no.com> Jan 09 2009
- "Denis Koroskin" <2korden gmail.com> Jan 09 2009
- yes <yes no.com> Jan 09 2009
- downs <default_357-line yahoo.de> Jan 10 2009
- "Jarrett Billingsley" <jarrett.billingsley gmail.com> Jan 09 2009
- "Jarrett Billingsley" <jarrett.billingsley gmail.com> Jan 09 2009
- "Jarrett Billingsley" <jarrett.billingsley gmail.com> Jan 10 2009
What does this code do? It looks like a ThreadPool use case but I might be wrong.
I tried to recursively make an object distribute its calculations. The calculations should take at least a minute.
Jan 09 2009
On Sat, 10 Jan 2009 07:32:43 +0300, yes <yes no.com> wrote:What does this code do? It looks like a ThreadPool use case but I might be wrong.
I tried to recursively make an object distribute its calculations. The calculations should take at least a minute.
Hm... I guess I understand now, you need a task parallelization, right? In order to do this, you usually separate the task into several independent sub-tasks, put into a queue and wait until it is finished. For example, we have a list of files to download from remote server. For each file, we create a new connection and retrieve it independently (and, possibly, out of order): import tango.core.ThreadPool; void execute() { auto pool = new ThreadPool!(string)(10); // create a thread pool with 10 threads string[] fileList = loadFileList(); foreach (string filePath; fileList) { pool.append(&download, filePath); // append the task } pool.finish(); // wait until all the tasks complete } void download(string fileName) { // ... } In this example, I create a lot of threads (more than hardware runs concurrently) because the CPU is not a bottleneck. You may experience different results in different CPU workaload, though. Hope that helps.
Jan 09 2009
Does Phobos also do threadpools? Somehow I liked the idea of an Object like agent smith, just duplicate yourself when the task is too big. ( ^_^ )Hm... I guess I understand now, you need a task parallelization, right? In order to do this, you usually separate the task into several independent sub-tasks, put into a queue and wait until it is finished. For example, we have a list of files to download from remote server. For each file, we create a new connection and retrieve it independently (and, possibly, out of order): import tango.core.ThreadPool; void execute() { auto pool = new ThreadPool!(string)(10); // create a thread pool with 10 threads string[] fileList = loadFileList(); foreach (string filePath; fileList) { pool.append(&download, filePath); // append the task } pool.finish(); // wait until all the tasks complete } void download(string fileName) { // ... } In this example, I create a lot of threads (more than hardware runs concurrently) because the CPU is not a bottleneck. You may experience different results in different CPU workaload, though. Hope that helps.
Jan 09 2009
Jarrett Billingsley wrote:On Sat, Jan 10, 2009 at 12:30 AM, Jarrett Billingsley <jarrett.billingsley gmail.com> wrote:On Sat, Jan 10, 2009 at 12:18 AM, yes <yes no.com> wrote:Does Phobos also do threadpools?
efficiently. Downs has implemented such a thing in scrapple.tools, but it requires a patch to Phobos to make it run at a reasonable speed.
Ffffffff.. ignore this, I was thinking about fiber pools. I don't think there's any technical reason why a ThreadPool couldn't be written for Phobos, but there isn't one provided by default.
Scrapple.Tools has a Threadpool too :)
Jan 10 2009
On Sat, Jan 10, 2009 at 12:30 AM, Jarrett Billingsley <jarrett.billingsley gmail.com> wrote:On Sat, Jan 10, 2009 at 12:18 AM, yes <yes no.com> wrote:Does Phobos also do threadpools?
From what I understand, not without modification. At least, not efficiently. Downs has implemented such a thing in scrapple.tools, but it requires a patch to Phobos to make it run at a reasonable speed.
Ffffffff.. ignore this, I was thinking about fiber pools. I don't think there's any technical reason why a ThreadPool couldn't be written for Phobos, but there isn't one provided by default.
Jan 09 2009
On Sat, Jan 10, 2009 at 12:18 AM, yes <yes no.com> wrote:Does Phobos also do threadpools?
From what I understand, not without modification. At least, not efficiently. Downs has implemented such a thing in scrapple.tools, but it requires a patch to Phobos to make it run at a reasonable speed.Somehow I liked the idea of an Object like agent smith, just duplicate yourself when the task is too big. ( ^_^ )
Hehe.
Jan 09 2009
On Sat, Jan 10, 2009 at 6:39 AM, downs <default_357-line yahoo.de> wrote:Jarrett Billingsley wrote:On Sat, Jan 10, 2009 at 12:30 AM, Jarrett Billingsley <jarrett.billingsley gmail.com> wrote:On Sat, Jan 10, 2009 at 12:18 AM, yes <yes no.com> wrote:Does Phobos also do threadpools?
efficiently. Downs has implemented such a thing in scrapple.tools, but it requires a patch to Phobos to make it run at a reasonable speed.
Ffffffff.. ignore this, I was thinking about fiber pools. I don't think there's any technical reason why a ThreadPool couldn't be written for Phobos, but there isn't one provided by default.
Scrapple.Tools has a Threadpool too :)
NO! *SLAM*
Jan 10 2009