www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Crazy, sad but ... would you use D for your own facebook or pinterest?

reply aberba <karabutaworld gmail.com> writes:
I'm not building Facebook/pinterest but I'm trying to work on a 
platform like "pinterest-like" but on a small scale. I want it to 
be easy to write, fast, ... you know. D is obviously that (IMO).

About scalability, would you recommend D(vibe.d initially) for 
long run (techically, generally, currently)? Why? (Brutal 
honesty).
Jan 02
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 2 January 2017 at 21:49:03 UTC, aberba wrote:
 I'm trying to work on a platform like "pinterest-like" but on a 
 small scale.
I did something similar for work about five or six years ago. Used D, went well. The company changed direction though while it was still small, so I'll never know if it would have scaled or not.
Jan 02
parent reply aberba <karabutaworld gmail.com> writes:
On Tuesday, 3 January 2017 at 00:04:33 UTC, Adam D. Ruppe wrote:
 On Monday, 2 January 2017 at 21:49:03 UTC, aberba wrote:
 I'm trying to work on a platform like "pinterest-like" but on 
 a small scale.
I did something similar for work about five or six years ago. Used D, went well. The company changed direction though while it was still small, so I'll never know if it would have scaled or not.
Nice. How fast were you able to get it to a mature state? I have been prototyping with vibe.d (dub) for sometime now. This might be obvious but ... compilation and linking slows the process... its not much problem though.
Jan 03
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 3 January 2017 at 11:59:46 UTC, aberba wrote:
 Nice. How fast were you able to get it to a mature state?
About six months to get everything we wanted working well. The initial core was done in about two (including me writing necessary D libraries to support it, this was done before vibe.d was out so I wrote all my own), then we had to add features, styles, polish, etc. over the next several months.
 I have been prototyping with vibe.d (dub) for sometime now.

 This might be obvious but ... compilation and linking slows the 
 process... its not much problem though.
My compile+link time at one point hit 15 seconds so I optimized some templates and got it down to 7. Still kinda slow but not *that* bad.
Jan 03
next sibling parent reply aberba <karabutaworld gmail.com> writes:
On Wednesday, 4 January 2017 at 04:10:30 UTC, Adam D. Ruppe wrote:

 About six months to get everything we wanted working well. The 
 initial core was done in about two (including me writing 
 necessary D libraries to support it, this was done before 
 vibe.d was out so I wrote all my own), then we had to add 
 features, styles, polish, etc. over the next several months.
Did you implement some form of clustering (session, storage, caching)?
Jan 03
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 4 January 2017 at 07:08:25 UTC, aberba wrote:
 Did you implement some form of clustering (session, storage, 
 caching)?
I cheated (well, some would say "made a reasonable decision not to reinvent the wheel"): I ran two copies of the application and both talked to the same database (which itself has a replication feature for failover, so any one server could go down at a time without breaking things, but two down at once would most likely have been a problem). Session storage was done as a special case of database setup. The application itself didn't know and didn't need to know about the server setup. We never hit a point where this was problematic. If it took off to millions of users, perhaps things would need to change, but it worked fine with our actual scale (we made a modest profit for the years it stayed up, so I don't consider it a failure per se, but as I said, the company eventually pivoted).
Jan 04
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/3/17 11:10 PM, Adam D. Ruppe wrote:
 My compile+link time at one point hit 15 seconds so I optimized some
 templates and got it down to 7. Still kinda slow but not *that* bad.
How large was the codebase? Thx! -- Andrei
Jan 04
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 4 January 2017 at 12:44:08 UTC, Andrei Alexandrescu 
wrote:
 How large was the codebase? Thx! -- Andrei
wc reports about 80,000 lines. But it isn't line count that affects build speed in D: it is the content of those lines. When I optimized it, the source code actually got *bigger*, but I simplified templates and moved some stuff to runtime functions which build much, much faster than complicated template+CTFE instantiations. See, the program used a lot of code generation so I'd just add a function to the class and all the web API and most the HTML components would be auto-generated from the function signature. By the time the D program was up to 10,000 lines, we already had more relevant functionality than the 100,000 line PHP program we were migrating away from, mostly thanks to the more efficient abstractions and generating boilerplate automatically. (Though, granted, the php did some stuff we didn't care about too, the feature sets were mostly comparable.) However, adding a one line function to the source could result in a dozen templates being instantiated, which slows compile and makes for some huge binaries. When I changed it, it would do some metadata generation in the template, then pass off to as many reused instantiations as possible (so instead of generateFunction!foo doing everything, it would forward to generateFunctionArgumentCode!(ParameterType!foo) (well, we didn't have ParameterType back then, std.traits wasn't even around IIRC until after like 2012 and I did the bulk of this in 2010, but my homemade code did the same thing). Then, the compiler can reuse the same thing for common types like int and string. Also, these would generate data tables that a runtime function could load instead of *being* the whole function, cutting the generated code down significantly.) Actually, I started all templated, then moved to a hybrid template + TypeInfo-like setup for various reasons... something to keep in mind as we talk about heavily templating druntime... (The performance difference at runtime was negligible, though I didn't dive into it much, since the performance of most the code was never under pressure; when things got slow, it was because of a few specific functions. My CSS thing was slow, so I cached it, problem solved. A few database queries were slow, so I fixed the SQL, problem solved. The gains from fixing those 10-line hotspots were always the major fixes we needed.)
Jan 04
parent Jacob Carlborg <doob me.com> writes:
On 2017-01-04 15:04, Adam D. Ruppe wrote:

 wc reports about 80,000 lines. But it isn't line count that affects
 build speed in D: it is the content of those lines.
Yeah, for projects like DWT and Tango where very few compile time features are used it's really quick to compile. DWT takes around 6 seconds for me to compile, just below 200 000 lines. -- /Jacob Carlborg
Jan 04
prev sibling next sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Mon, 02 Jan 2017 21:49:03 +0000, aberba wrote:

 I'm not building Facebook/pinterest but I'm trying to work on a platform
 like "pinterest-like" but on a small scale. I want it to be easy to
 write, fast, ... you know. D is obviously that (IMO).
 
 About scalability, would you recommend D(vibe.d initially) for long run
 (techically, generally, currently)? Why? (Brutal honesty).
I'd consider what languages future employees are likely to know and how much time it would take them to become proficient with D. I'd consider what dependencies I will likely have and whether there are existing D libraries for them. Then I'd put together a demo and see how it works. Use ab(1) (the Apache benchmark tool) to estimate scaling naively, and then implement a more application-specific benchmarking tool to see in more depth. I'd do that iteratively, so I'm not spending weeks on a demo that can't handle two simultaneous database connections or falls over if you look at it funny. I'm likely to want investors, so a functional demo is useful, even if I scrap it after seed funding or Series A. I'd do the same with any toolkit or language at that stage. For instance, I'm at an early-stage startup right now, and I spent a bit of effort comparing several different frameworks we could use for services and RPCs. This isn't so much an answer, but then, I don't know your usecase well enough, and I don't know what technologies you plan to use. For instance, you might want to use Hadoop for file storage. Or MongoDB + GridFS. Or MySQL with a limit on file sizes. Or a scale-out filesystem like Qumulo. These are quite different options with different levels of support in D.
Jan 02
parent reply aberba <karabutaworld gmail.com> writes:
On Tuesday, 3 January 2017 at 00:51:04 UTC, Chris Wright wrote:
 On Mon, 02 Jan 2017 21:49:03 +0000, aberba wrote:

 I'm not building Facebook/pinterest but I'm trying to work on 
 a platform like "pinterest-like" but on a small scale. I want 
 it to be easy to write, fast, ... you know. D is obviously 
 that (IMO).
 
 About scalability, would you recommend D(vibe.d initially) for 
 long run
 (techically, generally, currently)? Why? (Brutal honesty).
I'd consider what languages future employees are likely to know and how much time it would take them to become proficient with D.
Ha ha. That is a potential problem. But I will figure things out.
I'd consider what dependencies I will likely have and
 whether there are existing D libraries for them.
That's my fear. I'm much into using already baked stuff.
 a more application-specific benchmarking tool to see in more 
 depth.
Will look into that.
 I'm likely to want investors, so a functional demo is useful, 
 even if I scrap it after seed funding or Series A.

 I'd do the same with any toolkit or language at that stage. For 
 instance, I'm at an early-stage startup right now, and I spent 
 a bit of effort comparing several different frameworks we could 
 use for services and RPCs.
I'm currently planning on funding initially myself. And support in future with content promotion fees... mini mini ads :) I feel getting investors will limit my ability to craft it ATM.
 This isn't so much an answer, but then, I don't know your 
 usecase well enough, and I don't know what technologies you 
 plan to use. For instance, you might want to use Hadoop for 
 file storage. Or MongoDB + GridFS. Or MySQL with a limit on 
 file sizes. Or a scale-out filesystem like Qumulo. These are 
 quite different options with different levels of support in D.
Planning to use mysql and mongoDB(for the flexible part). I haven't research much about File Systems but I planning to use amazon s3. Basically I want to handle few burden as possible. Could use some advice though. It will be based my my country alone (local audience). But local hosting services are not reliable (down time issues).
Jan 03
next sibling parent Chris Wright <dhasenan gmail.com> writes:
On Tue, 03 Jan 2017 11:47:54 +0000, aberba wrote:
 Could use some advice though.
http://vibed.org/ is the framework you'll probably be using. It has mysql and mongodb support. http://code.dlang.org/ says there's an s3 client that works with vibe: http://code.dlang.org/packages/vibe-s3 It's got some scary warnings about its alpha status, so try it out before committing.
Jan 03
prev sibling parent aberba <karabutaworld gmail.com> writes:
On Tuesday, 3 January 2017 at 11:47:54 UTC, aberba wrote:
 On Tuesday, 3 January 2017 at 00:51:04 UTC, Chris Wright wrote:
 On Mon, 02 Jan 2017 21:49:03 +0000, aberba wrote:
 Planning to use mysql and mongoDB(for the flexible part). I 
 haven't research much about File Systems but I planning to use 
 amazon s3. Basically I want to handle few burden as possible.

 Could use some advice though. It will be based my my country 
 alone (local audience). But local hosting services are not 
 reliable (down time issues).
After thinking through things, I might go for Heroku with addons for various DBs to abstract load balancing and scalability. I way to administrate as few areas as possible.
Jan 05
prev sibling parent Mengu <mengukagan gmail.com> writes:
On Monday, 2 January 2017 at 21:49:03 UTC, aberba wrote:
 I'm not building Facebook/pinterest but I'm trying to work on a 
 platform like "pinterest-like" but on a small scale. I want it 
 to be easy to write, fast, ... you know. D is obviously that 
 (IMO).

 About scalability, would you recommend D(vibe.d initially) for 
 long run (techically, generally, currently)? Why? (Brutal 
 honesty).
i'd suggest the language that you know the best, the language that will not block your way and build barriers so you can focus on building your product. maybe later you can port it to D or build some new services in D.
Jan 03