www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DSource

reply Kyle Furlong <kylefurlong gmail.com> writes:
Okay, so we just got dug, and DSource has been down for more than a 
week. Does anyone else see the problem here? People will be turning away 
as I write this because the main repository of open source development 
in D has a "FATAL: connection limit exceeded for non-superusers" message 
on the front page.

While I appreciate the work done to create and maintain dsource.org, a 
week is not acceptable. I would be happy to help in any capacity to get 
even a mirror up asap.
Apr 25 2006
next sibling parent reply Walter Bright <newshound digitalmars.com> writes:
Kyle Furlong wrote:
 Okay, so we just got dug, and DSource has been down for more than a 
 week. Does anyone else see the problem here? People will be turning away 
 as I write this because the main repository of open source development 
 in D has a "FATAL: connection limit exceeded for non-superusers" message 
 on the front page.
 
 While I appreciate the work done to create and maintain dsource.org, a 
 week is not acceptable. I would be happy to help in any capacity to get 
 even a mirror up asap.

digitalmars.com saw roughly a 6 fold increase in traffic yesterday.
Apr 25 2006
parent reply Kyle Furlong <kylefurlong gmail.com> writes:
Walter Bright wrote:
 Kyle Furlong wrote:
 Okay, so we just got dug, and DSource has been down for more than a 
 week. Does anyone else see the problem here? People will be turning 
 away as I write this because the main repository of open source 
 development in D has a "FATAL: connection limit exceeded for 
 non-superusers" message on the front page.

 While I appreciate the work done to create and maintain dsource.org, a 
 week is not acceptable. I would be happy to help in any capacity to 
 get even a mirror up asap.

digitalmars.com saw roughly a 6 fold increase in traffic yesterday.

Thats exciting, but what does that have to do with the fact that dsource has been down for much longer than just yesterday? It would have been nice to have some of that increased traffic funneled to a working dsource. What we got is in essence a PR disaster.
Apr 25 2006
parent reply Lars Ivar Igesund <larsivar igesund.net> writes:
Kyle Furlong wrote:

 Walter Bright wrote:
 Kyle Furlong wrote:
 Okay, so we just got dug, and DSource has been down for more than a
 week. Does anyone else see the problem here? People will be turning
 away as I write this because the main repository of open source
 development in D has a "FATAL: connection limit exceeded for
 non-superusers" message on the front page.

 While I appreciate the work done to create and maintain dsource.org, a
 week is not acceptable. I would be happy to help in any capacity to
 get even a mirror up asap.

digitalmars.com saw roughly a 6 fold increase in traffic yesterday.

Thats exciting, but what does that have to do with the fact that dsource has been down for much longer than just yesterday? It would have been nice to have some of that increased traffic funneled to a working dsource. What we got is in essence a PR disaster.

The problems Dsource is having is not really traffic related (although high traffic don't help), it is a technical problem in Trac wherein it don't release it's db connections the way it should. You might help resolve the problem by finding out where in the Trac/Python code the problem is, I know Brad would be very happy to find it.
Apr 25 2006
next sibling parent reply =?ISO-8859-1?Q?Jari-Matti_M=E4kel=E4?= <jmjmak utu.fi.invalid> writes:
Lars Ivar Igesund wrote:

 The problems Dsource is having is not really traffic related (although high
 traffic don't help), it is a technical problem in Trac wherein it don't
 release it's db connections the way it should. You might help resolve the
 problem by finding out where in the Trac/Python code the problem is, I know
 Brad would be very happy to find it.

Has this already been fixed in the 0.10dev version of trac? -- Jari-Matti
Apr 25 2006
parent reply Lars Ivar Igesund <larsivar igesund.net> writes:
Jari-Matti Mäkelä wrote:

 Lars Ivar Igesund wrote:
 
 The problems Dsource is having is not really traffic related (although
 high traffic don't help), it is a technical problem in Trac wherein it
 don't release it's db connections the way it should. You might help
 resolve the problem by finding out where in the Trac/Python code the
 problem is, I know Brad would be very happy to find it.

Has this already been fixed in the 0.10dev version of trac?

AFAIK, DSource is one of very few Trac enabled sites using Postgres as it's DB backend. Apparently there are no similar problems with the default SQLite backend. Therefore it has been somewhat difficult getting those who know the code best to look at it.
Apr 25 2006
parent reply =?UTF-8?B?SmFyaS1NYXR0aSBNw6RrZWzDpA==?= <jmjmak utu.fi.invalid> writes:
Lars Ivar Igesund wrote:

 AFAIK, DSource is one of very few Trac enabled sites using Postgres as it's
 DB backend. Apparently there are no similar problems with the default
 SQLite backend. Therefore it has been somewhat difficult getting those who
 know the code best to look at it.

Sad to hear. Postgres might be more elegant, but I use sqlite myself for various reasons. It's very easy to dump the db, for example. Is there a good reason to use Postgres in the first place? -- Jari-Matti
Apr 25 2006
parent reply Brad Anderson <brad dsource.org> writes:
Jari-Matti Mäkelä wrote:
 Is there a
 good reason to use Postgres in the first place?
 

I was getting ready for Digg-type traffic, and I have seen Trac db's stored in SQLite lock during write (yes, the whole db locks). So I chose Postgres. I have heard of using one database and separating the Trac tables for a project into different schemas - the problem that was observed separate of dsource's actually went away doing this. This is a big task and I don't have a lot of time right now. A few weeks ago when I had some time, the problem wasn't as pronounced... Murphy. I have tried some settings in postgresql.conf to limit connection time-outs and they're not working yet. Plus, I didn't see nearly the traffic Walter talks about getting on his site, because the link to dsource is reasonably obscure. Some of that lack of traffic was the brief periods when we were down, but I would have seen hits in the logs, just with errors. Also, we weren't down for an entire week as the OP suggests. Apache resets itself, so we were down no longer than 2 hours at a time, and then were fine for most likely 12 hrs, until it happened again. BA
Apr 25 2006
next sibling parent Kyle Furlong <kylefurlong gmail.com> writes:
Brad Anderson wrote:
 Jari-Matti Mäkelä wrote:
 Is there a
 good reason to use Postgres in the first place?

I was getting ready for Digg-type traffic, and I have seen Trac db's stored in SQLite lock during write (yes, the whole db locks). So I chose Postgres. I have heard of using one database and separating the Trac tables for a project into different schemas - the problem that was observed separate of dsource's actually went away doing this. This is a big task and I don't have a lot of time right now. A few weeks ago when I had some time, the problem wasn't as pronounced... Murphy. I have tried some settings in postgresql.conf to limit connection time-outs and they're not working yet. Plus, I didn't see nearly the traffic Walter talks about getting on his site, because the link to dsource is reasonably obscure. Some of that lack of traffic was the brief periods when we were down, but I would have seen hits in the logs, just with errors. Also, we weren't down for an entire week as the OP suggests. Apache resets itself, so we were down no longer than 2 hours at a time, and then were fine for most likely 12 hrs, until it happened again. BA

I guess I got unlucky in that every time I looked in on the site in the past week it was down.
Apr 25 2006
prev sibling parent reply Walter Bright <newshound digitalmars.com> writes:
Brad Anderson wrote:
 Plus, I didn't see nearly the traffic Walter talks about getting on his site,
 because the link to dsource is reasonably obscure.

That's fixed now.
Apr 25 2006
parent Brad Anderson <brad dsource.org> writes:
Walter Bright wrote:
 Brad Anderson wrote:
 Plus, I didn't see nearly the traffic Walter talks about getting on
 his site,
 because the link to dsource is reasonably obscure.

That's fixed now.

sweet, thanks
Apr 26 2006
prev sibling parent Kyle Furlong <kylefurlong gmail.com> writes:
Lars Ivar Igesund wrote:
 Kyle Furlong wrote:
 
 Walter Bright wrote:
 Kyle Furlong wrote:
 Okay, so we just got dug, and DSource has been down for more than a
 week. Does anyone else see the problem here? People will be turning
 away as I write this because the main repository of open source
 development in D has a "FATAL: connection limit exceeded for
 non-superusers" message on the front page.

 While I appreciate the work done to create and maintain dsource.org, a
 week is not acceptable. I would be happy to help in any capacity to
 get even a mirror up asap.


has been down for much longer than just yesterday? It would have been nice to have some of that increased traffic funneled to a working dsource. What we got is in essence a PR disaster.

The problems Dsource is having is not really traffic related (although high traffic don't help), it is a technical problem in Trac wherein it don't release it's db connections the way it should. You might help resolve the problem by finding out where in the Trac/Python code the problem is, I know Brad would be very happy to find it.

I understand all this. I never thought that dsource was down due to the digg effect. I wish that had been the case! At least some of them would have seen the site. I was well aware of the connections issue and the length of down time which had been going on previous to the whole digg thing.
Apr 25 2006
prev sibling parent reply Hasan Aljudy <hasan.aljudy gmail.com> writes:
Kyle Furlong wrote:
 Okay, so we just got dug, and DSource has been down for more than a 
 week. Does anyone else see the problem here? People will be turning away 
 as I write this because the main repository of open source development 
 in D has a "FATAL: connection limit exceeded for non-superusers" message 
 on the front page.
 
 While I appreciate the work done to create and maintain dsource.org, a 
 week is not acceptable. I would be happy to help in any capacity to get 
 even a mirror up asap.

Has it been down for a week? I remember navigating dsource a few days ago and it was working!
Apr 25 2006
parent reply Lars Ivar Igesund <larsivar igesund.net> writes:
Hasan Aljudy wrote:

 Kyle Furlong wrote:
 Okay, so we just got dug, and DSource has been down for more than a
 week. Does anyone else see the problem here? People will be turning away
 as I write this because the main repository of open source development
 in D has a "FATAL: connection limit exceeded for non-superusers" message
 on the front page.
 
 While I appreciate the work done to create and maintain dsource.org, a
 week is not acceptable. I would be happy to help in any capacity to get
 even a mirror up asap.

Has it been down for a week? I remember navigating dsource a few days ago and it was working!

No, it hasn't been down at all, in the correct way of speech, but the Trac part of the site that use the underlying database, causes connection leaks that makes those pages inaccesible until they are cleaned up. At that point it also affects the forums. The connections also cause some thrashing, meaning that svn may or may not be somewhat accesible too, but in general, svn has been mostly unaffected.
Apr 25 2006
parent reply lightoze <lightoze_member pathlink.com> writes:
Lars Ivar Igesund says...
No, it hasn't been down at all, in the correct way of speech, but the Trac
part of the site that use the underlying database, causes connection leaks
that makes those pages inaccesible until they are cleaned up. At that point
it also affects the forums. The connections also cause some thrashing,
meaning that svn may or may not be somewhat accesible too, but in general,
svn has been mostly unaffected.

May be this can be temporary solved by setting low connection timeout...?
Apr 25 2006
parent reply clayasaurus <clayasaurus gmail.com> writes:
lightoze wrote:
 Lars Ivar Igesund says...
 No, it hasn't been down at all, in the correct way of speech, but the Trac
 part of the site that use the underlying database, causes connection leaks
 that makes those pages inaccesible until they are cleaned up. At that point
 it also affects the forums. The connections also cause some thrashing,
 meaning that svn may or may not be somewhat accesible too, but in general,
 svn has been mostly unaffected.

May be this can be temporary solved by setting low connection timeout...?

or use the SQLite backend?
Apr 25 2006
parent James Dunne <james.jdunne gmail.com> writes:
clayasaurus wrote:
 lightoze wrote:
 
 Lars Ivar Igesund says...

 No, it hasn't been down at all, in the correct way of speech, but the 
 Trac
 part of the site that use the underlying database, causes connection 
 leaks
 that makes those pages inaccesible until they are cleaned up. At that 
 point
 it also affects the forums. The connections also cause some thrashing,
 meaning that svn may or may not be somewhat accesible too, but in 
 general,
 svn has been mostly unaffected.

May be this can be temporary solved by setting low connection timeout...?

or use the SQLite backend?

In my experience, SQLite has huge problems with concurrency. I've tried developing for it and it's really difficult to get multithreaded database access code to work. -- -----BEGIN GEEK CODE BLOCK----- Version: 3.1 GCS/MU/S d-pu s:+ a-->? C++++$ UL+++ P--- L+++ !E W-- N++ o? K? w--- O M-- V? PS PE Y+ PGP- t+ 5 X+ !R tv-->!tv b- DI++(+) D++ G e++>e h>--->++ r+++ y+++ ------END GEEK CODE BLOCK------ James Dunne
Apr 26 2006