www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - mysql-native: API Refresh RC

reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
I've been working on a big refresh of mysql-native's API, to take care 
of various issues that have appeared with it. It involves some major 
breaking changes (although I've tried to keep old interfaces around for 
the moment, but marked deprecated), so I wanted to post it before 
committing to it so those interested have a change to take a look, give 
feedback, catch problems, etc.

Summary of these changes:

API overhauled for better safety, reliability and ease-of-use. 
Deprecated and replaced entire Command struct with better design. Better 
handling of null. Various bugs fixed and more rigorously tested.

------

For right now, the changes are in a separate fork, here:

https://github.com/Abscissa/mysql-native-experimental

The readme there has sample code and an overview of the new interface.

Changelog: 
https://github.com/Abscissa/mysql-native-experimental/blob/master/CHANGELOG.md

API ref: http://semitwist.com/mysql-native-docs/v0.2.0-preview1

-------

So take a look, let me know if there's any big issues with it. If all 
looks good, this will soon be released as mysql-native v0.2.0.
Jan 29
next sibling parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig+d outerproduct.org> writes:
Am 30.01.2017 um 03:56 schrieb Nick Sabalausky:
 (...)

 So take a look, let me know if there's any big issues with it. If all
 looks good, this will soon be released as mysql-native v0.2.0.
What about directly going for 1.0.0? At least after it has gotten enough real-world exposure, I'd say that the first API overhaul is a good opportunity for that.
Jan 29
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 01/30/2017 02:49 AM, Sönke Ludwig wrote:
 What about directly going for 1.0.0? At least after it has gotten enough
 real-world exposure, I'd say that the first API overhaul is a good
 opportunity for that.
Good point.
Jan 30
prev sibling next sibling parent reply Daniel Kozak via Digitalmars-d-announce writes:
Wow :) Maybe it is time to go back from mysql-lited to mysql-native :)


Dne 30.1.2017 v 03:56 Nick Sabalausky via Digitalmars-d-announce napsal(a):
 I've been working on a big refresh of mysql-native's API, to take care 
 of various issues that have appeared with it. It involves some major 
 breaking changes (although I've tried to keep old interfaces around 
 for the moment, but marked deprecated), so I wanted to post it before 
 committing to it so those interested have a change to take a look, 
 give feedback, catch problems, etc.

 Summary of these changes:

 API overhauled for better safety, reliability and ease-of-use. 
 Deprecated and replaced entire Command struct with better design. 
 Better handling of null. Various bugs fixed and more rigorously tested.

 ------

 For right now, the changes are in a separate fork, here:

 https://github.com/Abscissa/mysql-native-experimental

 The readme there has sample code and an overview of the new interface.

 Changelog: 
 https://github.com/Abscissa/mysql-native-experimental/blob/master/CHANGELOG.md

 API ref: http://semitwist.com/mysql-native-docs/v0.2.0-preview1

 -------

 So take a look, let me know if there's any big issues with it. If all 
 looks good, this will soon be released as mysql-native v0.2.0.
Jan 30
parent reply aberba <karabutaworld gmail.com> writes:
On Monday, 30 January 2017 at 08:09:18 UTC, Daniel Kozak wrote:
 Wow :) Maybe it is time to go back from mysql-lited to 
 mysql-native :)
I doubt that. Mysql-lited has support for using a struct as a Schema for DB queries. The as and optional properties are so useful for project I'm working on. Unless mysql-native has that.
Jan 30
parent Daniel Kozak via Digitalmars-d-announce writes:
Dne 30.1.2017 v 10:53 aberba via Digitalmars-d-announce napsal(a):

 On Monday, 30 January 2017 at 08:09:18 UTC, Daniel Kozak wrote:
 Wow :) Maybe it is time to go back from mysql-lited to mysql-native :)
I doubt that. Mysql-lited has support for using a struct as a Schema for DB queries. The as and optional properties are so useful for project I'm working on. Unless mysql-native has that.
I do not use any of these :)
Jan 30
prev sibling next sibling parent Andrea Fontana <nospam example.com> writes:
On Monday, 30 January 2017 at 02:56:27 UTC, Nick Sabalausky wrote:
 I've been working on a big refresh of mysql-native's API, to 
 take care of various issues that have appeared with it. It 
 involves some major breaking changes (although I've tried to 
 keep old interfaces around for the moment, but marked 
 deprecated), so I wanted to post it before committing to it so 
 those interested have a change to take a look, give feedback, 
 catch problems, etc.
Nice!
Jan 30
prev sibling next sibling parent reply Dejan Lekic <dejan.lekic gmail.com> writes:
On Monday, 30 January 2017 at 02:56:27 UTC, Nick Sabalausky wrote:
 I've been working on a big refresh of mysql-native's API, to 
 take care of various issues that have appeared with it. It 
 involves some major breaking changes (although I've tried to 
 keep old interfaces around for the moment, but marked 
 deprecated), so I wanted to post it before committing to it so 
 those interested have a change to take a look, give feedback, 
 catch problems, etc.
It is all really nice, but I think it would be nice to have an API that is as close to the C API as possible, for those developers out there familiar with the MySQL C API.
Jan 30
parent Daniel Kozak via Digitalmars-d-announce writes:
Dne 30.1.2017 v 10:14 Dejan Lekic via Digitalmars-d-announce napsal(a):

 On Monday, 30 January 2017 at 02:56:27 UTC, Nick Sabalausky wrote:
 I've been working on a big refresh of mysql-native's API, to take 
 care of various issues that have appeared with it. It involves some 
 major breaking changes (although I've tried to keep old interfaces 
 around for the moment, but marked deprecated), so I wanted to post it 
 before committing to it so those interested have a change to take a 
 look, give feedback, catch problems, etc.
It is all really nice, but I think it would be nice to have an API that is as close to the C API as possible, for those developers out there familiar with the MySQL C API.
No. Why? It does not make sense, almost nobody use C API directly. Everybody I know use some library build around C API.
Jan 30
prev sibling next sibling parent Gary Willoughby <dev nomad.so> writes:
On Monday, 30 January 2017 at 02:56:27 UTC, Nick Sabalausky wrote:
 I've been working on a big refresh of mysql-native's API, to 
 take care of various issues that have appeared with it. It 
 involves some major breaking changes (although I've tried to 
 keep old interfaces around for the moment, but marked 
 deprecated), so I wanted to post it before committing to it so 
 those interested have a change to take a look, give feedback, 
 catch problems, etc.

 [...]
Can we use semantic versioning correctly here and get an increment on the major version please. I'm so sick of seeing D libs in endless alpha/beta!
Jan 30
prev sibling next sibling parent reply Suliman <evermind live.ru> writes:
Plz update dub package on code.dlang.org
Feb 01
next sibling parent reply Suliman <evermind live.ru> writes:
Am I right understand that Connection instance should created at 
constructor and be one single for all class (and it will be 
reused by fibers) or am I wrong?

If yes, where I should to close it? To put
scope(exit) c.close();

In destructor?

If yes, does it behavior should be same as for working in vibed 
pool and without?
Feb 01
parent reply Suliman <evermind live.ru> writes:
On Wednesday, 1 February 2017 at 14:06:39 UTC, Suliman wrote:
 Am I right understand that Connection instance should created 
 at constructor and be one single for all class (and it will be 
 reused by fibers) or am I wrong?

 If yes, where I should to close it? To put
 scope(exit) c.close();

 In destructor?

 If yes, does it behavior should be same as for working in vibed 
 pool and without?
If I right understand class MysqlDB do not throw any exceptions, So I can't understand how to handle wrong connection. if(connection is null) { try // useless { connection = mydb.lockConnection(); } catch(Exception e) { writeln(e.msg); } }
Feb 01
next sibling parent reply Suliman <evermind live.ru> writes:
Also I can't understand what is SQL Command and what exec is 
doing if it's returning ulong?
Feb 01
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 02/01/2017 01:54 PM, Suliman wrote:
 Also I can't understand what is SQL Command and what exec is doing if
 it's returning ulong?
"struct Command" should not be used. It is old, and a bad design. This new release attempts to replace it with a better design. Hopefully, "struct Command" will be deleted in a later release. The "exec" functions are for commands like INSERT, UPDATE, DELETE, CREATE, etc (it is *not* for SELECT). It is for things that do NOT return actual rows of data. The "exec" functions return "rows affected" - the number of rows that were affected by your INSERT, or UPDATE, etc. Usually people ignore that number, but it's information the server sends back, and is sometimes useful to some people. For example, SQL administration tools usually tell you "# rows affected" after you run an INSERT/UPDATE/etc. If you are doing a SELECT, then you do NOT use "exec", you use "query" for SELECT. "query" returns a set of rows. Summary: --------- SELECT: Use query() or querySet() or queryRow(), etc. INSERT/UPDATE/DELETE/CREATE/DROP: Use exec(). Return value tells you how many rows were added/changed/deleted/etc.
Feb 01
parent Suliman <evermind live.ru> writes:
On Thursday, 2 February 2017 at 04:04:15 UTC, Nick Sabalausky 
wrote:
 On 02/01/2017 01:54 PM, Suliman wrote:
 Also I can't understand what is SQL Command and what exec is 
 doing if
 it's returning ulong?
"struct Command" should not be used. It is old, and a bad design. This new release attempts to replace it with a better design. Hopefully, "struct Command" will be deleted in a later release. The "exec" functions are for commands like INSERT, UPDATE, DELETE, CREATE, etc (it is *not* for SELECT). It is for things that do NOT return actual rows of data. The "exec" functions return "rows affected" - the number of rows that were affected by your INSERT, or UPDATE, etc. Usually people ignore that number, but it's information the server sends back, and is sometimes useful to some people. For example, SQL administration tools usually tell you "# rows affected" after you run an INSERT/UPDATE/etc. If you are doing a SELECT, then you do NOT use "exec", you use "query" for SELECT. "query" returns a set of rows. Summary: --------- SELECT: Use query() or querySet() or queryRow(), etc. INSERT/UPDATE/DELETE/CREATE/DROP: Use exec(). Return value tells you how many rows were added/changed/deleted/etc.
Plz add this mention to readme.
Feb 02
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 02/01/2017 10:34 AM, Suliman wrote:
 On Wednesday, 1 February 2017 at 14:06:39 UTC, Suliman wrote:
 Am I right understand that Connection instance should created at
 constructor and be one single for all class (and it will be reused by
 fibers) or am I wrong?

 If yes, where I should to close it? To put
 scope(exit) c.close();

 In destructor?

 If yes, does it behavior should be same as for working in vibed pool
 and without?
I'm not sure I understand the question here. Whether you should use a vibed connection pool or not is completely up to you and your program. And the choice of when to create the connection, and when to close the connection, is also up to your own program's needs. If you're using a connection pool, you shouldn't need to worry about closing the connection. The whole point is that the connections stay open until you need to use one again. When your program ends, then connections will close by themselves.
 If I right understand class MysqlDB do not throw any exceptions, So I
 can't understand how to handle wrong connection.

 if(connection is null)
 {
      try // useless
      {
          connection = mydb.lockConnection();
      }
      catch(Exception e)
      {
          writeln(e.msg);
      }
 }
mydb.lockConnection() does create a new connection if it needs to. And that WILL throw an exception if there's a problem connecting to the DB server. So your code above WILL catch an exception if the connection information (server address/port/login/etc) is wrong. Side note: That reminds me, I need to rename the MysqlDB class (and the mysql.db module). It's a connection pool (using vibe-d's connection pool), but their names make that very unclear.
Feb 01
parent reply Suliman <evermind live.ru> writes:
mydb.lockConnection() does create a new connection if it needs 
to. And that WILL throw an exception if there's a problem 
connecting to the DB server. So your code above WILL catch an 
exception if the connection information (server 
address/port/login/etc) is wrong.
But it does not. I am getting Access Violation instead of the exception if connection credentials is wrong: Authentication failure: Access denied for user 'root' '111.111.111.111' (using password: YES) object.Error (0): Access Violation ---------------- 0x004436C0 in void database.Database.connect() at D:\code\CMS\source\database.d(34) 0x00403130 in _Dmain at D:\code\CMS\source\app.d(17) 0x00593C6F in D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv 0x00593C33 in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll() 0x00593B34 in _d_run_main 0x004433CC in main at D:\code\CMS\source\app.d(7) 0x005F0929 in mainCRTStartup 0x769262C4 in BaseThreadInitThunk 0x774D0FD9 in RtlSubscribeWnfStateChangeNotification 0x774D0FA4 in RtlSubscribeWnfStateChangeNotification Program exited with code 1
Feb 02
next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 02/02/2017 03:23 AM, Suliman wrote:
 mydb.lockConnection() does create a new connection if it needs to. And
 that WILL throw an exception if there's a problem connecting to the DB
 server. So your code above WILL catch an exception if the connection
 information (server address/port/login/etc) is wrong.
But it does not. I am getting Access Violation instead of the exception if connection credentials is wrong: Authentication failure: Access denied for user 'root' '111.111.111.111' (using password: YES) object.Error (0): Access Violation ---------------- 0x004436C0 in void database.Database.connect() at D:\code\CMS\source\database.d(34) 0x00403130 in _Dmain at D:\code\CMS\source\app.d(17) 0x00593C6F in D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv 0x00593C33 in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll() 0x00593B34 in _d_run_main 0x004433CC in main at D:\code\CMS\source\app.d(7) 0x005F0929 in mainCRTStartup 0x769262C4 in BaseThreadInitThunk 0x774D0FD9 in RtlSubscribeWnfStateChangeNotification 0x774D0FA4 in RtlSubscribeWnfStateChangeNotification Program exited with code 1
Ahh, ok, that's a bug. Will fix.
Feb 02
next sibling parent reply Suliman <evermind live.ru> writes:
ResultSet querySet(Connection conn, string sql, 
ColumnSpecialization[] csa = null)

Could you explain last parameter?

`ColumnSpecialization[] csa = null`. I can't understand how to 
use it.

Could you give me your skype?

Also I think it's better to remove old deprecated methods at all, 
because it's add only mess.
Feb 02
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 02/02/2017 04:33 AM, Suliman wrote:
 ResultSet querySet(Connection conn, string sql, ColumnSpecialization[]
 csa = null)

 Could you explain last parameter?

 `ColumnSpecialization[] csa = null`. I can't understand how to use it.
The vast majority of the time, you don't need to worry about that parameter, just omit it. It's just for if the fields you're pulling from the DB are very, very large and you want to handle the data as it comes in, rather than waiting for all the data to download. If you want to know more, it's in the documentation here: http://semitwist.com/mysql-native-docs/v0.2.0-preview1/mysql/protocol/extra_types/ColumnSpecialization.html
 Also I think it's better to remove old deprecated methods at all,
 because it's add only mess.
They're still there for new just for backwards compatibility. But they are deprecated and will be removed in a later release.
Feb 02
prev sibling parent reply aberba <karabutaworld gmail.com> writes:
On Thursday, 2 February 2017 at 08:38:32 UTC, Nick Sabalausky 
wrote:
 On 02/02/2017 03:23 AM, Suliman wrote:
 [...]
But it does not. I am getting Access Violation instead of the exception if connection credentials is wrong: Authentication failure: Access denied for user 'root' '111.111.111.111' (using password: YES) object.Error (0): Access Violation ---------------- 0x004436C0 in void database.Database.connect() at D:\code\CMS\source\database.d(34) 0x00403130 in _Dmain at D:\code\CMS\source\app.d(17) 0x00593C6F in D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv 0x00593C33 in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll() 0x00593B34 in _d_run_main 0x004433CC in main at D:\code\CMS\source\app.d(7) 0x005F0929 in mainCRTStartup 0x769262C4 in BaseThreadInitThunk 0x774D0FD9 in RtlSubscribeWnfStateChangeNotification 0x774D0FA4 in RtlSubscribeWnfStateChangeNotification Program exited with code 1
Ahh, ok, that's a bug. Will fix.
* Moreover, how do I close a connection or does it auto close? * Does it support mysql_real_escape_string() like in php? This factor-in the db encoding to do he appriate encoding for '/\" ...
Feb 02
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 02/02/2017 07:10 AM, aberba wrote:
 * Moreover, how do I close a connection or does it auto close?
It closes in its destructor (although AIUI there are times when dtors don't get run). But it can be closed manually with Connection.close();
 * Does it support mysql_real_escape_string() like in php? This factor-in
 the db encoding to do he appriate encoding for '/\" ...
There is a function like that one that was contributed, mysql.escape.mysql_escape: https://github.com/Abscissa/mysql-native-experimental/blob/master/source/mysql/escape.d#L4 But, it's better to use prepared statements, because then no escaping is needed, and there's no worry about forgetting to escape: Prepared prepared = prepare(connection, "SELECT * FROM `someTable` WHERE i=? AND s=?"); prepared.setArgs(7, "hello"); ResultRange results = prepared.query(); The prepared statement is reference counted and will automatically be released when there are no references to it left.
Feb 02
prev sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 02/02/2017 03:23 AM, Suliman wrote:
 mydb.lockConnection() does create a new connection if it needs to. And
 that WILL throw an exception if there's a problem connecting to the DB
 server. So your code above WILL catch an exception if the connection
 information (server address/port/login/etc) is wrong.
But it does not. I am getting Access Violation instead of the exception if connection credentials is wrong: Authentication failure: Access denied for user 'root' '111.111.111.111' (using password: YES) object.Error (0): Access Violation ---------------- 0x004436C0 in void database.Database.connect() at D:\code\CMS\source\database.d(34) 0x00403130 in _Dmain at D:\code\CMS\source\app.d(17) 0x00593C6F in D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv 0x00593C33 in void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll() 0x00593B34 in _d_run_main 0x004433CC in main at D:\code\CMS\source\app.d(7) 0x005F0929 in mainCRTStartup 0x769262C4 in BaseThreadInitThunk 0x774D0FD9 in RtlSubscribeWnfStateChangeNotification 0x774D0FA4 in RtlSubscribeWnfStateChangeNotification Program exited with code 1
I'm unable to reproduce this problem. Are you trying to use the connection after it fails to connect? It looks like that's what's happening, you're using a null reference.
Feb 02
prev sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 02/01/2017 05:04 AM, Suliman wrote:
 Plz update dub package on code.dlang.org
It's deliberately not in the main mysql-mative package just yet. I will put it there once people have a chance to try this, and it becomes clear there aren't any big problems with the redesign. For now, this preview can be used like this: $ git clone https://github.com/Abscissa/mysql-native-experimental.git $ dub add-local mysql-native-experimental And then, when you're doing using this preview: $ dub remove-local mysql-native-experimental
Feb 01
prev sibling next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
Made a couple more long-needed changes while I'm at it:

https://github.com/Abscissa/mysql-native-experimental
Tag: v0.2.0-preview2

- For better clarity, renamed `mysql.db.MysqlDB` to `mysql.pool.MySqlPool`.

- Package mysql.connection no longer acts as a package.d, publicly 
importing other modules. To import all of mysql-native, use `import mysql;`
Feb 01
parent reply Suliman <evermind live.ru> writes:
On Thursday, 2 February 2017 at 05:28:10 UTC, Nick Sabalausky 
wrote:
 Made a couple more long-needed changes while I'm at it:

 https://github.com/Abscissa/mysql-native-experimental
 Tag: v0.2.0-preview2

 - For better clarity, renamed `mysql.db.MysqlDB` to 
 `mysql.pool.MySqlPool`.

 - Package mysql.connection no longer acts as a package.d, 
 publicly importing other modules. To import all of 
 mysql-native, use `import mysql;`
Thanks! Could you explain about pool. I googled about it, and still can't understand when it up new connections? How can I imagine what connection is? Because without it hard to understand what difference between connections with and without pool. Am I right understand that if I use pool I can create connection instance one time in DB class constructor end every new connection will be created on demand?
Feb 01
next sibling parent reply Suliman <evermind live.ru> writes:
On Thursday, 2 February 2017 at 06:50:34 UTC, Suliman wrote:
 On Thursday, 2 February 2017 at 05:28:10 UTC, Nick Sabalausky 
 wrote:
 Made a couple more long-needed changes while I'm at it:

 https://github.com/Abscissa/mysql-native-experimental
 Tag: v0.2.0-preview2

 - For better clarity, renamed `mysql.db.MysqlDB` to 
 `mysql.pool.MySqlPool`.

 - Package mysql.connection no longer acts as a package.d, 
 publicly importing other modules. To import all of 
 mysql-native, use `import mysql;`
Thanks! Could you explain about pool. I googled about it, and still can't understand when it up new connections? How can I imagine what connection is? Because without it hard to understand what difference between connections with and without pool. Am I right understand that if I use pool I can create connection instance one time in DB class constructor end every new connection will be created on demand?
Ok, I read articles about pool, as I understood it's depend on of the implementation. For example method `close` in pool mode should not close connection, but return it to pool. Could you tell about your implementation. Also actual question is can I open connection in constructor (during class instance creation) ?
Feb 01
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 02/02/2017 02:22 AM, Suliman wrote:
 Am I right understand that if I use pool I can create connection
 instance one time in DB class constructor end every new connection
 will be created on demand?
No. You create the pool once (wherever/whenever you want to). Then, every time you want to use the database you obtain a connection by calling MySqlPool.lockConnection.
 Ok, I read articles about pool, as I understood it's depend on of the
 implementation. For example method `close` in pool mode should not close
 connection, but return it to pool.
Calling 'close' will always close the connection. If you got you connection from the pool, then it will automatically return to the pool when you're no longer using it. No need to do anything special for that.
 Could you tell about your implementation. Also actual question is can I
 open connection in constructor (during class instance creation) ?
You can open a connection in a constructor if you wish. Or wherever you want to.
Feb 02
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 2/2/17 1:50 AM, Suliman wrote:
 On Thursday, 2 February 2017 at 05:28:10 UTC, Nick Sabalausky wrote:
 Made a couple more long-needed changes while I'm at it:

 https://github.com/Abscissa/mysql-native-experimental
 Tag: v0.2.0-preview2

 - For better clarity, renamed `mysql.db.MysqlDB` to
 `mysql.pool.MySqlPool`.

 - Package mysql.connection no longer acts as a package.d, publicly
 importing other modules. To import all of mysql-native, use `import
 mysql;`
Thanks! Could you explain about pool. I googled about it, and still can't understand when it up new connections? How can I imagine what connection is? Because without it hard to understand what difference between connections with and without pool. Am I right understand that if I use pool I can create connection instance one time in DB class constructor end every new connection will be created on demand?
Just to answer some of this, because I had questions about the pool as well. The ConnectionPool is a mechanism to allow a limited resource (or a resource you want to limit, depending how you look at it) to be pooled for use. It's basically a free-list. It also abstracts away the details of opening a connection (i.e. server ip, username, password). It's nice because it makes opening a connection cleaner and straightforward. The resulting connection can only be used in one fiber at a time, and the connection pool itself can only be used in one thread ever (there is no sharing of connection pools between threads). The resulting connection struct is reference counted, so it automatically releases back to the pool when it goes out of scope. One issue I had with MysqlDB (now MySqlPool) is that it doesn't allow you to actually put a limit on the connections. In other words, you can't say "only allow 50 simultaneous connections". The effect is that you save the connection initialization, but you can't put a hard cap on connections used in your thread. If you have 10,000 fibers running at once, then you may get 10,000 connections to the database that are left open. If each of those is a socket, you are consuming a lot of resources during down times. Not to mention that the allocation of the 10,000th connection has to first go on a linear search through the 9,999 other connections to find an open one (this seems like it could be solved better with a linked-list freelist?). See here: https://github.com/mysql-d/mysql-native/issues/74 BTW, I solved this by copying MysqlDB into my local project and adding the appropriate parameter to the constructor :) -Steve
Feb 02
parent reply Suliman <evermind live.ru> writes:
On Thursday, 2 February 2017 at 14:08:19 UTC, Steven 
Schveighoffer wrote:
 On 2/2/17 1:50 AM, Suliman wrote:
 On Thursday, 2 February 2017 at 05:28:10 UTC, Nick Sabalausky 
 wrote:
 Made a couple more long-needed changes while I'm at it:

 https://github.com/Abscissa/mysql-native-experimental
 Tag: v0.2.0-preview2

 - For better clarity, renamed `mysql.db.MysqlDB` to
 `mysql.pool.MySqlPool`.

 - Package mysql.connection no longer acts as a package.d, 
 publicly
 importing other modules. To import all of mysql-native, use 
 `import
 mysql;`
Thanks! Could you explain about pool. I googled about it, and still can't understand when it up new connections? How can I imagine what connection is? Because without it hard to understand what difference between connections with and without pool. Am I right understand that if I use pool I can create connection instance one time in DB class constructor end every new connection will be created on demand?
Just to answer some of this, because I had questions about the pool as well. The ConnectionPool is a mechanism to allow a limited resource (or a resource you want to limit, depending how you look at it) to be pooled for use. It's basically a free-list. It also abstracts away the details of opening a connection (i.e. server ip, username, password). It's nice because it makes opening a connection cleaner and straightforward. The resulting connection can only be used in one fiber at a time, and the connection pool itself can only be used in one thread ever (there is no sharing of connection pools between threads). The resulting connection struct is reference counted, so it automatically releases back to the pool when it goes out of scope. One issue I had with MysqlDB (now MySqlPool) is that it doesn't allow you to actually put a limit on the connections. In other words, you can't say "only allow 50 simultaneous connections". The effect is that you save the connection initialization, but you can't put a hard cap on connections used in your thread. If you have 10,000 fibers running at once, then you may get 10,000 connections to the database that are left open. If each of those is a socket, you are consuming a lot of resources during down times. Not to mention that the allocation of the 10,000th connection has to first go on a linear search through the 9,999 other connections to find an open one (this seems like it could be solved better with a linked-list freelist?). See here: https://github.com/mysql-d/mysql-native/issues/74 BTW, I solved this by copying MysqlDB into my local project and adding the appropriate parameter to the constructor :) -Steve
Do you mean that every new fiber open new connection? And when fiber is yield it's return connection to the pool? Could you explain real case if rangification of ResultSet http://semitwist.com/mysql-native-docs/v0.2.0-preview1/mysql/result/ResultSet.html Does it's mean that I can write foreach(x;result.empty) ? Or how to use it?
Feb 02
parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 02/02/2017 09:46 AM, Suliman wrote:
 Could you explain real case if rangification of ResultSet

 http://semitwist.com/mysql-native-docs/v0.2.0-preview1/mysql/result/ResultSet.html


 Does it's mean that I can write
 foreach(x;result.empty) ? Or how to use it?
.empty just checks whether the range is empty, it returns true/false. You can't iterate over that. But yes, you can iterate over the ResultSet itself. Of course, ResultSet is random-access, so you can also index it like an array: "resultset[0]" returns the first row.
Feb 02
parent Suliman <evermind live.ru> writes:
On Thursday, 2 February 2017 at 22:59:38 UTC, Nick Sabalausky 
wrote:
 On 02/02/2017 09:46 AM, Suliman wrote:
 Could you explain real case if rangification of ResultSet

 http://semitwist.com/mysql-native-docs/v0.2.0-preview1/mysql/result/ResultSet.html


 Does it's mean that I can write
 foreach(x;result.empty) ? Or how to use it?
.empty just checks whether the range is empty, it returns true/false. You can't iterate over that. But yes, you can iterate over the ResultSet itself. Of course, ResultSet is random-access, so you can also index it like an array: "resultset[0]" returns the first row.
But how by looking at docs I can understand what data can be iterable, and what do not?
Feb 02
prev sibling parent Chris Wright <dhasenan gmail.com> writes:
On Thu, 02 Feb 2017 06:50:34 +0000, Suliman wrote:

 On Thursday, 2 February 2017 at 05:28:10 UTC, Nick Sabalausky wrote:
 Made a couple more long-needed changes while I'm at it:

 https://github.com/Abscissa/mysql-native-experimental Tag:
 v0.2.0-preview2

 - For better clarity, renamed `mysql.db.MysqlDB` to
 `mysql.pool.MySqlPool`.

 - Package mysql.connection no longer acts as a package.d,
 publicly importing other modules. To import all of mysql-native, use
 `import mysql;`
Thanks! Could you explain about pool. I googled about it, and still can't understand when it up new connections?
A database connection is just a socket. Problem 1: database connections are expensive to set up. (Network connections in general are expensive -- in my testing, an AWS EC2 t2.micro instance can handle ~80 connections per second. I was primarily testing RPC / REST API systems, which is a fair bit more than 'open a socket, send a byte, receive a byte, close the socket'.) Problem 2: database connections only offer one stream of data each direction. Problem 3: my application needs to handle a bunch of requests that might occur at the same time, each of which talks to the database. Solution: we'll have an object that manages a set of connections. It can reuse a connection, and it guarantees a connection is not reused when a previous user is still using it. It might automatically close a connection after enough time without that connection being used. It will automatically open a new connection when it has no free connections to hand out.
Feb 02
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
https://github.com/Abscissa/mysql-native-experimental
Tag: v0.2.0-preview3

Just a few doc updates this time:

- Docs now include the `mysql.db.MysqlDB` to `mysql.pool.MySqlPool` 
change from preview2
- Clarified "Prepared" vs "PreparedImpl"
- Clarified "exec" vs "query"
- Rewrite the docs for ResultSet and ResultRange.

New docs:
http://semitwist.com/mysql-native-docs/v0.2.0-preview3
Feb 06
next sibling parent Suliman <evermind live.ru> writes:
On Monday, 6 February 2017 at 20:32:24 UTC, Nick Sabalausky wrote:
 https://github.com/Abscissa/mysql-native-experimental
 Tag: v0.2.0-preview3

 Just a few doc updates this time:

 - Docs now include the `mysql.db.MysqlDB` to 
 `mysql.pool.MySqlPool` change from preview2
 - Clarified "Prepared" vs "PreparedImpl"
 - Clarified "exec" vs "query"
 - Rewrite the docs for ResultSet and ResultRange.

 New docs:
 http://semitwist.com/mysql-native-docs/v0.2.0-preview3
Big thanks! I am very glad that driver becoming better! I have question about DB connection pattern if you could give any advices I would be very glad http://stackoverflow.com/questions/42083976/database-connection-pattern
Feb 06
prev sibling parent Suliman <evermind live.ru> writes:
 Nick, could you explain how connection working if I am close it 
after opening and only then do request to DB. See comments on SO.
Feb 07