Project

General

Profile

Actions

Support #1673

closed

Postgres connection pool in multi-threaded server

Added by Jarek Honkisz about 11 years ago. Updated about 9 years ago.

Status:
Closed
Priority:
High
Assignee:
Target version:
-
Start date:
02/05/2013
Due date:
% Done:

0%

Estimated time:

Description

Hi

*I'm using WT3.2.3, boost 1.52 and postgres db(8.4) - all in multi-threaded server.

Till now I used:*

dbo::backend:Postgres connection(Singleton::Instance().database());

session->setConnection(connection);

*for each session for connect to the database, but now I want to use connection pool.

So I prepared some classes:*

class DboFixture

{

public:

DboFixture(){}

void initializeConnectionPool() {

dbo::SqlConnection* connection = new dbo::backend::Postgres(Singleton::Instance().database());

connectionPool_ = new dbo::FixedSqlConnectionPool(connection, Singleton::Instance().postgresPool());

};

~DboFixture() {

delete connectionPool_;

};

dbo::SqlConnectionPool& getConnectionPool(){

return *connectionPool_;

}

private:

dbo::SqlConnectionPool *connectionPool_;

};

  • and main.cpp looks like:*

DboFixture pgConnectionPool;

class SongWWWApplication : public WApplication

{

public:

SongWWWApplication(const WEnvironment& env)

: WApplication(env)

{

root()->addWidget(new MainView("/"));

#ifdef ENABLE_STATSD_MONITORING

Monitor_Gauge("server","sessions",sessionNumber());

#endif

}

~SongWWWApplication()

{

#ifdef ENABLE_STATSD_MONITORING

Monitor_Gauge("server","sessions",sessionNumber());

#endif

}

};

WApplication* createApplication(const WEnvironment& env)

{

return new SongWWWApplication(env);

}

int main(int argc, char argv) {

std::string settings_file;

try {

WServer server(argv[0]);

server.setServerConfiguration(argc, argv, WTHTTP_CONFIGURATION);

Singleton::Instance().readConfiguration("settings.xml");

pgConnectionPool.initializeConnectionPool();

server.addEntryPoint(Application, createApplication, DefaultEntryPointStr);

if (server.start()) {

int sig = WServer::waitForShutdown(argv[0]);

std::cerr << "Shutdown (signal = " << sig << ")" << std::endl;

server.stop();

if (sig == SIGHUP) {

std::cerr << "\nReread configuration: settings.xml\n";

WServer::restart(argc, argv, environ);

}

}

}

catch (WServer::Exception& e) {

std::cerr << e.what() << "\n";

return 1;

}

catch (std::exception& e) {

std::cerr << "exception: " << e.what() << "\n";

return 1;

}

return 0;

}

MainSession is member of MainView class:

class MainSession : public dbo::Session

{

public:

MainSession(dbo::SqlConnectionPool& connectionPool);

void setUser(dbo::ptr user);

dbo::ptr user() const { return user_; }

private:

dbo::ptr user_;

};

MainSession::MainSession(dbo::SqlConnectionPool& connectionPool)

{

setConnectionPool(connectionPool);

mapClass("user");

...

};

and is initialized like:

extern DboFixture pgConnectionPool;

MainView::MainView(const std::string& basePath)

: m_base_path(basePath), m_session(pgConnectionPool.getConnectionPool())

{

...

};

I tested my web app with "https://strucr.com" crawler (4-12 threads with no delay between request) and with old connection to db everythink was ok, but with connection pool I had very offten deadlock (server not responding) or segmentation false.

deadlock:

threads - http://screencast.com/t/DkYhNcLr6TW

thread (_lll_lock_wait) - http://screencast.com/t/kx3HOoGy

rest threads (*Gl_pool) looks similar - http://screencast.com/t/mVk3Plr7Y

*What is the best way to initialize each session with connection pool?

If needed I can give more information.*

*best regrads

Musicplayr*

PS. I'm sorry for duplicated http://redmine.webtoolkit.eu/issues/1672

Actions #1

Updated by Koen Deforche about 11 years ago

  • Status changed from New to InProgress
  • Target version set to 3.3.0
Actions #2

Updated by Koen Deforche about 11 years ago

  • Status changed from InProgress to Feedback

Hey,

We aren't able to reproduce this problem. The Connection pool implementation is also fairly straight forward so I can't spot any obvious problem with it. What I did do is improve the Transaction.C handling so that we are certain that we are not leaking connections are returning the same connection twice to the connection pool. However, if this change gives another problem then perhaps this is concealing a more serious defect in Dbo Sessions and Transactions (related to connection pools).

One thing that I researched is the thread-safety of libpq. There should not be a problem with the assumptions made by Wt: that you can concurrently use different connections but that each connection should be used by only one thread at a time. However, I did read about compile flags for building libpq itself that enable it to be reentrant. You can check whether PQisthreadsafe() returns 1 ?

Another obvious thing to verify is that Wt is built with multi-threading support, which is stated at configuration time of Wt ( Enabling multi threading).

Regards,

koen

Actions #3

Updated by Koen Deforche about 11 years ago

  • Target version changed from 3.3.0 to 3.3.1
Actions #4

Updated by Koen Deforche over 10 years ago

Hey,

Any feedback on this issue? Did you find a solution, did it resolve by itself or is this still an issues as far as you know?

koen

Actions #5

Updated by Koen Deforche over 10 years ago

  • Target version changed from 3.3.1 to 3.3.2
Actions #6

Updated by Koen Deforche about 10 years ago

  • Target version deleted (3.3.2)
Actions #7

Updated by Koen Deforche about 9 years ago

  • Status changed from Feedback to Closed
Actions

Also available in: Atom PDF