Wt

As a workaround, I tried to set a global mutex on this part of code in the WResource.

Now I have a 'Wt: Exception while streaming resourceOperation requires an active transaction' that shows up after about 15 resources fetch.

Ok, let's focus on the first issue.

Here is the full source code:

- Resource: https://github.com/epoupon/lms/blob/master/src/ui/resource/CoverResource.cpp

- Database::Track::getById here : https://github.com/epoupon/lms/blob/master/src/database/Track.cpp#L163

• CoverArt::Grabber::getFromTrack : https://github.com/epoupon/lms/blob/master/src/cover/CoverArtGrabber.cpp#L76

Doing this in my resource code seems to solve the problem (dunno yet why it didn't work the other day...)

{
   std::unique_lock<std::mutex> lock(_mutex);
   Wt::Dbo::Transaction transaction(_db.getSession());
   Database::Track::pointer track = Database::Track::getById(_db.getSession(), trackId);
   covers = CoverArt::Grabber::getFromTrack(track);
}

Hey,

There is still a risk that you are allowing concurrent request to a particular 'covers', which may be dangerous if this is a Wt::Dbo::ptr, or contains Wt::Dbo::ptr's. Since these ptr's lazy load, or may lazy-load associated objects, you still have the risk of concurrency issues.

Koen

Ok, here is a reduced test case of my custom resource :

                {
                        static std::size_t i = 0;
                        LMS_LOG(MOD_UI, SEV_DEBUG) << "Resource request " << i++;

                        Wt::Dbo::Transaction transaction(_db.getSession());

                        Database::Track::pointer track = Database::Track::getById(_db.getSession(), trackId);
                        sleep(1);
                }

In the Track class:

typedef Wt::Dbo::ptr<Track> pointer;

Track::pointer
Track::getById(Wt::Dbo::Session& session, id_type id) {
   return session.find<Track>().where("id = ?").bind(id);
}

As you can see, I don't use the track's contents.

Output:

10.0.1.100 - - [2015-Mar-11 19:09:54.400105] "POST /?wtd=bFkLLzSE3ubYY9X0 HTTP/1.1" 200 1528
[2015-Mar-11 19:09:54.400155] 7202 - [info] "WebRequest: took 9.31ms"
[2015-03-11 19:09:54] [UI] [DEBUG] Resource request 0
[2015-03-11 19:09:54] [UI] [DEBUG] Resource request 1
[2015-03-11 19:09:54] [UI] [DEBUG] Resource request 2
[2015-03-11 19:09:54] [UI] [DEBUG] Resource request 3
[2015-03-11 19:09:54] [UI] [DEBUG] Resource request 4
[2015-Mar-11 19:09:54.426496] 7202 [/ bFkLLzSE3ubYY9X0] [error] "Wt: Exception while streaming resourceA collection for 'select ""id"", ""version"", ""track_number"", ""disc_number"", ""name"", ""artist_name"", ""release_name"", ""duration"", ""date"", ""original_date"", ""genre_list"", ""path"", ""last_write"", ""checksum"", ""has_cover"" from ""track""  where (id = ?)' is already in use. Reentrant statement use is not yet implemented."
[2015-Mar-11 19:09:54.426626] 7202 [/ bFkLLzSE3ubYY9X0] [error] "Wt: fatal error: A collection for 'select ""id"", ""version"", ""track_number"", ""disc_number"", ""name"", ""artist_name"", ""release_name"", ""duration"", ""date"", ""original_date"", ""genre_list"", ""path"", ""last_write"", ""checksum"", ""has_cover"" from ""track""  where (id = ?)' is already in use. Reentrant statement use is not yet implemented."
[2015-Mar-11 19:09:54.426714] 7202 - [info] "WebController: Removing session bFkLLzSE3ubYY9X0"

I don't understand this behavior since I am not iterating over a Wt::Dbo::collection nor using lazy loading?

But you now removed the lock again?

Yes indeed: I tried to reproduce a simpler test case in order for you to see what I am doing and why I think there is something wrong.

Just have a transaction that is used to get some unused data from the database and a sleep.

After a very few calls, I get the error.

I do not use collections nor lazy loadings. Everything is done in the transaction scope.

A Wt::Dbo::Session is not thread-safe. So it will not work from a resource like this, without a mutex. The error you get is predictable since two threads will now simultaneously try to use the same connection (and that won't work at all --- that's not something that can be fixed: database connections are single threaded by nature).

Koen

Ok, since upgrading Wt made the bug appears I missed that point!

Just to be sure: each WApplication (session) will be run by at most one thread?

I mean, the pool of threads is just used to make several sessions run simultaneously?

Then binding a Wt::Dbo::Session to the private data of a Wt::Wapplication is safe?

A WApplication (and its widget tree) is protected by the WApplication::UpdateLock. Before Wt handles a browser event, it will grab the updatelock, so indeed access to a single WApplication is exclusive even if a Wt application typically spawns multiple threads. The threads are indeed mainly used to handle requests for different sessions simultaneously.

WResources are special, in the sense that they are objects that are served by the http server. Many http connections can be querying the same WResource simultaneously. The handleRequest method of a WResource is called by the httpd without grabbing the WApplication UpdateLock. It is thus not safe to use data from the WResource from handleRequest, and it is not safe neither to use data from WApplication in handleRequest. State data related to one particular http request is intended to be stored in Http::Request::continuation().

So 'yes', binding a Dbo::Session to WApplication is safe in the sense that if you're allowed to access the WApplication object, you will have exclusive access. But it is not safe to use WApplication from handleRequest (since the update lock will not be held).

Wim.