Project

General

Profile

NKHow to properly utilize the WLogSink class?

Added by Nikita Kornilov 23 days ago

My particular goal is to rotate the logs produced by Wt. The current temporary solution is to rely on systemd and logrotate as suggested here. But nonetheless I would like to use something less elaborate.

The WLogSink seems to be just the right class for the job. The spdlog library appears to handle the log rotation in just three lines of code and has a header-only version. Perfect, seems like all that is needed to be done is to wrap the spdlog API in a WLogSink-derived class.

There is still a problem, however. According to the library overview Wt may spawn multiple processes when handling multiple sessions. And as far as I understand spdlog is not equipped to take care of this particular case, it can not operate on a single file from multiple processes. On the other hand the aforementioned documentation article states that "when using the built-in httpd, only one process is used in total". Thus I have a few questions:

  1. If I stick with the built-in httpd, can I use the spdlog or any other logging library not capable of handling multiple processes? Is the built-in httpd guaranteed not to spawn any subprocesses at all? If that is true, is the only mandatory requirement for a third-party logging library in this case is to be able to handle just the multiple threads within a single process? I would really like this one to work, as it is the simplest solution.

  2. In the multiple processes case, as far as I can tell my options are:

  • Use some other library that can handle it;
  • Rely on some system services like the systemd and logrotate mentioned above;
  • Create a separate process for logging. If I understand it correctly it could be handled in the following manner:
int main(int argc, char **argv)
{
    // Start the logging process before WRun is called
    int result = WRun (argc, argv, [handle_to_the_logging_process] (const Wt::WEnvironment& env) {
      return std::make_unique<HelloApplication>(env, handle_to_the_logging_process);
    });
    // Stop the logging process after WRun is finished
    return result;
}
  • Simply use the Wt::Dbo library? At least the SQlite backend should be capable of handling multiple processes as far as I understand. This solves the multiprocessing problem only though, the logic for rotating and limiting the log file needs to be manually implemented in this case.
  • Use a separate log file per session? Having dozens, hundreds or even thousands of small log files does not sound like an elegant solution at all. Well, these files can be merged later I suppose, but I am not sure this is worth the effort.

Are the any other possible issues stemming from the multiple process usage?

  1. Is Wt itself not fully equipped to handle logging from the multiple processes? Having a glimpse at the code of WLogger I haven't noticed anything that would suggest that it can safely write to a single file from multiple processes. And in particular the documentation for the WLogger::setUseLock function states that "For applications running in a dedicated process mode this may not be sufficient".

Replies (4)

RM RE: How to properly utilize the WLogSink class? - Added by Romain Mardulyn 23 days ago

Hi Nikita,

  1. Yes, the built-in httpd will only have one process as long as it is configured to be in shared-process mode (which is the default).

  2. I am not certain of why spdlog cannot operate on a single file from multiple processes, but if the issue is concurrency, you could use the inter-process lock from Boost to ensure that only 1 process at the time logs to spdlog.

  3. Yes, when using multiple processes, some logs will rarely be on the same line, followed by an empty line. The default Wt logger does not handle concurrent logging between different processes well.

NK RE: How to properly utilize the WLogSink class? - Added by Nikita Kornilov 23 days ago

Yes, the built-in httpd will only have one process as long as it is configured to be in shared-process mode (which is the default).

Excellent, that is a relief, I really didn't want to use any of these complicated solutions I listed, at least for now.

... you could use the inter-process lock from Boost to ensure that only 1 process at the time logs to spdlog

I had a brief look at the documentation of Boost.Interprocess, and it's that simple? Looks fascinating, all these elaborate setups I came up with could seemingly be replaced with a few lines of code from that library? Just curious, wouldn't adding it to the WLogger's source code solve that concurrency issue, considering Wt already depends on Boost?

RM RE: How to properly utilize the WLogSink class? - Added by Romain Mardulyn 23 days ago

We originally planned to do it, but this ended up being harder for Wt than it looked. This is because inter-process lock creates files. While this is quite simple to handle when creating your own application using Wt, it is much more difficult for Wt itself to ensure those files are deleted when Wt is closed, crashed or at least when it restarts. This gets even more difficult when you need to take into account that several Wt server with different versions of Wt could be running in the same environment.

NK RE: How to properly utilize the WLogSink class? - Added by Nikita Kornilov 23 days ago

I see, thank you for the quick responses Romain!

    (1-4/4)