Tracking logs on edx-platform

rsyslog

edx-platform tracking logs are currently configured to send to an rsyslog process. Rsyslog can be configured to forward tracking logs to various places, including specific files or other servers.

More info on rsyslog configuration:

• https://man7.org/linux/man-pages/man5/rsyslog.conf.5.html

• https://www.thegeekdiary.com/understanding-the-etc-rsyslog-conf-file-for-configuring-system-logging/

• https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/deployment_guide/s1-basic_configuration_of_rsyslog

Alternative Logging Handlers

While SysLogHandler is the class being used for platform tracking logs, that setting can be overwritten to support various other log handlers.

Some alternative classes include:

• SocketHandler

  • Useful for sending data to a remote machine

  • Pickles data prior to sending, so the remote machine must be able to unpickle

• FileHandler

  • Allows user to specify which file to write logs to

  • Eats up disk space if there is no separate log rotation process

• RotatingFileHandler

  • Similar to FileHandler, but handles process of log rotation

Fluentd

Fluentd is an open source log collector that can be run alongside an application. Because rsyslog cannot run inside of a docker container, fluentd can be used as an alternative log processor.

edx-platform’s current reliance on rsyslog does not easily translate to running in a container, which is why fluentd is being explored as an alternative.

More info on fluentd:

• Docker installation

• Fluentd configuration file

• Input sources

• Output options