Fluent Patterns for OE logs - Forum - OpenEdge General - Progress Community

Fluent Patterns for OE logs

 Forum

Fluent Patterns for OE logs

This question has suggested answer(s)

Wondering if there's anyone that has done the legwork of creating fluentd/bit patterns for OE logs such as the database, CASOE and PASOE processes? Thanks!

All Replies
  • I think you’ll be able to get a config in the PASOE Docker container (on ESD?). The PASOE agent  log format is very similar to CAS.
     
     
  • Database for fluentd:

    <source>

     @type tail

     format /^\[(?<time>\d{4}\/\d{2}\/\d{2}@\d{2}:\d{2}:\d{2}\.\d{3}\+\d{4})\] P-(?<pid>\d+)\s+T-(?<thread>\d+)\s+(?<severity>\S) (?<connection.type>(SQLSRV2)|([a-zA-Z]+))(\s*)(?<connection.id>[^:\s]+): .(?<messagenumber>[\d-]+).(\s+)(?<message>.*)$/

     path /some/db.lg

     pos_file /var/log/td-agent/somedb.lg.pos

     read_from_head true

     tag somedb

    </source>

    Clientlog for fluentd:

    <source>

     @type tail

     format /^\[(?<time>\d{2}/\d{2}/\d{2}@\d{2}:\d{2}:\d{2}\.\d{3}\+\d{4})\] P-(?<pid>\d+) T-(?<thread>\d+) (?<loglevel>\d+) (?<executionenvironment>[^\s\\]+) (?<logentrytype>[^\s\\]+)\s+(?<message>.*)$/

     path /some/client.log

     pos_file /var/log/td-agent/someclient.log.pos

     read_from_head true

     tag someclient

     path_key logfile

    </source>

    For pasoe logfile processing with fluentd and for clientlog parsing you may be interested in this, which handles multiline messages:

    github.com/.../fluent-plugin-mergecommon

    I haven't looked at the parsing provided by progress recently, does it now handle multi-line messages? I wouldn't mind switching to fluent-bit if it's supported now.

  • Thanks guys, great help from you both!