Logstash Grok filters for OpenEdge - Forum - OpenEdge Deployment - Progress Community

Logstash Grok filters for OpenEdge

 Forum

Logstash Grok filters for OpenEdge

  • We are currently in the process of implementing an ELK stack for log file archival, searching, etc. and the hard(est) thing we had to do was create the GROK filters for Logstash. The format of the log files across the AppServer broker, Server, NameServer and Database are all slightly different.

    I'm posting them here, in hopes that they will be useful to other developers that need to achieve the same thing, as well as to get some comments/feedback on them from anyone else that has implemented the same.

    OpenEdge Database:

    ^%{OE_TIMESTAMP:[@metadata][timestamp]}\s*%{OE_PROCESS_ID:pid}\s*%{OE_THREAD_ID:thread_id}\s*%{WORD:message_type}\s+%{WORD:db_process_type}\s*(%{OE_USER_ID:user_id})?\:\s+\(%{OE_MESSAGE_NUM:progress_message_num}\)\s+%{GREEDYDATA:message_text}

    OpenEdge AppServer/Webspeed:

    ^%{OE_TIMESTAMP:[@metadata][timestamp]}\s*%{OE_PROCESS_ID:pid}\s*%{OE_THREAD_ID:thread_id}\s*%{BASE10NUM:logging_level}\s+%{NOTSPACE:exec_env}\s*%{NOTSPACE:exec_subsys}\s*%{GREEDYDATA:message_text}\s*\(%{BASE10NUM:progress_message_num}\)$

    or


    ^%{OE_TIMESTAMP:[@metadata][timestamp]}\s*%{OE_PROCESS_ID:pid}\s*%{OE_THREAD_ID:thread_id}\s*%{BASE10NUM:logging_level}\s+%{NOTSPACE:exec_env}\s*%{NOTSPACE:exec_subsys}\s*%{GREEDYDATA:message_text}

    We have also defined the following types specifically for the patterns:

    OE_PROCESS_ID \P\-[0-9]*
    OE_THREAD_ID \T\-[a-zA-Z0-9\-]*
    OE_TIMESTAMP %{SYSLOG5424SD}
    OE_USER_ID %{BASE16FLOAT}
    OE_DB_PROCESS_TYPE %{WORD}
    OE_DB_MESSAGE_TYPE [FI]
    OE_MESSAGE_NUM \(?%{BASE16FLOAT}\)?

    Hopefully someone else finds this information useful.

  • Thanks! This is very useful since devising these is always a bit of a pain.