LogViewer - Failed to read data from logs

[srshankirth]

Hi,

Please see below configuration tried on our end for the sample log file. The exception is displayed as "Failed to read logs" and (No date field,
log cannot be merged) while running the log viewer. Attached screenshot. we are trying to read the logs from 2 servers and display it.
Could you please check and advise on this to resolve. Let us know if additional info is needed.

log-paths = {
sandstone = {
file = "E:/SSTLogs/.lo"
host = ["aptinthubt01","aptinthubu02"]
format = {
type: RegexLogFormat
regex: "(?\d{2}/\d{2}/\d{2}) + (?.+)"
locale: "en_UK"
charset: UTF-8
fields: [
{ name: "date", type: "date" },
{ name: "msg", type: "message" },
]
}
}
}

Log file as below:
"04/03/21 23:59:48 1197770875 TcpipListener 21002-030021 SkLoggedEvent Http request [BlvWebRequest (TF2TVB)] rejected with status=[404], reason=[File not found] , explanation=[Document /About/files/SuiKitchenSinkLibrary/logo.png not found.]";

Referred URL:
https://github.com/sevdokimov/log-viewer/blob/master/_docs/configuration.md

[srshankirth]

@sevdokimov ,
Could you please share your insights on this.

[srshankirth]

@sevdokimov ,

Could you please advise on this. awaiting for your response.

[sevdokimov]

@srshankirth sorry for the delay.
There are 2 problems
1.) format={...} should be specified in logs = [..] section, not in `log-paths = {...} . "log-paths" contains just shortcuts to real path.
2.) "date" field should include both the date and the time. Also "dateFieldName" and "datePattern" should be provided.

Here is the correct configuration:

logs = [
  {
      path: "E:/SSTLogs/.lo"
      format = {
        type: RegexLogFormat
        regex: "(?<date>\\d{2}/\\d{2}/\\d{2} \\d{2}:\\d{2}:\\d{2}) +(?<msg>.+)"
        locale: "en_UK"
        dateFieldName: "date"
        datePattern: "dd/MM/yy HH:mm:ss"
        charset: UTF-8
        fields: [
          { name: "date", type: "date" },
          { name: "msg", type: "message" },
        ]
      }
  }
]

log-paths = {
  sandstone = {
    file = "E:/SSTLogs/.lo"
    host = ["aptinthubt01","aptinthubu02"]
}

}

Note: logs=[..] section with the format must be present on all nodes.

[sevdokimov]

Log4J log format is simpler in this case:

      format = {
        type: Log4jLogFormat
        pattern: "%d{dd/MM/yy HH:mm:ss} %msg%n"
        locale: "en_UK"
        charset: UTF-8
      }

[srshankirth]

Hi @sevdokimov,

Thanks for sharing the configuration.

We have tried in the config and mentioned in both host as below. However, we are unable to read the log file from both servers T01/U02. (Note: we are able to read the log file from only one server - T01, but not from another host -
logsear.docx
U02) Please find attached screenshot for reference. Need your suggestion on this.

logs = [
{
path: "E:/SST_Logs/.lo"
format = {
type: RegexLogFormat
regex: "(?\d{2}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) +(?.+)"
locale: "en_UK"
dateFieldName: "date"
datePattern: "dd/MM/yy HH:mm:ss"
charset: UTF-8
fields: [
{ name: "date", type: "date" },
{ name: "msg", type: "message" },
]
}
}
]

log-paths = {
sandstone = {
file = "E:/SST_Logs/.lo"
host = ["osbinthubt01","osbinthubu02"]
}
}

[srshankirth]

Hi @sevdokimov ,
Could you please advise on this which I was tried with the given config. awaiting for your response.

[srshankirth]

Hi @sevdokimov ,

We are able to read the log files from both servers as expected with the configuration that you have shared. Thanks for the assistance.

I would need to disable or remove the Download option from the top right end menu. Attached screenshot for reference.

could you please assist on this to do the needful.

I am not sure since we have the Jar files available within "


Lib" folder. I came across the below files where the logic related to download is available under log-viewer-cli -> https://github.com/sevdokimov/log-viewer/blob/master/log-viewer-cli/assets/etc/linux-desktop/log-viewer.desktop

log-viewer-cli/src/test/java/com/logviewer/tests/pages/LogPage.java
log-viewer-cli/src/test/java/com/logviewer/tests/web/DownloadLogTest.java

[srshankirth]

Hi @sevdokimov ,

I am using below regex (consider screenshot) and trying to extract only the Green highlighted box , no need to pull or extract the yellow highlighted box (attached screenshot). could you please advise at earliest on this comment to achieve to pull only the green highlighted box log data and also advise for comment added on 26/03/2024.

logs = [
{
path: "E:/SST_Logs/.lo"
format = {
type: RegexLogFormat
regex: "(?\d{2}/\d{2}/\d{2} \d{2}:\d{2}:\d{2} \d{8,10} [A-Za-z]{7}[(][A-Za-z]{2}[)] [A-Za-z0-9]{10} [A-Za-z]{4}:\d{9,10}) +(?.+)"
locale: "en_UK"
dateFieldName: "date"
datePattern: "dd/MM/yy HH:mm:ss"
charset: UTF-8
fields: [
{ name: "date", type: "date" },
{ name: "msg", type: "message" },
]
}
}
]

[srshankirth]

Hi @sevdokimov ,

Request you to suggest on this to proceed further as this is urgent request on my end.

[srshankirth]

Hi @sevdokimov ,
could you please suggest at earliest on my previous 2 comments/queries.

[sevdokimov]

Hi @srshankirth

Sorry for the delay.

There is no options to hide Download button, I'll add this option later.

trying to extract only the Green highlighted box , no need to pull or extract the yellow highlighted box (attached screenshot).

Do you mean the messages containing "User: 280834234" should be displayed, and other messages should be hidden, correct?
You cannot hide messages using a regex in logs=[ {format="..."} ] this regexp is responsible for parsing the log. All log messages should be parsed. You can add a filter to hide messages in the UI.

You can click "Add Filter" -> Text Filter

Then select "Regexp" checkbox and type a regexp for filtering messages. This regexp will be applied to all messages after parsing, only matched events will be displayed.

After adding this filter, the browser URL will contain a parameter like &filters=86f9072721cd45fed8d5b041c3487157 , this is a hash of the filter panel state, you can add the URL to browser bookmarks to save the filter for the next opening.

[srshankirth]

Hi @sevdokimov ,
Thanks for your response on this.

Do you mean the messages containing "User: 280834234" should be displayed, and other messages should be hidden, correct?
"YES"

Please find attached document with screenshots along with my expectation/requirement.

could you please advise if there is a way to achieve instead of filtering from UI. Looking forward on your response.
logview_10-04-2024.docx

[sevdokimov]

It seems to be text filter -> enabling checkbox Regexp -> didn’t work.

I think something wrong with the regexp. In "User: \d+" the space after ":" should be removed. If you attach a sample of the log lines in a text form, I'll create a valid regexp.

In other words, the below based pattern messages should be read/display in UI from logs prior to filtering from logviewer UI.

I'm not clear that you mean. Adding the filter in UI will remove all unmatched lines from the view, you will see matched lines only.

[srshankirth]

Hi @sevdokimov ,
Thanks Again for the rapid response. The filtering is working now as expected, space caused the problem.

I understood the filter in UI will remove all unmatched lines. Can you please let me know if there is a way to read the logs based on below text form and display in UI instead of applying filter after the data is rendered in UI.

Text form as below:
24/03/24 23:12:40 2331240937 Session(IB) BVFS3JYDVF User:2707715134

[sevdokimov]

The filter is configured in UI, but it performs filtering on the backend. Unmatched lines don't go to UI.