Convenient to view the job logs

[Pandas886]

Error logs for directories like sink jobs or materialized views can only be found by viewing the running logs of the computing nodes from a large amount of logs. It is hoped that a web interface or SQL query can be provided to search for error logs related to specified sink names or materialized view names.

[fuyufjh]

Hi, thanks for bringing this up. We do record the most critical logs in rw_event_logs

rw_event_logs
Contains information about events, including event IDs, timestamps, event types, and additional information if available.

By "critical" I mean the errors that caused streaming jobs to fail and recovery. I guess the error in your case is not that critical, for example, a corrupted value in input data. These non-critical errors will be ignored, and a log will be printed into stderr.

A few approaches can mitigate the problem.

1. Store more error logs in rw_event_logs beside stderr. It's possible, but our engineers need to carefully pick the error types and do rate limit, in order to prevent it from overwhelming the internal storage.
2. Strictly evaluate the expression when creating a sink. When streaming is running, we prefer to continue the execution than failure. However, when a sink is being created, we might better just let it fail if any non-critical errors (such as corrupted input values), to notice users the problem (such as wrong column definition) and get fixed.

Looking forward to your feedback!

[xxchan]

I think there are two orthogonal things:

1. Easier to troubleshooting MV/sink issues. Currently we rely on logs heavily. We are striving to improve the UX, and trying to make most users don't need the logs to troubleshoot most issues. @fuyufjh mentioned some possible approaches in this aspect.
2. Easier to browse/search the logs, e.g., a "web interface" mentioned by you. Even we achieved 1, we might still need logs to diagnose some complex issues. But I think there are many existing solutions for logging search, like Loki and Elasticserach. You can use those tools to analyze RisingWave's log. For separation of concerns, it isn't too meaningful to have built-in log analyze abilities in RisingWave. How do you think?