Integrating with OSLog

[lukeredpath]

I’m currently using SwiftLog in my iOS app mainly because of how easy it is to connect to Pulse, however with the new Xcode 15 structured logging support in the console I’m wondering if should switch to use OSLog instead.

Is there an easy way of using OSLog as my main logging API but still connecting this up to Pulse?

[kean]

SwiftLog supports OSLog as one of its logging backends. If you are already using SwiftLog, it should be easy to add OSLog to the list of backends.

In case someone who is not using SwiftLog yet stumbles upon this thread, there is an alternative way to implement this without too much code change using LoggerStore/events.

[lukeredpath]

I was hoping to eliminate the SwiftLog dependency - it’s recommended to use OSLog directly rather than through a wrapper library. I was wondering if there was a way to stream OSLog entries into the logged store.

Maybe I could just use Pulse for network logging and use OSLog for other logging.

[kean]

Maybe I could just use Pulse for network logging and use OSLog for other logging.

I think that would make the most sense. I've been prioritizing network logging for a while now, and you'll see it more and more in the upcoming versions. The text-based logs are there primarily as a way to insert important text-based messages in between your network logs to give them some context.

[vdka]

It might be possible to integrate OSLog directly and show logs within Pulse by utilizing OSLogStore.getEntries(with:at:matching:) on a background thread. I am going to explore this and open a PR if it pans out.

[vdka]

@kean The good news is, I believe if this is something you'd rather keep external to Pulse that should be achievable with a single API addition! The bad news is that there are some limitations OSLog (or perhaps more specifically OSLogStore) would introduce, I outline those below.

If you want OSLog support to be external (similar to how support for apple/swift-log is today), then this is something that can be achieved simply by adding createdAt: Date? = nil as a parameter to LoggerStore.storeMessage (#199) allowing us to write external log entries into Pulse, including those from OSLog, in theory even those from OSLog features such as signposts.

Now, the limitations:

• No file, function, & line
  • In spite public statements from Apple that at least the file & line numbers are captured, it appears to not be the case for Swift code, or at least these values are not exposed through OSLogEntry's retrieved using OSLogStore (FB12316843)
  • If Pulse were to support pulling from OSLog directly these params would likely need to be made optional and UI updated to reflect these values being unavailable from certain sources.
• No way to specify custom metadata for the log object
  • The Swift OSLog API supports something it calls attributes but those are specific to the values provided by interpolation.
  • There is a set of predefined 'metadata' for a log entry, namely the properties of OSLogEntryFromProcess and OSLogEntryWithPayload.

import Foundation
import OSLog

@available(iOS 15.0, macOS 10.15, tvOS 15.0, watchOS 8.0, *)
class OSLogStoreSynchronizer {
    let osLogStore: OSLogStore
    let pulseStore: LoggerStore

    var subsystemAllowList: [String] = [Bundle.main.bundleIdentifier].compactMap({ $0 })
    var mapLogLevel: (OSLogEntryLog.Level) -> LoggerStore.Level
    var onRead: ((Error?) async -> Void)? = nil

    var timer: DispatchSourceTimer?

    var lastLogStoreError: Error?

    var predicate: NSPredicate?
    var lastPosition: OSLogPosition?

    let queue = DispatchQueue(label: "com.github.kean.pulse.OSLogStoreSynchronizer")

    init(
        pulseStore: LoggerStore,
        subsystemAllowList: [String] = [Bundle.main.bundleIdentifier].compactMap({ $0 }),
        mapLogLevel: @escaping (OSLogEntryLog.Level) -> LoggerStore.Level = defaultOSLogLevelMapping,
        onRead: ((Error?) async -> Void)? = nil
    ) throws {
        self.osLogStore = try OSLogStore(scope: .currentProcessIdentifier)
        self.pulseStore = pulseStore
        self.mapLogLevel = mapLogLevel
        self.subsystemAllowList = subsystemAllowList
        self.onRead = onRead
    }

    deinit {
        timer?.cancel()
    }

    func readOSLogEntries(into pulseStore: LoggerStore) throws {
        var lastOSLogEntry: OSLogEntryLog?
        let entries = try osLogStore.getEntries(with: [], at: lastPosition, matching: predicate)
        for entry in entries {
            if let payload = entry as? OSLogEntryWithPayload,
               !subsystemAllowList.contains(payload.subsystem) && !subsystemAllowList.isEmpty {
                continue
            }
            switch entry {
            case let entry as OSLogEntryLog:
                pulseStore.storeMessage(
                    createdAt: entry.date,
                    label: entry.category,
                    level: mapLogLevel(entry.level),
                    message: entry.composedMessage,
                    metadata: ["thread_id": .stringConvertible(entry.threadIdentifier)],
                    file: "<unknown>",
                    function: "<unknown>",
                    line: 0
                )
                lastOSLogEntry = entry

            default:
                continue
            }
        }
        if let lastOSLogEntry {
            /// NOTE: lastPosition doesn't seem to filter the entries retrieved, we provide it anyway, but rely upon `predicate`
            self.predicate = NSPredicate(format: "date > %@", lastOSLogEntry.date as NSDate)
            self.lastPosition = osLogStore.position(date: lastOSLogEntry.date)
        }
        print("Did read")
    }

    func startSyncing(every interval: TimeInterval = 30) {
        let timer = DispatchSource.makeTimerSource(queue: queue)
        self.timer = timer
        timer.schedule(wallDeadline: .now(), repeating: interval)
        timer.setEventHandler { [weak self] in
            guard let self else { return }
            do {
                try readOSLogEntries(into: pulseStore)
                if let onRead {
                    Task {
                        await onRead(nil)
                    }
                }
            } catch {
                lastLogStoreError = error
                if let onRead {
                    Task {
                        await onRead(error)
                    }
                }
            }
        }
        timer.resume()
    }

    func stop() {
        timer?.cancel()
    }

    static func defaultOSLogLevelMapping(_ level: OSLogEntryLog.Level) -> LoggerStore.Level {
        switch level {
        case .undefined: return .trace
        case .debug: return .debug
        case .info: return .info
        case .notice: return .notice
        case .error: return .error
        case .fault: return .critical
        @unknown default: return .info
        }
    }
}

[kean]

Hey, thanks for looking into it, @vdka. I merged the PR with createdAt – it's a nice addition regardless of how it's going to be used. I didn't know OSLog now (finally!) provides a programmatic way to access the logs, which is great.

I don't know the best way to approach OSLog. There is definitely going to be more desire to use it now. I'm speaking from my own experience: I started using OSLog in Pulse 14.0.0-beta.2 for debugging remote logging.

I can think of a couple of possible scenarios. If someone wants to use Pulse and OSLog together and save the same messages in both, there is already a solution: swift-log. I know Apple said there were some reasons to use OSLog directly (maybe %{private}@?), but I don't know what they are and don't really care because it solves my use cases as is.

I'm pivoting Pulse to focus on network logging and debugging (mocking, etc) in version 14.0. The logs were never the main priority anyway. I think of them as a way to log important network-related messages to add some context. I have a couple of ideas on how to make it easier to trace network logs, but I'm still struggling to complete all the baseline features. So, for this use-case, making sure Pulse and OSLog store the same is a no-goal.

However, Pulse could provide a better OSLog integration. It could have a separate view/tab/option to show OSLog messages directly from OSLogStore and provide some facilities for exporting the logs. So, basically, instead of using OSLogStoreSynchronizer, it would just read directly from the store. Of course, it won't enable the "mixing" of network tasks and logs. OSLogStoreSynchronizer has an advantage that it create first-class Pulse messages.

[kean]

But I'm sure folks already have solutions in place, whether with swift-log or their ad-hoc systems. Pulse was never designed to be used on its own – it doesn't write anything in the Xcode console, which you kind of want to do in all cases.