Is it possible to use both Kamon Client and Prometheus Client to same port?

[ZviMints]

build.sbt:

  val kamonVersion = "2.5.1"
  lazy val kamon = Seq(
    "io.kamon" %% "kamon-bundle" % kamonVersion,
    "io.kamon" %% "kamon-prometheus" % kamonVersion
  )

application.conf:

kamon.prometheus {
  start-embedded-http-server = false
  include-environment-tags = true
  embedded-server {
    hostname = 0.0.0.0
    port = 9095
  }
}

kamon.instrumentation.akka.http {
  server.metrics {
    enabled = yes
  }
}

plugins.sbt:

addSbtPlugin("io.kamon" % "sbt-kanela-runner" % "2.0.6")

ServiceApp:

object ServiceApp extends App with StrictLogging {

  PrometheusService.startServer(port = 9095)
  KamonService.start()

  val actorSystem = ActorSystem(UserGuardian(onSigterm), "BrandSafetyPhoenixService")
  actorSystem.whenTerminated.map(_ => {
    logger.info("[ActorSystem terminated] Stopping Prometheus...")
    PrometheusService.stopServer()
    KamonService.stop()
    System.exit(1)
  })

  def onSigterm(): Future[Done] = {
    logger.info("[CoordinatedShutdown] ServiceStop")
    PrometheusService.stopServer()
    KamonService.stop()
    Future.successful(Done)
  }
}

PrometheusService:

object PrometheusService extends LazyLogging {

  private[this] var server: HTTPServer = _

  def startServer(port: Int): Unit = {

    DefaultExports.initialize()

    logger.info(s"Started Prometheus Server on: http://0.0.0.0:$port")
    server = new HTTPServer(port)
  }

  def stopServer(): Unit = {
    logger.info(s"Stopping Prometheus Server")
    server.stop()
  }
}

KamonService:

object KamonService extends StrictLogging {
  def start(): Unit = {
    Kamon.init
    logger.info("Kamon Initialized")
  }

  def stop(): Future[Unit] = {
    logger.info(s"Stopping Kamon Server")
    Kamon.stop()
  }
}

The app is Akka based.

When I'm running with the above setup, I'm getting the regular app metrics on http://0.0.0.0:9095/metrics and firing request, i manage to see my app metrics

# TYPE brand_safety_num_of_requests counter
brand_safety_num_of_requests_total 1.0

BUT. I do NOT see any akka metrics.

When I'm changing the following configuration from false to true:

start-embedded-http-server = true

and comment out the following line:

// PrometheusService.startServer(port = 9095)

I get the Akka metrics but CANNOT find app (brand_safety_num_of_requests_total) metrics, i.e:

http_server_requests_total{port="8080",interface="0.0.0.0",http_status_code="2xx",component="akka.http.server"} 1.0

Any ideas why they override each other? how can I fix it?

I want to send metrics from both clients to same ports.

Thanks!

[ZviMints]

Why use two clients and not just Kamon?

Since I do not want to change the current implementation of all the metrics on this service, but create easy integration with Akka Actors module and Akka HTTP modules which Kamon provides, and remain the custom user metrics from Prometheus io client.