Migrate scala-client to SDK.v3 + libuast.v3

[bzz]

An umbrella issue for client migration to new SDK.

Takes similar approach to bblfsh/client-python. but aims to avoid having all the changes in a single PR, so will be broken down.

• SDK.v2 - parse API
  • parse() impl, that uses new protocol.v2
  • decode() impl, so the binary wire protocol can be read (on Go side)
  • root() impl, so the decoded (on Go side) UAST can be exposed to JVM
  • load() impl, so the Go-backed UAST can be transferred to host JVM memory Node.load() implementation #91
  • encode() impl. Reverse: JVM memory objects transferred to Go
  • expose mode argument for parse (native, annotated, semantic) v2: expose parse mode #111
• SDK.v2 - filter API
  • iterator() impl, so UAST can be iterated with and without loading it to JVM Iterator implementation #106
  • filter() impl, so XPath filtering works Filter implementation #109
    take early feedback on API into account
  • Context impl (JVM-side factory), so multiple concurrent filter() calls from the client are supported
• SDK.v2 - list languages API
  Support for languages/version request (as soon as Language aliases and languages protocol v2 sdk#400 is released)
• doc: add JNI debugging instructions Docs and leaks #115
• doc: update API examples in README and CLI tool
• investigate JNI warnings moved to v2: investigate all warnings from -Xcheck:jni #112
• enable timeouts at v2 API again moved to v2: expose custom gRPC connection timeouts #100
• CI: refactor to multiple modular jobs (tests, releases per-platform, etc) moved to Improve the CI #87

[bzz]

New API example

Protocol.v2 response is protobuf-like encoded binary blob, so libuast now provides a method to decode() it on the Go side. Decoding will result only in in-memory UAST representation on the Go side, thus in order to be exposed to the host environment (JVM) as objects - the root node is also needs to load()ed.

Examples below use explicit type excessively in order to highlight the API internals, real applications \w same code will be more concise.

decode + load (Go->JVM)

import org.bblfsh.client.v2.*

// parse
//// read the binary UASTv2
val resp = client.parse(fileName, fileContent)

//// decode it, UAST is backed by Go
val ctx: Context = resp.uast.decode()
val rootNode: Node = ctx.root()

//// load the UAST from Go to JVM
val rootJvm: Map[String, _] = rootNode.load()

Iterate UAST (JVM)

// iterate the tree from root
def processRecursive(node: Any): Unit = node match {
  case m: Map[_, _] => {
    for ((k, v) <- m) {
      println(k)
      processRecursive(v)
    }
  }
  case s: Seq[_] => {
    for (e <- s) {
      processRecursive(e)
    }
  }
  case a: Any => println(a)
}

processRecursive(rootJvm)

Query (Go->JVM)

Print every node

val nodes: List[Node] = ctx.filter(rootNode, "//*")
nodes.foreach { n =>
    println(s"Node: $n")
}

[dennwc]

Maybe we can call NodeExt just Node (and same for Context)? Native nodes will be Map[string, _] (or whatever), so we won't need a native node type (opposed to NodeExt).

Also, the Context work is mentioned in the filter, but it is required to make decode work, so it probably belongs to that task group.

[bzz]

Thank you for the suggestion! I assumed that it will be simpler to make the decode part work first, with a global context.

[bzz]

#83 (comment) updated to include better type names and USAT iteration example.

[bzz]

A pre-release RC1 was cut from the current state of v2 migration https://github.com/bblfsh/scala-client/releases/tag/v2.0-rc1

Update: RC1 is blocked by #98 and will be re-published once that is merged.

Update: RC1 has been published.

[bzz]

Update: RC3 has been published.

[bzz]

The only missing part before doing v2.0.0 release is a developer documentation!
Will submit a PR first thing tomorrow and start preparing a release notes.

[bzz]

Everything seems to be ready for initial release 🎉