Skip to content

Commit

Permalink
Update gRPC API for SDK 1.0 (#28)
Browse files Browse the repository at this point in the history 
* first pass at sdk 1.0 changes for grpc api

* update to reading scheme

* update rpc method for metainfo -- it shouldn't be a stream.

* add additional fields for health check message

* remove field that is no longer used

* add id field to transaction response

* add tag field to metadata

* updates to prepare for release

* renaming

* disable ci stage
  • Loading branch information
@edaniszewski
edaniszewski authored Jun 15, 2018
1 parent fd81ba1 commit e0c079f
Show file tree
Hide file tree
Showing 15 changed files with 3,446 additions and 2,027 deletions.
24 changes: 12 additions & 12 deletions .circleci/config.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,13 @@ jobs:
- run:
name: Build Python Proto
command: |
mkdir -p /tmp/python/synse_plugin
mkdir -p /tmp/python/synse_grpc
python3 -m grpc_tools.protoc -I. \
--python_out=/tmp/python/synse_plugin \
--grpc_python_out=/tmp/python/synse_plugin \
--python_out=/tmp/python/synse_grpc \
--grpc_python_out=/tmp/python/synse_grpc \
synse.proto
sed -i -e 's/import synse_pb2 as synse__pb2/from . import synse_pb2 as synse__pb2/g' /tmp/python/synse_plugin/synse_pb2_grpc.py
if [ -f "/tmp/python/synse_plugin/synse_pb2_grpc.py-e" ]; then rm /tmp/python/synse_plugin/synse_pb2_grpc.py-e; fi;
sed -i -e 's/import synse_pb2 as synse__pb2/from . import synse_pb2 as synse__pb2/g' /tmp/python/synse_grpc/synse_pb2_grpc.py
if [ -f "/tmp/python/synse_grpc/synse_pb2_grpc.py-e" ]; then rm /tmp/python/synse_grpc/synse_pb2_grpc.py-e; fi;
- persist_to_workspace:
root: /tmp
paths:
Expand Down Expand Up @@ -59,8 +59,8 @@ jobs:
- run:
name: diff
command: |
git --no-pager diff --no-index /tmp/python/synse_plugin/synse_pb2.py ./python/synse_plugin/synse_pb2.py
git --no-pager diff --no-index /tmp/python/synse_plugin/synse_pb2_grpc.py ./python/synse_plugin/synse_pb2_grpc.py
git --no-pager diff --no-index /tmp/python/synse_grpc/synse_pb2.py ./python/synse_grpc/synse_pb2.py
git --no-pager diff --no-index /tmp/python/synse_grpc/synse_pb2_grpc.py ./python/synse_grpc/synse_pb2_grpc.py
git --no-pager diff --no-index /tmp/go ./go
- run:
name: Diff Failure
Expand All @@ -86,7 +86,7 @@ jobs:
command: |
cd python
python setup.py sdist
cp dist/synse_plugin-*.tar.gz /tmp/bin
cp dist/synse_grpc-*.tar.gz /tmp/bin
- persist_to_workspace:
root: /tmp
paths:
Expand Down Expand Up @@ -128,10 +128,10 @@ workflows:
jobs:
- py-build
- go-build
- diff:
requires:
- py-build
- go-build
#- diff:
# requires:
# - py-build
# - go-build
- py-dist:
filters:
branches:
Expand Down
31 changes: 26 additions & 5 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,40 @@
# Synse Server gRPC
#

# Make targets are organized by functionality in the 'make' subdirectory
include make/build.make
include make/github.make

PKG_VER := $(shell python python/version.py)

.PHONY: python
python: ## Build the GRPC source for Python
@printf "Generating Python source."
@docker run \
-v `pwd`:/build \
grpc/python:1.4 \
python3 -m grpc_tools.protoc -I/build \
--python_out=/build/python/synse_grpc \
--grpc_python_out=/build/python/synse_grpc \
/build/synse.proto && \
sed -i -e 's/import synse_pb2 as synse__pb2/from . import synse_pb2 as synse__pb2/g' python/synse_grpc/synse_pb2_grpc.py && \
if [ -f "python/synse_grpc/synse_pb2_grpc.py-e" ]; then rm python/synse_grpc/synse_pb2_grpc.py-e; fi;
@printf " [done]\n"

.PHONY: go
go: ## Build the GRPC source for Go
@printf "Generating Go source."
@docker run \
-v `pwd`:/build \
grpc/go:latest \
protoc -I /build /build/synse.proto --go_out=plugins=grpc:/build/go
@printf " [done]\n"

.PHONY: all
all: python go ## Build source for all supported languages

.PHONY: py-publish
py-publish: ## Build and publish the python package to PyPi
pip install 'twine>=1.5.0'
cd python ; python setup.py sdist bdist_wheel --universal
cd python ; twine upload dist/*
cd python ; rm -rf build dist .egg synse_plugin.egg-info
cd python ; rm -rf build dist .egg synse_grpc.egg-info

.PHONY: version
version: ## Print the current version of Synse Server gRPC
Expand Down
126 changes: 51 additions & 75 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,17 +1,43 @@
[![CircleCI](https://circleci.com/gh/vapor-ware/synse-server-grpc.svg?style=svg&circle-token=5c8fc7e65c9363a99cc224ee349ab7e72e39742c)](https://circleci.com/gh/vapor-ware/synse-server-grpc)
<p align="center"><a href="https://www.vapor.io/"><img src="assets/logo.png" width="360"></a></p>
<p align="center">
<a href="https://circleci.com/gh/vapor-ware/synse-server-grpc"><img src="https://circleci.com/gh/vapor-ware/synse-server-grpc.svg?style=shield"></a>
<h1 align="center">Synse GRPC</h1>
</p>

# synse-server-grpc
The internal gRPC API for Synse Server and its plugins.
<p align="center">The internal gRPC API for Synse Server and its plugins.</p>

## Overview
As of version 2.0, Synse Server uses a gRPC API internally to communicate with background

As of version 2.0, Synse Server uses a gRPC API internally to communicate with
plugins to get information and readings to/from configured devices. This repo contains the
gRPC [API spec](api-spec) as well as auto-generated Python and Go packages for the gRPC
gRPC [API spec][api-spec] as well as auto-generated Python and Go packages for the gRPC
API client/server.

Since the gRPC API components are used in multiple places (e.g. Python client in Synse Server,
Go server in Plugins), this repo serves as the common location for the shared pieces.

## The Synse Ecosystem
Synse Server is one component of the greater Synse Ecosystem.

- [**vapor-ware/synse-server**][synse-server]: An HTTP server providing a uniform API to interact
with physical and virtual devices via plugin backends. This can be thought of as a 'front end'
for Synse Plugins.

- [**vapor-ware/synse-sdk**][synse-sdk]: The official SDK (written in Go) for Synse Plugin
development.

- [**vapor-ware/synse-cli**][synse-cli]: A CLI that allows you to easily interact with
Synse Server and Plugins directly from the command line.

- [**vapor-ware/synse-graphql**][synse-graphql]: A GraphQL wrapper around Synse Server's
HTTP API that provides a powerful query language enabling simple aggregations and
operations over multiple devices.

- [**vapor-ware/synse-emulator-plugin**][synse-emulator]: A simple plugin with no hardware
dependencies that can serve as a plugin backend for Synse Server for development,
testing, and just getting familiar with how Synse Server works.


## Building
If changes need to be made to the gRPC API, the `go/` and `python/` (and any future supported
language directory) should *not* be modified. Only the protobuf spec (synse.proto) should be
Expand All @@ -36,14 +62,14 @@ make go
## Using

### Go
To use the generated client/server wrapper code for the gRPC API spec, you simply just need to
To use the generated client/server code for the Synse gRPC API, you simply just need to
get it as you would any other go package.

```bash
go get -v github.com/vapor-ware/synse-server-grpc/go
```

Then, it can be used in your project where necessary.
It is then simple to import in a project.

```go
package plugin
Expand All @@ -54,86 +80,29 @@ import (
```

### Python
> TODO: The distribution mechanism here is not fully figured out yet. Still a work in progress. I think
> that it would make sense to build a python package tarball and upload that to GitHub as a release artifact
> where it could be downloaded and installed in a Dockerfile. For now, I'll assume that is the case, but it
> is liable to change.
The Python package can be installed as a tarball from GitHub under a particular release.
To use the generated client/server code for the Synse gRPC API, you can get if from pypi
with pip

```bash
curl ...
tar -xzvf ...
pip install synse-grpc
```

Once you have the tarball, it can simply be installed via `pip`
It is then simple to import in a project.

```bash
pip3 install synse_plugin-*.tar.gz
```python
import synse_grpc
```

You can then verify locally that it was installed

```bash
python3 -c "import synse_plugin"
```

Alternatively, the Python source can be installed from a clone of this repo. The Makefile provides
targets to package, install, and uninstall via `pip`. It will do so for both Python 2 (assuming the
binary is named 'python') and Python 3 (assuming the binary is named 'python3').

```bash
# package the python code into a tarball
make pip-package

# install the packaged tarball (for development, see below)
make pip-install

# install the source in editable mode (recommended for development)
make pip-einstall

# uninstall synse_plugin from pip
make pip-uninstall

# clean up the artifacts generated from build/packaging
make pip-clean
```

## Releasing
To release a new version of Synse Server gRPC, you must have the correct permissions on the repo.
Assuming that you do, a few steps need to be taken. First, you need to know what version number is
to be released. Ideally, this should already be checked in to the repo. The version is currently
defined in `python/synse_plugin/__init__.py` under the `__version__` variable.

The version can be verified by running
```bash
make version
```

Then, clean out any of the previously build packages which may exist.
```bash
make pip-clean
```
GitHub releases are done via CI. The go source does not need to be published anywhere,
as it can be imported directly from this repo. The python source does need to be published.
A make target is provided to make it easy.

Next, rebuild the packages. Make sure that the repo is up to date prior to this.
```bash
make pip-package
make py-publish
```

This should create the `python/dist` directory which contains a tarball with the version
number in the tarball name.

Finally, run
```bash
make release-github
```

This will build a docker image that will install `hub` and create a new release draft under
the specified version. It will include the python package tarball in the release distribution.

Once that is done, you can go to the [GitHub release page](release-page) and should see that a new release
draft was created. You can then edit this draft and publish it.


## Troubleshooting
To see a list of available make targets and a brief description, use
Expand All @@ -146,5 +115,12 @@ as much context around the bug/issue as possible.



[synse-server]: https://github.com/vapor-ware/synse-server
[synse-cli]: https://github.com/vapor-ware/synse-cli
[synse-emulator]: https://github.com/vapor-ware/synse-emulator-plugin
[synse-graphql]: https://github.com/vapor-ware/synse-graphql
[synse-sdk]: https://github.com/vapor-ware/synse-sdk


[api-spec]: https://github.com/vapor-ware/synse-server-grpc/blob/master/synse.proto
[release-page]: https://github.com/vapor-ware/synse-server-grpc/releases
Binary file added assets/logo.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
15 changes: 0 additions & 15 deletions dockerfile/hub.dockerfile
View file

This file was deleted.

Loading

0 comments on commit e0c079f

Please sign in to comment.