Skip to content

Commit

Permalink
Err fields (#165)
Browse files Browse the repository at this point in the history 
* Enable extra fields in primary error returns with new function

* Also add automatic code/status field setting option

* Changes to readme and additional test cases

* Cleanup code/status issues
  • Loading branch information
@Calebjh
Calebjh authored May 8, 2019
1 parent eb154b0 commit 6438755
Show file tree
Hide file tree
Showing 7 changed files with 354 additions and 72 deletions.
100 changes: 57 additions & 43 deletions gateway/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -169,35 +169,69 @@ func (s *myService) MyMethod(req *MyRequest) (*MyResponse, error) {
### Response Format
Unless another format is specified in the request `Accept` header that the service supports, services render resources in responses in JSON format by default.

Services must embed their response in a Success JSON structure.
By default for a successful RPC call only the proto response is rendered as JSON, however for a failed call a special format is used, and by calling special methods the response can include additional metadata.

The Success JSON structure provides a uniform structure for expressing normal responses using a structure similar to the Error JSON structure used to render errors. The structure provides an enumerated set of codes and associated HTTP statuses (see Errors below) along with a message.
The `WithSuccess(ctx context.Context, msg MessageWithFields)` function allows you to add a `success` block to the returned JSON.
By default this block only contains a message field, however arbitrary key-value pairs can also be included.
This is included at top level, alongside the assumed `result` or `results` field.

The Success JSON structure has the following format. The results tag is optional and appears when the response contains one or more resources.
Ex.
```json
{
"success": {
"status": <http-status-code>,
"code": <enumerated-error-code>,
"foo": "bar",
"baz": 1,
"message": <message-text>
},
"results": <service-response>
}
```

The `results` content follows the [Google model](https://cloud.google.com/apis/design/standard_methods): an object is returned for Get, Create and Update operations and list of objects for List operation.
The `WithError(ctx context.Context, err error)` function allows you to add an extra `error` to the `errors` list in the returned JSON.
The `NewWithFields(message string, kvpairs ...interface{})` function can be used to create this error, which then includes additional fields in the error, otherwise only the error message will be included.
This is included at top level, alongside the assumed `result` or `results` field if the call succeeded despite the error, or alone otherwise.

Ex.
```json
{
"errors": [
{
"foo": "bar",
"baz": 1,
"message": <message-text>
}
],
"results": <service-response>
}
```

To return an error with fields and fail the RPC, return an error from `NewResponseError(ctx context.Context, msg string, kvpairs ...interface{})` or `NewResponseErrorWithCode(ctx context.Context, c codes.Code, msg string, kvpairs ...interface{})` to also set the return code.

The function `IncludeStatusDetails(withDetails bool)` allows you to include the `success` block with fields `code` and `status` automatically for all responses,
and the first of the `errors` in failed responses will also include the fields.
Note that this choice affects all responses that pass through `gateway.ForwardResponseMessage`.

To allow compatibility with existing systems, the results tag name can be changed to a service-defined tag. In this way the success data becomes just a tag added to an existing structure.
Ex:
```json
{
"success": {
"status": <http-status-code>,
"code": <enumerated-error-code>,
"message": <message-text>
},
"results": <service-response>
}
```

#### Example Success Responses
#### Example Success Responses With IncludeStatusDetails(true)

Response with no results
```json
{
"success": {
"status": 201,
"status": "CREATED",
"message": "Account provisioned",
"code": "CREATED"
"code": 201
}
}
```
Expand All @@ -206,9 +240,9 @@ Response with results
```json
{
"success": {
"status": 200,
"status": "OK",
"message": "Found 2 items",
"code": "OK"
"code": 200
},
"results": [
{
Expand All @@ -235,9 +269,9 @@ Response for get by id operation
```json
{
"success": {
"status": 200,
"status": "OK",
"message": "object found",
"code": "OK"
"code": 200
},
"results": {
"account_id": 4,
Expand All @@ -252,9 +286,9 @@ Response with results and service-defined results tag `rpz_hits`
```json
{
"success": {
"status": 200,
"status": "OK",
"message": "Read 360 items",
"code": "OK"
"code": 200
},
"rpz_hits": [
{
Expand Down Expand Up @@ -304,33 +338,6 @@ operators using the one defined in [filter.go](filter.go).
filter_Foobar_List_0 = gateway.DefaultQueryFilter
```

## Errors

### Format
Method error responses are rendered in the Error JSON format. The Error JSON format is similar to the Success JSON format.

The Error JSON structure has the following format. The details tag is optional and appears when the service provides more details about the error.
```json
{
"error": {
"status": <http-status-code>,
"code": <enumerated-error-code>,
"message": <message-text>
},
"details": [
{
"message": <message-text>,
"code": <enumerated-error-code>,
"target": <resource-name>,
},
...
],
"fields": {
"field1": [<message-1>, <message-2>, ...],
"field2": ...,
}
}
```

### Translating gRPC Errors to HTTP

Expand Down Expand Up @@ -370,6 +377,9 @@ You can find sample in example folder. See [code](example/cmd/gateway/main.go)

The idiomatic way to send an error from you gRPC service is to simple return
it from you gRPC handler either as `status.Errorf()` or `errors.New()`.
If additional fields are required, then use the
`NewResponseError(ctx context.Context, msg string, kvpairs ...interface{})` or
`NewResponseErrorWithCode(ctx context.Context, c codes.Code, msg string, kvpairs ...interface{})` functions instead.

```go
import (
Expand All @@ -380,4 +390,8 @@ import (
func (s *myServiceImpl) MyMethod(req *MyRequest) (*MyResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method is not implemented: %v", req)
}

func (s *myServiceImpl) MyMethod2(ctx context.Context, req *MyRequest) (*MyResponse, error) {
return nil, NewResponseErrorWithCode(ctx, codes.Internal, "something broke on our end", "retry_in", 30)
}
```
79 changes: 73 additions & 6 deletions gateway/errors.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package gateway
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"math"
Expand Down Expand Up @@ -87,12 +88,16 @@ func (h *ProtoErrorHandler) StreamHandler(ctx context.Context, headerWritten boo
}

func (h *ProtoErrorHandler) writeError(ctx context.Context, headerWritten bool, marshaler runtime.Marshaler, rw http.ResponseWriter, err error) {
const fallback = `{"error":[{"message":"%s"}]}`
var fallback = `{"error":[{"message":"%s"}]}`
if setStatusDetails {
fallback = `{"error":[{"message":"%s", "code":500, "status": "INTERNAL"}]}`
}

st, ok := status.FromError(err)
if !ok {
st = status.New(codes.Unknown, err.Error())
}
statusCode, statusStr := HTTPStatus(ctx, st)

details := []interface{}{}
var fields interface{}
Expand All @@ -119,16 +124,25 @@ func (h *ProtoErrorHandler) writeError(ctx context.Context, headerWritten bool,
if fields != nil {
restErr["fields"] = fields
}
if setStatusDetails {
restErr["code"] = statusCode
restErr["status"] = statusStr
}

errs, _ := errorsAndSuccessFromContext(ctx)
errs, _, overrideErr := errorsAndSuccessFromContext(ctx)
restResp := &RestErrs{
Error: errs,
}
restResp.Error = append(restResp.Error, restErr)
if !overrideErr {
restResp.Error = append(restResp.Error, restErr)
} else if setStatusDetails && len(restResp.Error) > 0 {
restResp.Error[0]["code"] = statusCode
restResp.Error[0]["status"] = statusStr
}
if !headerWritten {
rw.Header().Del("Trailer")
rw.Header().Set("Content-Type", marshaler.ContentType())
rw.WriteHeader(HTTPStatus(ctx, st))
rw.WriteHeader(statusCode)
}

buf, merr := marshaler.Marshal(restResp)
Expand Down Expand Up @@ -210,6 +224,35 @@ func WithError(ctx context.Context, err error) {
grpc.SetTrailer(ctx, md)
}

// NewResponseError sets the error in the context with extra fields, to
// override the standard message-only error
func NewResponseError(ctx context.Context, msg string, kvpairs ...interface{}) error {
md := metadata.Pairs("error", fmt.Sprintf("message:%s", msg))
if len(kvpairs) > 0 {
fields := make(map[string]interface{})
for i := 0; i+1 < len(kvpairs); i += 2 {
k, ok := kvpairs[i].(string)
if !ok {
grpclog.Infof("Key value for error details must be a string")
continue
}
fields[k] = kvpairs[i+1]
}
b, _ := json.Marshal(fields)
md.Append("error", fmt.Sprintf("fields:%q", b))
}
grpc.SetTrailer(ctx, md)
return errors.New(msg) // Message should be overridden in response writer
}

// NewResponseErrorWithCode sets the return code and returns an error with extra
// fields in MD to be extracted in the gateway response writer
func NewResponseErrorWithCode(ctx context.Context, c codes.Code, msg string, kvpairs ...interface{}) error {
SetStatus(ctx, status.New(c, msg))
NewResponseError(ctx, msg, kvpairs...)
return status.Error(c, msg)
}

// WithSuccess will save a MessageWithFields into the grpc trailer metadata.
// This success message will then be inserted into the return JSON if the
// ResponseForwarder is used
Expand All @@ -223,14 +266,35 @@ func WithSuccess(ctx context.Context, msg MessageWithFields) {
grpc.SetTrailer(ctx, md)
}

func errorsAndSuccessFromContext(ctx context.Context) (errors []map[string]interface{}, success map[string]interface{}) {
// WithCodedSuccess wraps a SetStatus and WithSuccess call into one, just to make things a little more "elegant"
func WithCodedSuccess(ctx context.Context, c codes.Code, msg string, args ...interface{}) error {
WithSuccess(ctx, NewWithFields(msg, args))
return SetStatus(ctx, status.New(c, msg))
}

func errorsAndSuccessFromContext(ctx context.Context) (errors []map[string]interface{}, success map[string]interface{}, errorOverride bool) {
md, ok := runtime.ServerMetadataFromContext(ctx)
if !ok {
return nil, nil
return nil, nil, false
}
errors = make([]map[string]interface{}, 0)
var primaryError map[string]interface{}
latestSuccess := int64(-1)
for k, vs := range md.TrailerMD {
if k == "error" {
err := make(map[string]interface{})
for _, v := range vs {
parts := strings.SplitN(v, ":", 2)
if parts[0] == "fields" {
uq, _ := strconv.Unquote(parts[1])
json.Unmarshal([]byte(uq), &err)
} else if parts[0] == "message" {
err["message"] = parts[1]
}
}
primaryError = err
errorOverride = true
}
if strings.HasPrefix(k, "error-") {
err := make(map[string]interface{})
for _, v := range vs {
Expand Down Expand Up @@ -266,5 +330,8 @@ func errorsAndSuccessFromContext(ctx context.Context) (errors []map[string]inter
}
}
}
if errorOverride {
errors = append([]map[string]interface{}{primaryError}, errors...)
}
return
}
49 changes: 49 additions & 0 deletions gateway/errors_test.go
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package gateway

import (
"bytes"
"context"
"encoding/json"
"fmt"
Expand All @@ -13,6 +14,7 @@ import (

"github.com/grpc-ecosystem/grpc-gateway/runtime"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/status"
)

Expand All @@ -39,6 +41,53 @@ func TestProtoMessageErrorHandlerUnknownCode(t *testing.T) {
}
}

func TestProtoMessageErrorHandlerWithDetails(t *testing.T) {
IncludeStatusDetails(true)
defer IncludeStatusDetails(false)
err := status.Error(codes.NotFound, "overridden error")
v := &RestErrs{}

b := &bytes.Buffer{}
enc := json.NewEncoder(b)
enc.Encode(map[string]interface{}{"bar": 2})
md := runtime.ServerMetadata{
HeaderMD: metadata.Pairs(
// "status-code", CodeName(codes.NotFound),
),
TrailerMD: metadata.Pairs(
"error", "message:err message",
"error", fmt.Sprintf("fields:%q", string(b.Bytes()))),
}
ctx := runtime.NewServerMetadataContext(context.Background(), md)

rw := httptest.NewRecorder()
ProtoMessageErrorHandler(ctx, nil, &runtime.JSONBuiltin{}, rw, nil, err)

if ct := rw.Header().Get("Content-Type"); ct != "application/json" {
t.Errorf("invalid content-type: %s - expected: %s", ct, "application/json")
}
if rw.Code != http.StatusNotFound {
t.Errorf("invalid http status code: %d - expected: %d", rw.Code, http.StatusNotFound)
}

if err := json.Unmarshal(rw.Body.Bytes(), v); err != nil {
t.Fatalf("failed to unmarshal response: %s", err)
}

if v.Error[0]["message"] != "err message" {
t.Errorf("invalid message: %s", v.Error[0]["message"])
}
if v.Error[0]["bar"].(float64) != 2 {
t.Errorf("invalid bar field: %d", v.Error[0]["bar"])
}
if v.Error[0]["status"] != CodeName(codes.NotFound) {
t.Errorf("invalid status name: %s - expected: %s", v.Error[0]["status"], CodeName(codes.NotFound))
}
if v.Error[0]["code"].(float64) != http.StatusNotFound {
t.Errorf("invalid status code: %d - expected: %d", v.Error[0]["code"], http.StatusNotFound)
}
}

func TestProtoMessageErrorHandlerUnimplementedCode(t *testing.T) {
err := status.Error(codes.Unimplemented, "service not implemented")
v := new(RestErrs)
Expand Down
Loading

0 comments on commit 6438755

Please sign in to comment.