Skip to content

Commit

Permalink
support new _value field for nesting
Browse files Browse the repository at this point in the history
  • Loading branch information
PS-davetemplin committed Jul 12, 2019
1 parent 6e62e3a commit 8b0ec80
Show file tree
Hide file tree
Showing 4 changed files with 9 additions and 9 deletions.
11 changes: 5 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ Written by Dave Templin

# Overview
Bigboy is basically a **SQL-TO-JSON** tool that extracts data from SQL Server, PostgreSQL, or MySQL databases.
It is designed to handle extremely high data extraction rates by running multiple concurrent queries.
The tool provides a simple configuration model for managing any number of extractions.
It is designed to handle extremely high data extraction rates (multi-million rows / gigabyte-range) which is achieved by running multiple concurrent extraction queries from a configurable thread pool.
The tool provides a simple configuration model for managing any number of data extractions.
It also exposes a simple and minimal command-line interface (CLI) that works great for adhoc or batch/cron use-cases.

## Features
Expand All @@ -31,8 +31,7 @@ fetch, prefetch

## Transforms
nest, script, split, timezone
_ for value only nesting

special field names: _parent, _value


# Reference
Expand Down Expand Up @@ -127,7 +126,7 @@ The following examples assume there is a target named `log` with a single paramt
| `bigboy log 2017-07-21` | 7/21/2017 at midnight GMT
| `bigboy log "2017-07-21 15:00:00"` | 7/21/2017 at 3pm GMT
| `bigboy log today` | Midnight GMT of the current day
| `bigboy log yesterday` | Midnight GMT of the previous day
| `bigboy log yesterday` | Midnight GMT of the previous day

The following examples assume there is a target named `sales` with two paramters of type `date` representing a date range for the extraction.

Expand All @@ -137,7 +136,7 @@ The following examples assume there is a target named `sales` with two paramters
| `bigboy sales 2017-07-21 2d` | Midnight GMT of the previous day.


> The time zone database needed by LoadLocation may not be present on all systems, especially non-Unix systems. LoadLocation looks in the directory or uncompressed zip file named by the ZONEINFO environment variable, if any, then looks in known installation locations on Unix systems, and finally looks in $GOROOT/lib/time/zoneinfo.zip.
> The time-zone database needed by LoadLocation may not be present on all systems, especially non-Unix systems. LoadLocation looks in the directory for an uncompressed zip file named by the ZONEINFO environment variable, if any, then looks in known installation locations on Unix systems, and finally looks in $GOROOT/lib/time/zoneinfo.zip.


Expand Down
1 change: 1 addition & 0 deletions build-all.cmd
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
@ECHO OFF
CALL build linux
CALL build mac
CALL build windows
2 changes: 1 addition & 1 deletion constants.go
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
package main

const version = "1.0.0"
const version = "1.1.0"
4 changes: 2 additions & 2 deletions nest.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ func queryNest(db *sql.DB, nest *Nest, list []map[string]interface{}) error {
if val, ok := child["_parent"]; ok {
if to_uint64(val) == to_uint64(parent[nest.ParentKey]) {
delete(child, "_parent")
if _, ok := child["_"]; ok {
slice = append(slice, child["_"])
if _, ok := child["_value"]; ok {
slice = append(slice, child["_value"])
} else {
slice = append(slice, child)
}
Expand Down

0 comments on commit 8b0ec80

Please sign in to comment.