Skip to content

Latest commit

 

History

History
412 lines (336 loc) · 17.3 KB

README.md

File metadata and controls

412 lines (336 loc) · 17.3 KB
Raw

geojson-p2p-db

Build Status

A p2p database for geojson map data (based on osm-p2p-db)

geojson-p2p-db shares a lot of the same features as osm-p2p-db, including replication, forking, append-only logs, and changesets. But geojson-p2p-db differs from osm-p2p-db in several ways.

  • OpenStreetMap (and osm-p2p-db) stores spatial data as nodes, ways, and relations. GeoJSON stores spatial data as Points, LineStrings, Polygons with Multi modifiers for each (MultiPoint, MultiLinestring, MultiPolygon).

  • Nodes in osm-p2p-db are used to maintain topology with other features. There is no concept of topology in geojson-p2p-db.

  • GeoJSON stores addtional metadata in a properties object for each Feature. OSM and osm-p2p-db generally use tags.

Every spatial entry in geojson-p2p-db is stored as a GeoJSON Feature with geometry and properties objects. Each Feature or Geometry type in a FeatureCollection or GeometryCollection must be stored in separate entries in geojson-p2p-db. It is not possible to store a FeatureCollection or GeometryCollection as a single entry in geojson-p2p-db. This makes storing spatial indexes and streaming much easier. It's sort of similar to the idea of Newline Delimited JSON.

Usage

var hyperlog = require('hyperlog')

var level = require('level')
var db = {
  log: level('/tmp/geojson-p2p/log'),
  index: level('/tmp/geojson-p2p/index')
}
var fdstore = require('fd-chunk-store')
var storefile = '/tmp/geojson-p2p/kdb'

var gjdb = require('../')
var gj = gjdb({
  log: hyperlog(db.log, { valueEncoding: 'json' }),
  db: db.index,
  store: fdstore(4096, storefile)
})

if (process.argv[2] === 'create') {
  var value = JSON.parse(process.argv[3])
  gj.create(value, function (err, key, node) {
    if (err) console.error(err)
    else console.log(key)
  })
} else if (process.argv[2] === 'query') {
  var q = process.argv.slice(3).map(csplit)
  gj.query(q, function (err, pts) {
    if (err) console.error(err)
    else pts.forEach(function (pt) {
      console.log(pt)
    })
  })
}

function csplit (x) { return x.split(',').map(Number) }

Now we can add a few GeoJSON features and search within a bounding box

$ mkdir /tmp/geojson-p2p
$ node db.js create '{"type":"Feature","id":"5155edcbb6f3e2a4","properties":{"i-am":"a-point"},"geometry":{"type":"Point","coordinates":[-123.027648,48.695492]}}'
3b3842791d71c865
$ node db.js create '{"type":"Feature","id":"a073bf01dfca3ee3","properties":{"i-am":"a-line"},"geometry":{"type":"LineString","coordinates":[[-123.147125,48.522062],[-123.063354,48.582966],[-122.994689,48.489306],[-122.86972,48.568429]]}}'
0f16be2e09c144f4
$ node db.js query 48.672486,48.726529,-123.075371,-122.997093
{
  "type": "Feature",
  "id": "3b3842791d71c865",
  "properties": {
    "i-am": "a-point"
  },
  "geometry": {
    "type": "Point",
    "coordinates": [
      -123.027648,
      48.695492
    ]
  },
  "version": "74731c8381df7648eb8709e1d272240643c339a6f270420fc53c1ad23040228b"
}

API

DB

Create a new geojson-p2p database

Parameters

  • opts Object
    • opts.log string a hyperlog with a valueEncoding of json
    • opts.db string a levelup instace to store index data
    • opts.store string an abstract-chunk-store instance
    • opts.kv string an optional hyperkv instance, if not specified, one will be created (optional, default 'kv')

Examples

var hyperlog = require('hyperlog')
var level = require('level')
var fdstore = require('fd-chunk-store')
var gjdb = require('geojson-p2p-db')

var gj = gjdb({
  log: hyperlog(level('log'), { valueEncoding: 'json' }),
  db: level('index'),
  store: fdstore(4096, '/tmp/geojson-p2p/kdb')
})

create

Store a new geojson feature or changeset from value. cb(err, id, node) is returned with the generated id and the node from the underlying hyperlog.

Parameters

  • value Object A GeoJSON Feature object containing geometry, properties and changeset properties or a changeset object with a type='changeset' and tags properties. tags.comment is recommended for storing text describing the changeset. Note: The GeoJSON specification allows an id property on Feature objects. This property will be added or destructively overwritten in the geojson-p2p-db to ensure uniqueness.
  • opts Object Options to pass to hyperkv. (optional, default {})
  • cb Function A callback function with the parameters err, id, node where id is the generated id and the node from the underlying hyperlog.

Examples

gj.create({ type: 'changeset', tags: { comment: 'This is a new changeset' }},
function (err, id, node) {
  if (err) throw err
  var feat = {
    type: "Feature",
    properties: {
      beep: 'boop'
    },
    geometry: {
      type: 'Point',
      coordinates: [-123.027648, 48.695492]
    },
    changeset: id
  }
  gj.create(feat, function (err, id, node) {
    if (err) console.error(err)
    console.log('Id', id)
    console.log('Node', node)
  })
})

put

Replace a document key from value. The document will be created if it does not exist. cb(err, node) is returned with the node from the underlying hyperlog.

Parameters

  • key string Id of a document to replace with value.
  • value Object A GeoJSON Feature object containing geometry, properties and changeset properties or a changeset object with a type='changeset' and tags properties. tags.comment is recommended for storing text describing the changeset. Note: The GeoJSON specification allows an id property on Feature objects. This property will be added or destructively overwritten in the geojson-p2p-db to ensure uniqueness.
  • opts Object Options to pass to hyperkv. (optional, default {})
  • cb Function A callback function with the parameters err, node with the node from the underlying hyperlog.

Examples

gj.create({ type: 'changeset', tags: { comment: 'This is a new changeset' }},
function (err, id, node) {
  if (err) throw err
  var feat = {
    type: "Feature",
    properties: {
      beep: 'boop'
    },
    geometry: {
      type: 'Point',
      coordinates: [-123.027648, 48.695492]
    },
    changeset: id
  }
  gj.create(feat, function (err, id, node) {
    if (err) console.error(err)
    console.log('Id', id)
    console.log('Node', node)
    feat.properties = {
      boop: 'beep'
    }
    feat.changeset = id
    gj.put(id, feat, function (err, node) {
      console.log('New node', node)
    })
  })
})

del

Mark the document at key as deleted. cb(err, node) is returned with the node from the underlying hyperlog.

Parameters

  • key string Id of the document to mark as deleted.
  • opts Object Options to pass to hyperkv. (optional, default {})
  • cb Function A callback function with the parameters err, node with the node from the underlying hyperlog.

Examples

gj.create({ type: 'changeset', tags: { comment: 'This is a new changeset' }},
          function (err, id, node) {
            if (err) throw err
            var feat = {
              type: "Feature",
              properties: {
                beep: 'boop'
              },
              geometry: {
                type: 'Point',
                coordinates: [-123.027648, 48.695492]
              },
              changeset: id
            }
            gj.create(feat, function (err, id, node) {
              if (err) console.error(err)
              console.log('Id', id)
              console.log('Node', node)
              gj.del(id, function (err, node) {
                if (err) throw err
                gj.get(id, function (err, node) {
                  if (err) throw err
                  console.log('Deleted', node)
                })
              })
            })
          })

batch

Atomically put or delete an array of documents as rows

Parameters

  • rows Array<Object> Array of row to put or delete.
    • rows[].type string Type of transaction. Either 'put' or 'del'.
    • rows[].key string The id of the document to transact.
    • rows[].links Array<string> An array of links to ancestor ids.
    • rows[].value Object For put, the document to store.
  • opts Object Options to pass to hyperkv (optional, default {})
  • cb Function A callback function with the parameters err, nodes with the nodes from the underlying hyperlog.

Examples

// With existing changeset id of 'A'
var rows = [
  { type: 'put', value: { type: 'Feature', properties: {}, geometry: { type: 'Point', coordinates: [-122.85, 48.52] }, changeset: 'A' } },
  { type: 'put', value: { type: 'Feature', properties: {}, geometry: { type: 'Point', coordinates: [-122.90, 48.60] }, changeset: 'A' } }
]
gj.batch(rows, function (err, nodes) {
if (err) throw err
console.log(nodes)
})

get

Get the documents with the id key.

Parameters

  • key string The id of the documents to retrieve.
  • opts Object Options to pass to hyperkv. (optional, default {})
  • cb Function A callback function with the parameters err, docs where docs is an object mapping hyperlog hashes to current document values. If a document has been deleted, it will only have the properties { id: <id>, version: <version>, deleted: true }.

Examples

gj.create({ type: 'changeset', tags: { comment: 'This is a new changeset' } },
          function (err, id, node) {
            if (err) throw err
            var feat = {
              type: "Feature",
              properties: {
                beep: 'boop'
              },
              geometry: {
                type: 'Point',
                coordinates: [-123.027648, 48.695492]
              },
              changeset: id
            }
            gj.create(feat, function (err, id, node) {
              if (err) console.error(err)
              console.log('Id', id)
              gj.get(id, function (err, nodes) {
                if (err) throw err
                console.log('Nodes', nodes)
              })
            })
          })

query

Query the database using latitude/longitude bounds.

Parameters

  • q Array<Array<Number>> An array of [[ minLat, maxLat], [minLng, maxLng]] coordinate pairs to specify a bounding box.
  • opts Object Options to pass to kdb-tree-store query. (optional, default {})
  • cb Function A callback function with the parameters err, res where res is an array of documents each containing an id property and a version property which is the hash key from the underlying hyperlog. Deleted documents will only have the properties { id: <id>, version: <version>, deleted: true }.

Examples

// With existing changeset id of 'A'
var rows = [
  { type: 'put', value: { type: 'Feature', properties: {}, geometry: { type: 'Point', coordinates: [-122.85, 48.52] }, changeset: 'A' } },
  { type: 'put', value: { type: 'Feature', properties: {}, geometry: { type: 'Point', coordinates: [-122.90, 48.70] }, changeset: 'A' } }
]

gj.batch(rows, function (err, nodes) {
  if (err) throw err
  console.log('Created', nodes)
  gj.query([ [ 48.50, 48.60 ], [ -122.89, -122.80 ] ], function (err, res) {
    if (err) throw err
    console.log('Results', res)
  })
})

queryStream

Return a readable object stream of query results contained in the query q.

Parameters

  • q Array<Array<Number>> An array of [[ minLat, maxLat], [minLng, maxLng]] coordinate pairs to specify a bounding box.
  • opts Object Options to pass to kdb-tree-store query. (optional, default {})

getChanges

Get a list of document version ids in a changeset by the changeset id key

Parameters

  • key string Id of changeset.
  • opts
  • cb Function? An optional callback function with the parameters err, versions where versions is an array of changeset ids. If no callback is specified, a readable object stream is returned.

Examples

gj.create({ type: 'changeset', tags: { comment: 'This is a new changeset' }},
          function (err, changesetId, node) {
            if (err) throw err
            var rows = [
              { type: 'put', value: { type: 'Feature', properties: {}, geometry: { type: 'Point', coordinates: [-122.85, 48.52] }, changeset: changesetId } },
              { type: 'put', value: { type: 'Feature', properties: {}, geometry: { type: 'Point', coordinates: [-122.90, 48.60] }, changeset: changesetId } }
            ]
            gj.batch(rows, function (err, nodes) {
              if (err) throw err
              gj.getChanges(changesetId, function (err, nodes) {
                if (err) throw err
                console.log(nodes)
              })
            })
          })

License

Copyright 2017 Nick Peihl

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.