From c69ad059fae1fe83bb3c90f1533be5a12d6809e4 Mon Sep 17 00:00:00 2001 From: Riordan Date: Mon, 11 Apr 2016 13:59:11 -0400 Subject: [PATCH 1/9] wip --- ids.md | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) create mode 100644 ids.md diff --git a/ids.md b/ids.md new file mode 100644 index 0000000..1c4a9b0 --- /dev/null +++ b/ids.md @@ -0,0 +1,70 @@ +# IDs in Mapzen Search + +Depending on how you use the service, system identifiers can play play a big part in how you use Mapzen Search / Pelias. All places in Mapzen Search have a _global identifier_, known as a `gid`. Each matching record returned from a [`/search`](search), [`/autocomplete`](autocomplete), or [`/reverse`](reverse) geocoding request will have a `gid` field that corresponds to that place's information in the system. + +Identifiers in Mapzen Search may change from week-to-week. They should not be considered stable, unless otherwise noted in the sources section below. Because of the nature of some of the data sources we work with, IDs may change without notice at any given time. + +A record's `gid`'s in Mapzen Search looks like: +`"gid": "whosonfirst:locality:101750367",` + + +## Rules for using `gid`'s + +**1. Do not use Mapzen Search `gid`'s as permanent identifiers for places.** + +`gid`'s in Mapzen Search can change because several of our data sources also lack permanent IDs. Some of our sources do not provide ways to track [...]. + + +**2. The structure of identifiers in Mapzen Search may change.** + + +## Source-specific `gid` structures + + + + + + + + + +
Source Examples Structure Notes
Who's on First +
  • `whosonfirst:locality:101750367`
  • `whosonfirst:country:85633793`
+
+ + +| Source | Examples | Structure | Notes | +| ------ | --------- | ----------- | -------- | +| _Who's on First_ | | whosonfirst + + +### _Who's on First_ | `whosonfirst` + +`"gid": "whosonfirst:locality:101750367"` + +A Who's on First identifier consists of 3 elements, it's source (`whosonfirst`), its [placetype](https://github.com/whosonfirst/whosonfirst-placetypes), and the number that is its [unique 64-bit numeric ID](https://github.com/whosonfirst/whosonfirst-data/#portability). + +[Placetypes](https://github.com/whosonfirst/whosonfirst-placetypes) are consistent between Mapzen Search and Who's on First. + +The numeric identifier at the end of a Who's on First `gid` _may be considered a stable identifier_ for the corresponding Who's on First record, and is likely to be used as an identifier for other systems using Who's on First. However, Mapzen Search requires that a record's type be known before the record may be retrieved from `/place`. + +> ✅ Valid: [`/v1/place?ids=whosonfirst:country:85633793`](http://pelias.github.io/compare/#/v1/place%3Fids=whosonfirst:country:85633793) + +> ❌ Invalid (missing placetype): [`/v1/place?ids=whosonfirst:85633793`](http://pelias.github.io/compare/#/v1/place%3Fids=whosonfirst:85633793) + +### OpenStreetMap | `openstreetmap` + +Results coming from OpenStreetMap take the form of: + +### OpenAddresses | `openaddresses` + +### Geonames | `geonames` + + + + +## How we use `gid`'s in Mapzen Search + +### Why does each record have multiple multiple fields with `gid`? + +### Why does each record have an `id` field in addition to `gid`? From 0e5e284b7eedaf1335b0e9e0bc5723517f67f610 Mon Sep 17 00:00:00 2001 From: Riordan Date: Thu, 14 Apr 2016 14:28:30 -0400 Subject: [PATCH 2/9] word finishes --- ids.md | 37 ++++++++++++++++++------------------- 1 file changed, 18 insertions(+), 19 deletions(-) diff --git a/ids.md b/ids.md index 1c4a9b0..e958991 100644 --- a/ids.md +++ b/ids.md @@ -12,30 +12,16 @@ A record's `gid`'s in Mapzen Search looks like: **1. Do not use Mapzen Search `gid`'s as permanent identifiers for places.** -`gid`'s in Mapzen Search can change because several of our data sources also lack permanent IDs. Some of our sources do not provide ways to track [...]. - +`gid`'s in Mapzen Search can change because several of our data sources also lack permanent IDs. Some of our sources do not provide ways to track changes between each release of the data. This is particularly true for OpenAddresses, where there are no general ID's for each address. With OpenStreetMap it is possible for an entry to be superseded by another place, but for there to be no indicator of what the replacement place is. **2. The structure of identifiers in Mapzen Search may change.** +This is not necessarily the final configuration of ID's in Mapzen Search. We reserve the right change their structure at some time in the future. - -## Source-specific `gid` structures - - - - - - +**3. The Mapzen Search ID for a place may not be the same exact ID in other systems.** +While some of our sources do have persistent ID's (e.g. Node/Way ID's in OpenStreetMap or all Who's on First ID's), we augment them to fit into our taxonomy and add the source dataset. While the numeric component of a Mapzen Search record coming from Who's on First can be used as its ID, you can't just use a Who's on First ID to get the record from Mapzen Search. - -
Source Examples Structure Notes
Who's on First -
  • `whosonfirst:locality:101750367`
  • `whosonfirst:country:85633793`
-
- - -| Source | Examples | Structure | Notes | -| ------ | --------- | ----------- | -------- | -| _Who's on First_ | | whosonfirst +## Source-specific `gid` structures ### _Who's on First_ | `whosonfirst` @@ -54,7 +40,20 @@ The numeric identifier at the end of a Who's on First `gid` _may be considered a ### OpenStreetMap | `openstreetmap` + Results coming from OpenStreetMap take the form of: +`"gid": "openstreetmap::id:"` + +There's a lot to unpack about that. + +``: corresponds to the core OpenStreetMap data type for that place, or [Element](http://wiki.openstreetmap.org/wiki/Elements). These are the geometries that data about a place is attached to in OpenStreetMap. There are 3 elements in OpenStreetMap: +* [Nodes](http://wiki.openstreetmap.org/wiki/Node), which are points corresponding to a latitude and longitude on earth. Nodes have positive numbers as their ID numbers in OpenStreetMap. +* [Ways](http://wiki.openstreetmap.org/wiki/Way), which are ordered collections of points that can make up a line or a polygon +* [Relations](http://wiki.openstreetmap.org/wiki/Relation), which are arbitrary collections of points, lines, and relations. + + + +Records coming from OpenStreetMap can take the form of ### OpenAddresses | `openaddresses` From 59a89e54e290da0fa2b494066be77cf439df94a6 Mon Sep 17 00:00:00 2001 From: rmglennon Date: Thu, 12 May 2016 10:35:11 -0700 Subject: [PATCH 3/9] add some info on IDs to response --- response.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/response.md b/response.md index ae9bdee..f34ae2a 100644 --- a/response.md +++ b/response.md @@ -60,12 +60,17 @@ Each item in this list will contain all the information needed to find it in hum Additionally, `/reverse` queries will have a `distance` parameter, which is the distance, in meters, from the query point. -## Notable features +## Notable elements ### `gid` -This is a "global id" that can be used to reference a result with the [/place](place.md) endpoint. It consists of an identifier for the dataset, a layer, and finally an `id` for the individual record. This `id` corresponds to stable ids from datasets wherever possible (such as the ID of an OpenStreetMap Node or Way). Not all datasets have stable ID's (in particular OpenAddresses records), so for OpenAddresses, Mapzen Search includes the name of the source dataset in OpenAddresses as a part of the ID scheme. +All places in Mapzen Search have a global identifier, known as a `gid`. Each matching record returned from a [`/search`](search), [`/autocomplete`](autocomplete), or [`/reverse`](reverse) geocoding request has a `gid` field. -The `gid` is also used to retrieve full details on a particular result from the `place` endpoint. [Full details](place.md) on ID schemes are part of the `/place` endpoint. +The `gid` consists of an identifier for the dataset, a layer, and an `id` for the individual record. + +There are some important considerations for using IDs in Mapzen Search: + +- You should not create your own ID values. +- ID values are not stable and you should not store them for future use. You should only use IDs at the time when you receive the search results. One valid use for the `gid` is to retrieve full details on a particular result from the `place` endpoint. ### `label` The `label` is a human-friendly representation of the place, ready to be displayed to an end user. The label field attempts to use a format that is right for the region the result is in, although Mapzen Search only supports a few countries at the moment. From cabf79a39793272ba6afdbb5d0f2fa1bc6174434 Mon Sep 17 00:00:00 2001 From: rmglennon Date: Fri, 13 May 2016 11:33:18 -0700 Subject: [PATCH 4/9] add parse bullet point --- response.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/response.md b/response.md index f34ae2a..bade229 100644 --- a/response.md +++ b/response.md @@ -70,7 +70,8 @@ The `gid` consists of an identifier for the dataset, a layer, and an `id` for th There are some important considerations for using IDs in Mapzen Search: - You should not create your own ID values. -- ID values are not stable and you should not store them for future use. You should only use IDs at the time when you receive the search results. One valid use for the `gid` is to retrieve full details on a particular result from the `place` endpoint. +- ID values may not be stable. +- You should not attempt to parse ID values for information or store them for future use. You should only use IDs at the time when you receive the search results. One valid use for the `gid` is to retrieve full details on a particular result from the `place` endpoint. ### `label` The `label` is a human-friendly representation of the place, ready to be displayed to an end user. The label field attempts to use a format that is right for the region the result is in, although Mapzen Search only supports a few countries at the moment. From 9dd41c11456a5d1548df9ddf0a5b7aaa8bd8009f Mon Sep 17 00:00:00 2001 From: Rhonda Glennon Date: Fri, 13 May 2016 11:49:06 -0700 Subject: [PATCH 5/9] make gid consistent --- response.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/response.md b/response.md index bade229..3fc301c 100644 --- a/response.md +++ b/response.md @@ -65,13 +65,15 @@ Additionally, `/reverse` queries will have a `distance` parameter, which is the ### `gid` All places in Mapzen Search have a global identifier, known as a `gid`. Each matching record returned from a [`/search`](search), [`/autocomplete`](autocomplete), or [`/reverse`](reverse) geocoding request has a `gid` field. -The `gid` consists of an identifier for the dataset, a layer, and an `id` for the individual record. +The `gid` consists of a `layer` (such as `address` or `country`), an identifier for the original data source (`openstreetmap` or `openaddresses`), and an `id` for the individual record corresponding to the original source idenfier, where possible. This information is also available as properties on the individual results as `layer`, `source`, and `source_id`. -There are some important considerations for using IDs in Mapzen Search: +:rotating_light: -- You should not create your own ID values. -- ID values may not be stable. -- You should not attempt to parse ID values for information or store them for future use. You should only use IDs at the time when you receive the search results. One valid use for the `gid` is to retrieve full details on a particular result from the `place` endpoint. +Follow these guidelines regarding the `gid`: + +- You should not create your own `gid` strings. +- `gid` values may not be consistent across releases. +- You should not attempt to parse `gid` strings for information or store them for future use. You should only use `gid` at the time when you receive the search results. One valid use for the `gid` is to retrieve full details on a particular result from the `/place` endpoint. ### `label` The `label` is a human-friendly representation of the place, ready to be displayed to an end user. The label field attempts to use a format that is right for the region the result is in, although Mapzen Search only supports a few countries at the moment. From 3b3188e7802456e432d2d8f499a70fdd1f5d5f6b Mon Sep 17 00:00:00 2001 From: Rhonda Glennon Date: Fri, 13 May 2016 11:51:02 -0700 Subject: [PATCH 6/9] remove space --- response.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/response.md b/response.md index 3fc301c..138c6d9 100644 --- a/response.md +++ b/response.md @@ -67,9 +67,7 @@ All places in Mapzen Search have a global identifier, known as a `gid`. Each mat The `gid` consists of a `layer` (such as `address` or `country`), an identifier for the original data source (`openstreetmap` or `openaddresses`), and an `id` for the individual record corresponding to the original source idenfier, where possible. This information is also available as properties on the individual results as `layer`, `source`, and `source_id`. -:rotating_light: - -Follow these guidelines regarding the `gid`: +:rotating_light: Follow these guidelines regarding the `gid`: - You should not create your own `gid` strings. - `gid` values may not be consistent across releases. From 06bf55d8533927e7255d921db786429ddb667887 Mon Sep 17 00:00:00 2001 From: Rhonda Glennon Date: Fri, 13 May 2016 11:52:12 -0700 Subject: [PATCH 7/9] heading formatting --- response.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/response.md b/response.md index 138c6d9..ff85bed 100644 --- a/response.md +++ b/response.md @@ -67,7 +67,7 @@ All places in Mapzen Search have a global identifier, known as a `gid`. Each mat The `gid` consists of a `layer` (such as `address` or `country`), an identifier for the original data source (`openstreetmap` or `openaddresses`), and an `id` for the individual record corresponding to the original source idenfier, where possible. This information is also available as properties on the individual results as `layer`, `source`, and `source_id`. -:rotating_light: Follow these guidelines regarding the `gid`: +#### :warning: Follow these guidelines regarding the `gid`: - You should not create your own `gid` strings. - `gid` values may not be consistent across releases. From 5b6397215b0a5c4cf12c167bccf10a1924121ab3 Mon Sep 17 00:00:00 2001 From: Julian Simioni Date: Fri, 13 May 2016 15:35:48 -0400 Subject: [PATCH 8/9] Remove ids page Honestly, just the notes on the responses page are enough. We can always bring this back. --- ids.md | 69 ---------------------------------------------------------- 1 file changed, 69 deletions(-) delete mode 100644 ids.md diff --git a/ids.md b/ids.md deleted file mode 100644 index e958991..0000000 --- a/ids.md +++ /dev/null @@ -1,69 +0,0 @@ -# IDs in Mapzen Search - -Depending on how you use the service, system identifiers can play play a big part in how you use Mapzen Search / Pelias. All places in Mapzen Search have a _global identifier_, known as a `gid`. Each matching record returned from a [`/search`](search), [`/autocomplete`](autocomplete), or [`/reverse`](reverse) geocoding request will have a `gid` field that corresponds to that place's information in the system. - -Identifiers in Mapzen Search may change from week-to-week. They should not be considered stable, unless otherwise noted in the sources section below. Because of the nature of some of the data sources we work with, IDs may change without notice at any given time. - -A record's `gid`'s in Mapzen Search looks like: -`"gid": "whosonfirst:locality:101750367",` - - -## Rules for using `gid`'s - -**1. Do not use Mapzen Search `gid`'s as permanent identifiers for places.** - -`gid`'s in Mapzen Search can change because several of our data sources also lack permanent IDs. Some of our sources do not provide ways to track changes between each release of the data. This is particularly true for OpenAddresses, where there are no general ID's for each address. With OpenStreetMap it is possible for an entry to be superseded by another place, but for there to be no indicator of what the replacement place is. - -**2. The structure of identifiers in Mapzen Search may change.** -This is not necessarily the final configuration of ID's in Mapzen Search. We reserve the right change their structure at some time in the future. - -**3. The Mapzen Search ID for a place may not be the same exact ID in other systems.** -While some of our sources do have persistent ID's (e.g. Node/Way ID's in OpenStreetMap or all Who's on First ID's), we augment them to fit into our taxonomy and add the source dataset. While the numeric component of a Mapzen Search record coming from Who's on First can be used as its ID, you can't just use a Who's on First ID to get the record from Mapzen Search. - - -## Source-specific `gid` structures - - -### _Who's on First_ | `whosonfirst` - -`"gid": "whosonfirst:locality:101750367"` - -A Who's on First identifier consists of 3 elements, it's source (`whosonfirst`), its [placetype](https://github.com/whosonfirst/whosonfirst-placetypes), and the number that is its [unique 64-bit numeric ID](https://github.com/whosonfirst/whosonfirst-data/#portability). - -[Placetypes](https://github.com/whosonfirst/whosonfirst-placetypes) are consistent between Mapzen Search and Who's on First. - -The numeric identifier at the end of a Who's on First `gid` _may be considered a stable identifier_ for the corresponding Who's on First record, and is likely to be used as an identifier for other systems using Who's on First. However, Mapzen Search requires that a record's type be known before the record may be retrieved from `/place`. - -> ✅ Valid: [`/v1/place?ids=whosonfirst:country:85633793`](http://pelias.github.io/compare/#/v1/place%3Fids=whosonfirst:country:85633793) - -> ❌ Invalid (missing placetype): [`/v1/place?ids=whosonfirst:85633793`](http://pelias.github.io/compare/#/v1/place%3Fids=whosonfirst:85633793) - -### OpenStreetMap | `openstreetmap` - - -Results coming from OpenStreetMap take the form of: -`"gid": "openstreetmap::id:"` - -There's a lot to unpack about that. - -``: corresponds to the core OpenStreetMap data type for that place, or [Element](http://wiki.openstreetmap.org/wiki/Elements). These are the geometries that data about a place is attached to in OpenStreetMap. There are 3 elements in OpenStreetMap: -* [Nodes](http://wiki.openstreetmap.org/wiki/Node), which are points corresponding to a latitude and longitude on earth. Nodes have positive numbers as their ID numbers in OpenStreetMap. -* [Ways](http://wiki.openstreetmap.org/wiki/Way), which are ordered collections of points that can make up a line or a polygon -* [Relations](http://wiki.openstreetmap.org/wiki/Relation), which are arbitrary collections of points, lines, and relations. - - - -Records coming from OpenStreetMap can take the form of - -### OpenAddresses | `openaddresses` - -### Geonames | `geonames` - - - - -## How we use `gid`'s in Mapzen Search - -### Why does each record have multiple multiple fields with `gid`? - -### Why does each record have an `id` field in addition to `gid`? From fe1e82fa84d1c8260b6e47d3022aa920c6cfe253 Mon Sep 17 00:00:00 2001 From: Rhonda Glennon Date: Fri, 13 May 2016 12:48:46 -0700 Subject: [PATCH 9/9] adding a few changes from another branch --- response.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/response.md b/response.md index ff85bed..a21ccea 100644 --- a/response.md +++ b/response.md @@ -1,12 +1,12 @@ # Search results -When requesting results from Mapzen Search, you will always get back `GeoJSON` results, unless something goes terribly wrong, in which case you'll get an error message. +When requesting results from Mapzen Search, you will always get back GeoJSON results, unless something goes terribly wrong, in which case you'll get an error message. Tip: You can go to http://geojson.org/geojson-spec.html to learn more about the GeoJSON data format specification. The top-level structure to every response looks like this: -```json +``` { "geocoding":{...}, "type":"FeatureCollection", @@ -15,13 +15,13 @@ The top-level structure to every response looks like this: } ``` -##List of `features` returned +##List of features returned The `features` property of the result is where you will find the list of results that best matched your input parameters. Each item in this list will contain all the information needed to find it in human-readable format in the `properties` block, as well as computer friendly coordinates in the `geometry` property. -``` json +``` { "type": "Feature", "geometry": { @@ -58,20 +58,20 @@ Each item in this list will contain all the information needed to find it in hum }, ``` -Additionally, `/reverse` queries will have a `distance` parameter, which is the distance, in meters, from the query point. +Additionally, [/reverse](reverse.md) queries will have a `distance` parameter, which is the distance, in meters, from the query point. -## Notable elements +## Notable features ### `gid` -All places in Mapzen Search have a global identifier, known as a `gid`. Each matching record returned from a [`/search`](search), [`/autocomplete`](autocomplete), or [`/reverse`](reverse) geocoding request has a `gid` field. +All places in Mapzen Search have a global identifier, known as a `gid`. Each matching record returned from a [/search](search.md), [/autocomplete](autocomplete.md), or [/reverse](reverse.md) geocoding request has a `gid` field. -The `gid` consists of a `layer` (such as `address` or `country`), an identifier for the original data source (`openstreetmap` or `openaddresses`), and an `id` for the individual record corresponding to the original source idenfier, where possible. This information is also available as properties on the individual results as `layer`, `source`, and `source_id`. +The `gid` consists of a `layer` (such as `address` or `country`), an identifier for the original data source (such as `openstreetmap` or `openaddresses`), and an `id` for the individual record corresponding to the original source idenfier, where possible. This information is also available as properties on the individual results as `layer`, `source`, and `source_id`. #### :warning: Follow these guidelines regarding the `gid`: - You should not create your own `gid` strings. -- `gid` values may not be consistent across releases. -- You should not attempt to parse `gid` strings for information or store them for future use. You should only use `gid` at the time when you receive the search results. One valid use for the `gid` is to retrieve full details on a particular result from the `/place` endpoint. +- `gid` strings may not be consistent across releases. +- You should not attempt to parse `gid` strings for information or store them for future use. You should only use `gid` at the time when you receive the search results. One valid use for the `gid` is to retrieve full details on a particular result from the [/place](place.md) endpoint. ### `label` The `label` is a human-friendly representation of the place, ready to be displayed to an end user. The label field attempts to use a format that is right for the region the result is in, although Mapzen Search only supports a few countries at the moment. @@ -79,14 +79,14 @@ The `label` is a human-friendly representation of the place, ready to be display ### `confidence` The confidence score is an estimation of how accurately this result matches the query. -For the `/reverse` endpoint, the confidence score is determined solely by its distance from the coordinate specified. Closer results get a higher score. +For the [/reverse](reverse.md) endpoint, the confidence score is determined solely by its distance from the coordinate specified. Closer results get a higher score. -For the `/search` endpoint, it primarily takes into account how well properties in the result match what was expected from parsing the input text. For example, if the input text looks like an address, but the house number of the result doesn't match the house number that was parsed from the input text, the confidence score will be lower. +For the [/search](search.md) endpoint, it primarily takes into account how well properties in the result match what was expected from parsing the input text. For example, if the input text looks like an address, but the house number of the result doesn't match the house number that was parsed from the input text, the confidence score will be lower. Additionally, the confidence score can optionally be biased along with other results, like test scores in a classroom might be graded on a curve. This takes into account both the property matches described above and the distance between results. This relative scoring is enabled on Mapzen Search, but can be turned off when hosting your own Pelias instance. ### `bbox` -`Feature`s coming from _Who's on First_ and OpenStreetMap ways will often have their own `"bbox"` elements. This `"bbox"` is at the same level as `"properties"`. If present, it describes the geographic extent of the feature (e.g. the screen size necessary to show all of California without needing to send the precise polygon geometry). This should be treated as separate from the `bbox` that describes the entire `FeatureCollection`. +Features from Who's on First and OpenStreetMap often have their own `bbox` elements. This `bbox` is at the same level as `properties`. If present, it describes the geographic extent of the feature, such as the screen size necessary to show all of California without needing to send the precise polygon geometry. This should be treated as separate from the `bbox` that describes the entire `FeatureCollection`. ## Result count