Stephen Hess
73ef71d863
moved helper functions outside of setup scope
8 years ago
Stephen Hess
5148198e1d
added helper functions
8 years ago
Stephen Hess
ac0776e2be
match error handling to controller/search
8 years ago
Stephen Hess
9da4b9d236
standardized on name to match controller/search
8 years ago
Stephen Hess
ece98a2ae6
converted to let/const
8 years ago
Stephen Hess
6425cc6444
added request retries for ES request timeouts
8 years ago
Stephen Hess
a4a6a691c8
cleaned up log message parts
8 years ago
Stephen Hess
96f9d12ff5
rewrote tests to proxyquire service/search to reduce complexity
...
added a few more tests for coverage, removed unused things from unit/mock/backend
8 years ago
Stephen Hess
fbe0de386e
added ending ']'
8 years ago
Stephen Hess
bf62a1844b
added tests for verifying request retry behavior
...
limited retry behavior to errors with status 408 (request timeout). this also reduces reliance on unit/mock/backend and unit/mock/query.
8 years ago
Stephen Hess
995b6109e3
removed used variable
8 years ago
Stephen Hess
0121d3db2a
get # of retries from API config instead of ES client
8 years ago
Stephen Hess
67bba11001
removed object indirection in favor of function for easier proxyquire
8 years ago
Stephen Hess
e83c12cd0e
added test for early bail when errors are present
...
simplified conditionals using lodash `.get`
8 years ago
Stephen Hess
40e6b1b3b7
renamed variable for clarity
8 years ago
Stephen Hess
fe0457cb8b
removed maxRetries default
8 years ago
Stephen Hess
4ed4b8b357
added retry support for ES requests that timeout
8 years ago
Stephen Hess
8c249c26f4
initialize esclient in routes/v1, eliminating need for src/backend
8 years ago
Stephen Hess
00234265e6
always inject backend instead of require'ing inside
8 years ago
Stephen Hess
de2ff8cad7
removed debug
8 years ago
Diana Shkolnikov
ec91f4055f
add controller logging
8 years ago
Peter Johnson
1451e9038e
change all log labels to 'api'
8 years ago
Diana Shkolnikov
9ff383cc2b
change the query module interfaces back to simple functions
8 years ago
Diana Shkolnikov
3888ae0385
change query module interfaces to allow for additional meta data
8 years ago
Stephen Hess
cc5c9f8468
disable calling GeodisambiguationQuery
8 years ago
Stephen Hess
2a9c25f8c4
added condition that `res.data` must be non-empty to return early
8 years ago
Stephen Hess
4952a0ed2b
added condition to exit early if there are already results in `res`
...
in order to accommodate falling back to the existing search strategy, the search controller must exit early if the `res.data` already exists.
8 years ago
Julian Simioni
2557c68368
Use index name from config in controllers
8 years ago
Alec Vulfson
31adf09870
Using new layers view to filter layers
8 years ago
Julian Simioni
23ff51fd57
Add custom access_log handlers that remove user info
...
These handlers will remove sensitive query info from the logs when DNT
is enabled
9 years ago
Julian Simioni
3c1d758229
Fix timeout error handling
...
Timeout errors come back from the Elasticsearch module as Error objects,
so instead of pushing the entire object onto the errors array, we should
just push the message.
I'm not sure if there are other cases where errors are just strings,
errors that aren't objects with a message property are handled as
before.
9 years ago
Vesa Meskanen
a2e96987fc
Remove expensive JSON.stringify from api debug logging calls
9 years ago
Julian Simioni
79883594b8
Use clean.layers to determine types
...
This allows removing the types middleware and simplifies some code.
9 years ago
Julian Simioni
ef78b2e0fd
Filter source and layer by new fields
...
This is functionally the same code as before, except that it uses the
new source and layer mapping.
9 years ago
Vesa Meskanen
269dff9dd1
Add query/response logging in debug mode to place controller, too
9 years ago
Vesa Meskanen
899d520cd7
Debug log full queries and responses between Pelias and Elasticsearch
9 years ago
Diana Shkolnikov
42d940f8c8
Add simple normalizer (lowercase + remove punctuation)
9 years ago
Diana Shkolnikov
54187dde67
Add dedupe middleware
...
Dedupe middleware removes __exact__ dupes and truncates the results
to the specified size.
9 years ago
Julian Simioni
0e8f4253c0
Return multiple results in place when osm node and way share an id
...
It turns out the _type parameter to the Elasticsearch
[multiget](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-multi-get.html )
API does not allow an array of possible values. We were depending on its
ability to search multiple types to allow searching for OSM nodes and
ways.
But, since this doesn't work we essentially have to do it ourselves.
There is also the problem that OSM nodes and ways share an ID space. So
a gid such as `osm:venue:5` could in theory correspond to 2 records.
It seems like the only nice thing to do in that case is return both
results.
This PR "unrolls" such queries. For example, in the case of
`osm:venue:5`, the sanitisers will return the following array of objects
to be turned into multiget queries:
```
[{
id: 5,
types: ["osmway", "osmnode"]
}]
```
Before, this would turn into a multiget query with only one entry, like
this:
```
{
"docs": [
{
"_index": "pelias",
"_type": [
" osmnode",
"osmway"
],
"_id": 5
}
]
}
```
now it would look like this:
{
"docs": [
{
"_index": "pelias",
"_type": "osmnode",
"_id": 5
},
{
"_index": "pelias",
"_type": "osmnode",
"_id": 5
}
]
}
TLDR you might get back more records from /place than the number of ids
you specified, but at least you will definitely get back what you are
looking for.
9 years ago
Peter Johnson
c609ae3ccf
http friendly status codes. resolves #321
9 years ago
Stephen Hess
cb7d22df91
switched ES lookup field from "type" -> "_type", fixes issue with /place lookup on Eiffel Tower gid
9 years ago
Julian Simioni
a2cc31f142
Leave a note for future us
...
A small essay on Elasticsearch queries and fields and types.
9 years ago
Julian Simioni
2505e92a62
Expect an array of types from _ids sanitiser
...
This doesn't have any effect by itself but allows for the 3-part gid
sanitiser to possibly return multiple types (i.e. in the case of
osm:venue:1000)
9 years ago
Diana Shkolnikov
2b22d611bd
Fix lost confidence score
9 years ago
Peter Johnson
348d2e70e2
more conformance
9 years ago
Peter Johnson
7c46aed4a7
expose error messages in geojson
9 years ago
Peter Johnson
6ab81ffca5
set response data/meta on instead of
9 years ago
Peter Johnson
e726636c3d
merge with master
9 years ago
Julian Simioni
5d7f7ee4c0
Add /status controller
...
It's at the root level, and simply returns 200 with 'status: ok' as the
body.
9 years ago
Julian Simioni
f6fea21782
Use hasOwnProperty instead of comparing against undefined
9 years ago