Julian Simioni
a7932d0b8c
feat(log): move retry info to structured logs
6 years ago
Julian Simioni
d681a114d6
feat(log): Remove most unstructured controller logs
...
These are nose now, the structured logs have much better info
6 years ago
Julian Simioni
c91fd58ec6
fix(log): replace brittle string extraction
6 years ago
Julian Simioni
8c37ee63dd
feat(log): Add JSON log for Elasticsearch queries
...
This adds a structured and detailed log line for each Elasticsearch
query.
It includes information like the total number of Elasticsearch hits, how
long Elasticsearch took to process the request, query parameters, etc.
This is extremely useful for later analysis as the structured nature of
the query allows for powerful filtering.
6 years ago
Lily He
9405a8f370
correct timer timings for controller debug logs
7 years ago
Lily He
12a1dcba08
update controller debug timers to new format
7 years ago
Stephen Hess
eb4a896272
removed debug
7 years ago
Stephen Hess
568a258406
code cleanup, shorter function syntax
7 years ago
Stephen Hess
147d76f9d7
added controller for searching with ids
7 years ago
Stephen Hess
52a6ec68ff
added support for coarse reverse geocoding
...
several things are in this commit:
- coarse reverse controller / pip service
- inject "conditional execute" predicate into search service to shortcut execution conditions
- added coarse reverse controller to precede standard reverse controller
- lots of tests!
8 years ago
Stephen Hess
6918d866b8
added comments clarifying retry logic
8 years ago
Stephen Hess
73ef71d863
moved helper functions outside of setup scope
8 years ago
Stephen Hess
5148198e1d
added helper functions
8 years ago
Stephen Hess
ece98a2ae6
converted to let/const
8 years ago
Stephen Hess
a4a6a691c8
cleaned up log message parts
8 years ago
Stephen Hess
96f9d12ff5
rewrote tests to proxyquire service/search to reduce complexity
...
added a few more tests for coverage, removed unused things from unit/mock/backend
8 years ago
Stephen Hess
fbe0de386e
added ending ']'
8 years ago
Stephen Hess
bf62a1844b
added tests for verifying request retry behavior
...
limited retry behavior to errors with status 408 (request timeout). this also reduces reliance on unit/mock/backend and unit/mock/query.
8 years ago
Stephen Hess
995b6109e3
removed used variable
8 years ago
Stephen Hess
0121d3db2a
get # of retries from API config instead of ES client
8 years ago
Stephen Hess
67bba11001
removed object indirection in favor of function for easier proxyquire
8 years ago
Stephen Hess
e83c12cd0e
added test for early bail when errors are present
...
simplified conditionals using lodash `.get`
8 years ago
Stephen Hess
40e6b1b3b7
renamed variable for clarity
8 years ago
Stephen Hess
fe0457cb8b
removed maxRetries default
8 years ago
Stephen Hess
4ed4b8b357
added retry support for ES requests that timeout
8 years ago
Stephen Hess
8c249c26f4
initialize esclient in routes/v1, eliminating need for src/backend
8 years ago
Stephen Hess
00234265e6
always inject backend instead of require'ing inside
8 years ago
Diana Shkolnikov
ec91f4055f
add controller logging
8 years ago
Peter Johnson
1451e9038e
change all log labels to 'api'
8 years ago
Diana Shkolnikov
9ff383cc2b
change the query module interfaces back to simple functions
8 years ago
Diana Shkolnikov
3888ae0385
change query module interfaces to allow for additional meta data
8 years ago
Stephen Hess
cc5c9f8468
disable calling GeodisambiguationQuery
8 years ago
Stephen Hess
2a9c25f8c4
added condition that `res.data` must be non-empty to return early
8 years ago
Stephen Hess
4952a0ed2b
added condition to exit early if there are already results in `res`
...
in order to accommodate falling back to the existing search strategy, the search controller must exit early if the `res.data` already exists.
8 years ago
Julian Simioni
2557c68368
Use index name from config in controllers
8 years ago
Alec Vulfson
31adf09870
Using new layers view to filter layers
8 years ago
Julian Simioni
23ff51fd57
Add custom access_log handlers that remove user info
...
These handlers will remove sensitive query info from the logs when DNT
is enabled
9 years ago
Julian Simioni
3c1d758229
Fix timeout error handling
...
Timeout errors come back from the Elasticsearch module as Error objects,
so instead of pushing the entire object onto the errors array, we should
just push the message.
I'm not sure if there are other cases where errors are just strings,
errors that aren't objects with a message property are handled as
before.
9 years ago
Vesa Meskanen
a2e96987fc
Remove expensive JSON.stringify from api debug logging calls
9 years ago
Julian Simioni
79883594b8
Use clean.layers to determine types
...
This allows removing the types middleware and simplifies some code.
9 years ago
Julian Simioni
ef78b2e0fd
Filter source and layer by new fields
...
This is functionally the same code as before, except that it uses the
new source and layer mapping.
9 years ago
Vesa Meskanen
899d520cd7
Debug log full queries and responses between Pelias and Elasticsearch
9 years ago
Diana Shkolnikov
42d940f8c8
Add simple normalizer (lowercase + remove punctuation)
9 years ago
Diana Shkolnikov
54187dde67
Add dedupe middleware
...
Dedupe middleware removes __exact__ dupes and truncates the results
to the specified size.
9 years ago
Peter Johnson
c609ae3ccf
http friendly status codes. resolves #321
9 years ago
Diana Shkolnikov
2b22d611bd
Fix lost confidence score
9 years ago
Peter Johnson
348d2e70e2
more conformance
9 years ago
Peter Johnson
7c46aed4a7
expose error messages in geojson
9 years ago
Peter Johnson
6ab81ffca5
set response data/meta on instead of
9 years ago
Julian Simioni
f6fea21782
Use hasOwnProperty instead of comparing against undefined
9 years ago