Julian Simioni
7e4559fdc2
fix(sanitizer): Trim whitespace in addressit queries
...
This is a followup PR to https://github.com/pelias/api/pull/1171 and
https://github.com/pelias/api/pull/1170 .
Apparently we have two different `text` sanitizers, and autocomplete
queries were treating a single space as valid input. This had a
particularly bad outcome as it would end up generating queries (see an
[example](https://gist.github.com/orangejulius/2cc26c7eed39311b6eaf1fb0175c13e6 )) that had no main query clause.
This caused them to match basically every document in the index. Looking
at the geocode.earth slowlog, these queries took **__8 seconds per
shard__**.
6 years ago
Julian Simioni
40e6054523
chore(geo_common): assert that a specific exception was thrown
...
Without checking the message when asserting an exception is thrown, it's
possible for the test to pass when undesired behavior is occuring.
6 years ago
Julian Simioni
1553dfb103
fix(geo_common): check for invalid bbox where min=max
...
This condition will cause Elasticsearch to throw an error, we should
catch it outselves first.
The error is more friendly than the case where min>max, but still an
error.
Connects https://github.com/pelias/api/pull/1050
6 years ago
Julian Simioni
76bc5c654d
fix(geo_common): check bbox parameters are within range
...
If bounding box lat/lon values are outside the correct range,
Elasticsearch throws very alarming errors.
With a little validation code we can provide more friendly and
actionable error messages.
Fixes https://github.com/pelias/pelias/issues/750
6 years ago
Julian Simioni
76d88b62b4
fix(tests): Sanitizer tests should use faked config
...
Otherwise a user's own pelias.json settings can cause tests to fail
6 years ago
Horace Williams
c7ae37980d
Remove Quattroshapes deprecation notice
...
Fixes #1132
The Quattroshapes source has been deprecated from
some time now, and any users for whom the deprecation
notice was helpful have presumably either upgraded
or moved on by now.
7 years ago
Julian Simioni
9a0f182fb2
fix(whitespace): Trim whitespace and quotes before checking text length
...
Previously, our text sanitizer code did not trim whitespace before
checking that the string was non-empty. This lead to strings consisting
only of whitespace being treated as valid. Not all our downstream
services (such as libpostal) accept whitespace-only input, so this
causes a rather harsh error.
This PR builds upon the code in https://github.com/pelias/api/pull/1170
and moves the trimming code above the nonEmptyString check. Now, a
whitespace-only input string produces the normal error for empty input.
Fixes https://github.com/pelias/api/issues/1158
7 years ago
Peter Johnson
d38d4b1fa8
feat(text_sanitizer): trim whitespace and quotation marks from a range of natural languages
7 years ago
Julian Simioni
204f5297de
Remove all leading newlines in files
7 years ago
Julian Simioni
02aac0c13e
Remove use stricts
7 years ago
Joxit
551f0227db
Transliterates Saint and Sainte in ES schema see pelias/schema#268
7 years ago
Joxit
d187e7d950
Fix for #1077 , fail to search `Saint..` cities in structured queries
7 years ago
Rhys Kidd
1dfc6988ed
Update to current Pelias contact email address
...
Per https://github.com/pelias/pelias/tree/master/announcements/2018-01-02-pelias-update
replace all @mapzen.com email addresses with the current Pelias contact email address.
grep used to confirm no '@mapzen.com' remain.
7 years ago
Stephen Hess
b5e48afb35
remove text-analyzer address operation
7 years ago
Peter Backx
f16565df9e
Verify min and max longitude and latitude
...
Otherwise this can potentially lead to a fairly weird Elasticsearch error mentioned in https://github.com/pelias/pelias/issues/656
7 years ago
Julian Simioni
bc6401842c
Add friendly, coarse specific error message to reverse
...
This error message is updated to mention the coarse-reverse only nature
of Geonames support removal. It also links to the
[tracking ticket](https://github.com/pelias/acceptance-tests/pull/447 )
for Geonames removal which should help provide more context to users.
7 years ago
Julian Simioni
d577d42d9f
Support geonames non-coarse reverse in deprecation sanitizer
7 years ago
Stephen Hess
18e7673dc1
added empire to coarse translation
7 years ago
Stephen Hess
e70a668056
added support for continent, ocean, and marinearea placetypes
...
`coarse` now maps to these as well
7 years ago
Stephen Hess
42097fafb2
treat undefined text-analyzer response as no house number available
7 years ago
Stephen Hess
30b7290990
added stub request_language sanitizer that just allows lang param
7 years ago
Lily He
cafb7d9277
created new debug helper class and debug sanitizer
8 years ago
Lily He
bf234f5539
flag_bool expects parameter opts.paramName
8 years ago
Lily He
a5cf44bef4
clarify _text sanitizer test message
8 years ago
Lily He
8e45af1e49
adapt defer_to_addressit to new sanitizer format
8 years ago
Lily He
db56ce45c3
remove exporting sanitize for testing
8 years ago
Lily He
9a88d5a28b
unmerge nearby sanitizers from reverse sanitizer_list
8 years ago
Lily He
07c7e7d483
test reverse, nearby, place sanitizer wrappers w/ proxyquire
8 years ago
Lily He
80a3a259ef
remove callback in sanitizeAll.runAllChecks for sync processing
8 years ago
Lily He
a27ac0fc88
use strict for ecmascript 6 syntax, fix typos
8 years ago
Lily He
b9de4b6788
additional test to verify runAllCheck works for sanitizeAll
8 years ago
Lily He
279b70b7d7
typo: delete mentions of joi in sanitizers
8 years ago
Lily He
670c673419
call runAllChecks for nearby, reverse, and place sanitizing
8 years ago
Lily He
1f527be3e3
call runAllChecks for search, search_fallback, autocomplete & structured sanitizing
8 years ago
Lily He
af906c05a8
sanitizeAll validates req.query parameters, export async runAllChecks function
8 years ago
Lily He
a683e27a87
removed 'parsed_text' from list of expected params for _text
8 years ago
Lily He
368c0aa4cb
added expected parameters list to _text_addressit
8 years ago
Lily He
13a7b42de7
Standardize sanitizers to export setup function that returns sanitize and expected function
8 years ago
Stephen Hess
2b70b41a01
move _text sanitizer
8 years ago
Stephen Hess
85b831b161
removed iso2_to_iso3 and city_name_standardizer from wrapper
8 years ago
Stephen Hess
5dc9737caa
removed text-analyzer call from _text sanitizer
8 years ago
Stephen Hess
40ddc93bbf
added conditional fallback to addressit
8 years ago
Stephen Hess
30cd30236a
don't call libpostal if sources=whosonfirst
8 years ago
Stephen Hess
ac7f5c445c
moved libpostal lower in sanitizers to after sources
8 years ago
Lily He
57b133ba42
check and set default for undefined boundary circle radius in reverse
8 years ago
Lily He
5504240016
prevent overriding raw.focus.point.lat/lon
8 years ago
Lily He
40ca17b886
passed down peliasConfig.api schema to sanitizers
8 years ago
Lily He
2e2ae9b005
added location_bias sanitizer to autocomplete, search, and structured_geocoding
8 years ago
Lily He
f67664c3e0
location bias feature - sets default focus.point
8 years ago
Stephen Hess
d788d6a491
always set boundary.circle regardless of layers
8 years ago