Julian Simioni
7e4559fdc2
fix(sanitizer): Trim whitespace in addressit queries
...
This is a followup PR to https://github.com/pelias/api/pull/1171 and
https://github.com/pelias/api/pull/1170 .
Apparently we have two different `text` sanitizers, and autocomplete
queries were treating a single space as valid input. This had a
particularly bad outcome as it would end up generating queries (see an
[example](https://gist.github.com/orangejulius/2cc26c7eed39311b6eaf1fb0175c13e6 )) that had no main query clause.
This caused them to match basically every document in the index. Looking
at the geocode.earth slowlog, these queries took **__8 seconds per
shard__**.
6 years ago
Stephen Hess
6fd3c011fb
remove unused require
7 years ago
Stephen Hess
1e003efe28
used lodash.extend instead of extend module
7 years ago
Lily He
368c0aa4cb
added expected parameters list to _text_addressit
8 years ago
Lily He
13a7b42de7
Standardize sanitizers to export setup function that returns sanitize and expected function
8 years ago
Stephen Hess
40ddc93bbf
added conditional fallback to addressit
8 years ago
Stephen Hess
3763d0b503
renamed `sanitise` -> `sanitize`
8 years ago
Stephen Hess
768843b0fb
delete `clean.parsed_text` if falling back to addressit
8 years ago
Stephen Hess
46d58cafb0
renamed `_text_autocomplete` to `_text_addressit`
8 years ago
Stephen Hess
b612b2750e
added addressit support for autocomplete via separatee text sanitiser
...
this commit combines the other sanitiser/_text.js and addressit parser logic from text-analyzer into one module for easier integration until such time that libpostal is ready for autocomplete
8 years ago