Previously, our text sanitizer code did not trim whitespace before
checking that the string was non-empty. This lead to strings consisting
only of whitespace being treated as valid. Not all our downstream
services (such as libpostal) accept whitespace-only input, so this
causes a rather harsh error.
This PR builds upon the code in https://github.com/pelias/api/pull/1170
and moves the trimming code above the nonEmptyString check. Now, a
whitespace-only input string produces the normal error for empty input.
Fixes https://github.com/pelias/api/issues/1158
The parameter in clean.types being set by `helpers/text_parser.js` was
"from_address_parsing", but the code in `helper/types.js` was expecting
"from_address_parser". This commit makes both use "from_address_parser"
and adds a test.
The address parser currently does two things:
1.) make some intelligent guesses as to possible admin regions to
explicitly search against to improve the quality of results returned
2.) make some intelligent guesses as to when no part of the query needs
to search against anything other than admin regions. This somewhat
improves the quality of results returned but mostly improves the speed
of the Elasticsearch query since it's searching significantly fewer
recoords.
These two concerns are now split into two separate methods within the
query_parser helper module. They are mostly independent today, but don't
have to be in the future.
Modifying these sanitiser tests became extremely hard because almost all
of them were looping over lots of individual test cases, which places
assumptions about the common behavior of potentialy very different test
cases, as well as making assertions about huge swaths of output when
only a small amount of that output was really under test.
Hopefully these changes will make our tests easier to modify, and not
really lose any ability to catch bugs.
The address parser currently does two things:
1.) make some intelligent guesses as to possible admin regions to
explicitly search against to improve the quality of results returned
2.) make some intelligent guesses as to when no part of the query needs
to search against anything other than admin regions. This somewhat
improves the quality of results returned but mostly improves the speed
of the Elasticsearch query since it's searching significantly fewer
recoords.
These two concerns are now split into two separate methods within the
query_parser helper module. They are mostly independent today, but don't
have to be in the future.
Modifying these sanitiser tests became extremely hard because almost all
of them were looping over lots of individual test cases, which places
assumptions about the common behavior of potentialy very different test
cases, as well as making assertions about huge swaths of output when
only a small amount of that output was really under test.
Hopefully these changes will make our tests easier to modify, and not
really lose any ability to catch bugs.
Julian Simioni
Peter Johnson
Lily He
Stephen Hess
Harish Krishna
Harish Krishna