pelias/api - api - CogtoSrc

Author	SHA1	Message	Date
Julian Simioni	9a0f182fb2	fix(whitespace): Trim whitespace and quotes before checking text length Previously, our text sanitizer code did not trim whitespace before checking that the string was non-empty. This lead to strings consisting only of whitespace being treated as valid. Not all our downstream services (such as libpostal) accept whitespace-only input, so this causes a rather harsh error. This PR builds upon the code in https://github.com/pelias/api/pull/1170 and moves the trimming code above the nonEmptyString check. Now, a whitespace-only input string produces the normal error for empty input. Fixes https://github.com/pelias/api/issues/1158	6 years ago
Peter Johnson	d38d4b1fa8	feat(text_sanitizer): trim whitespace and quotation marks from a range of natural languages	6 years ago
Lily He	a683e27a87	removed 'parsed_text' from list of expected params for _text	7 years ago
Lily He	13a7b42de7	Standardize sanitizers to export setup function that returns sanitize and expected function	7 years ago
Stephen Hess	5dc9737caa	removed text-analyzer call from _text sanitizer	7 years ago
Stephen Hess	30cd30236a	don't call libpostal if sources=whosonfirst	7 years ago
Stephen Hess	3763d0b503	renamed `sanitise` -> `sanitize`	8 years ago
Stephen Hess	c18baaf775	fixed text analysis tests proxyquire is now used because the text_analyzer package requires node_postal which isn't guaranteed to be available	8 years ago
Stephen Hess	bd3b8e7bbe	switched to pelias-text-analyzer package since that responsibility has been moved to there	9 years ago
Julian Simioni	ef78b2e0fd	Filter source and layer by new fields This is functionally the same code as before, except that it uses the new source and layer mapping.	9 years ago
Julian Simioni	cb7b38bc75	Replace check-types depricated unemptyString with nonEmptyString See https://www.npmjs.com/package/check-types#what-changed-from-4x-to-5x	9 years ago
Julian Simioni	db895c05b7	Rename `from_address_parser` key to `from_text_parser`	9 years ago
Julian Simioni	b188a2046c	Fix mismatching name of types preference parameter The parameter in clean.types being set by `helpers/text_parser.js` was "from_address_parsing", but the code in `helper/types.js` was expecting "from_address_parser". This commit makes both use "from_address_parser" and adds a test.	9 years ago
Julian Simioni	59b70f7c7e	Rename helper/query_parser to helper/text_parser	9 years ago
Diana Shkolnikov	553f9780c5	Remove admin matching when address is not parsed	9 years ago
Julian Simioni	66e1cab007	Whitespace	9 years ago
Peter Johnson	e726636c3d	merge with master	9 years ago
Peter Johnson	10242a8682	rename unclean->raw	9 years ago
Peter Johnson	bf42290c7c	refactor sanitizers: first pass	9 years ago
Peter Johnson	22322eaa32	change param:input to param:text	9 years ago
Harish Krishna	86424b8bbe	Rename `/search?input`, replacing it with '/search?text' parameter instead changing parameter name input to text https://github.com/pelias/api/issues/172 Rebased by @orangejulius	9 years ago
Julian Simioni	7a6ac8541b	Separate concerns of address parser The address parser currently does two things: 1.) make some intelligent guesses as to possible admin regions to explicitly search against to improve the quality of results returned 2.) make some intelligent guesses as to when no part of the query needs to search against anything other than admin regions. This somewhat improves the quality of results returned but mostly improves the speed of the Elasticsearch query since it's searching significantly fewer recoords. These two concerns are now split into two separate methods within the query_parser helper module. They are mostly independent today, but don't have to be in the future.	9 years ago
Julian Simioni	9b94a0e4ef	Fix inconsistently named imported module	9 years ago
Julian Simioni	e71fb7cfd4	Whitespace	9 years ago
Julian Simioni	b9152dbe29	Separate concerns of address parser The address parser currently does two things: 1.) make some intelligent guesses as to possible admin regions to explicitly search against to improve the quality of results returned 2.) make some intelligent guesses as to when no part of the query needs to search against anything other than admin regions. This somewhat improves the quality of results returned but mostly improves the speed of the Elasticsearch query since it's searching significantly fewer recoords. These two concerns are now split into two separate methods within the query_parser helper module. They are mostly independent today, but don't have to be in the future.	9 years ago
Julian Simioni	4704583178	Fix inconsistently named imported module	9 years ago
Julian Simioni	fb0cf514e9	Whitespace	9 years ago
Harish Krishna	495c3251a4	moving all query parsing logic into its own helper. Tests to come	9 years ago
Harish Krishna	a8c090252a	if the query has a number - search addresses	9 years ago
Harish Krishna	346f187c3e	updateing isNumber regex	9 years ago
Harish Krishna	e1f3cfa0d3	no need to hit address parser if the input has just one token or two and no numbers Ex: new, new york, minneapolis etc dont need address parsing but 'starbucks 10010' might need it	9 years ago
Harish Krishna	ebd4a5a08a	splitOnDelim first followed by a space tokenizer	9 years ago
Harish Krishna	a4d09f8e12	no address parsing required if there are only 2 or less tokens	9 years ago
Harish Krishna	bdffe758d4	commenting out the overkill	9 years ago
Harish Krishna	96a21570be	fixing layers related bug - adding a default_layers_set flag to clean obj.	9 years ago
Harish Krishna	042772cf3b	only do address parsing when necessary	9 years ago
Harish Krishna	1d24890b27	introducing target_layer that gets set part of input_parsing - if input is less than 3 characters -> only hit admin layers - if input suggests that its not a street address -> avoid hitting address layers	9 years ago
Harish Krishna	5accecfa6c	adding postcode support for usa	10 years ago
Harish Krishna	46cc1a6569	disabling parse-address	10 years ago
Harish Krishna	0b5b1dce85	address parser initial pass - breaks 68 tests! (ignoring tests for now)	10 years ago
Harish Krishna	176f580215	using is-object	10 years ago
Harish Krishna	d2d76be8fa	changing a commented line to reflect reality	10 years ago
Harish Krishna	a003b7b9bf	strip out leading and trailing whitespace if any	10 years ago
Harish Krishna	84a9b3e20e	splitting on comma (trying to find/assume admin values)	10 years ago
Harish Krishna	32275aca56	split sanitizers into reusable pieces. renaming sanitise to suggest. using suggest sanitizer for search. adding reverse sanitizer. updating tests..	10 years ago