pelias/api - api - CogtoSrc

Commit Graph

Author	SHA1	Message	Date
Julian Simioni	b1107a0c8f	fix(autocomplete): detect the case where input text is unsubstantial It's possible for the `text` input to /v1/autocomplete to be of non-zero length after trimming whitespace and quotes, but still be insufficient to use for geocoding. One common case is that it contains only commas, slashes, or other delimiters. Our query logic currently does not handle this case, and will generate Elasticsearch queries that do not have a primary `must` clause and end up searching every document in the index. These queries are slow, take up cluster resources, and are not useful. By detecting unsubstantial inputs, we can prevent this.	6 years ago
Julian Simioni	7e4559fdc2	fix(sanitizer): Trim whitespace in addressit queries This is a followup PR to https://github.com/pelias/api/pull/1171 and https://github.com/pelias/api/pull/1170. Apparently we have two different `text` sanitizers, and autocomplete queries were treating a single space as valid input. This had a particularly bad outcome as it would end up generating queries (see an [example](https://gist.github.com/orangejulius/2cc26c7eed39311b6eaf1fb0175c13e6)) that had no main query clause. This caused them to match basically every document in the index. Looking at the geocode.earth slowlog, these queries took __8 seconds per shard__.	6 years ago
Julian Simioni	40e6054523	chore(geo_common): assert that a specific exception was thrown Without checking the message when asserting an exception is thrown, it's possible for the test to pass when undesired behavior is occuring.	6 years ago
Julian Simioni	1553dfb103	fix(geo_common): check for invalid bbox where min=max This condition will cause Elasticsearch to throw an error, we should catch it outselves first. The error is more friendly than the case where min>max, but still an error. Connects https://github.com/pelias/api/pull/1050	6 years ago
Julian Simioni	76bc5c654d	fix(geo_common): check bbox parameters are within range If bounding box lat/lon values are outside the correct range, Elasticsearch throws very alarming errors. With a little validation code we can provide more friendly and actionable error messages. Fixes https://github.com/pelias/pelias/issues/750	6 years ago
Julian Simioni	76d88b62b4	fix(tests): Sanitizer tests should use faked config Otherwise a user's own pelias.json settings can cause tests to fail	6 years ago
Horace Williams	c7ae37980d	Remove Quattroshapes deprecation notice Fixes #1132 The Quattroshapes source has been deprecated from some time now, and any users for whom the deprecation notice was helpful have presumably either upgraded or moved on by now.	7 years ago
Julian Simioni	9a0f182fb2	fix(whitespace): Trim whitespace and quotes before checking text length Previously, our text sanitizer code did not trim whitespace before checking that the string was non-empty. This lead to strings consisting only of whitespace being treated as valid. Not all our downstream services (such as libpostal) accept whitespace-only input, so this causes a rather harsh error. This PR builds upon the code in https://github.com/pelias/api/pull/1170 and moves the trimming code above the nonEmptyString check. Now, a whitespace-only input string produces the normal error for empty input. Fixes https://github.com/pelias/api/issues/1158	7 years ago
Peter Johnson	d38d4b1fa8	feat(text_sanitizer): trim whitespace and quotation marks from a range of natural languages	7 years ago
Julian Simioni	204f5297de	Remove all leading newlines in files	7 years ago
Julian Simioni	02aac0c13e	Remove use stricts	7 years ago
Joxit	551f0227db	Transliterates Saint and Sainte in ES schema see pelias/schema#268	7 years ago
Joxit	d187e7d950	Fix for #1077 , fail to search `Saint..` cities in structured queries	7 years ago
Rhys Kidd	1dfc6988ed	Update to current Pelias contact email address Per https://github.com/pelias/pelias/tree/master/announcements/2018-01-02-pelias-update replace all @mapzen.com email addresses with the current Pelias contact email address. grep used to confirm no '@mapzen.com' remain.	7 years ago
Stephen Hess	b5e48afb35	remove text-analyzer address operation	7 years ago
Peter Backx	f16565df9e	Verify min and max longitude and latitude Otherwise this can potentially lead to a fairly weird Elasticsearch error mentioned in https://github.com/pelias/pelias/issues/656	7 years ago
Julian Simioni	bc6401842c	Add friendly, coarse specific error message to reverse This error message is updated to mention the coarse-reverse only nature of Geonames support removal. It also links to the [tracking ticket](https://github.com/pelias/acceptance-tests/pull/447) for Geonames removal which should help provide more context to users.	7 years ago
Julian Simioni	d577d42d9f	Support geonames non-coarse reverse in deprecation sanitizer	7 years ago
Stephen Hess	18e7673dc1	added empire to coarse translation	7 years ago
Stephen Hess	e70a668056	added support for continent, ocean, and marinearea placetypes `coarse` now maps to these as well	7 years ago
Stephen Hess	42097fafb2	treat undefined text-analyzer response as no house number available	7 years ago
Stephen Hess	30b7290990	added stub request_language sanitizer that just allows lang param	7 years ago
Lily He	cafb7d9277	created new debug helper class and debug sanitizer	8 years ago
Lily He	bf234f5539	flag_bool expects parameter opts.paramName	8 years ago
Lily He	a5cf44bef4	clarify _text sanitizer test message	8 years ago
Lily He	8e45af1e49	adapt defer_to_addressit to new sanitizer format	8 years ago
Lily He	db56ce45c3	remove exporting sanitize for testing	8 years ago
Lily He	9a88d5a28b	unmerge nearby sanitizers from reverse sanitizer_list	8 years ago
Lily He	07c7e7d483	test reverse, nearby, place sanitizer wrappers w/ proxyquire	8 years ago
Lily He	80a3a259ef	remove callback in sanitizeAll.runAllChecks for sync processing	8 years ago
Lily He	a27ac0fc88	use strict for ecmascript 6 syntax, fix typos	8 years ago
Lily He	b9de4b6788	additional test to verify runAllCheck works for sanitizeAll	8 years ago
Lily He	279b70b7d7	typo: delete mentions of joi in sanitizers	8 years ago
Lily He	670c673419	call runAllChecks for nearby, reverse, and place sanitizing	8 years ago
Lily He	1f527be3e3	call runAllChecks for search, search_fallback, autocomplete & structured sanitizing	8 years ago
Lily He	af906c05a8	sanitizeAll validates req.query parameters, export async runAllChecks function	8 years ago
Lily He	a683e27a87	removed 'parsed_text' from list of expected params for _text	8 years ago
Lily He	368c0aa4cb	added expected parameters list to _text_addressit	8 years ago
Lily He	13a7b42de7	Standardize sanitizers to export setup function that returns sanitize and expected function	8 years ago
Stephen Hess	2b70b41a01	move _text sanitizer	8 years ago
Stephen Hess	85b831b161	removed iso2_to_iso3 and city_name_standardizer from wrapper	8 years ago
Stephen Hess	5dc9737caa	removed text-analyzer call from _text sanitizer	8 years ago
Stephen Hess	40ddc93bbf	added conditional fallback to addressit	8 years ago
Stephen Hess	30cd30236a	don't call libpostal if sources=whosonfirst	8 years ago
Stephen Hess	ac7f5c445c	moved libpostal lower in sanitizers to after sources	8 years ago
Lily He	57b133ba42	check and set default for undefined boundary circle radius in reverse	8 years ago
Lily He	5504240016	prevent overriding raw.focus.point.lat/lon	8 years ago
Lily He	40ca17b886	passed down peliasConfig.api schema to sanitizers	8 years ago
Lily He	2e2ae9b005	added location_bias sanitizer to autocomplete, search, and structured_geocoding	8 years ago
Lily He	f67664c3e0	location bias feature - sets default focus.point	8 years ago

1 2

78 Commits (libpostal_aus_unit_numbers)