pelias/api - api - CogtoSrc

Commit Graph

Author	SHA1	Message	Date
Julian Simioni	003c182df5	feat(log): add structured data to address parser logs These logs have been very confusing to us for a long time, and we have never bothered to collect enought data to understand them better.	6 years ago
Julian Simioni	b1107a0c8f	fix(autocomplete): detect the case where input text is unsubstantial It's possible for the `text` input to /v1/autocomplete to be of non-zero length after trimming whitespace and quotes, but still be insufficient to use for geocoding. One common case is that it contains only commas, slashes, or other delimiters. Our query logic currently does not handle this case, and will generate Elasticsearch queries that do not have a primary `must` clause and end up searching every document in the index. These queries are slow, take up cluster resources, and are not useful. By detecting unsubstantial inputs, we can prevent this.	6 years ago
Julian Simioni	7e4559fdc2	fix(sanitizer): Trim whitespace in addressit queries This is a followup PR to https://github.com/pelias/api/pull/1171 and https://github.com/pelias/api/pull/1170. Apparently we have two different `text` sanitizers, and autocomplete queries were treating a single space as valid input. This had a particularly bad outcome as it would end up generating queries (see an [example](https://gist.github.com/orangejulius/2cc26c7eed39311b6eaf1fb0175c13e6)) that had no main query clause. This caused them to match basically every document in the index. Looking at the geocode.earth slowlog, these queries took __8 seconds per shard__.	6 years ago
Julian Simioni	16667199cd	feat(geo_common): improve boundary.rect error message In the case where a min lat/lon is larger than a max lat/lon, the error message was a bit confusing as it did not show the actual property name or the values that are causing errors.	6 years ago
Julian Simioni	1553dfb103	fix(geo_common): check for invalid bbox where min=max This condition will cause Elasticsearch to throw an error, we should catch it outselves first. The error is more friendly than the case where min>max, but still an error. Connects https://github.com/pelias/api/pull/1050	6 years ago
Julian Simioni	f1afda469d	Move bbox min/max check to its own function	6 years ago
Julian Simioni	76bc5c654d	fix(geo_common): check bbox parameters are within range If bounding box lat/lon values are outside the correct range, Elasticsearch throws very alarming errors. With a little validation code we can provide more friendly and actionable error messages. Fixes https://github.com/pelias/pelias/issues/750	6 years ago
Julian Simioni	8a9c31e46e	Remove usage of util.format	6 years ago
Julian Simioni	1054a634de	chore(geo_common): refactor bbox min/max validation	6 years ago
Julian Simioni	3c33b98f8a	chore(geo_common): only set bbox coords if it validates	6 years ago
Horace Williams	ed0a96cff2	Migrate to internal iso3166 defnitions This is a somewhat roundabout fix to #1179, as a way to deal with the persistent npm ls and commit hook troubles we were dealing with due to dependencies of the iso3166 package. Additionally it should give us a faster definition of these ISO lookups, since the existing approaches were implemented using linear scans through an array rather than map-based lookups.	6 years ago
Horace Williams	c7ae37980d	Remove Quattroshapes deprecation notice Fixes #1132 The Quattroshapes source has been deprecated from some time now, and any users for whom the deprecation notice was helpful have presumably either upgraded or moved on by now.	6 years ago
Julian Simioni	9a0f182fb2	fix(whitespace): Trim whitespace and quotes before checking text length Previously, our text sanitizer code did not trim whitespace before checking that the string was non-empty. This lead to strings consisting only of whitespace being treated as valid. Not all our downstream services (such as libpostal) accept whitespace-only input, so this causes a rather harsh error. This PR builds upon the code in https://github.com/pelias/api/pull/1170 and moves the trimming code above the nonEmptyString check. Now, a whitespace-only input string produces the normal error for empty input. Fixes https://github.com/pelias/api/issues/1158	6 years ago
Peter Johnson	d38d4b1fa8	feat(text_sanitizer): trim whitespace and quotation marks from a range of natural languages	6 years ago
Julian Simioni	204f5297de	Remove all leading newlines in files	7 years ago
Julian Simioni	02aac0c13e	Remove use stricts	7 years ago
Joxit	551f0227db	Transliterates Saint and Sainte in ES schema see pelias/schema#268	7 years ago
Joxit	d187e7d950	Fix for #1077 , fail to search `Saint..` cities in structured queries	7 years ago
Rhys Kidd	1dfc6988ed	Update to current Pelias contact email address Per https://github.com/pelias/pelias/tree/master/announcements/2018-01-02-pelias-update replace all @mapzen.com email addresses with the current Pelias contact email address. grep used to confirm no '@mapzen.com' remain.	7 years ago
Stephen Hess	b5e48afb35	remove text-analyzer address operation	7 years ago
Peter Backx	f16565df9e	Verify min and max longitude and latitude Otherwise this can potentially lead to a fairly weird Elasticsearch error mentioned in https://github.com/pelias/pelias/issues/656	7 years ago
Julian Simioni	bc6401842c	Add friendly, coarse specific error message to reverse This error message is updated to mention the coarse-reverse only nature of Geonames support removal. It also links to the [tracking ticket](https://github.com/pelias/acceptance-tests/pull/447) for Geonames removal which should help provide more context to users.	7 years ago
Julian Simioni	d577d42d9f	Support geonames non-coarse reverse in deprecation sanitizer	7 years ago
Stephen Hess	6fd3c011fb	remove unused require	7 years ago
Stephen Hess	1e003efe28	used lodash.extend instead of extend module	7 years ago
Stephen Hess	42097fafb2	treat undefined text-analyzer response as no house number available	7 years ago
Stephen Hess	30b7290990	added stub request_language sanitizer that just allows lang param	7 years ago
Lily He	cafb7d9277	created new debug helper class and debug sanitizer	7 years ago
Lily He	bf234f5539	flag_bool expects parameter opts.paramName	7 years ago
Lily He	8e45af1e49	adapt defer_to_addressit to new sanitizer format	7 years ago
Lily He	db56ce45c3	remove exporting sanitize for testing	7 years ago
Lily He	9a88d5a28b	unmerge nearby sanitizers from reverse sanitizer_list	7 years ago
Lily He	80a3a259ef	remove callback in sanitizeAll.runAllChecks for sync processing	7 years ago
Lily He	a27ac0fc88	use strict for ecmascript 6 syntax, fix typos	7 years ago
Lily He	279b70b7d7	typo: delete mentions of joi in sanitizers	7 years ago
Lily He	670c673419	call runAllChecks for nearby, reverse, and place sanitizing	7 years ago
Lily He	1f527be3e3	call runAllChecks for search, search_fallback, autocomplete & structured sanitizing	7 years ago
Lily He	0486967526	add two more acceptable params for _geo_reverse	7 years ago
Lily He	af906c05a8	sanitizeAll validates req.query parameters, export async runAllChecks function	7 years ago
Lily He	a683e27a87	removed 'parsed_text' from list of expected params for _text	7 years ago
Lily He	368c0aa4cb	added expected parameters list to _text_addressit	7 years ago
Lily He	13a7b42de7	Standardize sanitizers to export setup function that returns sanitize and expected function	7 years ago
Stephen Hess	5dc9737caa	removed text-analyzer call from _text sanitizer	7 years ago
Stephen Hess	40ddc93bbf	added conditional fallback to addressit	7 years ago
Stephen Hess	30cd30236a	don't call libpostal if sources=whosonfirst	7 years ago
Stephen Hess	ac7f5c445c	moved libpostal lower in sanitizers to after sources	7 years ago
Lily He	c458eabf9b	delete unused _details sanitizer	7 years ago
Lily He	57b133ba42	check and set default for undefined boundary circle radius in reverse	7 years ago
Lily He	5504240016	prevent overriding raw.focus.point.lat/lon	7 years ago
Lily He	40ca17b886	passed down peliasConfig.api schema to sanitizers	7 years ago

1 2

91 Commits (f3b11a16ebb949c9fbfc4f64b0d09572725580c9)