pelias/api - api - CogtoSrc

Commit Graph

Author	SHA1	Message	Date
Julian Simioni	23ff51fd57	Add custom access_log handlers that remove user info These handlers will remove sensitive query info from the logs when DNT is enabled	9 years ago
Julian Simioni	3c1d758229	Fix timeout error handling Timeout errors come back from the Elasticsearch module as Error objects, so instead of pushing the entire object onto the errors array, we should just push the message. I'm not sure if there are other cases where errors are just strings, errors that aren't objects with a message property are handled as before.	9 years ago
Vesa Meskanen	a2e96987fc	Remove expensive JSON.stringify from api debug logging calls	9 years ago
Julian Simioni	79883594b8	Use clean.layers to determine types This allows removing the types middleware and simplifies some code.	9 years ago
Julian Simioni	ef78b2e0fd	Filter source and layer by new fields This is functionally the same code as before, except that it uses the new source and layer mapping.	9 years ago
Vesa Meskanen	269dff9dd1	Add query/response logging in debug mode to place controller, too	9 years ago
Vesa Meskanen	899d520cd7	Debug log full queries and responses between Pelias and Elasticsearch	9 years ago
Diana Shkolnikov	42d940f8c8	Add simple normalizer (lowercase + remove punctuation)	9 years ago
Diana Shkolnikov	54187dde67	Add dedupe middleware Dedupe middleware removes __exact__ dupes and truncates the results to the specified size.	9 years ago
Julian Simioni	0e8f4253c0	Return multiple results in place when osm node and way share an id It turns out the _type parameter to the Elasticsearch [multiget](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-multi-get.html) API does not allow an array of possible values. We were depending on its ability to search multiple types to allow searching for OSM nodes and ways. But, since this doesn't work we essentially have to do it ourselves. There is also the problem that OSM nodes and ways share an ID space. So a gid such as `osm:venue:5` could in theory correspond to 2 records. It seems like the only nice thing to do in that case is return both results. This PR "unrolls" such queries. For example, in the case of `osm:venue:5`, the sanitisers will return the following array of objects to be turned into multiget queries: ``` [{ id: 5, types: ["osmway", "osmnode"] }] ``` Before, this would turn into a multiget query with only one entry, like this: ``` { "docs": [ { "_index": "pelias", "_type": [ " osmnode", "osmway" ], "_id": 5 } ] } ``` now it would look like this: { "docs": [ { "_index": "pelias", "_type": "osmnode", "_id": 5 }, { "_index": "pelias", "_type": "osmnode", "_id": 5 } ] } TLDR you might get back more records from /place than the number of ids you specified, but at least you will definitely get back what you are looking for.	9 years ago
Peter Johnson	c609ae3ccf	http friendly status codes. resolves #321	9 years ago
Stephen Hess	cb7d22df91	switched ES lookup field from "type" -> "_type", fixes issue with /place lookup on Eiffel Tower gid	9 years ago
Julian Simioni	a2cc31f142	Leave a note for future us A small essay on Elasticsearch queries and fields and types.	9 years ago
Julian Simioni	2505e92a62	Expect an array of types from _ids sanitiser This doesn't have any effect by itself but allows for the 3-part gid sanitiser to possibly return multiple types (i.e. in the case of osm:venue:1000)	9 years ago
Diana Shkolnikov	2b22d611bd	Fix lost confidence score	9 years ago
Peter Johnson	348d2e70e2	more conformance	9 years ago
Peter Johnson	7c46aed4a7	expose error messages in geojson	9 years ago
Peter Johnson	6ab81ffca5	set response data/meta on instead of	9 years ago
Peter Johnson	e726636c3d	merge with master	9 years ago
Julian Simioni	5d7f7ee4c0	Add /status controller It's at the root level, and simply returns 200 with 'status: ok' as the body.	9 years ago
Julian Simioni	f6fea21782	Use hasOwnProperty instead of comparing against undefined	9 years ago
Julian Simioni	6dbb4f8b08	Expose list of document types sent to Elasticsearch	9 years ago
Julian Simioni	ade4ce8b17	Add types middleware This middleware looks at the list of types that will be sent to Elasticsearch, if it's an empty array, it sends an error response before Elasticsearch is even quieried, because Elasticsearch interprets an empty type array as "search anything" rather than the intended "don't search anything".	9 years ago
Julian Simioni	258fa4390e	Move cmd.type setting logic to types helper	9 years ago
Julian Simioni	82d3a6603c	Store sanitised types from layers parameter This moves the list of types created by sanitising the layer API parameter from clean.layers to clean.types.from_layers. In subsequent commits, types created from address parsing, and the yet-to-be-implemented source parameter will also live in the clean.types object. This will allow moving logic to set cmd.type out of controllers, and into separate logic that can be a littler smarter. Also, it will no longer require the clean.default_layers_set flag to be passed all around like a nasty global variable.	9 years ago
Julian Simioni	e71fb7cfd4	Whitespace	9 years ago
Diana Shkolnikov	f7044d2435	Add content for `attribution` static endpoint Fixes #212	9 years ago
Julian Simioni	6f0ad678eb	Add types middleware This middleware looks at the list of types that will be sent to Elasticsearch, if it's an empty array, it sends an error response before Elasticsearch is even quieried, because Elasticsearch interprets an empty type array as "search anything" rather than the intended "don't search anything".	9 years ago
Julian Simioni	1573f3a3fa	Move cmd.type setting logic to types helper	9 years ago
Julian Simioni	4037c49c4b	Store sanitised types from layers parameter This moves the list of types created by sanitising the layer API parameter from clean.layers to clean.types.from_layers. In subsequent commits, types created from address parsing, and the yet-to-be-implemented source parameter will also live in the clean.types object. This will allow moving logic to set cmd.type out of controllers, and into separate logic that can be a littler smarter. Also, it will no longer require the clean.default_layers_set flag to be passed all around like a nasty global variable.	9 years ago
Julian Simioni	fb0cf514e9	Whitespace	9 years ago
Diana Shkolnikov	562338635c	Confidence score calculation Fixes pelias/pelias#120	9 years ago
Diana Shkolnikov	db0e063d8c	Refactor controllers to be simple midddleware Refactor search and doc controllers to allow for post-processing middleware to handle geojson and response sending. Allows for a much more flexible routing scheme.	9 years ago
Julian Simioni	2fce64f75a	Move files	9 years ago
Julian Simioni	4e74e9b681	Whitespace	9 years ago
Harish Krishna	e5f783567f	remove all things that involved suggester including query mixer keeping the suggest sanitizer for suggest/nearby wherein lat/lon are required	9 years ago
Harish Krishna	96a21570be	fixing layers related bug - adding a default_layers_set flag to clean obj.	9 years ago
Harish Krishna	1d24890b27	introducing target_layer that gets set part of input_parsing - if input is less than 3 characters -> only hit admin layers - if input suggests that its not a street address -> avoid hitting address layers	9 years ago
Harish Krishna	b9c4c673f5	Revert "removing searchType overhead" This reverts commit `ee4aa1b97a`.	9 years ago
Harish Krishna	ee4aa1b97a	removing searchType overhead	9 years ago
Severyn Kozak	29838e5b8c	Move code that needs to run only once. controller/index.js -Move all of the index page HTML "initialization" code that only needs to run once, like markdown compilation and `style` prepending, out of the request handler.	10 years ago
Diana Shkolnikov	99ee2d95a3	Change to readFileSync and update test to use proxyquire	10 years ago
Diana Shkolnikov	1d45b20268	Load markdown file into memory and check if client requested html	10 years ago
Harish Krishna	14c718bf1c	passing in an object (req.clean) instead of a boolean. Plus updated test	10 years ago
Harish Krishna	a91cb23ea4	adding a new file DOCS.md (same as https://github.com/pelias/api/wiki/API-Endpoints ), modifying controller/index.js to return text/html instead of application/json by default. Using markdown module to convert MD to HTML. Fixing tests (removed test/jsonp.coffee)	10 years ago
Harish Krishna	c99c5f712a	adding a parameter called details (defaults to false for all endpoint except reverse) + fixing tests	10 years ago
Harish Krishna	ff9e9973fc	switching searchType from the default 'query_then_fetch' to 'dfs_query_then_fetch' because we have multiple shards in prod http://www.elasticsearch.org/blog/understanding-query-then-fetch-vs-dfs-query-then-fetch/	10 years ago
Harish Krishna	bcf823c519	cleanup + tests	10 years ago
Harish Krishna	fc7f6cb768	removing suggest_nearby, replacing suggest with suggest_multiple, commenting out a couple of tests for right now.	10 years ago
Harish Krishna	9afed3caef	adding query mixer for suggest and suggest/nearby	10 years ago

1 2 3

102 Commits (5964e57faab200eed4fe9eabb80b18d4d479b174)