Evaluate ranked search results | Elasticsearch API documentation

Query parameters

allow_no_indicesboolean
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
expand_wildcardsstring | array[string]
Whether to expand wildcard expression to concrete indices that are open, closed or both.
Values are all, open, closed, hidden, or none.
ignore_unavailableboolean
If true, missing or closed indices are not included in the response.
search_typestring
Search operation type

application/json

BodyRequired

requestsarray[object] Required
A set of typical search requests, together with their provided ratings.
Hide requests attributes Show requests attributes object
- idstring Required
- requestobject
  Hide request attributes Show request attributes object
  queryobject Required
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  External documentation
  sizenumber
- ratingsarray[object] Required
  List of document ratings
  Hide ratings attributes Show ratings attributes object
  _idstring Required
  _indexstring Required
  ratingnumber Required
  The document’s relevance with regard to this search request.
- template_idstring
- paramsobject
  The search template parameters.
  Hide params attribute Show params attribute object
  *object Additional properties
metricobject
Hide metric attributes Show metric attributes object
- precisionobject
  Precision at K (P@k)
  Hide precision attributes Show precision attributes object
  knumber
  Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.
  relevant_rating_thresholdnumber
  Sets the rating threshold above which documents are considered to be "relevant".
  ignore_unlabeledboolean
  Controls how unlabeled documents in the search results are counted. If set to true, unlabeled documents are ignored and neither count as relevant or irrelevant. Set to false (the default), they are treated as irrelevant.
- recallobject
  Recall at K (R@k)
  Hide recall attributes Show recall attributes object
  knumber
  Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.
  relevant_rating_thresholdnumber
  Sets the rating threshold above which documents are considered to be "relevant".
- mean_reciprocal_rankobject
  Mean Reciprocal Rank
  Hide mean_reciprocal_rank attributes Show mean_reciprocal_rank attributes object
  knumber
  Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.
  relevant_rating_thresholdnumber
  Sets the rating threshold above which documents are considered to be "relevant".
- dcgobject
  Discounted cumulative gain (DCG)
  Hide dcg attributes Show dcg attributes object
  knumber
  Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.
  normalizeboolean
  If set to true, this metric will calculate the Normalized DCG.
- expected_reciprocal_rankobject
  Expected Reciprocal Rank (ERR)
  Hide expected_reciprocal_rank attributes Show expected_reciprocal_rank attributes object
  knumber
  Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.
  maximum_relevancenumber Required
  The highest relevance grade used in the user-supplied relevance judgments.

Responses

200 application/json
Hide response attributes Show response attributes object
- metric_scorenumber Required
  The overall evaluation quality calculated by the defined metric
- detailsobject Required
  The details section contains one entry for every query in the original requests section, keyed by the search request id
  Hide details attribute Show details attribute object
  *object Additional properties
  Hide * attributes Show * attributes object
  metric_scorenumber Required
  The metric_score in the details section shows the contribution of this query to the global quality metric score
  unrated_docsarray[object] Required
  The unrated_docs section contains an _index and _id entry for each document in the search result for this query that didn’t have a ratings value. This can be used to ask the user to supply ratings for these documents
  Hide unrated_docs attributes Show unrated_docs attributes object
  _idstring Required
  _indexstring Required
  hitsarray[object] Required
  The hits section shows a grouping of the search results with their supplied ratings
  Hide hits attributes Show hits attributes object
  hitobject Required
  Hide hit attributes Show hit attributes object
  _idstring Required
  _indexstring Required
  _scorenumber Required
  ratingnumber | string | null
  One of:
  number-1number string-2string | null
  metric_detailsobject Required
  The metric_details give additional information about the calculated quality metric (e.g. how many of the retrieved documents were relevant). The content varies for each metric but allows for better interpretation of the results
  Hide metric_details attribute Show metric_details attribute object
  *object Additional properties
  Hide * attribute Show * attribute object
  *object Additional properties
- failuresobject Required
  Hide failures attribute Show failures attribute object
  *object Additional properties

GET /_rank_eval

GET /my-index-000001/_rank_eval
{
  "requests": [
    {
      "id": "JFK query",
      "request": { "query": { "match_all": {} } },
      "ratings": []
    } ],
  "metric": {
    "precision": {
      "k": 20,
      "relevant_rating_threshold": 1,
      "ignore_unlabeled": false
    }
  }
}

resp = client.rank_eval(
    index="my-index-000001",
    requests=[
        {
            "id": "JFK query",
            "request": {
                "query": {
                    "match_all": {}
                }
            },
            "ratings": []
        }
    ],
    metric={
        "precision": {
            "k": 20,
            "relevant_rating_threshold": 1,
            "ignore_unlabeled": False
        }
    },
)

const response = await client.rankEval({
  index: "my-index-000001",
  requests: [
    {
      id: "JFK query",
      request: {
        query: {
          match_all: {},
        },
      },
      ratings: [],
    },
  ],
  metric: {
    precision: {
      k: 20,
      relevant_rating_threshold: 1,
      ignore_unlabeled: false,
    },
  },
});

response = client.rank_eval(
  index: "my-index-000001",
  body: {
    "requests": [
      {
        "id": "JFK query",
        "request": {
          "query": {
            "match_all": {}
          }
        },
        "ratings": []
      }
    ],
    "metric": {
      "precision": {
        "k": 20,
        "relevant_rating_threshold": 1,
        "ignore_unlabeled": false
      }
    }
  }
)

$resp = $client->rankEval([
    "index" => "my-index-000001",
    "body" => [
        "requests" => array(
            [
                "id" => "JFK query",
                "request" => [
                    "query" => [
                        "match_all" => new ArrayObject([]),
                    ],
                ],
                "ratings" => array(
                ),
            ],
        ),
        "metric" => [
            "precision" => [
                "k" => 20,
                "relevant_rating_threshold" => 1,
                "ignore_unlabeled" => false,
            ],
        ],
    ],
]);

curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"requests":[{"id":"JFK query","request":{"query":{"match_all":{}}},"ratings":[]}],"metric":{"precision":{"k":20,"relevant_rating_threshold":1,"ignore_unlabeled":false}}}' "$ELASTICSEARCH_URL/my-index-000001/_rank_eval"

Request example

An example body for a `GET /my-index-000001/_rank_eval` request.

{
  "requests": [
    {
      "id": "JFK query",
      "request": { "query": { "match_all": {} } },
      "ratings": []
    } ],
  "metric": {
    "precision": {
      "k": 20,
      "relevant_rating_threshold": 1,
      "ignore_unlabeled": false
    }
  }
}

Query parameters

BodyRequired

Responses

ratingnumber | string | null