[FIXED] How to get multiple values output from multiple fields with one _search request?

Issue

On a performance testing project where ELK stack is used to collect and process API test data streams(samples/document go under one index) it would be good if we could get multiple aggregated results with only one _search request, like the count of successful requests AND average response time AND various percentiles, etc. Is it possible to create such a DSL query or do we have to execute multiple searches like one to get the count of successful requests, another one for percentiles, etc?

Solution

you sure can, this page of the documentation gives an example, but here it is for reference;

curl -X GET "localhost:9200/my-index-000001/_search?pretty" -H 'Content-Type: application/json' -d'
{
  "aggs": {
    "my-first-agg-name": {
      "terms": {
        "field": "my-field"
      }
    },
    "my-second-agg-name": {
      "avg": {
        "field": "my-other-field"
      }
    }
  }
}
'

Answered By - warkolm

Answer Checked By - Terry (PHPFixing Volunteer)

Read More

[FIXED] How Elasticsearch do filter aggregation

Issue

I need to do average aggregation, but I want to filter some values. In the below examples, I want filter the length=100, so I want to do average for length (with doc #1 and doc #2) and for width with all documents. So I expect to see length average as 9 and width average as 5. What should I do?

document example:

["id": 1, "length": 10, "width":8]
["id": 2, "length": 8, "width":2]
["id": 3, "length": 100, "width":5]

And In some other case, length may not exist, How about this case?

["id": 1, "length": 10, "width":8]
["id": 2, "length": 8, "width":2]
["id": 3, "width":5]

termAggregation.subAggregation(AggregationBuilders.avg("length").field("length"))
.subAggregation(AggregationBuilders.avg("width").field("width"));

Solution

Your aggregation query will look like below for excluding 100 from aggregation. You need to use filter aggregation and inside that avg as sub aggregation.

{
  "size": 0,
  "aggs": {
    "cal": {
      "filter": {
        "bool": {
          "must_not": [
            {
              "match": {
                "length": "100"
              }
            }
          ]
        }
      },
      "aggs": {
        "avg_length": {
          "avg": {
            "field": "length"
          }
        }
      }
    },
    "avg_width":{
      "avg": {
        "field": "width"
      }
    }
  }
}

Java code

AvgAggregationBuilder widthAgg = new AvgAggregationBuilder("avg_width").field("width");
        AvgAggregationBuilder lengthAgg = new AvgAggregationBuilder("avg_length").field("length");

        FilterAggregationBuilder filter = new FilterAggregationBuilder("cal",
                QueryBuilders.boolQuery().mustNot(QueryBuilders.matchQuery("length", "100")));
        filter.subAggregation(lengthAgg);
        
        SearchSourceBuilder ssb = new SearchSourceBuilder();
        ssb.aggregation(filter);
        ssb.aggregation(widthAgg);
        
        System.out.println(ssb.toString());

Response

"aggregations": {
    "avg_width": {
      "value": 5
    },
    "cal": {
      "meta": {},
      "doc_count": 3,
      "avg_length": {
        "value": 9
      }
    }
  }

Answered By - Sagar Patel

Answer Checked By - Gilberto Lyons (PHPFixing Admin)

Read More

[FIXED] How do I get `sort` and `bool => should` query to work in tandem?

Issue

I have the following query for fetching all products. What I'm trying to achieve is keep the out of stock products I.E. products with stock_sum = 0 at the bottom:

{
  "sort": [
    {
      "updated_at": {
        "order": "desc"
      }
    }
  ],
  "size": 10,
  "from": 0,
  "query": {
    "bool": {
      "should": [
        {
          "range": {
            "stock_sum": {
              "gte": 1,
              "boost": 5
            }
          }
        }
      ]
    }
  }
}

But with the above query sort seems to completely override should, which is how it's suppose to behave I guess. A couple of things that I tried are changing the should to must in this case the out of stock products, are left out completely (that's not what I want, I still want the out of stock products at the bottom).

Another approach is remove sort, and then the should query seems to have an effect, but again I need the sort. So my question is how do I get sort and bool => should query to work in tandem ? I.E. sort by updated_at but also keep the stock_sum = 0 at the bottom?

Solution

Using match_all and constant_score query in the same should clause and sorting first by _score by asc, then by updated_at by desc should work for your example. Here is an example query:

{
  "sort": [
    {
      "_score": {
        "order": "asc"
      }
    },
    {
      "updated_at": {
        "order": "desc"
      }
    }
  ]
  "query": {
    "bool": {
      "should": [
        {
          "match_all": {}
        },
        {
          "constant_score": {
             "filter": {
              "term": {
                "stock_sum": 0
              }
            },
            "boost": 10
          }
        }
      ]
    }
  }
}

Answered By - YD9

Answer Checked By - David Marino (PHPFixing Volunteer)

Read More

[FIXED] How to calculate gauss value?

Issue

I am just curious to know how this value came i applied formula but i think i am missing something can anybody tell me please.I am running single ELK stack version 7.16

POST sneaker/_search
{
  "query": {
    "function_score": {
      "functions": [
        {
          "gauss": {
            "price": {
              "origin": "300",
              "scale": "200"
            }
          }
        }
      ]
    }
  }
  , "explain": true
}

Query result

 "max_score" : 1.0,
    "hits" : [
      {
        "_shard" : "[sneaker][0]",
        "_node" : "29ds_f0VSM6_-eDNhdQPLw",
        "_index" : "sneaker",
        "_type" : "_doc",
        "_id" : "6",
        "_score" : 1.0,
        "_source" : {
          "brand" : "flite",
          "price" : 300,
          "rating" : 2,
          "release_date" : "2020-12-21"
        },
        "_explanation" : {
          "value" : 1.0,
          "description" : "function score, product of:",
          "details" : [
            {
              "value" : 1.0,
              "description" : "*:*",
              "details" : [ ]
            },
            {
              "value" : 1.0,
              "description" : "min of:",
              "details" : [
                {
                  "value" : 1.0,
                  "description" : "Function for field price:",
                  "details" : [
                    {
                      "value" : 1.0,
                      "description" : "exp(-0.5*pow(MIN[Math.max(Math.abs(300.0(=doc value) - 300.0(=origin))) - 0.0(=offset), 0)],2.0)/28853.900817779268)",
                      "details" : [ ]
                    }
                  ]
                },

I look for guassian distribution but it is different from this. I want to know how 28853.900817779268 value came

Solution

If you look on the official documentation for the gauss decay function, you'll find the following formula for computing the sigma:

Using scale = 200 and decay = 0.5 (default value if unspecified), we get the following:

-200^2 / (2 * ln (0.5)) = -28853.90081

which is what you're seeing in the explanation of the query.

Answered By - Val

Answer Checked By - Candace Johnson (PHPFixing Volunteer)

Read More