Java操作elasticSearch复杂查询以及解析数据以及索引保存数据

news/2024/7/7 5:53:35 标签: java, elasticsearch, 开发语言

Java操作elasticSearch复杂查询以及解析数据

银行测试库

es的银行测试库，看一个Kibana操作 然后用java检索解析这个数据

#聚合搜索 address 中包含 mill 的所有人的年龄分布以及平均薪资
GET bank/_search
{
  "query":{
    "match": {
      "address": "mill"
    }
  },
  "aggs": {
    "ageAgg": {
      "terms": {
        "field": "age",
        "size": 10
      }
    },
    "balanceAvg":{
      "avg":{
        "field": "balance"
      }
    }
  },
  "size": 0
}

分解思路实现

拆解操作数据

#聚合搜索 address 中包含 mill 的所有人的年龄分布以及平均年龄
GET bank/_search
{
“query”:{ “match”: { “address”: “mill” }
},
“aggs”: { “ageAgg”: { “terms”: { “field”: “age”, “size”: 10 } },

“balanceAvg”:{ “avg”:{ “field”: “balance” } } }, “size”: 0 }

构造一个查询器 指向索引

java">SearchRequest searchRequest = new SearchRequest();
//指定索引
searchRequest.indices("bank");

封装查询条件器

java">//指定DSL 检索条件
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//构造检索条件
searchSourceBuilder.query(QueryBuilders.matchQuery("address","mill"));
//按照年龄只分布进行聚合
TermsAggregationBuilder ageAgg = AggregationBuilders.terms("ageAgg").field("age").size(10);
searchSourceBuilder.aggregation(ageAgg);
//计算平均薪资
AvgAggregationBuilder balanceAvg = AggregationBuilders.avg("balanceAvg").field("blance");
searchSourceBuilder.aggregation(balanceAvg);

//打印检索条件 打印结果与Kibana核对
System.out.println("检索条件："+searchSourceBuilder);

检索条件：{"query":{"match":{"address":{"query":"mill","operator":"OR","prefix_length":0,"max_expansions":50,"fuzzy_transpositions":true,"lenient":false,"zero_terms_query":"NONE","auto_generate_synonyms_phrase_query":true,"boost":1.0}}},"aggregations":{"ageAgg":{"terms":{"field":"age","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},{"_key":"asc"}]}},"balanceAvg":{"avg":{"field":"blance"}}}}

封装的条件器置入查询器

java">searchRequest.source(searchSourceBuilder);

容器中的client调用查询：

java">//执行检索
SearchResponse search = client.search(searchRequest, GuilimallElasticSearchConfig.COMMON_OPTIONS);

解析查询结果

java">		System.out.println(search.toString());
//		Map map = JSON.parseObject(search.toString(), Map.class);
		//分析结果 查询结构
		SearchHits hits = search.getHits();
		SearchHit[] searchHits = hits.getHits();
		for (SearchHit hit: searchHits){
//			hit.getIndex();
//			hit.getId();
			String sourceAsString = hit.getSourceAsString();
			Accout accout = JSON.parseObject(sourceAsString, Accout.class);
			System.out.println(accout.toString());
		}
		//获取检索的分析信息
		Aggregations aggregations = search.getAggregations();
//		for (Aggregation aggregation : aggregations.asList()) {
//			System.out.println("当前聚合名字："+aggregation.getName());
//		}
		//分类聚合
		Terms ageAgg1 = aggregations.get("ageAgg");
		for (Terms.Bucket bucket : ageAgg1.getBuckets()) {
			String keyAsString = bucket.getKeyAsString();
			System.out.println("年龄：" + keyAsString + "人数："+bucket.getDocCount());
		}
		//平局值
		Avg balanceAvg1 = aggregations.get("balanceAvg");
		System.out.println("平均薪资"+ balanceAvg1.getValue());

{"took":1,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":4,"relation":"eq"},"max_score":5.4032025,"hits":[{"_index":"bank","_type":"account","_id":"970","_score":5.4032025,"_source":{"account_number":970,"balance":19648,"firstname":"Forbes","lastname":"Wallace","age":28,"gender":"M","address":"990 Mill Road","employer":"Pheast","email":"forbeswallace@pheast.com","city":"Lopezo","state":"AK"}},{"_index":"bank","_type":"account","_id":"136","_score":5.4032025,"_source":{"account_number":136,"balance":45801,"firstname":"Winnie","lastname":"Holland","age":38,"gender":"M","address":"198 Mill Lane","employer":"Neteria","email":"winnieholland@neteria.com","city":"Urie","state":"IL"}},{"_index":"bank","_type":"account","_id":"345","_score":5.4032025,"_source":{"account_number":345,"balance":9812,"firstname":"Parker","lastname":"Hines","age":38,"gender":"M","address":"715 Mill Avenue","employer":"Baluba","email":"parkerhines@baluba.com","city":"Blackgum","state":"KY"}},{"_index":"bank","_type":"account","_id":"472","_score":5.4032025,"_source":{"account_number":472,"balance":25571,"firstname":"Lee","lastname":"Long","age":32,"gender":"F","address":"288 Mill Street","employer":"Comverges","email":"leelong@comverges.com","city":"Movico","state":"MT"}}]},"aggregations":{"lterms#ageAgg":{"doc_count_error_upper_bound":0,"sum_other_doc_count":0,"buckets":[{"key":38,"doc_count":2},{"key":28,"doc_count":1},{"key":32,"doc_count":1}]},"avg#balanceAvg":{"value":null}}}
GulimallSearchApplicationTests.Accout(account_number=970, balance=19648, firstname=Forbes, lastname=Wallace, age=28, gender=M, address=990 Mill Road, employer=Pheast, email=forbeswallace@pheast.com, city=Lopezo, state=AK)
GulimallSearchApplicationTests.Accout(account_number=136, balance=45801, firstname=Winnie, lastname=Holland, age=38, gender=M, address=198 Mill Lane, employer=Neteria, email=winnieholland@neteria.com, city=Urie, state=IL)
GulimallSearchApplicationTests.Accout(account_number=345, balance=9812, firstname=Parker, lastname=Hines, age=38, gender=M, address=715 Mill Avenue, employer=Baluba, email=parkerhines@baluba.com, city=Blackgum, state=KY)
GulimallSearchApplicationTests.Accout(account_number=472, balance=25571, firstname=Lee, lastname=Long, age=32, gender=F, address=288 Mill Street, employer=Comverges, email=leelong@comverges.com, city=Movico, state=MT)
年龄：38人数：2
年龄：28人数：1
年龄：32人数：1
平均薪资25208.0

打印逐条记录时，可以把结构封装成一个model 借助一下：json.cn

在这里插入图片描述
在这里插入图片描述

完整操作：

java">	@ToString
		@Data
		static class Accout {

			private int account_number;
			private int balance;
			private String firstname;
			private String lastname;
			private int age;
			private String gender;
			private String address;
			private String employer;
			private String email;
			private String city;
			private String state;
		}

	@Test
	public void searchData() throws IOException {
		SearchRequest searchRequest = new SearchRequest();
		//指定索引
		searchRequest.indices("bank");
		//指定DSL 检索条件
		SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
		//构造检索条件

	/**
	 #聚合搜索 address 中包含 mill 的所有人的年龄分布以及平均年龄
	 GET bank/_search
	 {
	 "query":{ "match": { "address": "mill" }
	 },
	 "aggs": { "ageAgg": { "terms": { "field": "age", "size": 10 } },
	 "balanceAvg":{ "avg":{ "field": "balance" } } }, "size": 0 }
	 */

//		searchSourceBuilder.aggregation();
//		searchSourceBuilder.from();
//		searchSourceBuilder.size();
		searchSourceBuilder.query(QueryBuilders.matchQuery("address","mill"));
		//按照年龄只分布进行聚合
		TermsAggregationBuilder ageAgg = AggregationBuilders.terms("ageAgg").field("age").size(10);
		searchSourceBuilder.aggregation(ageAgg);
		//计算平均薪资
		AvgAggregationBuilder balanceAvg = AggregationBuilders.avg("balanceAvg").field("balance");
		searchSourceBuilder.aggregation(balanceAvg);

		//打印检索条件
 		System.out.println("检索条件："+searchSourceBuilder);


		searchRequest.source(searchSourceBuilder);
		//执行检索
		SearchResponse search = client.search(searchRequest, GuilimallElasticSearchConfig.COMMON_OPTIONS);

		//分析结果
//		searchRequest.
		System.out.println(search.toString());
//		Map map = JSON.parseObject(search.toString(), Map.class);
		//分析结果 查询结构
		SearchHits hits = search.getHits();
		SearchHit[] searchHits = hits.getHits();
		for (SearchHit hit: searchHits){
//			hit.getIndex();
//			hit.getId();
			String sourceAsString = hit.getSourceAsString();
			Accout accout = JSON.parseObject(sourceAsString, Accout.class);
			System.out.println(accout.toString());
		}
		//获取检索的分析信息
		Aggregations aggregations = search.getAggregations();
//		for (Aggregation aggregation : aggregations.asList()) {
//			System.out.println("当前聚合名字："+aggregation.getName());
//		}
		//分类聚合
		Terms ageAgg1 = aggregations.get("ageAgg");
		for (Terms.Bucket bucket : ageAgg1.getBuckets()) {
			String keyAsString = bucket.getKeyAsString();
			System.out.println("年龄：" + keyAsString + "人数："+bucket.getDocCount());
		}
		//平局值
		Avg balanceAvg1 = aggregations.get("balanceAvg");
		System.out.println("平均薪资"+ balanceAvg1.getValue());

	}

2.Java操作elasticSearch索引保存数据

Java操作elasticSearch索引保存数据

计划与实现

存储一个新索引students，然后保存文档

  • 借助Kibana:
GET /students/_search

结果：
{
  "error" : {
    "root_cause" : [
      {
        "type" : "index_not_found_exception",
        "reason" : "no such index [students]",
        "resource.type" : "index_or_alias",
        "resource.id" : "students",
        "index_uuid" : "_na_",
        "index" : "students"
      }
    ],
    "type" : "index_not_found_exception",
    "reason" : "no such index [students]",
    "resource.type" : "index_or_alias",
    "resource.id" : "students",
    "index_uuid" : "_na_",
    "index" : "students"
  },
  "status" : 404
}

索引不存在
  • 单元测试

索引数据的请求是个网络操作，所以会有异常处理。

java">//做一个学生对象
	//注解后setter getter
	@Data
	class Student{
		private String name;
		private Integer age;
		private String gender;
	}

	@Test
	public void indexData() throws IOException {
		//索引
		IndexRequest indexRequest = new IndexRequest("students");
		//数据id 不设置会自动生成
		indexRequest.id("1");

		Student student = new Student();
		student.setAge(18);
		student.setGender("男");
		student.setName("张铁蛋");
		//对象转换json
		String jsonString = JSON.toJSONString(student);
		//索引对象加入对象json  声明保存形式
		indexRequest.source(jsonString, XContentType.JSON);
		//用容器中导入的client 调用请求  索引对象 和 配置参数 这个配置参数是整合配置时搞定的
		IndexResponse index = client.index(indexRequest, GuilimallElasticSearchConfig.COMMON_OPTIONS);
		//index为相应数据
		System.out.println(index);
	}

执行：

2021-11-05 16:01:28.219  INFO 1548 --- [           main] c.a.g.s.GulimallSearchApplicationTests   : Started GulimallSearchApplicationTests in 18.417 seconds (JVM running for 19.985)
IndexResponse[index=students,type=_doc,id=1,version=1,result=created,seqNo=0,primaryTerm=1,shards={"total":2,"successful":1,"failed":0}]
2021-11-05 16:01:32.019  INFO 1548 --- [       Thread-9] o.s.s.concurrent.ThreadPoolTaskExecutor  : Shutting down ExecutorService 'applicationTaskExecutor'

Process finished with exit code 0
  • 在看一下Kibana:
GET /students/_search

结果集：

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "students",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "age" : 18,
          "gender" : "男",
          "name" : "张铁蛋"
        }
      }
    ]
  }
}
  • 保存成功

http://www.niftyadmin.cn/n/176462.html

相关文章

MongoDB数据库(1)

MongoDB数据库(1)

一、MongoDB简介 1、MongoDB介绍 MongoDB是为快速开发互联网Web应用而设计的数据库系统。 MongoDB的设许目标是极简、灵活、作为Web应用栈的一部分。 MongoDB的数据模型是面向文档的, 所谓文档是一种类似于JSON的结构，简单理解MongoDB这个数据库中存的是各种各样的…
阅读更多...
JAVA语言之Solr的工作原理以及如何管理索引库

JAVA语言之Solr的工作原理以及如何管理索引库

Solr的简介 Solr是一个独立的企业级搜索应用服务器，它对外提供类似于Web-service的API接口。用户可以通过http请求，向搜索引擎服务器提交一定格式的XML文件，生成索引；也可以通过Http Get操作提出查找请求，并得到XML格…
阅读更多...
uniapp开发微信小程序，路由跳转传参多种方式

uniapp开发微信小程序，路由跳转传参多种方式

方式一：//在起始页面跳转到test.vue页面并传递参数 uni.navigateTo({url: test?id1&nameuniapp });// 在test.vue页面接受参数 export default {onLoad: function (option) { //option为object类型，会序列化上个页面传递的参数console.log(option.i…
阅读更多...
银河麒麟v10sp2安装nginx

银河麒麟v10sp2安装nginx

nginx官网下载：http://nginx.org/download/ 银河麒麟系统请先检查yum源是否配置，若没有配置请参考：https://qdhhkj.blog.csdn.net/article/details/129680789 一、安装 1、yum安装依赖 yum install gcc gcc-c make unzip pcre pcre-devel …
阅读更多...
xss labs(11-14)

xss labs(11-14)

pass11这里是直接查看的源码的，结果发现也是有很多input标签，并且不知道为啥还有个标签的参数的我上一道题构造的表达式等我随便执行一个，他就没有了还是依照上一题的语句进行注入，结果发现还是只有一个回显了，而且对双…
阅读更多...
从零到亿学pytorch系列一：使用远程服务器在pycharm上运行简单的训练模型

从零到亿学pytorch系列一：使用远程服务器在pycharm上运行简单的训练模型

参考视频和代码来源详见up主Leo在这的b站教学视频：1、Pytorch的安装与环境配置【小学生都会的Pytorch】_哔哩哔哩_bilibili如何在pycharm上连接远程服务器详见：(7条消息) 如何使用租用的云服务器实现神经网络训练过程（超详细教程，…
阅读更多...
2023年3月广东软考中/高级报名在这里，高效备考

2023年3月广东软考中/高级报名在这里，高效备考

软考是全国计算机技术与软件专业技术资格（水平）考试（简称软考）项目，是由国家人力资源和社会保障部、工业和信息化部共同组织的国家级考试，既属于国家职业资格考试，又是职称资格考试。 系统集成…
阅读更多...
【Unity3D简单项目开发】疯狂点击01

【Unity3D简单项目开发】疯狂点击01

使用Unity3D的内置资源，制作一个简单的游戏项目，通过这个项目，掌握使用Unity3D开发简单游戏的一个基本流程。第一步，使用Unity Hub创建一个项目，选择一个Unity版本之后，点击创建即可。注意，在创…
阅读更多...
最新文章

Copyright @ 2022~2023