将ip转换为地理位置存入elasticsearch

有些需要将一些数据基于地址位置进行分析，比如哪些区域比较活跃，在什么时间范围内活跃，但是后端仅能获取ip地址，因此需要将ip地址转换为地理位置，幸运的是我们有开源的工具可以使用，maxmind/GeoIP2-java使用GeoLite2-City.mmdb库就可以由ip分析得到对应的经纬度，下面给出具体操作步骤：

1、从https://dev.maxmind.com/geoip/geoip2/geolite2/下载免费的GeoLite2-City库，但是准确度不如收费的geoip2-city库

2、创建elasticsearch索引

3、更新映射

4、将数据插入到elasticsearch,ip解析经纬度参考https://github.com/maxmind/GeoIP2-java，

代码语言：javascript复制

     //使用RestHighLevelClient BulkRequest批量插入数据
     @Test
	public void createSubBankIndex(){
    	List<LogDetail>  details = logDetailMapper.getLogDetails();
    	
    	//bulkRequest.timeout("2m");
		DatabaseReader reader = null;
		try {
			Resource resource = new ClassPathResource("GeoLite2-City.mmdb");
			InputStream file = resource.getInputStream();				
			reader = new DatabaseReader.Builder(file).build();
		} catch (IOException e1) {
			// TODO Auto-generated catch block
			e1.printStackTrace();
		}
		if(null == reader)
			return;
    	for(int i=0,len=details.size();i<len;){
    		int sz = len - i > 2000?2000: len - i;
    		List<LogDetail> subBanks = details.subList(i, sz i);
    		i  = sz;
        	BulkRequest bulkRequest = new BulkRequest("log_detail1104", "log_detail");
        	bulkRequest.timeout(TimeValue.timeValueMinutes(2));   
        	IndexRequest indexRequest = new IndexRequest("log_detail1104", "log_detail");
        	GeoLocation loc = new GeoLocation();
    		for(LogDetail b:subBanks){
    			GeoPoint gp = ip2Geo(reader,b.getIpAddress());
    			loc.setLat(gp.getLat());
    			loc.setLon(gp.getLon());
    			b.setLocation(loc);
    			//indexRequest.timeout(TimeValue.timeValueMinutes(2)); 
    			indexRequest.source(JSON.toJSONString(b), XContentType.JSON);
    			bulkRequest.add(indexRequest);			
    		}
    		
    		try {
    			highLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);
    			//highLevelClient.index(indexRequest, RequestOptions.DEFAULT);
    		} catch (IOException e) {
    			// TODO Auto-generated catch block
    			e.printStackTrace();
    		}	    		
    	}	
	}


    //将ip转换为地理位置信息
	private GeoPoint ip2Geo(DatabaseReader reader,String ip){
		try {
			InetAddress ipAddress = InetAddress.getByName(ip);

			// Replace "city" with the appropriate method for your database, e.g.,
			// "country".
			CityResponse response = reader.city(ipAddress);		


			Country country = response.getCountry();
			//System.out.println(country.getIsoCode());            // 'US'
			//System.out.println(country.getName());               // 'United States'
			System.out.println(country.getNames().get("zh-CN")); // '美国'

			Subdivision subdivision = response.getMostSpecificSubdivision();
			System.out.println(subdivision.getName());    // 'Minnesota'
			//System.out.println(subdivision.getIsoCode()); // 'MN'

			City city = response.getCity();
			System.out.println(city.getName()); // 'Minneapolis'

			Postal postal = response.getPostal();
			//System.out.println(postal.getCode()); // '55455'

			Location location = response.getLocation();
			System.out.println(location.getLatitude());  // 44.9733
			System.out.println(location.getLongitude()); // -93.2323	
			
			GeoPoint gp = new GeoPoint(location.getLatitude(),location.getLongitude());
			return gp;
		} catch (Exception e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		return null;
	}

5、运行后效果图

https java tcp/ip 网络安全 ElasticsearchService

0 人点赞