背景 由于现代社会迅速发展,AI模型可以做的事情也越来越多,大家的学习效率突飞猛进,那么一些同学也渐渐接触到python ,学习 selenium 处理一些自动化的事情,例如 爬取数据等等,那么作为研发同学,应该如何避免这些事情呢?
为什么要设置防刷?自动话爬取数据对正在进行的系统有哪些影响呢?
1、影响真实的用户,降低系统的处理效能
2、增加服务压力,造成不能预估的问题
3、避免恶意爬取,盗取数据
如何实现?
本次实现使用JDK21 、SpringBoot 3.1.6
首先定义一个注解@AccessLimit
这里只实现一个最简单的防刷,一些更加深入的应用,各位同学自行增加。
这里主要通过限流时间和最大请求数量,这并不能完全解决盗刷问题,只是展示一种方案
代码语言:java复制package com.lzmvlog.demoannotation.annotation;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
/**
* @author xiao.zhang zhang1591313226@163.com
* @since 2023-12-07
*/
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD})
public @interface AccessLimit {
/**
* 限流时间
*
* @return
*/
int seconds();
/**
* 最大请求次数
*
* @return
*/
int maxCount();
}
添加拦截器,这里使用HandlerInterceptor
,官方对HandlerInterceptor
得解释是:工作流接口,允许自定义处理程序执行链。应用程序可以注册任意数量的现有拦截器或自定义拦截器对于某些处理程序组,添加公共预处理行为无需修改每个处理程序实现。
package com.lzmvlog.demoannotation.handler;
import com.lzmvlog.demoannotation.annotation.AccessLimit;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import org.springframework.stereotype.Component;
import org.springframework.web.method.HandlerMethod;
import org.springframework.web.servlet.HandlerInterceptor;
import org.springframework.web.servlet.ModelAndView;
/**
* @author xiao.zhang zhang1591313226@163.com
* @since 2023-12-07
*/
@Component
public class LimitInterceptor implements HandlerInterceptor {
/**
* 模拟redis中的请求次数
*/
public Integer count = 0;
/**
* 处理程序执行前的拦截点。
*
* @param request current HTTP request
* @param response current HTTP response
* @param handler chosen handler to execute, for type and/or instance evaluation
* @return
* @throws Exception
*/
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
if (handler instanceof HandlerMethod) {
HandlerMethod hm = (HandlerMethod) handler;
//获取方法中的注解,看是否有该注解
AccessLimit accessLimit = hm.getMethodAnnotation(AccessLimit.class);
// 如果不包含注解直接放行
if (accessLimit == null) {
return true;
}
int seconds = accessLimit.seconds();
int maxCount = accessLimit.maxCount();
String url = request.getRequestURI();
System.out.println("请求地址:" url);
if (count < maxCount) {
//加1
count ;
} else {
return false;
}
}
return HandlerInterceptor.super.preHandle(request, response, handler);
}
/**
* 处理程序成功执行后的拦截点
*
* @param request current HTTP request
* @param response current HTTP response
* @param handler the handler (or {@link HandlerMethod}) that started asynchronous
* execution, for type and/or instance examination
* @param modelAndView the {@code ModelAndView} that the handler returned
* (can also be {@code null})
* @throws Exception
*/
@Override
public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView) throws Exception {
HandlerInterceptor.super.postHandle(request, response, handler, modelAndView);
}
/**
* 处理程序执行后的拦截点。
*
* @param request current HTTP request
* @param response current HTTP response
* @param handler the handler (or {@link HandlerMethod}) that started asynchronous
* execution, for type and/or instance examination
* @param ex any exception thrown on handler execution, if any; this does not
* include exceptions that have been handled through an exception resolver
* @throws Exception
*/
@Override
public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) throws Exception {
HandlerInterceptor.super.afterCompletion(request, response, handler, ex);
}
}
将拦截器进行配置
代码语言:java复制package com.lzmvlog.demoannotation.config;
import com.lzmvlog.demoannotation.handler.LimitInterceptor;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.InterceptorRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
/**
* @author xiao.zhang zhang1591313226@163.com
* @since 2023-12-07
*/
@Configuration
public class WebConfig implements WebMvcConfigurer {
private final LimitInterceptor limitInterceptor;
public WebConfig(LimitInterceptor limitInterceptor) {
this.limitInterceptor = limitInterceptor;
}
/**
* Add Spring MVC lifecycle interceptors for pre- and post-processing of
* controller method invocations and resource handler requests.
* Interceptors can be registered to apply to all requests or be limited
* to a subset of URL patterns.
*
* @param registry
*/
@Override
public void addInterceptors(InterceptorRegistry registry) {
registry.addInterceptor(limitInterceptor);
WebMvcConfigurer.super.addInterceptors(registry);
}
}
测试
代码语言:java复制package com.lzmvlog.demoannotation.controller;
import com.lzmvlog.demoannotation.annotation.AccessLimit;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.ResponseBody;
/**
* @author xiao.zhang zhang1591313226@163.com
* @since 2023-12-07
*/
@Controller
public class IndexController {
@AccessLimit(seconds = 5, maxCount = 5)
@GetMapping("/hello")
@ResponseBody
public String index() {
return "请求成功";
}
}
防刷只是服务安全种很小得一个防范举措,虽然解决不了大得问题,但是对于服务器压力得突发激增有一些效果,尤其是很多项目面对大众得情况下。各位同学慢慢学习,多多思考,希望大家都能每天有进步。