前言:周末闲来无事,玩了几局王者荣耀,突发奇想怎么获取到王者荣耀里面的英雄皮肤,本期分享一下如何通过matlab爬虫批量提取王者荣耀的英雄皮肤
关键字:王者荣耀、爬虫、Matlab
首先在度娘上找到王者荣耀的英雄主页
https://pvp.qq.com/web201605/herolist.shtml
通过chrome浏览器的F12,找到获取英雄的调用方法。
双击这个herolist.json,通过Notepad 打开可以看到里面对应的英雄详情
在matlab通过urlread函数获取到英雄列表情况,由于获取到的数据为char,需要转换为json格式,通过jsondecode函数实现转换
代码语言:javascript复制url = 'https://pvp.qq.com/web201605/js/herolist.json';
try
herolist = urlread(url);
catch
disp('提取英雄列表失败,请再次重试')
end
jsonData=jsondecode(herolist);
对每个英雄提取对应的ename、cname和skin_name变量,
代码语言:javascript复制[row, col] = size(jsonData);
hero_name = cell(row,1);
hero_number = cell(row,1);
hero_skin_number = cell(row,1);
for i = 1:row
hero_name{i} = jsonData{i,1}.cname;
hero_number{i} = jsonData{i,1}.ename;
try
skin_name = strsplit(jsonData{i,1}.skin_name, '|');
hero_skin_number{i} = length(skin_name);
catch
hero_skin_number{i} = 1;
end
end
下面开始要提取对应的图片,例如任意选择一个英雄,网址为:
https://pvp.qq.com/web201605/herodetail/508.shtml
右击选择图片-->检查,
可以看到对应的图片路径:
https://game.gtimg.cn/images/yxzj/img201606/heroimg/508/508-smallskin-2.jpg
根据上面的网址,可以得到对应的图片路径格式为:
https://game.gtimg.cn/images/yxzj/img201606/heroimg/{hero_number
}/{hero_number}-smallskin-{hero_skin_number}.jpg
通过webread函数提取到对应的图片,然后获取的图片通过imwrite保存成jpg文件。
代码语言:javascript复制onehero_link = strcat('http://game.gtimg.cn/images/yxzj/img201606/skin/hero-info/' ...
, num2str(hero_number{i}), '/', num2str(hero_number{i}), '-bigskin-', num2str(k),'.jpg');
try
image = webread(onehero_link);
catch
disp(['获取英雄',hero_name{i},'图片失败,请再次重试'])
end
imwrite(image,strcat(file_path_name_, num2str(k), '.jpg'));
通过urlread和webread在网络不好的情况下,可能会出现报错,所以通过try catch避免程序异常报错。对于每个英雄都单独保存在对应的名称文件夹下,通过mkdir创建文件夹。
完整的代码:
代码语言:javascript复制url = 'https://pvp.qq.com/web201605/js/herolist.json';
try
herolist = urlread(url);
catch
disp('提取英雄列表失败,请再次重试')
end
jsonData=jsondecode(herolist);
[row, col] = size(jsonData);
hero_name = cell(row,1);
hero_number = cell(row,1);
hero_skin_number = cell(row,1);
for i = 1:row
hero_name{i} = jsonData{i,1}.cname;
hero_number{i} = jsonData{i,1}.ename;
try
skin_name = strsplit(jsonData{i,1}.skin_name, '|');
hero_skin_number{i} = length(skin_name);
catch
hero_skin_number{i} = 1;
end
end
savepath = 'C:UsersxxxDocumentsMATLAB';
for i = 1:row
file_name = hero_name{i};
file_path_name = strcat(savepath,'',file_name);
file_path_name_ = strcat(file_path_name,'');
mkdir(file_path_name_);
for k = 1:hero_skin_number{i}
onehero_link = strcat('http://game.gtimg.cn/images/yxzj/img201606/skin/hero-info/' ...
, num2str(hero_number{i}), '/', num2str(hero_number{i}), '-bigskin-', num2str(k),'.jpg');
try
image = webread(onehero_link);
catch
disp(['获取英雄',hero_name{i},'图片失败,请再次重试'])
end
imwrite(image,strcat(file_path_name_, num2str(k), '.jpg'));
end
end
最后提取的结果如下
最喜欢的英雄程咬金结束本文
附上Python版本(推荐):
代码语言:javascript复制import os
import requests
url = 'https://pvp.qq.com/web201605/js/herolist.json'
herolist = requests.get(url) # 获取英雄列表json文件
herolist_json = herolist.json() # 转化为json格式
hero_name = list(map(lambda x: x['cname'], herolist.json())) # 提取英雄的名字
hero_number = list(map(lambda x: x['ename'], herolist.json())) # 提取英雄的编号
hero_skin_number = []
for i in herolist.json():
try:
hero_skin_number.append(len(i['skin_name'].split("|")))
except KeyError:
hero_skin_number.append(1)
# 下载图片
def downloadPic():
i = 0
for j in hero_number:
# 创建文件夹
os.mkdir("./" hero_name[i])
# 进入创建好的文件夹
os.chdir("./" hero_name[i])
i = 1
for k in range(1, hero_skin_number[i - 1] 1):
# 拼接url
onehero_link = 'http://game.gtimg.cn/images/yxzj/img201606/skin/hero-info/' str(j) '/' str(
j) '-bigskin-' str(k) '.jpg'
print(onehero_link)
im = requests.get(onehero_link) # 请求url
if im.status_code == 200:
open(str(k) '.jpg', 'wb').write(im.content) # 写入文件
os.chdir("../")
downloadPic()