selenium、chromedirver

代码语言：javascript复制

driver = webdriver.Chrome()
driver.get("https://www.zhipin.com/web/geek/job?query=数据分析&city=100010000")

获取当前页面信息，一定要等待网页加载完成，针对反爬措施，要点手动与自动相结合，手动登录（自动难以实现），翻页可以自动，一定要注意到driver始终获取的是当前页面最新信息

代码语言：javascript复制

elements = driver.find_elements(By.CSS_SELECTOR, ".job-card-wrapper")
jobinfo_csv(elements)

代码语言：javascript复制

def jobTaglist(taglist):
    strTaglist = ""
    for el in taglist:
        strTaglist  = el.text "*"
    return strTaglist
def jobinfo_csv(elements):
    jobInfo = {}
    for element in elements:
        jobInfo["job-name"] = element.find_element(By.CSS_SELECTOR, ".job-name").text
        jobInfo['job-area'] = element.find_element(By.CSS_SELECTOR, ".job-area").text
        jobInfo['salary'] = element.find_element(By.CSS_SELECTOR, ".salary").text
        t = element.find_element(By.CSS_SELECTOR, ".job-card-footer .tag-list").find_elements(By.XPATH, "./li")
        jobInfo['technologylist']= jobTaglist(t)
        taglist = element.find_element(By.CSS_SELECTOR, ".tag-list").find_elements(By.XPATH, "./li")
        jobInfo['taglist']= jobTaglist(taglist)
        companyTaglist = element.find_element(By.CSS_SELECTOR, ".company-tag-list").find_elements(By.XPATH, "./li")
        jobInfo['companyTaglist']= jobTaglist(companyTaglist)
        jobInfo['company-name'] = element.find_element(By.CSS_SELECTOR, ".company-name").text
        jobInfo['info-desc'] = element.find_element(By.CSS_SELECTOR, ".info-desc").text
        pddata = pd.DataFrame([jobInfo])
        pddata.to_csv('static.csv',mode='a',encoding='utf-8',header=False,index=False)

driver 登录

0 人点赞