我用 python 结合 selenium 编写了一个脚本来登录网站,然后从driver
to requests
这样我就可以继续使用requests
进行进一步的活动。
I used item = soup.select_one("div[class^='gravatar-wrapper-']").get("title")
此行用于检查脚本是否可以在一切完成后获取我的用户名。
这是我到目前为止的尝试:
import requests
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
url = "https://stackoverflow.com/users/login"
driver = webdriver.Chrome()
driver.get(url)
driver.find_element_by_css_selector("#email").send_keys("your_username")
driver.find_element_by_css_selector("#password").send_keys("your_password")
driver.find_element_by_css_selector("#submit-button").click()
driver_cookies = driver.get_cookies()
c = {c['name']:c['value'] for c in driver_cookies}
res = requests.get(driver.current_url,cookies=c)
soup = BeautifulSoup(res.text,"lxml")
item = soup.select_one("div[class^='gravatar-wrapper-']").get("title")
print(item)
driver.quit()
当我运行脚本时,它找不到用户名并给出 None 作为输出。
我怎样才能在之间传递cookieselenium
and requests
为了使用进行刮擦requests
我使用selenium登录后?
你已经走在正确的轨道上了。您现在需要做的就是让脚本稍等片刻以加载 cookie。您可以通过以下方式获得响应:
import time
import requests
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
url = "https://stackoverflow.com/users/login"
with webdriver.Chrome() as driver:
driver.get(url)
driver.find_element_by_css_selector("#email").send_keys("your_username")
driver.find_element_by_css_selector("#password").send_keys("your_password")
driver.find_element_by_css_selector("#submit-button").click()
time.sleep(5) #This is the fix
driver_cookies = driver.get_cookies()
c = {c['name']:c['value'] for c in driver_cookies}
res = requests.get(driver.current_url,cookies=c)
soup = BeautifulSoup(res.text,"lxml")
item = soup.select_one("div[class^='gravatar-wrapper-']").get("title")
print(item)
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)