Scrapy 解析方法中的**kwargs具体怎么用
IPFS
示例:
import scrapy
class MySpider(scrapy.Spider):
name = 'myspider'
start_urls = ['http://www.example.com']
def start_requests(self):
for url in self.start_urls:
headers = {'User-Agent': 'Mozilla/5.0'}
yield scrapy.Request(url, callback=self.parse, headers=headers)
def parse(self, response, **kwargs):
headers = kwargs.get('headers')
self.logger.info(f'User-Agent = {headers.get("User-Agent")}')
# your parsing logic here
喜欢我的作品吗?别忘了给予支持与赞赏,让我知道在创作的路上有你陪伴,一起延续这份热忱!
- 来自作者
- 相关推荐