属性与文本选择

Python Web 爬取

Thomas Laetsch

Data Scientist, NYU

用好冒号需要勇气

  • 使用 XPath：<xpath-to-element>/@attr-name
xpath = '//div[@id="uid"]/a/@href'
  • 使用 CSS 定位器：<css-to-element>::attr(attr-name)
css_locator = 'div#uid > a::attr(href)'
Python Web 爬取

文本提取

<p id="p-example">
  Hello world! 
  Try <a href="http://www.datacamp.com">DataCamp</a> today!
</p>
  • 在 XPath 中使用 text()
sel.xpath('//p[@id="p-example"]/text()').extract()

# result: ['\n Hello world!\n Try ', ' today!\n']

sel.xpath('//p[@id="p-example"]//text()').extract()

# result: ['\n Hello world!\n Try ', 'DataCamp', ' today!\n']
Python Web 爬取

文本提取

<p id="p-example">
  Hello world! 
  Try <a href="http://www.datacamp.com">DataCamp</a> today!
</p>
  • 对于 CSS 定位器，使用 ::text
sel.css('p#p-example::text').extract()

# result: ['\n Hello world!\n Try ', ' today!\n']

sel.css('p#p-example ::text').extract()

# result: ['\n Hello world!\n Try ', 'DataCamp', ' today!\n']
Python Web 爬取

限定冒号作用域

Python Web 爬取

Preparing Video For Download...