五月 26th, 2010
python 抓取页面
Python, by cnphpbb.
好久没用了。再把python拣起来,下面的代码只是测试下程序。
#coding=utf-8
import sys
import urllib2
import gzip
import StringIO
# 页面url
url = "http://www.915.com/news/201005/25-052512922010.html"
# 页面编码
page_encode = "utf8"
request = urllib2.Request(url)
request.add_header("Accept-encoding", "gzip")
usock = urllib2.urlopen(request)
page = usock.read()
# 处理gzip过的页面
if usock.headers.get('content-encoding', None) == 'gzip':
page = gzip.GzipFile(fileobj=StringIO.StringIO(page)).read()
# 转unicode(gbk/utf8)
if not isinstance(page, unicode):
page = unicode(page, page_encode)
print(page)
Tags: Python
Back Top
回复自“python 抓取页面”
发表回复