scrapy源码编译安装教程（scrapy库使用教程）
本文目录一览：
scrapy怎么安装？急！！
1.安装Python
安装完了记得配置环境，将python目录和python目录下的Scripts目录添加到系统环境变量的Path里。在cmd中输入python如果出现版本信息说明配置完毕（如下面截图）。python下载网站，。请选择对应的Python版本安装。我装了2.7
（记得设置环境变量）
2.安装lxml lxml是一种使用 Python 编写的库，可以迅速、灵活地处理 XML。下载地址 ，可用pip下载，
下载命令：python -m pip install lxml
3.安装setuptools 一般都已经安装了的，可在cmd中用 python -m pip list 查看是否已经安装，如果没安装，用下载命令：python -m pip install setuptools安装，下载地址
4.安装zope.interface
可在cmd中用下载命令：python -m pip install zope.interface
也可以使用第三步下载的setuptools来安装egg文件，现在也有exe版本，点击下载。
5.安装Twisted
可在cmd中用下载命令：python -m pip install Twisted，它是用Python实现的基于事件驱动的网络引擎框架。下载地址：
6.安装pyOpenSSL
pyOpenSSL是Python的OpenSSL接口， 可在cmd中用下载命令：python -m pip install pyOpenSSL安装，下载地址：
7.安装win32py 提供win32api，下载地址：
8.安装Scrapy
直接在cmd中输入easy_install scrapy回车即可。安装后在cmd命令行输入scrapy，如果出现下图，则安装成功。
怎么用anaconda安装scrapy？
anaconda中安装scrapyscrapy源码编译安装教程的方法：
步骤1：在安装Anaconda的情况下，只需在cmd窗口输入：conda install scrapy ，按回车就可以scrapy源码编译安装教程；
步骤2：检测scrapy是否安装成功,在cmd窗口输入scrapy回车查看scrapy源码编译安装教程；
步骤3：在pycharm中输入import scrapy ，没有报错，安装成功。
更多Python知识，请关注Python视频教程！！
如何在ubuntu中安装scrapy
Scrapy是Python开发的一个快速,高层次的屏幕抓取和web抓取框架scrapy源码编译安装教程，用于抓取web站点并从页面中提取结构化的数据。Scrapy用途广泛scrapy源码编译安装教程，可以用于数据挖掘、监测和自动化测试。 官网网站
1、安装如下软件
sudo apt-get install build-essential;
sudo apt-get install python-dev;
sudo apt-get install libxml2-dev;
sudo apt-get install libxslt1-dev;
sudo apt-get install python-setuptools;
2、安装Scrapy
sudo easy_install Scrapy;
wang@ubuntu:/usr/local/lib/python2.7/dist-packages$ sudo easy_install Scrapy
Searching for Scrapy
Best match: Scrapy 0.16.1
Processing Scrapy-0.16.1-py2.7.egg
Scrapy 0.16.1 is already the active version in easy-install.pth
Installing scrapy script to /usr/local/bin
Using /usr/local/lib/python2.7/dist-packages/Scrapy-0.16.1-py2.7.egg
Processing dependencies for Scrapy
Searching for lxml
Reading
Reading
Best match: lxml 3.0.1
Downloading
Processing lxml-3.0.1.tar.gz
Running lxml-3.0.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-qibAzL/lxml-3.0.1/egg-dist-tmp-mSvUVN
Building lxml version 3.0.1.
Building without Cython.
Using build configuration of libxslt 1.1.26
Building against libxml2/libxslt in the following directory: /usr/lib/x86_64-linux-gnu
warning: no files found matching '*.txt' under directory 'src/lxml/tests'
src/lxml/lxml.etree.c: In function ‘__pyx_f_4lxml_5etree__getFilenameForFile’:
src/lxml/lxml.etree.c:26310:7: warning: variable ‘__pyx_clineno’ set but not used [-Wunused-but-set-variable]
src/lxml/lxml.etree.c:26309:15: warning: variable ‘__pyx_filename’ set but not used [-Wunused-but-set-variable]
src/lxml/lxml.etree.c:26308:7: warning: variable ‘__pyx_lineno’ set but not used [-Wunused-but-set-variable]
src/lxml/lxml.etree.c: In function ‘__pyx_pf_4lxml_5etree_4XSLT_18__call__’:
src/lxml/lxml.etree.c:132608:81: warning: passing argument 1 of ‘__pyx_f_4lxml_5etree_12_XSLTContext__copy’ from incompatible pointer type [enabled by default]
src/lxml/lxml.etree.c:130569:52: note: expected ‘struct __pyx_obj_4lxml_5etree__XSLTContext *’ but argument is of type ‘struct __pyx_obj_4lxml_5etree__BaseContext *’
src/lxml/lxml.etree.c: In function ‘__pyx_f_4lxml_5etree__copyXSLT’:
src/lxml/lxml.etree.c:133997:79: warning: passing argument 1 of ‘__pyx_f_4lxml_5etree_12_XSLTContext__copy’ from incompatible pointer type [enabled by default]
src/lxml/lxml.etree.c:130569:52: note: expected ‘struct __pyx_obj_4lxml_5etree__XSLTContext *’ but argument is of type ‘struct __pyx_obj_4lxml_5etree__BaseContext *’
src/lxml/lxml.etree.c: At top level:
src/lxml/lxml.etree.c:12128:13: warning: ‘__pyx_f_4lxml_5etree_displayNode’ defined but not used [-Wunused-function]
src/lxml/lxml.etree.c: In function ‘__pyx_f_4lxml_5etree_11_BaseParser__parseDocFromFile’:
src/lxml/lxml.etree.c:86715:3: warning: ‘__pyx_r’ may be used uninitialized in this function [-Wuninitialized]
src/lxml/lxml.etree.c: In function ‘__pyx_f_4lxml_5etree_11_BaseParser__parseDoc’:
src/lxml/lxml.etree.c:86403:3: warning: ‘__pyx_r’ may be used uninitialized in this function [-Wuninitialized]
src/lxml/lxml.etree.c: In function ‘__pyx_f_4lxml_5etree_11_BaseParser__parseUnicodeDoc’:
src/lxml/lxml.etree.c:86093:3: warning: ‘__pyx_r’ may be used uninitialized in this function [-Wuninitialized]
src/lxml/lxml.etree.c: In function ‘__pyx_f_4lxml_5etree_11_BaseParser__parseDocFromFilelike’:
src/lxml/lxml.etree.c:86925:3: warning: ‘__pyx_r’ may be used uninitialized in this function [-Wuninitialized]
Adding lxml 3.0.1 to easy-install.pth file
Installed /usr/local/lib/python2.7/dist-packages/lxml-3.0.1-py2.7-linux-x86_64.egg
Searching for w3lib=1.2
Reading
Reading
Best match: w3lib 1.2
Downloading
Processing w3lib-1.2.tar.gz
Running w3lib-1.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-ZAXTgy/w3lib-1.2/egg-dist-tmp-aU3vpc
zip_safe flag not set; analyzing archive contents...
Adding w3lib 1.2 to easy-install.pth file
Installed /usr/local/lib/python2.7/dist-packages/w3lib-1.2-py2.7.egg
Searching for Twisted=8.0
Reading
Reading
Reading
Reading
Reading
Reading
Reading
Reading
Reading
Best match: Twisted 12.2.0
Downloading
Processing Twisted-12.2.0.tar.bz2
Running Twisted-12.2.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-kw897y/Twisted-12.2.0/egg-dist-tmp-sZWFYb
In file included from /usr/include/python2.7/Python.h:8:0,
from twisted/internet/_sigchld.c:9:
/usr/include/python2.7/pyconfig.h:1161:0: warning: "_POSIX_C_SOURCE" redefined [enabled by default]
/usr/include/features.h:215:0: note: this is the location of the previous definition
twisted/internet/_sigchld.c: In function ‘got_signal’:
twisted/internet/_sigchld.c:15:13: warning: variable ‘ignored_result’ set but not used [-Wunused-but-set-variable]
Adding Twisted 12.2.0 to easy-install.pth file
Installing mailmail script to /usr/local/bin
Installing conch script to /usr/local/bin
Installing pyhtmlizer script to /usr/local/bin
Installing twistd script to /usr/local/bin
Installing lore script to /usr/local/bin
Installing tkconch script to /usr/local/bin
Installing tapconvert script to /usr/local/bin
Installing ckeygen script to /usr/local/bin
Installing tap2rpm script to /usr/local/bin
Installing manhole script to /usr/local/bin
Installing trial script to /usr/local/bin
Installing cftp script to /usr/local/bin
Installing tap2deb script to /usr/local/bin
Installed /usr/local/lib/python2.7/dist-packages/Twisted-12.2.0-py2.7-linux-x86_64.egg
Finished processing dependencies for Scrapy
表示安装成功。
3、测试
scrapy shell
获取所有a标签
hxs.select('//a').extract()
参考资料
