这期内容当中小编将会给大家带来有关如何使用PHP 爬取网页,文章内容丰富且以专业的角度为大家分析和叙述,阅读完这篇文章希望大家可以有所收获。

成都创新互联公司主要从事成都网站建设、
成都做网站、网页设计、企业做网站、公司建网站等业务。立足成都服务上饶,十多年网站建设经验,价格优惠、服务专业,欢迎来电咨询建站服务:18982081108
1.file()函数
2.file_get_contents()函数
3.fopen()->fread()->fclose()模式
4.curl方式 (本人主要用这个)
5.fsockopen()函数 socket模式
6.插件(如:http://sourceforge.net/projects/snoopy/)
7.file()函数
2.使用file_get_contents方法实现,比较简单。
使用file_get_contents和fopen必须空间开启allow_url_fopen。方法:编辑php.ini,设置 allow_url_fopen = On,allow_url_fopen关闭时fopen和file_get_contents都不能打开远程文件。
$url="[http://news.sina.com.cn/c/nd/2016-10-23/doc-ifxwztru6951143.shtml](http://news.sina.com.cn/c/nd/2016-10-23/doc-ifxwztru6951143.shtml)";
$html=file_get_contents($url);
//如果出现中文乱码使用下面代码`
//$getcontent = iconv("gb2312", "utf-8",$html);
echo"";3.fopen()->fread()->fclose()模式 ,目前还没用过,看到了就先记下了
4.使用curl实现(本人一般使用这个)。
使用curl必须空间开启curl。方法:windows下修改php.ini,将extension=php_curl.dll前面的分号去掉,而且需 要拷贝ssleay32.dll和libeay32.dll到C:\WINDOWS\system32下;Linux下要安装curl扩展。
(.*?)<\/script>/',$res,$arr_all);//这个网页中数据通过js包过来,所以直接抓js就可以
preg_match_all('/"id"\:"(.*?)",/',$arr_all[1][1],$arr1);//从js块中匹配要的数据
$list = array_unique($arr1[1]);//(可省)保证不重复
//以下则是同理,循环则可
for($i=0;$i<=6;$i=$i+2){
$detail_url = 'ht*****em/'.$list[$i];
$detail_res = curl_get_contents($detail_url);
preg_match_all('/
基本
文件
流程
错误
SQL
调试
- 请求信息 : 2026-05-23 13:01:44 HTTP/1.1 GET : /article/pcdoj.html
- 运行时间 : 0.9379s ( Load:0.0063s Init:0.2800s Exec:0.6428s Template:0.0089s )
- 吞吐率 : 1.07req/s
- 内存开销 : 2,236.35 kb
- 查询信息 : 12 queries 5 writes
- 文件加载 : 36
- 缓存信息 : 0 gets 0 writes
- 配置加载 : 130
- 会话信息 : SESSION_ID=cm6egcudf3fdona20lqnl0jju4
- /www/wwwroot/tsicrk.com/index.php ( 1.09 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/ThinkPHP.php ( 4.61 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Think.class.php ( 12.26 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Storage.class.php ( 1.37 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Storage/Driver/File.class.php ( 3.52 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Mode/common.php ( 2.82 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Common/functions.php ( 53.56 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Hook.class.php ( 4.01 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/App.class.php ( 13.49 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Dispatcher.class.php ( 14.79 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Route.class.php ( 13.36 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Controller.class.php ( 11.23 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/View.class.php ( 7.59 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Behavior/BuildLiteBehavior.class.php ( 3.68 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Behavior/ParseTemplateBehavior.class.php ( 3.88 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Behavior/ContentReplaceBehavior.class.php ( 1.91 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Conf/convention.php ( 11.15 KB )
- /www/wwwroot/tsicrk.com/App/Common/Conf/config.php ( 2.14 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Lang/zh-cn.php ( 2.55 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Conf/debug.php ( 1.49 KB )
- /www/wwwroot/tsicrk.com/App/Home/Conf/config.php ( 0.31 KB )
- /www/wwwroot/tsicrk.com/App/Home/Common/function.php ( 3.33 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Behavior/ReadHtmlCacheBehavior.class.php ( 5.62 KB )
- /www/wwwroot/tsicrk.com/App/Home/Controller/ArticleController.class.php ( 6.02 KB )
- /www/wwwroot/tsicrk.com/App/Home/Controller/CommController.class.php ( 1.60 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Model.class.php ( 60.11 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Db.class.php ( 32.43 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Db/Driver/Pdo.class.php ( 16.74 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Cache.class.php ( 3.83 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Cache/Driver/File.class.php ( 5.87 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Template.class.php ( 28.16 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Template/TagLib/Cx.class.php ( 22.40 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Template/TagLib.class.php ( 9.16 KB )
- /www/wwwroot/tsicrk.com/App/Runtime/Cache/Home/7540f392f42b28b481b30614275e4e55.php ( 17.71 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Behavior/WriteHtmlCacheBehavior.class.php ( 0.97 KB )
- /www/wwwroot/tsicrk.com/ThinkPHP/Library/Behavior/ShowPageTraceBehavior.class.php ( 5.24 KB )
- [ app_init ] --START--
- Run Behavior\BuildLiteBehavior [ RunTime:0.000005s ]
- [ app_init ] --END-- [ RunTime:0.000028s ]
- [ app_begin ] --START--
- Run Behavior\ReadHtmlCacheBehavior [ RunTime:0.000270s ]
- [ app_begin ] --END-- [ RunTime:0.000289s ]
- [ view_parse ] --START--
- [ template_filter ] --START--
- Run Behavior\ContentReplaceBehavior [ RunTime:0.000050s ]
- [ template_filter ] --END-- [ RunTime:0.000070s ]
- Run Behavior\ParseTemplateBehavior [ RunTime:0.006028s ]
- [ view_parse ] --END-- [ RunTime:0.006051s ]
- [ view_filter ] --START--
- Run Behavior\WriteHtmlCacheBehavior [ RunTime:0.000178s ]
- [ view_filter ] --END-- [ RunTime:0.000193s ]
- [ app_end ] --START--
- 1064:You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ') LIMIT 1' at line 1
[ SQL语句 ] : SELECT `id`,`pid`,`navname` FROM `cx_nav` WHERE ( id= ) LIMIT 1
- 1064:You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ') LIMIT 1' at line 1
[ SQL语句 ] : SELECT `id`,`navname` FROM `cx_nav` WHERE ( id= ) LIMIT 1
- 1064:You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')' at line 1
[ SQL语句 ] : SELECT `id`,`navname` FROM `cx_nav` WHERE ( pid= )
- [8] Undefined index: pid /www/wwwroot/tsicrk.com/App/Home/Controller/ArticleController.class.php 第 47 行.
- [8] Undefined index: db_host /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Db.class.php 第 120 行.
- [8] Undefined index: db_port /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Db.class.php 第 121 行.
- [8] Undefined index: db_name /www/wwwroot/tsicrk.com/ThinkPHP/Library/Think/Db.class.php 第 122 行.

0.9379s
