PHP: How to download a webpage (aka web scrapping) with PHP PHP: Kaip parsisiųsti svetainės (taip pat žinomas kaip žiniatinklio metalo laužą) su PHP

Posted on 03. Posted on 03. Oct, 2009 by Dragos in Coding , PHP Spalis, 2009 Dragos ir kodavimo, PHP,

There are many ways of downloading web pages, or web content. Yra daug būdų atsisiųsti iš interneto puslapių, arba interneto turinio. Personally I like to use cURL for my web scrapping needs, but sometimes I also use fsockopen and file_get_contents . Asmeniškai aš norėčiau naudoti renka mano interneto poreikiai metalo laužą, bet kartais aš taip pat naudoti fsockopen ir file_get_contents.

Here are 3 different functions that will allow you to download web content. Čia yra 3 skirtingų funkcijų, kurios leis jums atsisiųsti interneto turinio.

cURL : curl:

 function getData($url) { funkcija getData ($ url) ( 
     if($url!='localhost' && $url!='http://localhost') { if ($ url! = 'localhost' & & $ url! = 'http://localhost') ( 
         $ch=curl_init(); $ ch = curl_init (); 
         curl_setopt($ch, CURLOPT_URL, $url); curl_setopt ($ ch, CURLOPT_URL, $ url); 
         curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); curl_setopt ($ ch, CURLOPT_RETURNTRANSFER, true); 
         curl_setopt($ch,CURLOPT_USERAGENT,"Mozilla/6.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.3"); curl_setopt ($ ch, CURLOPT_USERAGENT, "Mozilla/6.0 (Windows; U; Windows NT 5.1; en-US; rv: 1.9.0.1) Gecko/2008070208 Firefox/3.0.3"); 
         curl_setopt($ch, CURLOPT_FOLLOWLOCATION,3); curl_setopt ($ ch, CURLOPT_FOLLOWLOCATION, 3); 
         $result['data']=curl_exec($ch); $ result [ 'data'] = curl_exec ($ ch); 
         $result['error']=curl_error($ch); $ result [ 'error'] = curl_error ($ ch); 
         curl_close($ch); curl_close ($ ch); 
         return $result; return $ result; 
     } ) 
     else return $result['error']='err'; else return $ result [ 'error'] = "Err"; 
 } ) 

fsockopen fsockopen

 function getData($url) { funkcija getData ($ url) ( 
     $arr=parse_url($url); $ arr = parse_url ($ url); 
     $fp = fsockopen($arr['host'], 80, $errno, $errstr, 30); $ fp = fsockopen ($ arr [ 'host'], 80, $ errno, $ errstr, 30); 
     if(!$fp) { if (! $ fp) ( 
         return false; return false; 
     }else { ) else ( 
     // send headers / / Siųsti antraštes 
         $out = "GET ".fsockopen($arr['host'], 80, $errno, $errstr, 30)." HTTP/1.1\r\n"; $ out = "GET". fsockopen ($ arr [ 'host'], 80, $ errno, $ errstr, 30). "HTTP/1.1 \ r \ n"; 
         $out .= "Host: ".str_replace('http://'.$arr['host'],'',$url)."\r\n"; $ out .= "Host:". str_replace ( 'http://'. $ arr [ 'host'],'',$ URL). "\ r \ n"; 
         $out .= "User-Agent: FSOCKOPEN\r\n"; $ out .= "User-Agent: fsockopen \ r \ n"; 
         $out .= "Connection: Close\r\n\r\n"; $ out .= "Connection: Close \ r \ n \ r \ n"; 
         fwrite($fp, $out); fwrite ($ fp, $ out); 
         while(!feof($fp)) { while (! feof ($ fp)) ( 
             $contents .= fgets($fp, 4096); $ turinys .= fgets ($ fp, 4096); 
         }; ); 
         fclose($fp); fclose ($ fp); 
         return $contents; return $ turinys; 
     } ) 
 } ) 

file_get_contents file_get_contents

 function getData($url) { funkcija getData ($ url) ( 
 return file_get_contents($url); grįžti file_get_contents ($ url); 
 } ) 

As you see the easiest way of downloading web content is by using the file_get_contents function, but if you need more options, especially if you are working with the headers, then cURL is the best way to go for you. Kaip matote Lengviausias būdas atsisiųsti interneto turinio naudojant file_get_contents funkcija, bet jei jums reikia daugiau galimybių, ypač jei dirbate su antraštes, tada renka geriausia išeitis Jums.

No related posts. Nėra susijusių pranešimų.