PHP: How to download a webpage (aka web scrapping) with PHP PHP: Kako prenesti spletno stran (alias web razreza) z PHP

Posted on 03. Objavljeno dne 03. Oct, 2009 by Dragos in Coding , PHP Oktober 2009, ki ga Dragos v Coding, PHP

There are many ways of downloading web pages, or web content. Obstaja veliko načinov za prenos spletnih strani, ali spletne vsebine. Personally I like to use cURL for my web scrapping needs, but sometimes I also use fsockopen and file_get_contents . Osebno želim uporabljati cURL za moje potrebe spletne razrez, ampak včasih sem tudi uporabo fsockopen in file_get_contents.

Here are 3 different functions that will allow you to download web content. Tukaj so 3 različne funkcije, ki vam bo omogočilo, da naložite spletne vsebine.

cURL : cURL:

 function getData($url) { Funkcija getData ($ url) ( 
     if($url!='localhost' && $url!='http://localhost') { if ($ url! = 'localhost' & & $ url! = 'http://localhost') ( 
         $ch=curl_init(); $ ch = curl_init (); 
         curl_setopt($ch, CURLOPT_URL, $url); curl_setopt ($ ch, CURLOPT_URL, $ url); 
         curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); curl_setopt ($ ch, CURLOPT_RETURNTRANSFER, TRUE); 
         curl_setopt($ch,CURLOPT_USERAGENT,"Mozilla/6.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.3"); curl_setopt ($ ch, CURLOPT_USERAGENT, "Mozilla/6.0 (Windows; U; Windows NT 5.1; en-US; rv: 1.9.0.1) Gecko/2008070208 Firefox/3.0.3"); 
         curl_setopt($ch, CURLOPT_FOLLOWLOCATION,3); curl_setopt ($ ch, CURLOPT_FOLLOWLOCATION, 3); 
         $result['data']=curl_exec($ch); $ result [ 'podatki'] = curl_exec ($ ch); 
         $result['error']=curl_error($ch); $ result [ 'napaka'] = curl_error ($ ch); 
         curl_close($ch); curl_close ($ ch); 
         return $result; return $ rezultat; 
     } ) 
     else return $result['error']='err'; else return $ result [ 'napaka'] = 'napačno'; 
 } ) 

fsockopen fsockopen

 function getData($url) { Funkcija getData ($ url) ( 
     $arr=parse_url($url); $ arr = parse_url ($ url); 
     $fp = fsockopen($arr['host'], 80, $errno, $errstr, 30); $ fp = fsockopen ($ arr [ 'host'], 80, $ errno, $ errstr, 30); 
     if(!$fp) { if (! $ fp) ( 
         return false; return false; 
     }else { ) else ( 
     // send headers / / Pošlje glave 
         $out = "GET ".fsockopen($arr['host'], 80, $errno, $errstr, 30)." HTTP/1.1\r\n"; $ out = "GET". fsockopen ($ arr [ 'host'], 80, $ errno, $ errstr, 30). "HTTP/1.1 \ r \ n"; 
         $out .= "Host: ".str_replace('http://'.$arr['host'],'',$url)."\r\n"; $ iz .= "Host:". str_replace ( "http://". $ arr [ 'host'],'',$ url). "\ r \ n"; 
         $out .= "User-Agent: FSOCKOPEN\r\n"; $ iz .= "User-Agent: fsockopen \ r \ n"; 
         $out .= "Connection: Close\r\n\r\n"; $ iz .= "Connection: Close \ r \ n \ r \ n"; 
         fwrite($fp, $out); fwrite ($ fp, $ out); 
         while(!feof($fp)) { while (! feof ($ fp)) ( 
             $contents .= fgets($fp, 4096); $ vsebina .= fgets ($ fp, 4096); 
         }; ); 
         fclose($fp); fclose ($ fp); 
         return $contents; return $ vsebina; 
     } ) 
 } ) 

file_get_contents file_get_contents

 function getData($url) { Funkcija getData ($ url) ( 
 return file_get_contents($url); vrnitev file_get_contents ($ url); 
 } ) 

As you see the easiest way of downloading web content is by using the file_get_contents function, but if you need more options, especially if you are working with the headers, then cURL is the best way to go for you. Kot vidite je najlažji način za prenos spletnih vsebin s pomočjo file_get_contents funkcijo, če pa potrebujete več možnosti, še posebej, če delate z glavami, potem cURL je najboljša pot za vas.

No related posts. No related posts.