PHP: How to download a webpage (aka web scrapping) with PHP PHP: Cara untuk men-download halaman web (web alias berkelahi) dengan PHP

Posted on 03. Posted on 03. Oct, 2009 by Dragos in Coding , PHP Oct, 2009 oleh Dragos di Coding, PHP

There are many ways of downloading web pages, or web content. Ada banyak cara untuk men-download halaman web, atau konten web. Personally I like to use cURL for my web scrapping needs, but sometimes I also use fsockopen and file_get_contents . Secara pribadi saya suka menggunakan cURL untuk kebutuhan membuang web saya, tapi kadang-kadang saya juga menggunakan fsockopen dan file_get_contents.

Here are 3 different functions that will allow you to download web content. Berikut adalah 3 fungsi yang berbeda yang akan memungkinkan Anda untuk men-download konten web.

cURL : Curl:

 function getData($url) { fungsi getData ($ url) ( 
     if($url!='localhost' && $url!='http://localhost') { if ($ url! = 'localhost' & & $ url! = 'http://localhost') ( 
         $ch=curl_init(); $ ch = curl_init (); 
         curl_setopt($ch, CURLOPT_URL, $url); curl_setopt ($ ch, CURLOPT_URL, $ url); 
         curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); curl_setopt ($ ch, CURLOPT_RETURNTRANSFER, TRUE); 
         curl_setopt($ch,CURLOPT_USERAGENT,"Mozilla/6.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.3"); curl_setopt ($ ch, CURLOPT_USERAGENT, "Mozilla/6.0 (Windows; U; Windows NT 5.1; en-US; rv: 1.9.0.1) Gecko/2008070208 Firefox/3.0.3"); 
         curl_setopt($ch, CURLOPT_FOLLOWLOCATION,3); curl_setopt ($ ch, CURLOPT_FOLLOWLOCATION, 3); 
         $result['data']=curl_exec($ch); $ result [ 'data'] = curl_exec ($ ch); 
         $result['error']=curl_error($ch); $ result [ 'error'] = curl_error ($ ch); 
         curl_close($ch); curl_close ($ ch); 
         return $result; return $ hasil; 
     } ) 
     else return $result['error']='err'; else return $ result [ 'error'] = 'salah'; 
 } ) 

fsockopen fsockopen

 function getData($url) { fungsi getData ($ url) ( 
     $arr=parse_url($url); $ arr = parse_url ($ url); 
     $fp = fsockopen($arr['host'], 80, $errno, $errstr, 30); $ fp = fsockopen ($ arr [ 'host'], 80, $ errno, $ errstr, 30); 
     if(!$fp) { if (! $ fp) ( 
         return false; return false; 
     }else { ) else ( 
     // send headers / / Mengirim header 
         $out = "GET ".fsockopen($arr['host'], 80, $errno, $errstr, 30)." HTTP/1.1\r\n"; $ out = "GET". fsockopen ($ arr [ 'host'], 80, $ errno, $ errstr, 30). "HTTP/1.1 \ r \ n"; 
         $out .= "Host: ".str_replace('http://'.$arr['host'],'',$url)."\r\n"; $ out .= "Host:". str_replace ( 'http://'. $ arr [ 'host'],'',$ url). "\ r \ n"; 
         $out .= "User-Agent: FSOCKOPEN\r\n"; $ out .= "User-Agent: fsockopen \ r \ n"; 
         $out .= "Connection: Close\r\n\r\n"; $ out .= "Connection: Close \ r \ n \ r \ n"; 
         fwrite($fp, $out); fwrite ($ fp, $ out); 
         while(!feof($fp)) { while (! feof ($ fp)) ( 
             $contents .= fgets($fp, 4096); $ contents .= fgets ($ fp, 4096); 
         }; ); 
         fclose($fp); fclose ($ fp); 
         return $contents; return $ isi; 
     } ) 
 } ) 

file_get_contents file_get_contents

 function getData($url) { fungsi getData ($ url) ( 
 return file_get_contents($url); return file_get_contents ($ url); 
 } ) 

As you see the easiest way of downloading web content is by using the file_get_contents function, but if you need more options, especially if you are working with the headers, then cURL is the best way to go for you. Seperti yang Anda lihat cara termudah men-download konten web adalah dengan menggunakan fungsi file_get_contents, tapi jika Anda membutuhkan lebih banyak pilihan, terutama jika Anda bekerja dengan header, kemudian Curl adalah cara terbaik untuk pergi untuk Anda.

No related posts. No related posts.