PHP: How to download a webpage (aka web scrapping) with PHP PHP: Kuidas alla laadida veebilehel (teise nimega web lammutamiseks) PHP

Posted on 03. Postitatud 03. Oct, 2009 by Dragos in Coding , PHP Oktoober, 2009 Dragos in kodeerimise, PHP

There are many ways of downloading web pages, or web content. On mitmeid viise alla laadida veebilehti või veebi sisu. Personally I like to use cURL for my web scrapping needs, but sometimes I also use fsockopen and file_get_contents . Isiklikult mulle meeldib kasutada Curl minu web lammutamiseks peab, kuid mõnikord olen ka kasutada fsockopen ja file_get_contents.

Here are 3 different functions that will allow you to download web content. Siin on 3 eri funktsioone, mis võimaldavad alla laadida veebi sisu.

cURL : Curl:

 function getData($url) { funktsiooni getData ($ url) (
    if($url!='localhost' && $url!='http://localhost') { if ($ url! = 'localhost' & & $ url! = 'http://localhost') (
        $ch=curl_init(); $ ch = curl_init ();
        curl_setopt($ch, CURLOPT_URL, $url); curl_setopt ($ ch, CURLOPT_URL, $ url);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); curl_setopt ($ ch, CURLOPT_RETURNTRANSFER, true);
        curl_setopt($ch,CURLOPT_USERAGENT,"Mozilla/6.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.3"); curl_setopt ($ ch, CURLOPT_USERAGENT, "Mozilla/6.0 (compatible; Windows NT 5.1; en-US; rv: 1.9.0.1) Gecko/2008070208 Firefox/3.0.3");
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION,3); curl_setopt ($ ch, CURLOPT_FOLLOWLOCATION, 3);
        $result['data']=curl_exec($ch); $ tulemus [ 'data'] = curl_exec ($ ch);
        $result['error']=curl_error($ch); $ tulemus [ 'ERROR'] = curl_error ($ ch);
        curl_close($ch); curl_close ($ ch);
        return $result; return $ tulemus;
    } )
    else return $result['error']='err'; else return $ tulemus [ 'ERROR'] = 'viga';
} ) 

fsockopen fsockopen

 function getData($url) { funktsiooni getData ($ url) (
    $arr=parse_url($url); $ arr = parse_url ($ url);
    $fp = fsockopen($arr['host'], 80, $errno, $errstr, 30); $ fp = fsockopen ($ arr [ 'host'], 80, $ errno, $ errstr, 30);
    if(!$fp) { if (! $ fp) (
        return false; return false;
    }else { ) else (
    // send headers / / Send headers
        $out = "GET ".fsockopen($arr['host'], 80, $errno, $errstr, 30)." HTTP/1.1\r\n"; $ out = "GET". fsockopen ($ arr [ 'host'], 80, $ errno, $ errstr, 30). "HTTP/1.1 \ r \ n";
        $out .= "Host: ".str_replace('http://'.$arr['host'],'',$url)."\r\n"; $ out .= "Host:". str_replace ( 'http://'. $ arr [ 'host'],'',$ url). "\ r \ n";
        $out .= "User-Agent: FSOCKOPEN\r\n"; $ out .= "User-Agent: FSOCKOPEN \ r \ n";
        $out .= "Connection: Close\r\n\r\n"; $ out .= "Connection: Close \ r \ n \ r \ n";
        fwrite($fp, $out); fwrite ($ fp, $ out);
        while(!feof($fp)) { while (! feof ($ fp)) (
            $contents .= fgets($fp, 4096); $ sisu .= fgets ($ fp, 4096);
        }; );
        fclose($fp); fclose ($ fp);
        return $contents; return $ sisu;
    } )
} ) 

file_get_contents file_get_contents

 function getData($url) { funktsiooni getData ($ url) (
return file_get_contents($url); tagasi file_get_contents ($ url);
} ) 

As you see the easiest way of downloading web content is by using the file_get_contents function, but if you need more options, especially if you are working with the headers, then cURL is the best way to go for you. Nagu näete Lihtsaim viis alla laadida veebi sisu on kasutada file_get_contents funktsiooni, aga kui sul on vaja rohkem võimalusi, eriti kui te töötate päised, siis Curl on parim viis minna teile.

Related posts: Seonduvad postitused:

1. JavaScript: Send function as a parameter to another function (callbacks) Javascript: Saada funktsiooni parameeter teise funktsiooni (kutse,)