PHP Screen Scraping

I had a request recently for help with scraping a little content from http://www.newyork411.com :)

So here we go. This time I made a quick PHP class with some basic functions to grab the source fo the page as well as fetchBetween, fetchAfter, fetchAll, etc. You can get the latest version of the class at http://www.bradino.com/downloads/cScrape.txt - be sure to rename it to cScrape.php. If there is an interest I can continue to develop this class a tool for screens craping with PHP.

Anyway so here we go scraping all the companies from this page http://www.newyork411.com/Ad_Agencies_Production_Companies/category-cid-50553.htm as well as the details of each company, found by clicking on the company.

Step 1 - Initialize the class and fetch the page:

include ('cScrape.php');

$scrape = new Scrape();

$url = 'http://www.newyork411.com/Ad_Agencies_Production_Companies/category-cid-50553.htm';

$scrape->fetch($url);

$data = $scrape->removeNewlines($scrape->result);

Step 2 - find your anchor and get the chunk of html that contains what you want

$data = $scrape->fetchBetween('<table width="490" border="0" cellpadding="3"','</table>',$data,true);

$rows = $scrape->fetchAllBetween('<TR','</tr>',$data,true);

Step 3 - parse out the individual values and print out the first record for demo

foreach ($rows as $id => $row){
   
    $record = array();
   
    $cells = $scrape->fetchAllBetween('<td','</td>',$row,true);
   
    $record['company'] = strip_tags($cells[1]);
   
    $url = 'http://www.newyork411.com' . $scrape->fetchBetween('<a href="','">',$cells[1],false);
   
    $url = str_replace(' ','%20',$url);
   
    $scrape->fetch($url);
   
    $data2 = $scrape->removeNewlines($scrape->result);
   
    $data2 = $scrape->fetchBetween('<div id="tabText">','</div>',$data2,true);
   
    $data2 = $scrape->fetchAfter('</table>',$data2,false);

    $details = explode('<br />',$data2);
   
    $record['address'] = $details[0];
   
    $location = explode(',',$details[1]);
   
    $record['city'] = trim($location[0]);
   
    $location = explode(' ',trim($location[1]));
   
    $record['state'] = trim($location[0]);
   
    $record['zip'] = trim($location[1]);
   
    for($i=2; $i<=5; $i++){
       
        $detail = trim($details[$i]);
       
        if(substr($detail,0,6)=='Phone:') $record['phone'] = str_replace('Phone: ','',$detail);
       
        else if(substr($detail,0,4)=='Fax:') $record['fax'] = str_replace('Fax: ','',$detail);
       
        else if(substr($detail,0,4)=='Web:') $record['web'] = strip_tags(str_replace('Web: ','',$detail));
       
        else if(substr($detail,0,6)=='Email:') $record['email'] = strip_tags(str_replace('Email: ','',$detail));
       
    }
   
    print_r($record);
   
    die();
}

  • Digg
  • TwitThis
  • del.icio.us
  • Netvouz
  • description
  • Reddit
  • Furl
  • NewsVine
  • Simpy
  • Slashdot
  • Spurl
  • StumbleUpon
  • YahooMyWeb
  • TailRank
  • Technorati
  • Facebook
  • Google
  • LinkedIn
  • Live
  • MySpace
  • Ping.fm
  • Yahoo! Buzz
  • E-mail this story to a friend!


Home | PHP | PHP Screen Scraping

You’re currently reading “ PHP Screen Scraping ,” an entry on BRADINO

Published:
6.2.08 / 2pm
Category:
PHP, Screen Scraping


Follow me on Twitter