PHP Screen Scraping

I had a request recently for help with scraping a little content from http://www.newyork411.com :)

So here we go. This time I made a quick PHP class with some basic functions to grab the source fo the page as well as fetchBetween, fetchAfter, fetchAll, etc. You can get the latest version of the class at http://www.bradino.com/downloads/cScrape.txt - be sure to rename it to cScrape.php. If there is an interest I can continue to develop this class a tool for screens craping with PHP.

Anyway so here we go scraping all the companies from this page http://www.newyork411.com/Ad_Agencies_Production_Companies/category-cid-50553.htm as well as the details of each company, found by clicking on the company.

Step 1 - Initialize the class and fetch the page:

PLAIN TEXT

PHP:

include ('cScrape.php');
$scrape = new Scrape();
$url = 'http://www.newyork411.com/Ad_Agencies_Production_Companies/category-cid-50553.htm';
$scrape->fetch($url);
$data = $scrape->removeNewlines($scrape->result);

Step 2 - find your anchor and get the chunk of html that contains what you want

PLAIN TEXT

PHP:

$data = $scrape->fetchBetween('<table width="490" border="0" cellpadding="3"','</table>',$data,true);
$rows = $scrape->fetchAllBetween('<TR','</tr>',$data,true);

Step 3 - parse out the individual values and print out the first record for demo

PLAIN TEXT

PHP:

foreach ($rows as $id => $row){
$record = array();
$cells = $scrape->fetchAllBetween('<td','</td>',$row,true);
$record['company'] = strip_tags($cells[1]);
$url = 'http://www.newyork411.com' . $scrape->fetchBetween('<a href="','">',$cells[1],false);
$url = str_replace(' ','%20',$url);
$scrape->fetch($url);
$data2 = $scrape->removeNewlines($scrape->result);
$data2 = $scrape->fetchBetween('<div id="tabText">','</div>',$data2,true);
$data2 = $scrape->fetchAfter('</table>',$data2,false);
$details = explode('<br />',$data2);
$record['address'] = $details[0];
$location = explode(',',$details[1]);
$record['city'] = trim($location[0]);
$location = explode(' ',trim($location[1]));
$record['state'] = trim($location[0]);
$record['zip'] = trim($location[1]);
for($i=2; $i<=5; $i++){
$detail = trim($details[$i]);
if(substr($detail,0,6)=='Phone:') $record['phone'] = str_replace('Phone: ','',$detail);
else if(substr($detail,0,4)=='Fax:') $record['fax'] = str_replace('Fax: ','',$detail);
else if(substr($detail,0,4)=='Web:') $record['web'] = strip_tags(str_replace('Web: ','',$detail));
else if(substr($detail,0,6)=='Email:') $record['email'] = strip_tags(str_replace('Email: ','',$detail));
}
print_r($record);
die();
}