Category Archives: PHP

PHP (recursive acronym for “PHP: Hypertext Preprocessor”) is a widely-used Open Source general-purpose scripting language that is especially suited for Web development and can be embedded into HTML.

URL Multiple Query Parameters Encoded with HTML Entities

I came across a situation where a URL with multiple query parameters was encoded using htmlentities() and PHP was not recognizing the query parameters using $_GET. A common case for encoding urls using htmlentities() is to use them inside XML documents.

So a url with multiple query parameters, encoded using htmlentities() would look like this:

http://www.bradino.com/?color=white&size=medium&quantity=3

and when that url is accessed the second and third query parameters are not recognized because instead of separating the subsequent variables with an & that character gets converted into &. I could not find a good way to resolve this, so basically I just encoded the query string back to normal using html_entity_decode() and then slammed the parameters back into the $_GET array using parse_str().

$query = html_entity_decode($_SERVER['QUERY_STRING']);
 
parse_str($query,$_GET);

There must be a better way! Anyone come across this before?

PHP Screen Scraping Class

After some positive feedback I have decided to continue to develop the PHP Screen Scraping class. This post will server as the permanent home for the class.

Download PHP Screen Scraping Class

Updates

20009-07-30 Added setHeader() function

CakePHP Missing Database Table Error

I am baking a new project management application at work and added a couple new tables to the database today. When I went into the console to bake the new models, they were not in the list…

php /path/cake/console/cake.php bake all -app /path/app/

So I manually typed in the model name and I got a missing database table for model error. I checked and double-checked and the database table was named properly. Turns out that some files inside the /app/tmp/cache/ folder were causing Cake not to recognize that I had added new tables to my database. Once I deleted the cache files cake instantly recognized my new database tables and I was baking away!

rm -Rf /path/app/tmp/cache/cake*

Screen Scraping Twitter

I got an email today asking for help to scrape Twitter. In particular, to be able to login. So I am going to show everyone, NOT to encourage anyone to violate Twitters terms of use but as an educational blog post about how PHP and cURL can be used to post variables and store cookies.

Again, I am using the cScrape class I wrote, which you can download.

Step 1
First go to twitter.com and look at the source code of the login to get the form field names and the form post location. You will see that the form posts to https://twitter.com/session and the username and password fields are session[username_or_email] and session[password] respectively.

Step 2
Now you are ready to login. So using the fetch function in the Scrape class you create an associative array to contain the form values you want to post. The other thing you will need to do is uncomment the lines for CURLOPT_COOKIEFILE and CURLOPT_COOKIEJAR. Cookies will be required to stay logged in and scrape around. The paths to the cookie files need to be writable by your app. Also you will need to uncomment the line about CURLOPT_FOLLOWLOCATION.

$data = array('session[username_or_email]' => "bradino", 'session[password]' => "secret");
 
$scrape->fetch('https://twitter.com/sessions',$data);

Step 1.5
Oops that didn’t work. All I got back was 403 Forbidden: The server understood the request, but is refusing to fulfill it. Ahhh I see another variable called authenticity_token I bet Twitter was looking for that. So let’s back up and first hit twitter.com to get the authenticity_token variable, and then make the login post request with that variable included in our array of parameters.

$scrape->fetch('https://twitter.com');
 
$data = array('session[username_or_email]' => "bradino", 'session[password]' => "secret");
 
$data['authenticity_token'] = $scrape->fetchBetween('name="authenticity_token" type="hidden" value="','"',$scrape->result);
 
$scrape->fetch('https://twitter.com/sessions',$data);
 
echo $scrape->result;

So that’s basically it. Now you are logged in and can scrape around and request other pages as you normally would. Sorry it wasn’t a longer post. I really do enjoy this kind of stuff so if anyone has a request, hit me up.

Errors?
1) Make sure that you are properly parsing the token variable
2) Make sure that you uncommented the lines about CURLOPT_COOKIEFILE and CURLOPT_COOKIEJAR, those options need to be enabled and be sure the path set is writable by your application
3) Make sure that the path to the cookie file is writable and that it is getting data written to it
4) If you get a message about being redirected you need to uncomment the line about CURLOPT_FOLLOWLOCATION, that option needs to be enabled true

Zipcodes in CSV Generation

When exporting to CSV format, then opening in a spreadsheet program like Excel zipcodes that start with a zero or zeroes have the preceding zeros stripped off. Obviously it is because the spreadsheet sees that column as integers and preceding zeros in integers are useless.

A quick and dirty trick to force Excel (hopefully you are using OpenOffice) to display the full zipcode, we wrap it in double quotes and put an equal sign in front of it, to force it to be a string like this:

$zipcode = 00123;
 
$data = '="' . $zipcode . '"' ;

So if you are doing the straight query to CSV export, using the fputcsv function it would look something like this. Basically just overwrite the value in the row and then continue along.

while ($row = mysql_fetch_assoc($query)){
 
	$row['zipcode'] = '="'.$row['zipcode'].'"';
 
	fputcsv($output, $row);
}

Spanish Character Translation

If you have a quick and dirty need to translate Spanish Characters into HTML codes you could use this function. Basically just uses an array where the key is the Spanish Character and the Value is the corresponding HTML code. This makes it easy to extend to cover any other character translation you need.

function handleSpanishCharacters($input){
 
	$translate = array();
	$translate['Á'] = "Á";
	$translate['É'] = "É";
	$translate['Í'] = "Í";
	$translate['Ó'] = "Ó";
	$translate['Ú'] = "Ú";
	$translate['Ñ'] = "Ñ";
	$translate['Ü'] = "Ü";
	$translate['á'] = "á";
	$translate['é'] = "í";
	$translate['í'] = "é";
	$translate['ó'] = "ó";
	$translate['ú'] = "ú";
	$translate['ñ'] = "ñ";
	$translate['ü'] = "ü";
	$translate['¿'] = "¿";
	$translate['¡'] = "¡";
	$translate['«'] = "«";
	$translate['»'] = "»";
	$translate['€'] = "€";
 
	$search = array_keys($translate);
 
	$replace = array_values($translate);
 
	return str_replace($search,$replace,$input);
 
}

2147483647

I had someone ask me the other day why they were getting 2147483647 as the value for some operation. Any time you see this number, it should be a red flag that you are hitting the ceiling of the values that can be the 4 byte integer.

So here in this classic newbie mistake, grooming a phone number by casting it to an integer.

$phone = '6195551234';
 
echo (int) $phone;

Will result in 2147483647.

You also see 2147483647 sometimes in database records where the field type is INT and the value you are inserting is greater than 2147483647. Hope this helps!

Google Analytics API

Unfortunately Google Analytics does not have an API yet… However I will show you a quick and dirty way to get the data you need. The idea is basically to setup an automated report from within Google Analytics, which will email a CSV file attachment, that you can download and parse with a script. For this example I will use the Zend Mail class because I love Zend Mail and it is free.

So the first step is to schedule the report. For this simple example we will use the default Visitors Report. So login to your Google Analytics account, click Visitors on the main left nav, then click the Email icon up at the top of the report, see screenshot below:

GA Screenshot
Google Analytics Screenshot

Now let’s get to the code. First step is to connect to your POP mail server and get the new messages. Don’t forget to download the latest version of Zend at http://framework.zend.com.

include ('Zend/Mail/Storage/Pop3.php');
 
$params = array('host' => "mail.domain.com", 'user' => "info@domain.com", 'password' => "secret");
 
$mail = new Zend_Mail_Storage_Pop3($params);
 
$num = $mail->countMessages();

In a production environment, you would want some logic to verify that the number of messages is greater than 0, loop through the messages, etc but for this tutorial I have omitted these items for brevity. So next we need to download the first message and get its parts.

$message = $mail->getMessage(1);
 
$parts = $message->countParts();

Next we spin through the parts, looking for the attachment.

for($b=1; $b<=$parts; $b++){
 
	$part = $message->getPart($b);
 
	$headers = $part->getHeaders();
 
	if (!empty($headers['Content-Disposition']) || !empty($headers['content-disposition'])){

Then we parse the CSV file to an array for simple looping:

$csv = base64_decode($part->getContent());
 
$lines = explode("\n",$csv);
 
$data = array();
 
foreach ($lines as $line){
 
	$data[] = explode(',',$line);					
 
}

The previous code checks the email account, downloads the CSV attachment and parses it to an array. That is all going to stay the same but the matching below will change depending on your needs. For this simplistic example, I am going to get the number of visits from yesterday.

$yesterday = date('F j',mktime(0, 0, 0, date('m'), date('d')-1, date('Y')));
 
foreach ($data as $row){
 
	if ($row[0] == '"'.$yesterday){
 
		$visits = $row[2];
 
		echo "VISITS: {$visits}";
 
	}
 
}

So that’s basically it. Of course in production you would probably want to write the Google Analytics data to a database or whatever, but you get the point! Hopefully Google Analytics will get an API soon, but until then, this works like a champ!

FTP Upload File

Need to transfer files via FTP in some automated or scheduled manner. It is actually quite easy to do using FTP. Following is a simplistic PHP FTP example to get you started.

Replace ‘hostname’, ‘username’, and ‘password’ with real FTP credentials, then we try and connect to the FTP server.

//FTP CONNECT
 
$connect = ftp_connect('hostname');
 
$login = ftp_login($connect, 'username', 'password');

if an error occurred connecting to the FTP server, then we can stop here:

//CONNECTION ERROR
 
if ((!$connect) || (!$login)) {
 
    echo "FTP Connection Failed";
 
    exit;
 
}

Otherwise let’s proceed and upload a file via FTP. Be sure to replace ‘/path/to/file.ext’ with the full path to your file, as well as ‘file.ext’ with the filename you would like to use when you FTP up the file.

//UPLOAD THE FILE
 
$file = fopen('/path/to/file.ext', 'r');
 
$upload = ftp_fput($connect, 'file.ext', $file, FTP_ASCII);
 
fclose($file);
 
ftp_close($connect);

and finally let’s perform a simple check to see if the file was uploaded successfully.

// UPLOAD ERROR
 
if (!$upload) {
 
    echo "FTP Upload Failed";
 
} 
 
// UPLOAD SUCCESS
 
else {
 
    echo "FTP Upload Success";
 
}

SQL SELECT WHERE IN

Here is a basic sql statement where you are looking for shirts that are red, white or blue.

[mysql]SELECT * FROM `shirts` WHERE `color`=’red’ OR `color`=’white’ OR `color`=’blue’[/mysql]

Here is the same request, more efficiently constructed using IN

[mysql]SELECT * FROM `shirts` WHERE `color` IN (‘red’,’white’,’blue’)[/mysql]

If you are like me and dynamically create SQL statements using PHP then you get the idea of how easy it would be to create this statement. Let’s say you have an array of possible value.

$colors = array('red','white','blue');
 
$sql = "SELECT * FROM `shirts` WHERE `color` IN ('" . implode("','", $colors) . "')" ;