Scrapelist The art of content scraping

Display Steam games list on your website

Here is a simple PHP function to scrape your games list off your Steam profile. You can select games from the recent, all, and wishlist tab lists.

By looking at the source code of your profile's games list, you'll notice that every game title is wrapped in the h4 element. No other content on the page uses the h4 element so it is easy to extract the titles. Using the preg_match_all() function you can grab all text between the h4 tags and store it in an array.

There are also a few XML sources for Steam community data that include Player Profile, Player Game Stats, Leaderboards, and Groups. Also available is a Team Fortress 2 API

Usage:
$steamname is your Steam nickname id that is in the URL of your public profile. http://steamcommunity.com/id/nickname
$list argument can accept all, recent, or wishlist

Display Array data for test:

print_r( steam_games_list('nickname','recent') );

The function:

function steam_games_list($steamname,$list="all"){

     //only allow input of exactly recent|all|wishlist
     $list = (preg_match('/^recent$|^all$|^wishlist$/',$list))?$list:"all";

     //wishlist url is different
     if($list == "wishlist"){
     $url = "http://steamcommunity.com/id/".$steamname."/wishlist/";
     }else{
     $url = "http://steamcommunity.com/id/".$steamname."/games/?tab=".$list;
     }

     $games = file_get_contents($url);		

     //all game titles are in H4, no other H4 on page
     preg_match_all("|<h4>(.*?)</h4>|",$games,$match);
     $games = $match[0];

     //remove h4 and other possible tags from name, just in case
     foreach($games as $key=>$val){
	$games[$key] = trim(strip_tags($val));
     }

     return $games;
}

Useful PHP functions for scraping content

file_get_contents()

Reads an entire file or URL into a string. Some hosts may disallow this function and you'll have to use cURL instead. You can do more with cURL anyways.

Example:

$page_contents = file_get_contents("http://example.com/page.html");

preg_match()

Used to capture a piece of content within a matched regular expression.

Example:

preg_match("/href=(.*?)title=(.*?)/",$page_contents,$match);

$match will have an array of content matched in order from each (.*?).

preg_match_all()

Matches your regular expression many times in your content to produce an array of matches. Useful for getting a list of items that have the same string pattern around them. Such as your Steam games list.

Example:

preg_match("/<h4>(.*?)</h4>/",$page_contents,$match);

..will create an array list of all h4 headings.

print_r() or var_dump()

Display your data and arrays to test if your code is getting the content you want.

str_replace() or preg_replace()

Replaces strings in content. Good for cleaning up extra junk or modifying content. str_ matches literal strings, and preg_ can use regular expressions.

Example:

$content = str_replace("<h4>","<h3>",$content)

Other useful features of PHP

Traverse the DOM and SimpleXML parser