I need a response to be identifiable after it fulfilled a request.

For smaller requests, I'm using the code below and it's working fine.

For larger requests, I could write the results to disk with sequential file naming for post sequential compilation, rather than storing all the results in memory.

I'm thinking the best way to handle it dynamically, would be to create a function to write sequentially named files to disk, based on a defined size/memory threshold, otherwise, handle via an in-memory associative array as below.

Thoughts?

function request_callback($response,$info) {
global $rc_result ;
// get last character of URL to enable indexing of results
// e.g., $url[] = http://mydomain.com?request=1, $url[] = http://mydomain.com?request=2 , etc.
$index_id = $info[url][strlen($info[url])-1];
$rc_result[$index_id] = $response ;
}

$rc = new RollingCurl("request_callback");
$rc->window_size = 10;
foreach ($urls as $url) {
$request = new Request($url);
$rc->add($request);
}
$rc->execute();

ksort($rc_result,SORT_NUMERIC);
print_r($rc_result);

Who cares whether it's technically forking, threading, or otherwise….it works as advertised.

One small issue on the blog presentation. While I know it should be obvious that the current code lives on google, I think it would be advantages (from a visual quickscan standpoint) to replace the old code in the black area (on top of the blog post) with the current meat and potatoes, end result, functionality from the example on Google:

<code>
/*
authored by Josh Fraser (http://www.joshfraser.com)
released under Apache License 2.0
*/

// a little example that fetches a bunch of sites in parallel and echos the page title and response info for each request

require("RollingCurl.php");

// top 20 sites according to alexa (11/5/09)
$urls = array("http://www.google.com",
"http://www.facebook.com",
"http://www.yahoo.com",
"http://www.youtube.com",
"http://www.live.com",
"http://www.wikipedia.com",
"http://www.blogger.com",
"http://www.msn.com",
"http://www.baidu.com",
"http://www.yahoo.co.jp",
"http://www.myspace.com",
"http://www.qq.com",
"http://www.google.co.in",
"http://www.twitter.com",
"http://www.google.de",
"http://www.microsoft.com",
"http://www.google.cn",
"http://www.sina.com.cn",
"http://www.wordpress.com",
"http://www.google.co.uk");

function request_callback($response, $info) {
// parse the page title out of the returned HTML
if (eregi ("<title>(.*)</title>", $response, $out)) {
$title = $out[1];
}
echo "$title
";
print_r($info);
echo "<hr>";
}

$rc = new RollingCurl("request_callback");
$rc->window_size = 20;
foreach ($urls as $url) {
$request = new Request($url);
$rc->add($request);
}
$rc->execute();
</code>

And provide the direct Google trunk link for those who are not used to working with a repository:

http://code.google.com/p/rolling-curl/source/brow...

I know for myself, when I'm scanning a bunch of sites for the right solution, it's nice to see a quick visual reference.

Cheers.

About the only thing I can think of at the moment is flagging each URL in the array and recursively processing until they are either flagged as completed or error. I will keep you posted.

I added a reference to the parent class in the Rolling Curl constructer:
function __construct($callback = null,$cls=null) {
$this->parentClass = $cls;
$this->callback = $callback;

To target the callback function of the parent class, I replaced:
// Send the return values to the callback function.
$callback = $this->callback;
$callback($output, $info);
With:
$this->callback= str_replace('$this->','',$this->callback); //remove '$this->' from the string
eval('$this->parentClass->'.$this->callback.';');

In my Parent Class, I called rolling curl like this:
$callback = '$this->attributeHTMLScraper($output,$info,"/html/body//a")';
$this->rc = new RollingCurl($callback,$this);

As an alternative, I tried the below but could not get the $vArgs array to show up in the attributeHTMLScraper function.
$vArgs = array($response,$info,'/html/body//a');
call_user_func_array($this->parentClass->attributeHTMLScraper,$vArgs );

If anyone knows a more elegant way to do this, I would be much appreciative. Still deep in that learning phase.

Doesn't Work: ($response, $info is null, $scrape works):
$callback = $this->attributeHTMLScraper($response,$info,$scrape="/html/body//a");
$this->rc = new RollingCurl($callback);

Here is the attributeHTMLScraper method in the parent class:
public function attributeHTMLScraper($response,$info,$scrape)
{
var_dump($response); var_dump($info); //Both Null
$dom = new DOMDocument();
$dom->loadHTML($reponse);

$xpath = new DOMXPath($dom);
$hrefs = $xpath->evaluate($scrape);

if(!is_null($hrefs)):
for ($i = 0; $i < $hrefs->length; $i++):
$href = $hrefs->item($i);
$url = $href->getAttribute('href');
$result[] =$url;
endfor;
endif;

//Return a simple variable if 1 value is returned. Else return an array
if(count($result)==1):
return $result[0];
else:
return $result;
endif;

return $result;
}

How can I pass these parent methods to the callback variable?