SRL Archive - Web Development

I currently running into problems when my php scripts run too long (probably the same issue as the cronjob).

[php]
<?php
set_time_limit(60);
$num = 615;
for($n=$num; $n < ($num + 50); $n++)
{
$url = 'http://services.runescape.com/m=itemdb_rs/3450_obj_sprite.gif?id='.$n;

$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$page_data = curl_exec ($ch);
curl_close ($ch);

if(preg_match('/Page not found/', $page_data)){
echo $n." –> 404<br />";
continue;
}

$i = file_get_contents($url);
$f = file_put_contents('******.'.gif', $i);

echo $n.' success<br />';
set_time_limit(30);
}
?>
[/php]This is downloading me all the sprites from the GE database. However without limiting the loop, it would run through 60 items before stopping and printing the data. So, the time limit sets don't seem to help at all and there's ~25,000 ID's, maybe half which don't have sprites though. Doing it every 50 would take a while.

Any ideas?

[PHP]<?php
set_time_limit(60);
$num = 615;
for($n=$num; $n < ($num + 50); $n++)
{
$url = 'http://services.runescape.com/m=itemdb_rs/3450_obj_sprite.gif?id='.$n;

$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$page_data = curl_exec ($ch);
curl_close ($ch);

if(preg_match('/Page not found/', $page_data)){
echo $n." –> 404<br />";
continue;
}

$i = file_get_contents($url);
$f = file_put_contents('******.gif', $i);
usleep(1);
echo $n.' success<br />';
}
?>[/PHP]

There seemed to be a syntax error in your script, perhaps you should check this out http://semlabs.co.uk/journal/object-oriented-curl-class-with-multi-threading ?

From: smurg
[php]
<?php
set_time_limit(60);
$num = 615;
for($n=$num; $n < ($num + 50); $n++)
{
$url = 'http://services.runescape.com/m=itemdb_rs/3450_obj_sprite.gif?id='.$n;

$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$page_data = curl_exec ($ch);
curl_close ($ch);

if(preg_match('/Page not found/', $page_data)){
echo $n." –> 404<br />";
continue;
}

$i = file_get_contents($url);
$f = file_put_contents('******.'.gif', $i);

echo $n.' success<br />';
set_time_limit(30);
}
?>
[/php]

As the PHP documentation for set_time_limit says, put 0 as a parameter and there is no limit.

[php]$url = 'http://services.runescape.com/m=itemdb_rs/3450_obj_sprite.gif?id='.$n;[/php]

has a problem, since that .gif file can start either with 3459 as well (and probably with something else).

Also, why the use of CURL and then file_get_contents? Use only file_get_contents and test the size of the result in order to find out, if there's data or not.

[php]$f = file_put_contents('******.'.gif', $i);[/php]

Is weird. Apparently you don't need/use $f, so remove that and

[php]'******.'.gif'[/php]

doesn't work either. This overwrites the previous file everytime. If this is the desired action, at least do it this way:

[php]'******.gif'[/php]

Good luck!

Last, take

[php]set_time_limit(30);[/php]

away, it doesn't reset the counter as I think you think it does :p

From: superuser
As the PHP documentation for set_time_limit says, put 0 as a parameter and there is no limit.

[php]$url = 'http://services.runescape.com/m=itemdb_rs/3450_obj_sprite.gif?id='.$n;[/php]has a problem, since that .gif file can start either with 3459 as well (and probably with something else).

Also, why the use of CURL and then file_get_contents? Use only file_get_contents and test the size of the result in order to find out, if there's data or not.

[php]$f = file_put_contents('******.'.gif', $i);[/php]Is weird. Apparently you don't need/use $f, so remove that and

[php]'******.'.gif'[/php]doesn't work either. This overwrites the previous file everytime. If this is the desired action, at least do it this way:

[php]'******.gif'[/php]Good luck!

Last, take

[php]set_time_limit(30);[/php]away, it doesn't reset the counter as I think you think it does :p

I didn't want to set it to 0 just in case.

Are you talking about the 3450 number in front of the sprite? I've only seen it that number before but i'll double check.

I used the f commands because i was copying examples of code and that worked lol. I like cURL though.

In ***.gif, I edited out concatenating it with my variable.

————-

The local root php.ini didn't work until I edited it to php5.ini and now it does. Still the apache timeout is like 300 seconds tho. For now I'm doing a redirect every so often with $_GET data lol, so ghetto.