SRL Archive - General

Pure Bash GE Update grabber/text notification


moparisthebest

2011-03-09 00:14:23

This thing grabs your GE offer updates, through a proxy if you wish, and sends you a text message if the percentage done has increased by a threshold you specify. It only has a few requirements that must be installed before running, but it checks them and lets you know if you don't have them. It also only logs in when the cookie expires, and so is faster and uses less bandwidth than other solutions. Silabsoft said it can only be run every 2 minutes, and I now enforce this in the script, and for me personally I have it set for a cronjob to do so.

Some credit should be extended to cumescape for his PHP GE Tool, which I basically took the curl requests from. It saved me a lot of packet sniffing, so I appreciate it. :)

Note:
Everyone seems to think this checks GE prices, it DOES NOT! It checks your personal GE offers and outputs text similar to:
2 completed offers
Buying 20 Air rune at 5, 100% complete, costing 100
Selling 10 Air rune at 5, 100% complete, earning 50

It will also optionally send you an email, text message, or notify you any way you want when your completion percent changes by an amount you specify (currently 10)


#!/bin/bash

# rs user/pass
username=my_rs_user
password=my_rs_pass

# notification command, called like $notify "$message"
notify="sendEmail -f from@gmail.com -t to@yourdomain.com -s smtp.gmail.com -o tls=yes -xu username -xp password -m"

# only notify you if the completion percentage of your offer changed by this much
percent_diff=10

# other settings
proxy="--socks5 localhost:1081"
cookie_file=/tmp/cookies.txt
percent_file=/tmp/percents
timestamp_file1=/tmp/limit
timestamp_file2=/tmp/timestamp
user_agent="Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.04 (lucid) Firefox/3.6.13"

# you can customize your call to curl here, but you shouldn't need to touch below this
ge_curl() {
curl -s -o - $proxy -b "$cookie_file" -c "$cookie_file" -A "$user_agent" "$@"
}

# check we have all pre-requisites before attempting to run
all_programs=0
for prog in curl touch grep echo xpath sed cut date sendEmail
do
command -v "$prog" >/dev/null 2>&1 || { echo "This program requires $prog but it's not installed." >&2; all_programs=1; }
done

if [ "$all_programs" -ne 0 ]
then
echo "We don't have all the required programs, aborting..." >&2
exit 1
fi

# this shouldn't be ran more often than every 2 minutes, ensure this here
# if you feel lucky, you can lower the 120 second limit below, but silabsoft says they will ban you...
touch -d '-120 second' $timestamp_file1
if [ -f $timestamp_file2 ] && [ $timestamp_file2 -nt $timestamp_file1 ]
then
echo "This program has been ran less than 120 seconds ago, aborting..."
exit 1
fi
touch $timestamp_file2

cookie_expiration=$(grep toolbar_hash "$cookie_file" | cut -f5)
if [ "$cookie_expiration" = "" ] || [ $cookie_expiration -lt $(date +%s) ]
then
echo "Valid cookie doesn't exist or has expired. Logging in again"
rm -f "$cookie_file"

login_result=$(ge_curl -d "password=$password&username=$username&rem=0&mod=toolbar&ssl=0&dest=authorise.ws" "https://secure.runescape.com/m=weblogin/login.ws?mod=toolbar&ssl=0&expired=0&dest=authorise.ws")
#echo "login_result: $login_result"
if [ "$(echo $login_result | grep 'Invalid Login or Password')" != "" ]
then
echo "Invalid username/password. After 3 attempts they will add a captcha and this will fail for 24 hours regardless. Login failed, aborting."
rm -f "$cookie_file"
exit 1
else
echo "Login successful."
fi
auth_result=$(ge_curl -d "remember=remember&submit=login" "http://services.runescape.com/m=toolbar/authorise.ws")
#echo "auth_result: $auth_result"
fi


geupdate=$(ge_curl "http://services.runescape.com/m=toolbar/geupdate.ws")
#geupdate=$(cat ~/newge.xml)
#echo -e $geupdate > ~/newge.xml
if [ "$(echo $geupdate | grep '<DYNAMIC_MENU>')" = "" ]
then
echo "Error getting GE Update, aborting."
exit 1
fi
result=$(echo $geupdate | xpath -q -e 'DYNAMIC_MENU/BUTTON/DEFAULT_BUTTON_TEXT' | sed "s/<[^>]*>//g")"\n"
# I am also going to get rid of the trailing ' gp so far' at the end to save space (for text messages)
result=$result$(echo $geupdate | xpath -q -e 'DYNAMIC_MENU/MENU/MENU_ITEM/CAPTION' | sed "s/<[^>]*>//g" | sed "s/ gp so far//g")
# hack for switching from dash to bash
result=$(echo -e "$result")
echo "$result"

# now parse out % dones and save them to file for comparison
percents=$(echo "$result" | grep '%' | sed "s/%.*//" | sed "s/.*, //")
line_num=1
for new_percent in $percents
do
old_percent=$(sed -n ${line_num}p $percent_file)
diff_percent=$(($new_percent - $old_percent))
#echo "line $line_num, new_percent: $new_percent, old_percent: $old_percent, diff_percent: $diff_percent"
if [ "$diff_percent" -ge "$percent_diff" ] || [ "$diff_percent" -lt 0 ] || [[ "$new_percent" -eq 100 && "$old_percent" -ne 100 ]]
then
#echo "Sending update."
echo "$percents" > $percent_file
$notify "$result"
break
fi
line_num=$(($line_num + 1))
done

Post any questions about it here. (this should run in windows with cygwin as well)

RAM

2011-03-09 00:33:29

:thumbsup:

Flight

2011-03-09 01:10:05

I had found something like this on the android market, GE price watcher? I don't know how either one works, but it's an app for android phones that notifies you of a GE update/Items % changes, ect… Is this the basic idea?

Echo_

2011-03-09 09:07:45

Ooh cool, kinda reminds me of that the IRC application Lothy made. I definitely want to try this out :D

Harry

2011-03-09 09:20:19

Why does it require your Runescape username and password? GE database allows anonymous access last time I recall…

Method

2011-03-09 09:38:27

From: Harry
Why does it require your Runescape username and password? GE database allows anonymous access last time I recall…


How else are you supposed to grab the status of your account's offers?

moparisthebest

2011-03-09 10:20:59

I just edited the script slightly, because I introduced a small bug into this last night where it would send an update every time when it reached 100%, I have fixed that, it will now only send an update the first time it reaches 100%, regardless of your percent_diff. Also I enforce a 2 minute limit so you don't run it too often and get temporarily blocked by Jagex.

I think I'm done with it, let me know if you guys have any problems or requests.

From: Harry
Why does it require your Runescape username and password? GE database allows anonymous access last time I recall…


This doesn't check GE prices, it checks your personal GE offers and notifies you of changes. No need to be suspicious about putting your user/pass in there, since it's just a script you can easily read exactly what is being done with it. In this case, you will see your user/pass are only used on this one line:
     login_result=$(ge_curl -d "password=$password&username=$username&rem=0&mod=toolbar&ssl=0&dest=authorise.ws" "https://secure.runescape.com/m=weblogin/login.ws?mod=toolbar&ssl=0&expired=0&dest=authorise.ws")
Which calls the function:
ge_curl() {
curl -s -o - $proxy -b "$cookie_file" -c "$cookie_file" -A "$user_agent" "$@"
}
So as you can see, it is only sent over SSL directly to secure.runescape.com, the only people you can trust your user/pass with. Feel safer now? :)

edit:
Everyone seems to think this checks GE prices, it DOES NOT! It checks your personal GE offers and outputs text similar to:
2 completed offers
Buying 20 Air rune at 5, 100% complete, costing 100
Selling 10 Air rune at 5, 100% complete, earning 50

It will also optionally send you an email, text message, or notify you any way you want when your completion percent changes by an amount you specify (currently 10)

weequ

2011-03-09 10:34:10

It uses the information used by runescape toolbar.

Bebe

2011-03-09 11:50:45

I tried Spel's and Silab's so I'll give yours a try too. The email idea is a nice feature cause I can just set it to my vtext. Thanks mopar :)

Harry

2011-03-09 19:13:45

I didn't feel unsafe, I just misunderstood the reasoning for it, as I thought it was a global item tracker and not a "your offers" tracker. Really cool.

moparisthebest

2011-03-09 21:19:40

From: Bebe
The email idea is a nice feature cause I can just set it to my vtext.


That's exactly what I do, which is why I keep it under 160 characters so I can receive it as a text message! :)

From: Harry
I didn't feel unsafe, I just misunderstood the reasoning for it, as I thought it was a global item tracker and not a "your offers" tracker. Really cool.


No problem, I didn't realize until I typed all that out that everyone thought it was just a GE price checker, which would be highly suspicious asking for your pass. :D