1036 views

VB6 Betfair Web Scraping API

Categories > Code Snippets

Page 1

posts 1–11 of 11

birchy

superstar - member

230 posts

+0 permalink

Attached is what i call "Bespoke API". It is a VB6 .bas module containing my home-grown API for scraping the Betfair web server. This DOES NOT use Betfair's free API.

Note that this particular module is moulded around greyhound racing and has been chopped from my infamous Dog Bot. Horse racing and other sports can be added with a few copy/pastes and a little work with browser tools such as Fiddler, Live HTTP Headers or IEHttpHeaders.

The following functions are included:

--- API ---
Login()
Logout()
GetAccountBalance()
GetDogsMenu()
GetMarket()
PlaceBets()
GetProfitAndLoss()
GetMarketBets()
(each function returns a delimited string which can be parsed using the Split() and GetSubstring() functions)

--- TOOLS ---
GetSubstring() - Used for parsing strings
SetBetfairOdds() - Rounds a given number to nearest betfair increment. Has options for round up, round down and add or subtract "x" number of tics.
GetTicDifference() - Calculates the number of tics between the back odds and lay odds (the "spread"). I use this to decide whether to take the current odds or undercut the queued odds.
SaveFile() - Dump a string to a file. Use for debugging, data logging, etc, etc.
ShutDownPc() - I added this as my bots were often left unattended. Saves the electric bill!

There are no restrictions what-so-ever. Download my code, chop it, hack it, delete it, do what the hell you like with it, but don't pester me for any more spoon feeding.

And above all, GOOD LUCK!

FILE: BespokeAPI.zip

__________________

www.bespokebots.com

quote

07th November 2008 20:34 p.m.

birchy

superstar - member

230 posts

+0 permalink

An interesting problem came up recently which has been discussed previously: http://diybetfairbots.lefora.com/2008/07/02/website-scraping-login-failure/page1/

If you call the Login() function as your first hit on the Betfair site, it will fail repeatedly and/or the "Location" header that indicates a successful login is not present. The reason for this is cookie related. If you call Login() before any other pages, no cookies are present and this causes the failure. You can emulate this behaviour in Firefox or Internet Explorer by clearing your cookies just before you login. If you load ANY betfair page previous to logging in, it will work fine. Personally, i've never encountered the problem because my bots always load up a menu and a market before logging in. The solution is to simply make sure that you load at least one betfair page previous to logging in. Other than this, you do not need to worry about cookies as they are auto-handled by WinHTTP...which is the exact reason why we use it.

Here is a simple Login() procedure which can be copy/pasted into a VB6 project or any VBA application such as Excel, Access, etc:

' tools > references > check "Microsoft WinHTTP Services, version 5.1" > ok
Private Const USER_AGENT As String = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3"
Dim oHTTP As New WinHttp.WinHttpRequest

Public Function Login(username As String, password As String) As String
    Dim s(1) As String

    ' NOTE:
    ' load at least one page previous to calling Login() which will set a cookie
    ' Login() will fail if there is no cookie. WinHTTP auto-handle cookies.
    s(0) = GetHtml("http://www.betfair.com") ' (this is only temp to set cookie)
    If Len(username) > 0 And Len(password) > 0 Then
        ' set url + postdata
        s(0) = "https://www.betfair.com/account/login/LoginAction.do"
        s(1) = "username=" & username & "&password=" & password
        ' send request
        s(0) = GetHtml(s(0), s(1))
        ' check response
        If InStr(s(0), "LoginView.do?status=success") > 0 Then
            Login = "SUCCESS"
        Else
            Login = "FAIL"
        End If
    Else
        Login = "Username or Password blank!"
    End If
End Function

Private Function GetHtml(url As String, Optional postdata As String) As String
    On Error GoTo ExitError
    If Len(url) > 0 Then
        ' setup http
        With oHTTP
            .SetTimeouts 5000, 5000, 5000, 5000
            If Len(postdata) = 0 Then
                .Open "GET", url, False
            Else
                .Open "POST", url, False
                .SetRequestHeader "Content-Type", "application/x-www-form-urlencoded"
            End If
            .SetRequestHeader "User-Agent", USER_AGENT
            .Send postdata
            If .Status = "200" Or .Status = "302" Then
                ' return response
                GetHtml = .GetAllResponseHeaders & StrConv(.ResponseBody, vbUnicode)
            Else
                ' http error (usually 404 if site down)
                GetHtml = "HTTP ERROR " & .Status & vbCrLf & "Betfair may be off line."
            End If
        End With
    End If
    Exit Function
ExitError:
    GetHtml = "FUNCTION ERROR: " & Err.Description
    Err.Clear
    oHTTP.Abort
End Function

__________________

www.bespokebots.com

quote

08th November 2008 22:59 p.m.

lope

rookie - member

6 posts

+0 permalink

Thanks for the source!
it seems they have changed the login URL to: https://www.betfair.com/account/login/LoginRequest.do
I've updated that and made sure I load a page first, but i'm getting the error: "The requested header is not found"
I figured its also a good idea to make it look like we came from betfair.com: .SetRequestHeader "Referer", "http://www.betfair.com/"
Maybe we need more header info?
I'm running MS WinHTTP Services 5.0 by the way.
I'm not sure what you're doing in the getmarketbets function without seeing the strings you're working with, but it didn't compile. i changed s = StrConv(.ResponseBody, vbUnicode) to s(0) = StrConv(.ResponseBody, vbUnicode)
I used the Live HTTP headers on firefox (awesome tool) and looked for marketIDs etc, but could not find any. quite a lot seems to have changed, i could not find getBootstrapData.do or LoadMenuNodesAction.do etc in my headers.
I know this is a lot faster than controlling a web browser control, but its a lot less forgiving. also i'm not sure how you manage to figure out which of these GETs are necessary for the pages we need.
Do you have any tips on this?

quote

18th January 2009 02:05 a.m.

birchy

superstar - member

230 posts

+0 permalink

Yes. Learn to program. This whole http programming malarkey isn't one of those script kiddies' copy and paste tutorials. You need to be comfortable with at least one programming language and even then, the http stuff can be confusing at first. Forget VB6 unless you're already FLUENT in it (you wouldn't be asking for help if you were) and learn a modern language. Microsoft killed VB6 about 10 years ago. If you plan on sticking with Windows, learn C# or VB.Net. If you want operating system flexibility, learn Java or Python or PHP or whatever takes your fancy.

And go to Specsavers:

Download my code, chop it, hack it, delete it, do what the hell you like with it, but don't pester me for any more spoon feeding
-birchy

__________________

www.bespokebots.com

quote

18th January 2009 06:44 a.m.

lope

rookie - member

6 posts

+0 permalink

I am fluent in VB6, been coding in it 12 years but been trying to get away from it. been coding PHP for 2 years, and C# for a month :) hehe

I saw your disclaimer, I just thought you might have an updated version of your code since making your last post. I prefer not to reinvent the wheel.
Thanks for the friendly reply.

quote

18th January 2009 11:12 a.m.

inksmithy

regular - member

184 posts

+0 permalink

No go on birchy, don't hold back, tell him what you really think!

birchy has a point though lope, vb6 is getting a bit old, although interestingly, it seems a lot of the vb6 stuff seems to be applicable to VBA in excel.

I wouldn't have a clue about it though - I once tried to write a lottery number picker in VB6 and (at the time) had so few clues about writing code that I received an error message telling me that the module I was working in was too big.

What had happened was that I had written a long and involved method for each individual number which could be drawn out of the barrel, then copy/pasted it below, editing where applicable. The error that I recieved was telling me that I had exceeded the number of lines permitted in a VB module. As I recall, the file was over a megabyte uncompressed.

That was many years ago mind you. I would like to think I've moved on since then.

If you are interested in PHP and C#, have a look at the new BDP forums - vossie (bdp team member) over there has just posted a PHP framework for the Betfair API and he's posting what looks like incredibly useful articles for .NET development using the API.

__________________

Slashdot. It's like Digg on slow, but sensible.

quote

18th January 2009 13:19 p.m.

lope

rookie - member

6 posts

+0 permalink

Hey,

Yeah, i saw that PHP framework at the BDP forums, but it wanted me to login to download it, and they arent accepting forum registrations there :/

quote

18th January 2009 14:56 p.m.

inksmithy

regular - member

184 posts

+0 permalink

Aren't they? I'm posting away merrily at the moment. Have you tried logging in with your betfair username and password?

__________________

Slashdot. It's like Digg on slow, but sensible.

quote

18th January 2009 15:00 p.m.

inksmithy

regular - member

184 posts

+0 permalink

Something I saw when I was trawling through the old forum recently was a comment about that Failed: Requires_Funded_Account error you mentioned.

Try moving the funds from your main wallet to the Australian wallet and back. Apparantly the way it works is in order for an account to be seen as active, it needs to have had some sort of transaction within the last three months. I don't know whether that applies to you of course, but it might, so give it a go.

__________________

Slashdot. It's like Digg on slow, but sensible.

quote

18th January 2009 15:03 p.m.

birchy

superstar - member

230 posts

+0 permalink

Please start a new thread if you want to discuss this, this supposed to be "code snippets" and it's starting to look a bit messy.
Thanks.

Alan:
FWIW, my login code above works fine with only a small and obvious mod. Anyone with a little VB6 experience would be able to fix it within a few minutes...hence my frosty reply.