birchy's Blog

Re: Scraping Racing Post Live Odds

I don't use the Racing Post but they are probably using AJAX in the background. Rather than refreshing the whole page, they are probably using JavaScript's XMLHttpRequest() function and then inserting the relevant data. Take a look at the HTML and check the source code of any imported .js files.

Re: Betfair API performance

the margins are so tight and perfect that x-games is ... the perfect vehicle for earning bf commission lol

-fred77

I told you that about 2 years ago. You just don't listen to me, do you Fred. lol

Poker bot is currently on a downswing. Hope it also has an equal upswing at some point in the future. It's my biggest project ever but i HAVE learnt a lot along the way, including how to write firefox extensions.

I'm actually surprised that nobody has written a betfair bot in JavaScript and/or as a Firefox extension. Should be quite straight forward using XMLHttpRequest() and my standard hand-rolled XML technique.

Re: Betfair API performance

API is fine but the BDP site has been stuffed for me on and off for days, the link I gave doesn't currently work for me.

-fred77

Yes birchy I'm still botting. I thought you were too as I saw your Python API library download on your website. Are you finally making a killing with other online activites?

-fred77

Got promoted to engineering foreman at work so the extra £100 a week diverted my attention.

Finally finished off my sit n' go poker bot when i could be arsed (only took a year or so to complete the basics!). First few "brains" lost £80 over around 400 games which wasn't too bad for a first effort. Did a complete re-write last week - it totally owned the £1 buy-ins and has now stepped up to £2. Only played a few hundred games though, so not sure whether i've (accidentally) perfected the AI or it is simply the luckiest player in the world at present...

Re: Betfair API performance

Cheers Fred. I did wonder what was going on. Good to see you're still botting. ;) I've moved onto pastures new.

Betfair API performance

What's the situation with the API nowadays? I'm no longer botting Betfair, but have received several emails from RacingTraders regarding some "performance issues" at Betfair's end. Are they having server problems or is the exchange now saturated with bots?

Re: Horse Finishing Position Abbreviations

I thought it was an official API seeing as it was done by Vossie who is BDP staff or whatever. Having read it through, it's just web scraping code dressed up with a few fancy function names, so nothing new. But still, i guess it's easier than writing your own. :)

I wouldn't be surprised if betfair have got Vossie to release this because quite a few botters were already doing it. You know what betfair are like for moving the goal posts if a customer finds a profitable strategy...

Re: Horse Finishing Position Abbreviations

I like the API because it avoids some nasty Racing Post screen scraping (sorry Birchy, I know you're a scraping fan!).

No offence taken, i'm a believer in using whatever is easiest. Besides, i used to scrape the Sporting Life website cos it's easier than the RP. ha ha. So Timeform now has an official API? How much are Betfair charging for access? Or are they trying to get everyone on the bandwagon first?

Regarding the abbreviations:

dh = Dead Heat?
ds = disqualified?

These are my edumacated guesses but they're prob right.

Re: Fresh blood coming?

They've started a private section on the pokerai forum, so i guess they don't think we're intelligent enough.

Mind you, they can't seem to grasp the concept that the betfair games are negative value due to poor odds and commission but hey-ho, i've explained it and they're still not convinced. Best to let them lose a few quid, like WE did...

On the plus side, they're talking of a joint project and seeing as i'm the only one with some betfair experience, i seem to be chief advisor (at the moment) but they'll probably kick me out once they understand how betfair works and realise that i know absolutely nothing about AI. ha ha

Fresh blood coming?

There's a discussion just started on pokerai.org about Betfair sports betting. Could make for some interesting discussions because a lot of those guys are experts in all things AI...

http://www.pokerai.org/pf3/viewtopic.php?f=79&t=2442&hilit=

Re: How's everyone doing?

I gave up on Betfair. As soon as you find a winning method, they introduce things like data charges and "bet matching logic" which nullifies your edge. To me, it's quite obvious that winners are NOT welcome. Don't get me wrong though, it makes a lot of sense from a business point of view - a nice level playing field where nobody wins and nobody loses (significantly) keeps the commission rolling in...

Re: API request XML - hand rolled

Do you really find hand-rolling XML fun? Wierd!

-bennean

It's a performance thing.

Re: scraping prevention

Are you using the "Pragma: no-cache" header? If not, try it.
I'm no longer scraping betfair, but i remember that i always used the main loader url and not the one they use for refreshing:

http://uk.site.sports.betfair.com/betting/api/json/getBootstrapData.do?mi=xxxxx

Also note that i never used any of the url parameters. And do make sure that you're not using the "prevcache" parameter...although that should be obvious...

Re: scraping prevention

That's definitely a cacheing problem. What language/libraries are you using? Is the other PC running the same operating system?

You're not automating a web browser component are you?

Re: API Lite - what's the difference?

ALL of the Lite versions are available to the Free API at a rate of 60 p/m, this is why i was looking at them. It's been recently updated following a comment in the BDP forum.

Assuming that (for example) getMUBetsLite will give me the same information as getMUBets, then what IS the point of having two services that do the same thing?

API Lite - what's the difference?

I'm tinkering with the API again and i'm wondering what these "Lite" services are. Have RTFM, searched the BDP forums and done a fair bit of Googling but i can't find anything that actually explains how the "Lite" api's are any different.

Anyone care to share?

Re: API request XML - hand rolled

huh? according to what i'm seeing, this forum has 1200 posts in total?
1.01 that alan is a little inebriated. lol

Re: API request XML - hand rolled

It turns out that if you include a parameter, it must be set with a value. For instance, <locale></locale> will cause an error, however <locale>en</locale> is fine. Any empty parameters must be completely omitted rather than sent with no values. That's not how i understood it from the documentation but it does explain why Fred's slim version works ok. I decided to write my GetAllMarkets function to allow for any (or none) of the filters to be set.

Incidentally, how strict are the datetime formats accepted by the server? The documentation suggests something like "2009-04-09T19:52:28.972937Z", but i haven't (yet) found a one-hit way to create this in Python. Nearest i've got so far is along these lines:

fromDate = str(datetime.utcnow()).replace(" ", "T") + "Z"

Not very pretty but i suppose it works...although i'm not (yet) sure how i'd go about adding or subtracting to these date values. Does anyone have any suggestions?

Re: API request XML - hand rolled

I'm having trouble with getAllMarkets and there's no obvious reason why. According to the documentation, i don't need to specify anything other than the sessionToken, yet i am getting INTERNAL_ERROR when i send the following request to https://api.betfair.com/exchange/v5/BFExchangeService:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
 xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:xsd="http://www.w3.org/2001/XMLSchema"
 xmlns:ns2="http://www.betfair.com/publicapi/types/exchange/v5/"
 xmlns:ns1="http://www.betfair.com/publicapi/v5/BFExchangeService/">
 <SOAP-ENV:Body>
   <ns1:getAllMarkets>
    <ns1:request>
     <header>
      <sessionToken>ABCD...</sessionToken>
     </header>
     <locale></locale>
     <eventTypeIds>
      <int>0</int>
     </eventTypeIds>
     <countries>
      <Country></Country>
     </countries>
     <fromDate></fromDate>
     <toDate></toDate>
    </ns1:request>
   </ns1:getAllMarkets>
 </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

As far as i understand, this should give me a response containing every open market?

Re: API request XML - hand rolled

Yeah, i know that some other forum users have the opinion that we should spend our time developing strategies rather than dicking about with the API but i think i'm more addicted to the programming than i am to betfair. lol

Re: API request XML - hand rolled

I was having a few problems getting back into coding the API, so wondered if i was sending the right info. Then i read the manual and realised that i wasn't sending the SoapAction header. DOH!

What i'm doing at present is running gsoap to create the xml request files and then loading them into a global dictionary at startup, so all my requests are sat in less than 50kb of ram. It's then just a case of referencing the XML by it's key name (e.g. "login") and filling in the missing bits. Should end up with a nice lean API library that will be easy to maintain.

API request XML - hand rolled

I've decided to have another tickle at the official API but am slightly rusty. I'm using Python and only need half a dozen functions, so i am hand-rolling rather using ZSI. Now i have a small problem with the the XML format. According to the bdp samples, a login request should look like this:

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <soap:Body>
    <login xmlns="http://www.betfair.com/publicapi/v3/BFGlobalService/">
      <request>
        <locationId xmlns="">0</locationId>
        <password xmlns="">bfpassword</password>
        <productId xmlns="">82</productId>
        <username xmlns="">bfusername</username>
        <vendorSoftwareId xmlns="">0</vendorSoftwareId>
      </request>
    </login>
  </soap:Body>
</soap:Envelope>

HOWEVER, i'm using gSOAP (a C code generator which also spits out XML files), and it generates the following XML from the WSDL:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
 xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:xsd="http://www.w3.org/2001/XMLSchema"
 xmlns:ns2="http://www.betfair.com/publicapi/types/global/v3/"
 xmlns:ns1="http://www.betfair.com/publicapi/v3/BFGlobalService/">
 <SOAP-ENV:Body>
   <ns1:login>
    <ns1:request>
     <ipAddress></ipAddress>
     <locationId>0</locationId>
     <password></password>
     <productId>0</productId>
     <username></username>
     <vendorSoftwareId>0</vendorSoftwareId>
    </ns1:request>
   </ns1:login>
 </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

So the question is: should both of these requests work ok? Why are they different?

A return to botting

Ok, after a year or so out of botting, i'm about to re-embark on automating my cash donations instead of doing them manually.

Seeing as i'm now a big Linux fan, i've dumped VB6 and have been really enjoying Python. I've slapped together some scraping code and have a couple of "form" bots that automate some form studying techniques via the sporting life website. So far, one is showing a 7% ROI at official SP and considerably more at BSP, however it's early days and it's only picked 37 winners from 238 selections since 1st feb. The strike rate is not too impressive, although it has been picking out some nice priced winners, hence the decent ROI. I also have a couple of other ideas, so it's time for me to write myself a betfair scraping library in Python...

In the past, my VB6 bots were always dedicated to horse and greyhound racing, so in order to get a list of markets, i simply called the "Todays Card" urls from the menu. I now want to investigate other sports, so am wondering if there is a one-hit way of getting a list of ALL active markets on the exchange? I could probably call http://site.sports.betfair.com/Menu.do and then navigate from there, but it looks like it could get a bit messy, so i'm wondering if the Free API call to GetAllMarkets will return EVERY market?

I'm currently at the planning stage and everything seems very straight forward, other than obtaining a list of markets. I'm also open to other suggestions, such as recommended json parsing libraries and/or regexs, etc.

Flutter...

Anyone remember this?

http://www.guardian.co.uk/sport/2002/jan/16/horseracing.gregwood

Those were the days when you could actually make a profit from manual trading because most of the punters were less clued up. I only wish that i'd known then what i know now...i certainly wouldn't be working for a living. Those very first bots must have been like little gold mines.

Re: new guy

Can't help you with the soap stuff as i prefer web scraping, however you might find this website useful for decompiling .pyc files:

http://www.depython.net/

Re: Poker bots

Chill out, maaaaan. The article is regarding poker bots so totally unrelated to what WE do. For someone like myself who can program AND play poker, it is slightly interesting but doesn't share anything that we didn't know 5 years ago. It's actually a long way out of date....things have advanced significantly since the days when those methods were "cutting edge".