Remove this ad
avatar

snooki

rookie botter

Posts: 6 Member Since: February 22, 2013

Lead

April 27, 2013 18:03:53

Tags : :

Hello all,
Maybe some of the more advanced botters can offer some inside into the more general structure of bot programming. Specifically im now running into some problems with the amount of time it takes to update all the observed markets. So far im watching up to about 110 markets at the same time. Each market has its own "MarketFormWindow" with all sorts of data on it. Each form gets its marketdataupdate  by either 1 call to getcompletemarketpricescompressed alternating with 4 calls from getmarketpricescompressed, getMUBets, getmarkettradedvolumecompressed and getMarketprofitandloss. i do this alternating to double the priceupdates to 120 markets/min instead of the 60call/min limit however i find its totally useless to me as the single call only updates prices AND NOT any change in position in this market(which is only done at 60 Markets/min). I dont really want to use just the pricedatachange all by itself to enter(or exit) trades when i cant be sure about my current position in this market.How do you people handle this ??. My first idea was to use a "centralised" database or multidimensional array to hold ALL Marketdata and feed it to the datagridviews on the MarketForms. As my programming skills are still "developing" :) im wondering what would be the right path. I think the database idea is probably in the longterm the most flexible(also beeing able to save marketdata permanently could be a good idea( as was mentioned multiple times in different threads lately). Then again even that dosent change anything about the 60call/min position update problem as they only come half as fast as pricedatachange.

Well that was worded a bit complicated but maybe someone has been there already and knows what i mean.  It would be great if someone has some more inside than me and wants to share a bit of that. Also it would be greatly appreciated for a lead about how to go about my "central database" idea, never had anything to do with that programmaticly and wondering if that is something way beyond me(not even knowing what the letters SQL stand for) or "just a piece of cake".

thanks for all replys

snooki

Quote    Reply   
Remove this ad
Remove this ad
avatar

birchy

Betfair Elite

Posts: 591 Member Since:May 11, 2008

#1 [url]

April 27, 2013 19:19:25

I'm not really a fan of GUI's as (IMO) they add an extra layer of unnecessary complexity and can often slow down an otherwise fast program. As you are finding, it's the GUI programming that is giving you an (unnecessary) headache. Having said that, a GUI is essential for any commercial app or a bot that requires human intervention.

Your main options are:

1. Use Asynchronous requests, where you fire off all your requests at the same time and therefore receive all your responses within a tight time frame. This also prevents the GUI freezing up as the http stuff is effectively run as a different thread.

2. Use Synchronous requests, saving the data to a data object/variable and then updating the GUI when all info is received.

3. Scrape the website rather than use the API. You get much more info in a single call (i.e. latest prices AND PnL in the same request) and can refresh upto 10x a second before incurring data request charges. This is particularly useful if you are developing an app that mimics the website.

For a graphical app, Async requests will offer the best solution and least time delay between the requests. Personally, I'd scrape the website or wait for the JSON API to be completed. But then again I'd never use a GUI for a one-off project.

www.bespokebots.com

"This time next year Rodney, we'll be millionaires!"

Quote    Reply   
avatar

snooki

rookie botter

Posts: 6 Member Since:February 22, 2013

#2 [url]

April 27, 2013 20:16:15

ty for a quick response,
Yes im using async calls already. You are right the GUI adds certian problems but i use it to be able to see what the bot is doing at anytime in one look. Just in case, as im not really trusting my coding ability and i'm not comfortable yet letting bot run without beeing able to see exactly what its doing. It helps me a lot with testing things out and seeing errors, it also helps me in noticing market behavior change for new trading ideas. So no BIG problems with the GUI itself so far. Even with over 100 windows open performance is not an issue for me. Real issue is API stability latley. Nearly everyday calls get "lost" or time out and the program dosent handle that very well(yet).
Very interesting idea with the website scraping, I saw it mentioned here before but wasn't aware about the 10xrefresh/sec.Is there any Documentation about that ?
JSON API ..well i think we are many month if not years away from that.

Quote    Reply   
avatar

birchy

Betfair Elite

Posts: 591 Member Since:May 11, 2008

#3 [url]

April 27, 2013 20:27:26

Regarding web scraping, it's probably a good way to go nowadays as the web pages use JSON format data already, so it's a head start for the future API. Just enter "scraping" into the search box of this forum and you'll find plenty of discussion. Also, don't overlook the Betfair Lite website which is greatly simplified (and very easy to parse) but may hold enough info for your bot if you only need prices and PnL figures in one hit... https://lite.betfair.com/

To quote from http://www.betfair.com/aboutUs/Betfair.Charges/#charges5:

Market and private data requests made through the website are weighted 2. If you're using the website manually with a mouse, you shouldn't be affected by this charging scheme. If you're refreshing data on the website using an application and want to receive updates more than once or twice a second you may be affected depending on how many markets you read simultaneously and how much data (market prices, current bets, P&L etc.) is sent with each update.

www.bespokebots.com

"This time next year Rodney, we'll be millionaires!"

Quote    Reply   
avatar

snooki

rookie botter

Posts: 6 Member Since:February 22, 2013

#4 [url]

May 1, 2013 08:31:35

ty again for the links birchy, i will look into that as soon as i can.
Real quality responses from you everytime. I like this place :)

Quote    Reply   
Add Reply

Quick Reply

bbcode help