Remove this ad
avatar

blackmagic

bot addict

Posts: 70 Member Since:June 26, 2008

#21 [url]

February 5, 2013 16:41:49

@pic

I'm glad someone else is using a similar technique its a shame you are not having success at the moment with it. Can you elaborate on what you meant by a statistics friend said the data is not good?

This may seem obvious but what i found is that when you back test the data, making sure the strategy sees in backtest the exact same thing as what it sees in reality is hugely important. I was storing my data in a compressed format where i was only saving the prices when they changed but you have to be careful when you replay that data this way. Something else that can be different is what you call time, this can be taken as from the last time betfair updated or when you received it. This was minor but if you are using moving averages etc, it can make a difference. I trade aggressively so i can make sure that if in sim i traded then it is likely that i will have in live. At the end of each day i reconcile sim of that day with  production of that day and make sure that every trade is the same, if they are not then an extremely boring process of working out what the issue is and fixing it. 

The good thing about all this is from this point, you can actually be pretty stupid, machine learning techniques can simply try out a massive number of combinations of strategies for you. I was doing this before but found that when i found something that worked, it didn't make money in reality. The nice thing now is that when i do find something (which still takes a while) it actually makes money. Then second useful feature of not using passive orders is you are not limited to trading small size.  

It sounds to me that birchy uses some smart insights into the market to make money. I am unfortunately not very insightful so i use brute force. I have spoken to him a bit on PM and he really really knows his stuff. The help he has given me along the way has been really useful. Its the kind of experience that wracking your brain about something for 10 years will give you but i am as i say too lazy for that sort of business!

Quote    Reply   
Remove this ad
avatar

theshadow

rookie botter

Posts: 9 Member Since:February 3, 2013

#22 [url]

February 5, 2013 18:13:58

There are some very smart people out there indeed and I get the impression a few of those people are on here.

I came across a phd engineering/computer boffin and had a very interesting chat about mathematics, modelling and algorithms. He is currently trying to work out the algorithm that HSBC use on those keypad things. He reckons he has the last 3 digits sorted. If I bump into him again I'm going to challenge him to work out the bfsp algorithm. Anyone given that a go?

Quote    Reply   
avatar

blackmagic

bot addict

Posts: 70 Member Since:June 26, 2008

#23 [url]

February 6, 2013 22:17:19

Could you not just look at this?
http://www.betfair.com/help/Help.SP.Betting/

I think one of the most interesting things about trading is how many different angles you can approach it from. Horse racing is something i have spent a lot of time working on and tried countless different ideas. I wrote my masters thesis on the statistics relating to horse racing markets and have spent years running strategies and doing data analysis. I think that is why a lot of people you meet who have actually made money are pretty pessimistic in regards to what works. Its only because on aggregate you'll probably find they have tried a lot of different things over many years before getting something that works. It really is like pulling teeth. 

Quote    Reply   
avatar

theshadow

rookie botter

Posts: 9 Member Since:February 3, 2013

#24 [url]

February 12, 2013 20:43:01

I have read through the forum and its seems like im 5 years behind you fellas ... just read the scraping thread and it is 5 years old so feel like im a bit behind the times!

Im working my way through the tutorials that Birchy has kindly recommended and I have been able configure my windows machine and install and use beautiful soup.

My first project is to write a basic scraper that will dump some info from a website into excel or mysql ....

Quote    Reply   
avatar

pic

bot addict

Posts: 63 Member Since:July 28, 2011

#25 [url]

February 14, 2013 09:19:22


@blackmagic

I shown a sizeable sample of my data to a guy (who is pretty good in statistic, afaik) and he didn't find it of sufficient quality. The data were about football matches and the main complains were:
1. I didn't have complete set for a championship season,
2. 20% missed pre-match odds.

Both the points are true and they are due to the difficulties in collecting data I've already mentioned, in particular the limit of the free API. I should have many more accounts and split the load over them but this isn't (and hasn't never been) my "core business" :-)

Quote    Reply   
avatar

pic

bot addict

Posts: 63 Member Since:July 28, 2011

#26 [url]

February 14, 2013 09:35:27


I also like to put my 2 cents about collaboration:
it can't work because there isn't enough money.
In my experience most of the working strategies don't generate high profits and don't scale well over some limits. Therefore it is something that could be worth for a single developer in his spare time, but it couldn't reward the collaboration of a group of people ...

Perhaps I'm wrong and a smart team could still make big money with these exchanges, what do you think?

Quote    Reply   
avatar

guiness

rookie botter

Posts: 39 Member Since:August 28, 2012

#27 [url]

March 2, 2013 08:45:12

Hi, saw this topic a few weeks ago and meant to add my thoughts:

Firstly to 'plc', depending on what data you wanted, have you tried download the free volume data from betfair.. you can see everybet matched and the time and event it was matched on, have a look on the developers forum..
hang on:  http://data.betfair.com/

This won't give you the priceladder data at each moment in time however.
But you can also buy this data from the company Betfair work with which (is one of the links on the page..)

They have better access to the Betfair data than you will with your free API connection..[I think they have priceladder data but you'd need to check]

Another option is to open up several Betfair accounts, get one in your grans name or something.. then you can cain the API far more..

Quote    Reply   
avatar

guiness

rookie botter

Posts: 39 Member Since:August 28, 2012

#28 [url]

March 2, 2013 09:24:25


Reply about Birchy's initial point:

Your post was probably the most interesting post I've read in a betting forum.. It's nice to know what someone is really making out there and you seem to be a truthful guy..

1)RE: Complacency
Well it depends on what your purpose in life is. If you had a child and your sole purpose was to spend as much time with the child as possible and you just wanted enough money to feed/house and provide for them.. and the 36k from betting + money from job, was adequate.. then why waste time looking for new strategies..

But If eg your purpose in life is to 'be the best professional punter I can be' or become a millionaire, then no, you should definitely be looking at more strategies..  you don't know what market conditions may change and your  system may stop working suddenly..   and given that you've already gone down many blind alleys in 9 years, you are probably in a fantastic position to find more edges. Why not just find another 10 strategies?

Guys, I'm going to say something and I don't want to sound like an asshole..but it scared me to think how much effort Birchy put in over a long period of time, to be successful. 9 years. You know  also, to put things in perspective, when I was working on a trading floor in city, 90% of people that came on to the floor were not there in 6 months time.. but the few who stuck around.. some of these guys approx 25 years old were making 1-2 million a year.. [ I was not one of them btw ] .. these figures dwarf 36k..
I've spent 7 months on my betting robot, prototype 2 is almost finished, but it only looks from my recent tests and first prototype bot, as if I'll make £100 - £200 max a week, and probably alot less over time , as the people Im making money from will stop betting..   I saw an edge, then got sucked into the 'this will be easy money', and mis-underestimated how much time it would take me..
Sure I've learnt alot, in 7 months, Ive learnt Linux, Python + various modules, the Betfair API, databases.. but
I can't help thinking, these rewards are so small.. if Id spend the same time working on a trading strategies for real financial markets.. I'd be laughing.. at least I could make some 'real' money. The markets are bigger and less 'personal'..

From what I can see, the betting and trading markets are pretty similar.. it seems as if the same %age of people lose money and same make money consistently..  eg 95% lose money to 5% that make it over time.. and both groups have winning and losing streaks..
Just in these betting markets, with exception of big matches in Football and eg Tennis finals.. the money on the table is very small, for most successful people it seems you can just about make a good salary off them at best..

If you are a losing better, your habits would probably cause you to lose in the financial markets..
if you are making money betting, your habits would probably cause you to make money in the financial markets.. [obviously you would need to do research to find a different type of edge, and sport knowledge may have been your edge]..

Birchy, I know we already had a lil chat over this and I know you do already have some knowledge in financial markets..   but if I had acquired your 9 years expertise in this, I'd either do two things:
1) work my ass off to find another 10 strategies bringing in the same income you have
2) Leave bot running, and study and test as many financial market trading strategies [many are found in books eg moving average crossovers, surprisingly many work, just only at certain market conditions.. eg a bull or bear or sideways market..].. seperate the wheat from the chaff, find out what works and doesnt..  then trade these markets..  9 years time, with the same commitment and discipline youve showed in betting, you'd have to be a millionarre for sure.. 
You know, most people problem is they don't have a proven edge..your problem now is you cant scale up your proven edge..
But in trading if you can get from £1k to £2k.. its usually just as easy to get from £100k to £200k..
Thats not true in betting as markets too small usually. 

That's my thoughts

Out
!


Quote    Reply   
avatar

toy

rookie botter

Posts: 15 Member Since:June 2, 2008

#29 [url]

March 3, 2013 09:11:33

Guiness you are right!
I come from a trading background and I was developing tools for the trading floor (Option Arbitrage Strategies). But this was a firm with fairly large resources, I haven't looked into doing something as an individual because my perception has been "its for the professionals" (probably the wrong opinion). If you were to start a financial market project what platform would you use and what are the barriers of entry to this arena?

Quote    Reply   
Remove this ad
avatar

guiness

rookie botter

Posts: 39 Member Since:August 28, 2012

#30 [url]

March 4, 2013 19:05:38



Hi Toy,

Id say any arbritrage strategy regarding finanical markets will likely involve very quick transactions..
an arb may only exist for a fraction of a sec because nowadays many other people are also looking for them..with computers..

So if you were trying to compete by placing such arbs, youd be at a disadvantage and  you are correct, you don't have the same resources..  my guess is your company had a very fast connection to the exchange, possibly even being in a building very close by..  because a microsecsond might make all the difference.. maybe a nanosecond ?
I don't know too much about trading on this timeframe myself.

There are many different time-frames you can operate on.. e.g. a long term investor may be successful just by having 1 time a year where he sells old stock and buys new..  and he doesn't need the latest and fastest technology, if he made his trade a few weeks late, it probably wouldn't affect his overall profit and loss much at all..

But with your previous company.. a half a second delay, and there would be no point placing the arb probably..
it would have vanished..

I used to trade S&P500 future on approx  3-5 day timeframe.. if I was late to buy and sell by a few mins sometimes even an hour.. it was often ok..  Ive also traded on much shorter time frames, where I'd see a price and sometimes couple of secs later it was gone..a few minutes late would render the strategy I was using completely  useless in this case..

So the bottom line is actually.. where is your edge, your advantage? Obvious blazing fast access to the exchange is not something you have so just dont trade with an edge that relies on super high speed..
ps edge = statistically proven advantage
No matter how good your system, there will always be wins and losses, you just got to lose less money than you make..

Re: Edge, my guess is, you don't yet have one.. that's fine..  but now you have you stick in a lot of time and effort to find one.. ultimately, trading (and betting) is very personal, so whatever you find that works, you will no doubt customize it to suit your personality..

My advice:  Look up these guys, Larry Williams.. he's a genuine guy, still trades now and hes 70 years old..I had a mentor once who had made a few million from his old strategies ..[many of his old strategies are redundant, but he teaches his new ones]
He also has free eLetters.
Again, he says himself, majority of his students don't make money.. but he has had many students make their million using him as a trading coach.. most people don't commit enough to trading.. they dont do the work necessary..its not something you can just do in a a few months and become rich.. it takes time in the markets and going down many rabbit holes on the way.. most people going on trading course, go home and never practice the strategies they learn, very few test them

Also check out Van Tharp.. he's a bit to much into his psychology, but he can point you in direction to find an edge.. again can subscribe to newsletters.. I'd recommend buying a couple of his books..

Also another guy I personally know is Patrick Stockhausen.. he has a free subscriber email.. again he's quite into his psychology but he can give practical advice on how to find an edge..and he makes money himself and is a real trader.

Check out these three guys.

Regarding platform to trade on: doesnt really matter as long as it works and you can roughly get the prices you see..
When I was trading futures I registered with LIFFE and traded directly through a 3rd party company..  otherwise I just spreadbet..
Ignore all the shit about spreadbetting being 'tax free', virtually everybody who trades in any finacial market  loses money initially anyway.. so actually it will be a loss you can't claim off tax man..
I liked Worldspreads before they folded.. IG Index are quite nice, spreads used to be a little high.. maybe ok now.. GFT UK was quite nice too.. I dabbled with Finspeads but the slow execution of Finspreads pissed me off so I dont use them..

To be honest the cost of spreads aint going to be the thing that loses you money initially.. your own losing trades will be..
so until you get the hang of trading and find a proven edge.. I'd keep your trade minimal, leave your ego at the door, and build up some reliable statistics and trade history.. yes you will put £20 on something and make £60 and wish you put £10k on it then bought a car.. but betting big like this, initially anyway, will wipe you out quick..if you are too aggressive you could be right 9 times in a row but the 10th time can completely wipe your account if you bet too big..

Ignore all the shit and 'how to make money' tutorials from spreadbet companies.. you can go mental trying to learn every possible indicator.. just study the guys I mentioned above, buy a couple of Van Tharps books for £10 on Amazon..etc 

You can start with as little as a couple of hundred pounds with spreadbetting.. make sure you have enough to bankrole your trading system tho.. you may have a string of losses quite naturally and that shouldnt destroy your account 

And expect to work very hard to find an edge, even by following the guys yourself you still have to put in 99% of the work yourself..

If you want to trade automatically..   Im very interested in this also, but Ive never yet done it..so cant really advise:
I picked up  NinjaTrader (can get for free) and started learning that a while back..
there is also TradeStation.. they are the top two platforms from what I understand..
I recommend 'Way of The Turtle' book by Curtis Faith , talks about backtesting.. good stuff, easy to read

Personal note: Its easy to write something on a forum and pretend to be an expert. So I have to be honest with you: back in 2009 and again in 2010, I though I would soon in a couple of years ,be a millionarre. Id found edges that worked and had started to make, to me, large amounts,during start of 2009, I had previous built my account up and up and had approx 10k before 2009, I then built my account to £40k from 10k in 3 months, (only risking 5% a trade)..

I had backtested my strategies but didnt realised Id optimised them to a bear market, and in March time when markets turned round, I started to lose my profits bit by bit.. I pulled the plug when I was back to 10k..

I then went onto daytrade for about 7 months.. after the 5th month I started to make money.. I had a bad 7th month and myself, markets changed property (and all my friends I was trading with) lost alot of our profits..I ended up £500 down on my initial investment of £10k, not too bad but  I had office/desk expenses and food + rent costs for the 7 months.. Id have made more money working at MacDonalds :)
I  realised I was still trading blind and needed to backtest my strategies to check a real edge existed..
I then moved house and started doing other stuff, I never tested my strategies..2 years went by and now I think 'why did I stop'
I havent properly traded since then, and regret stopping. I hit a wall and stopped like everyone else.. but its those that continue that succeed.
basically mate, I haven't yet made money trading (or betting actually) though believe Im close. You just have to find an edge then work the edge. I have found edges in trading but did not realaise they only worked in certain market conditions..the question for me now is, is it worth all the work for another couple of years to start to make serious money.. or should I go and start a business in the real world or do something completely different..  Right now, Im dabbling in several things.. but I think the time must come soon where I just pick one drop everything else and focus on just that one thing..

P.s. I have met and known a number of successful traders that have made a good living..  but there is 1 or 2 for every 99 that failed.. so the truth is, YOU could likely definately be one of those people.. just how committed are you to being one of them?.. are you prepared to spend e.g. 5 years of your life and energy to figure out how to make money..

I'm sure some of the more experienced bettors on here would say there is a direct comparision to what Ive said about the trading markets to the betting markets..

Hope that helps


Quote    Reply   
avatar

birchy

Betfair Elite

Posts: 593 Member Since:May 11, 2008

#32 [url]

March 4, 2013 20:56:14

Well that WAS a long read but a subject I'm familiar with. I guess that having the right mind set to NOT be too bothered about a £1000 downswing is fairly critical. But you also have to be sure that you have a real edge and have the confidence in your strategy to absorb the losses, knowing that you will profit long term. And getting to that point requires a LOT of dedication. Much, much, much more dedication than most punters/botting wannabes realise. That is why most give up. There are obviously a few people out there that slung some code together and had a winning strategy within their first few attempts, but I can assure you that you'll only need one hand to count them.

I also dabble in stocks, shares and managed funds and have done so for about 15 years. I don't take it too seriously, however I DO have most of my savings in some form of "medium risk" type investments...I guess that's the gambler in me. For actual shares trading, I carry out long term trades based on common sense and some bullishness. For instance, I made a nice profit from the BP oil spills a few years ago. When the price dropped, I ploughed in most of my savings. My decision was based on the fact that their $20Bn clean up costs was a tiny fraction of their annual profits and the likelihood of them going out of business was very close to zero. Sure enough, 6 months later it was all forgotten and the price had risen enough to secure a 40% profit. I did a similar thing with Tesco in Jan 2012 after their disastrous Christmas. I initially lost money but sat tight and ended up taking a 15% profit many months later.

I've often wondered about automating these sort of trades, or at least automating the triggers but it's just another line on the never ending "TO DO" list...

www.bespokebots.com

"This time next year Rodney, we'll be millionaires!"

Quote    Reply   
avatar

worthapunt

rookie botter

Posts: 15 Member Since:January 29, 2012

#33 [url]

March 4, 2013 21:23:29

I don't know lots about financial markets but some points that come to mind:

  • Generally there is a charge for a transaction with a broker.
  • The minimum transaction is higher.
  • For spread betting it is hard to limit your liability.
  • There is a charge for 'live' data.
  • The competition is very fierce (e.g. people are building servers in the arctic to reduce latencies!)
I agree overall financial markets might be better as they are a 'zero-sum game' in the short term and a 'positive-sum game' in the long term, while betting is a 'negative-sum game' due to betfair's commission.

Quote    Reply   
avatar

toy

rookie botter

Posts: 15 Member Since:June 2, 2008

#34 [url]

March 8, 2013 08:02:20

Thanks Guiness, I'm not new to prop trading. In fact I have a position on right now. I normally trade oil and the major equity indices. I have had a lot of success in the past and a lot of failures. Up until now I have only spreadbet.

Thanks for the recommendations, I will be checking them out. I have taken the decision to go full time trading and will be looking for the best resources. I'm ordering a new computer and purchasing Ninja. My real question was how easy/difficult is it to program your own automated strategies, I guess I'm not really in a position to worry too much about that right now but would like to visit it in the future.

Quote    Reply   
avatar

worthapunt

rookie botter

Posts: 15 Member Since:January 29, 2012

#35 [url]

March 8, 2013 21:05:15

For some reason it took a long time for these posts to appear...when I posted I was replying to toy's 'barriers to entry post'. Anyway I remembered another 'barrier to entry'.

  • Financial trading coincides with most peoples working day. Sports are conveniently arranged to happen in people's leisure time.
Birchy, you say you are a gambler due to your 'medium risk investments'. I think you probably just have your eyes open and know your risks, most people implicitly have most their wealth in property (mortgage) yet don't even realise it's an investment. Similarly most people have some 'savings' in cash and don't realise they are losing 2-3% per year due to inflation at the moment (even in a 'high interest account').

Quote    Reply   
avatar

pic

bot addict

Posts: 63 Member Since:July 28, 2011

#36 [url]

March 16, 2013 20:21:42

@guiness:

thank you for your feedback.
About http://data.betfair.com/, I think it is not sufficient for testing most of the strategies, unfortunately.
About buying data, it is quite expensive ...
About having many different accounts, this might work but I haven't been so interested in collecting data (until now) to deploy such a system :)

Quote    Reply   
avatar

birchy

Betfair Elite

Posts: 593 Member Since:May 11, 2008

#37 [url]

March 16, 2013 20:46:02

Talking of historical data...I wonder how much data we're all "wasting" by running bots and not saving it? I, for one, don't save any of the data my bots use but often think that I should. The reason I don't save the data is because my strategies work with every betting market available, so it can be minutes in between refreshes. But what if we were ALL saving data to an agreed format which we could then compile into one massive, timestamped database?

www.bespokebots.com

"This time next year Rodney, we'll be millionaires!"

Quote    Reply   
avatar

guiness

rookie botter

Posts: 39 Member Since:August 28, 2012

#38 [url]

March 17, 2013 15:20:23



Reply to worthapunt:

"
I don't know lots about financial markets but some points that come to mind:
Generally there is a charge for a transaction with a broker.
The minimum transaction is higher.
For spread betting it is hard to limit your liability.
There is a charge for 'live' data.
The competition is very fierce (e.g. people are building servers in the arctic to reduce latencies!)
I agree overall financial markets might be better as they are a 'zero-sum game' in the short term and a 'positive-sum game' in the long term, while betting is a 'negative-sum game' due to betfair's commission.
"

1) Charge with broker..  sure there is..with shares you usually get charged a set price per each share or or with spreadbetting the charge is 'added' into the price..
but betfair rob you of the 5% or so .. and eg SBO bet again like spreadbetting, include the charge in the spread between the back and lay..  charges are always there
2) min transaction is higher.. yeah Id say that was probably true.. certainly cant put on £2 bets with most trading platforms
3) Spreadbetting, hard to limit liability...when you say 'spreadbetting' it doesnt matter weather it is spreadbetting or trading shares, the same issue applies, its just as easy or hard to limit liability in both.. you know, 

But you know, the lines are bluring. eg with betting markets, you can trade in and out of prices.. in which case you have exactly the same issues of limiting liability.. also with spreadbetting, there are also binary bets now which are fixed loss..
Though overall yeah Id understand your statement..  but before I knew about betting, I never had any problems limiting liability.. I got the odd 'slippage' perhaps missed the price I wanted by a point but consequencese were minor..
In fixed odds betting sure, you only stand to lose what you stake..
with trading, you can pretty much achieve limiting liability  if you follow one golden rule: 
'Know where you will get out, before you get in' and set an automatic 'stop-loss' on that getting out price (available on most platforms)
eg suppose I buy xyz at 35.50 a share, I may decide that if price drops to 29.99  I will take the lose and get out..  in 
which case, suppose I want to risk at most £500,  I can calculate how many shares of xyz I should buy so that should price drop to 29.99 and Im forced to sell, then my loss will be no more than £500..
This is 98% fool proof depending on the market you trade.. sometimes if liquidly drops and/or market moves very quick.. your protection stopped can get breached.. but if you are careful what markets you trade (eg 24 hour markets are better) then this breaching can usually be limited to 1 or 2 points and not make too much negative impact..

4) Charge for live data.. well... again it depends what data you need.. sure there can be, I knew traders that paid £1500 a month for Bloomberg subcription.. and you are prob right that the average trader spends more money on the average bettor for live data.. but again there are professional betting guys out there paying people to sit at Tennis games and get the most uptodate scores just to give them a slight edge.. or television live sports subscriptions..
Also many traders can trade just off the information from their trading platform.. most platforms include free data feeds  eg IGindex.. so..  Id say it was kinda equal..

5) More competitions:  Hmm.. Id actually disagree with you on this one. I'm not a betting expert, but I've spent some time now looking at markets + I have a couple of friends in professional syndicates.. Its hard to give solid facts and figures, but I actually think betting is more competative, there are less markets and there is far less money in most markets also..and relatively more people (given size of market) trying to make money/living from those markets.. 
In betting, everyone is trying to make money..It may sound strange but in trading, not everyone is trying to make money.. people are often transfering risk..
eg you may have a company that is a big consumer of oil.. who just wants to insure himself agains a future price fluctation.. so he buys larges qualities of eg Crude oil futures for 2015 now, so if prices does increase, he can still get it for the cheap price of today..  with many big players like this, I think it creates opportunites for other traders to come in and find ways to profit..
 
Not sure how relevent this next comment is: but it feels to me that betting on exhanges seems  more 'personal' and dog-eats-dog, trading seems more anonymous..  you know even Betfair, that indroduced themselves as the betting man's friend.. now are playing the dog-eats-dog games and imposing more charges on bettors.. fighting to get a bigger bit of the pie..
Betfair can also be considered another competitor for a bettor..
If not betfair exchanges, then you have Ladbrokes and Paddypower etc etc.. who will limit your account if you start making too much money (from them)
Betting is hard, you are constanly fighting people, even the brokers that you are betting through..
You have to find a winning strategy with an edge.. then that edge (which may be small) has to be enough to overcome the 'betting tax/charges' Betfair offer you  ..

Quote    Reply   
Remove this ad
Add Reply

Quick Reply

bbcode help