myrddin's Blog

Permissions are one of the big advantages linux has (or had) over windows; you are probably better off biting the bullet and learning about them, then doing things right from the start. If you get into the habit of working as root on a routine basis, it will pretty soon bite you in the bum. root really is 'god' on a *nix system - on windows, even the default Administrator is limited in what it can do - linux will quite happily let root irretrievably trash the system without so much as an 'are you sure?' (been there, done that). Always try to limit su / sudo use to the ABSOLUTE minimum required.

Can't help with the C++/Java scraping, sorry, but I just wondered how you were managing with online poker under linux. I remember you were keen on the betfair site a month or so ago; have you had any luck running their software under linux?

I think nadat hit the nail on the head - all these choices depend on your application. Work out what you want to do first and pick the tools to suit.

Depends what OS you're using (and Excel/VBA seems to imply Windows), but on linux, MySQL is a sensible choice - dead easy to set up and it's ubiquity means there is loads of information, help and advice available for free online.
Two years ago I would have said it was the only sensible choice, but lately I have been hearing a lot of good things about PostgreSQL, which seems to be undergoing something of a renaissance. This is a VERY powerful,enterprise level database - if I was starting from scratch today I would seriously consider it, and even with 25 GB of existing MySQL tables, I am thinking of migrating.
I use perl rather than PHP, and the main database library, DBI, is designed to be as agnostic as possible regarding back-end databases. This means you can swap your back-end with minimal code changes (providing the schema stays the same). Not sure, but I think PHP has a similar idea.

Heads up - there's an article on slashdot about high-end poker bots
(http://games.slashdot.org/article.pl?sid=08/07/03/214245),
mainly about the University of Alberta program
( http://poker.cs.ualberta.ca/ )
They are the guys who came up with a total solution to draughts, btw, so they know their stuff regarding theoretical game playing.

Just saw this on slashdot - sort of sums up botting.....
One day this guy is finally fed up with his middle-class existence and decides to do something about it. He calls up his best friend, who is a mathematical genius. "Look," he says, "do you suppose you could find some way mathematically of guaranteeing winning at the race track? We could make a lot of money and retire and enjoy life." The mathematician thinks this over a bit and walks away mumbling to himself.

A week later his friend drops by to ask the genius if he's had any success. The genius, looking a little bleary-eyed, replies, "Well, yes, actually I do have an idea, and I'm reasonably sure that it will work, but there a number of details to be figured out.

After the second week the mathematician appears at his friend's house, looking quite a bit rumpled, and announces, "I think I've got it! I still have some of the theory to work out, but now I'm certain that I'm on the right track."

At the end of the third week the mathematician wakes his friend by pounding on his door at three in the morning. He has dark circles under his eyes. His hair hasn't been combed for many days. He appears to be wearing the same clothes as the last time. He has several pencils sticking out from behind his ears and an almost maniacal expression on his face. "WE CAN DO IT! WE CAN DO IT!!" he shrieks. "I have discovered the perfect solution!! And it's so EASY! First, we assume that horses are perfect spheres in simple harmonic motion..."

If it was done well, I can see people using something like that. Not sure how much they would be prepared to pay, though, and the potential market is, as you say, pretty small. One thing that makes it even smaller is the diversity of languages people use for botting - you aren't tied by the betfair end, so people use Visual Basic, C#, ruby, python, java, php (some poor sods even use perl). It would have to be blindingly good to get people to switch.
I think if you are prepared to put in a lot of work for little or no return, you could get a reasonable level of uptake for something like this (especially if you are writing the stuff anyway for your own bots). As a stand-alone commercial project, I would say it's a non-starter.

Five, or even three, years ago you were probably right; writing code to interact with dynamic web pages was a specialised skill-set. Nowadays there are so many well-documented libraries and code examples kicking about it's pretty easy to hack together a working bot. (Only today, nadat has posted examples in C#, ruby and python). I have to agree with austinpodhorzer, if someone is at the stage of having viable ideas for automated strategies, they will learn the programming skills required. It's not always plain sailing, granted, but this forum, and fred's before it, are invaluable for the coding numptys among us (myself included).

Can't find any changes in selectionId for horses running in the UK and Ireland after a quick check,(in the past 14 months anyway), but I will cross-check with Sporting Life when I get a chance.
For US races they seem to assign new SelectionIds for each race, but I don't bet on those so it's not a problem. Seems strange that they treat these differently, maybe something to do with where they get the original race entry data from.

"Hardly a big deal I know, but why on earth would they change one of the seven? Do they often do this sort of thing? Do they hate us that much? "

I think this comes under Hanlon's Razor: Never attribute to malice that which is adequately explained by incompetence :-).

Having said that, there are a few places in my code where I make assumptions that SelectionIDs won't change, usually accompanied by comments like 'fix this to make it more robust', because I realised it WAS an assumption, but, touch wood, nothing has broken so far..

One thing I do is uniquely identify horses by the SelectionId, in fact I use it as the Primary Key in my Horse table. I did some initial testing a year ago which indicated this didn't change between races or seasons, but nothing in the documentation says this. I shall do some checking tonight to see if any have changed, and report back.

Thanks for the heads-up on Absolute, birchy, didn't know about that.
Packet sniffing on Windows 9.x used to be straightforward (there was some Italian University freeware that was blinding), but NT/W2k/XP was a pain. You needed some MS libraries that were only available for server versions, and I could never get it to work properly on my system.
Recently, MS made their sniffer (Microsoft Network Monitor) available as a free download, but I don't know how good it is.
I use tcpdump on linux which is very good and well established; perl has modules which make it painless, and I would imagine other languages do as well. It is also available for Windows, but I've never used it on that platform.
Anyway, getting the raw packets should be quite straightforward (did it for Paradise about 3 years ago), but I couldn't decrypt them so I gave up. Hope you have more luck.

I play on Absolute Poker, mainly because they have a Java client and I use linux (realtime plain text Hand Histories as well, birchy). Used to use Paradise under wine, but got fed up of the way it scattered files all over my home directory.
I noticed the same thing as bmsleight just after the US clampdown, the only Yanks who played were sharks prepared to make the effort. I don't think that's so true now though, a lot of fish are creeping back from over the pond as circumvention methods filter down..(I mentioned the football at a ring game the other night and no-one knew what I was on about).
I think the games would have to be much looser than they are before I would risk using a bot to play, but as birchy says, helper apps can be very useful. I used to slap all the hand histories into a database and use that to identify strong/weak players and such. It's very useful for analysing your own play as well, I found I was overplaying KK (used to treat it pretty much like AA), and JJ (ditto, playing it like QQ), and losing a lot on things like KQ and KJ suited when I hit a flush (you only ever get action against an Ace-Rag).
I don't use helper apps now, I would have to alter the code for the new site, and there are higher priorities on my coding time (horesracing bots especially). After this thread, though, it's going back on my to-do list, right under the v6 API upgrade....

Still using v5 here, an upgrade is one of those things on the 'to do' list that never gets done - no pressing reason for me at the moment. When they dropped v4 there was an announcement on bdp about 3 months in advance of the scheduled date (which ended up being 6 months before the actual date). So I would guess at least another year.

I think it depends a lot on your strategies; trading based / in running bots would probably benefit from being on a server closer to betfair, especially if your home ping times are as bad as mine. My own methods are not particularly time sensitive, so I run from home.
I don't think better uptime alone would justify hosting, basic packages are cheap but if you want control of your environment, good service, a London-based centre and guaranteed uptime that still costs.
I think a backup running on a mates system might be an idea - the only significant (> 2 hr) downtime I've had in two years was a local exchange problem, so another line to my house, even with a different provider, wouldn't have helped. I thought about that when you mentioned it, pretty sure my brother would be OK with the idea, but I know I'll never get round to it.
To be honest, Network downtime so far (> 1 yr) hasn't been a problem. (Betfair downtime is a different matter!). If it became a problem, I would think about other solutions but until then....

ooops.......
I just read some other posts, especially inksmithy's above mine. My comments about guis and oop weren't directed at him, just general waffle. Good to see new people here, in fact. The main thing I gained from Fred's page was a feeling that I was not alone, which is important when you are starting out.
Sorry, inksmithy, if my previous post looks like a flame, it wasn't intended to be. I'm sure you are no mug. Welcome to the brave new world of bot-building, and I hope you get as much support here as I did at Fred's.

Ahhh ... This is where you all got to.
I used to work for a bookies, first as a shop manager then in Head Office. One of the things I did was QA testing of betting software (a few years ago now, before betfair or even internet betting).
I used PCs to help with my own bet selection quite early on (Turbo Pascal on DOS), but even when online bookies and betfair came along I still placed my bets manually. I always realized that automating things was the way to go, but that seemed to need 'real' programming skills, at least in the early days.
It wasn't until about a year ago I looked at automation seriously again, and after finding Fred's site I saw that this was now within the capabilities of hobbyist programmers, so I took the plunge (Thanks, Fred!).
I use perl / MySql on linux for my bots, running against the free API. Most of it was slapped together as Proof of Concept, with a view to doing it 'properly' later; needless to say, all of those shabby hacks are still there a year later :-)
I don't think that you have to be a great programmer to make a successful bot, but you do need sound knowledge of mathematics and gambling. If you bet like a mug in real life, your optimized, modular, object-orientated, gui-driven shiny new bot is just going to be a (super efficient) mug punter :-)