• Hi Guest Just in case you were not aware I wanted to highlight that you can now get a free 7 day trial of Horseracebase here.
    We have a lot of members who are existing users of Horseracebase so help is always available if needed, as well as dedicated section of the fourm here.
    Best Wishes
    AR

Scraper/Method to obtain Top Speed and RPR from Racing Post Website into Excel

Your luck is in L Laugro1968 because the problem is simply the same one I solved last night when I found the cards would no longer download - RP have made a tiny change to their page code and I had to change the programs I use to suit.... so a quick look (I found a copy of the original code) and I realised this was a problem I already fixed.

Delete the existing files from the first download, then follow the install process with this replacement version and you should be okay. I'm attaching a copy of the download i got a few minutes ago using this program for reference.


Dave
Hi davejb davejb

I hope your system is doing well?

I have taken a break from the horses but i always keep hopping onto the forum to see what's going on.

I was grateful for the original script you kindly wrote and shared, and i am grateful for this.

Many thanks
Giuseppe
 
Hi Dave,

It works as a charm.
I am a member of a couple of great horse racing groups and analyse horse racing data from a couple of resources/software's.
I am a system bettor with SQL knowledge, which allows me to produce my own racecards.
I have been doing this for a couple of years now and am still impressed with the amount of great people I have met and still am meeting inside the community.

I hope that I'll be able to return the favour.

Thank you,
Laurent
 
Last edited:
Happy to oblige guys,
I just like to keep people's expectations in check a bit, as in the past I've had folk asking me to just knock up quite large programs as if it's a 15 minute job, and then getting miffed when I prefer to have a life :D

I quite enjoyed learning to use SQL many years ago as part of a database programming course I took, but never really got into using it for my own projects over the years - I prefer going to a lower level for access and having a bit more freedom with what I do with it. Luckily nice people produce languages like Python to make things easier - 20 years ago, for me, it was Visual Basic, before that Pascal and C++...... I think the most enjoyable time I had in programming was actually back in the early 80's learning machine code on various home computers - you could program yourself a sports car in 6 months back then, if you got lucky!

Dave
 
I’m far from stupid , once did one of those online IQ tests and it said I was in the same percentile as Einstein (yeah yeah) but when I see things like this it goes right over my head and I feel frustrated , how you guys understand all this what I see as jargon always amazes me, it would take me 6 months trying to make these scrapers work on my pc , never mind programming it myself. I thought Python was a snake.
 
Hi ArkRoyal ArkRoyal
The C64 was my favourite for coding on, as there was so much info about its internals available and the 6502 chip (called a 6510 in the C64 due to a very minor difference) was really simple compared to the Z80.... Loads of fun on it for a year or two before moving on. I even had a game out on it - a very badly selling one unfortunately! (No sports car for me until rather later in life as a result).

Dave
 
I win! A Sinclair ZX81...with...a...16K ram pack. Learned all the Z80 machine code and used it to create my first racing DB in the early '80s. I typed in all the results for Peter Walwyn, Henry Cecil, Dick Hern, Guy Harwood and a few others, taken from the form book, on that miserable plastic membrane keyboard it had and wrote another routine to produce statistics. Although I wasn't able to make much use of it at the time, it stood me in good stead for the second part of my career, about as far from horse racing as you could get, by the way, but that's another story.

And now for a cautionary tale about backing up databases. I have 250,000+ French races, scraped since 2011 and going back to 1982, in an SQL Server DB. I have a lot of routines written in T-SQL for managing the DB and for ad hoc queries that I use regularly. Because I know how easy it is to corrupt a DB with my sloppy programming, I got into the habit of backing everything up regularly. And it saved my bacon today.

I use a Mac with a Windows 7 virtual machine running Parallels on the OS X operating system. Apple's built-in Time Machine backs up OS X, including the Windows VM, every hour. My Windows VM is backed up in the cloud, also every hour, using Backblaze, which I've used for years with few problems, but I deliberately exclude my SQL databases from the Backblaze backups (the files are 2 GB+ in size, so it would be a significant drain on CPU resources, as they change every day). In the middle and on the last day of each month, I copy the SQL database to another disk, which requires stopping the SQL Server via Windows Services, I clone OS X, including the Windows VM, using Carbon Copy Cloner, into a bootable setup on a separate disk, which I test from time-to-time (no problems so far, as long as you disable Backblaze as soon as you fire up Windows in the clone - I'll go into the reasons why if anyone chooses this way of doing things).

And now for a dazzling example of my thinking-I know-it-all stupidity that occurred today. I've been promising myself for some time now that I would edit all the comments-in-running to determine pace factors and the race descriptions, which include sale details in selling (reclamé) races - which horse, how much and by whom the horse was claimed.

T-SQL allows you to have multiple algorithms in the same tab, so long as you comment out each algorithmic code using the "/*...*/" comments and only run the one you want by highlighting it. Paris-Turf doesn't have the same disciplined comments-in-running and sales reports that you have in the Racing Post. It's all very free-flowing and dependent on the style of who's writing them. Right, so I wanted to change the race description for a particular race to show that the horse was bought-in ("défendu") by the owner and not claimed (reclamé) by the current owner. This was the code I used:
/*
USE FrenchDB
UPDATE Links SET [Description] = 'A reclamer - 9.500 € - 1.600 metres - Corde a droite - Bon terrain - Temps total 1''44.67 - Reduction km 1''05.42. Adorian a ete defendu pour 12.888 €.'
WHERE RceID = 1145048

*/
Unfortunately, I didn't highlight the last line, so all 250,000+ races were updated with the same race description. If I had made no back-up provision, I would have been rightly fucked, but I was able to restore the complete DB using the last fortnightly copy (stopping SQL Server via Windows Services before I did so) and scraping the results for 13 days from the 1st November, allowing me to update the database up to today, with no problems. Sure, it took me all afternoon to do it, but compared to what I could have lost, that's nothing at all.

I have, arguably, the best French horse-racing database available today, thanks to a lot of hard work over the years. Unfortunately, this does not exonerate me from incredible idiocy when it comes to programming in T-SQL. So, other users of the facility: BE WARNED! You never really know how idiotic you can be!

Ray
 
I win! A Sinclair ZX81...with...a...16K ram pack. Learned all the Z80 machine code and used it to create my first racing DB in the early '80s. I typed in all the results for Peter Walwyn, Henry Cecil, Dick Hern, Guy Harwood and a few others, taken from the form book, on that miserable plastic membrane keyboard it had and wrote another routine to produce statistics. Although I wasn't able to make much use of it at the time, it stood me in good stead for the second part of my career, about as far from horse racing as you could get, by the way, but that's another story.

And now for a cautionary tale about backing up databases. I have 250,000+ French races, scraped since 2011 and going back to 1982, in an SQL Server DB. I have a lot of routines written in T-SQL for managing the DB and for ad hoc queries that I use regularly. Because I know how easy it is to corrupt a DB with my sloppy programming, I got into the habit of backing everything up regularly. And it saved my bacon today.

I use a Mac with a Windows 7 virtual machine running Parallels on the OS X operating system. Apple's built-in Time Machine backs up OS X, including the Windows VM, every hour. My Windows VM is backed up in the cloud, also every hour, using Backblaze, which I've used for years with few problems, but I deliberately exclude my SQL databases from the Backblaze backups (the files are 2 GB+ in size, so it would be a significant drain on CPU resources, as they change every day). In the middle and on the last day of each month, I copy the SQL database to another disk, which requires stopping the SQL Server via Windows Services, I clone OS X, including the Windows VM, using Carbon Copy Cloner, into a bootable setup on a separate disk, which I test from time-to-time (no problems so far, as long as you disable Backblaze as soon as you fire up Windows in the clone - I'll go into the reasons why if anyone chooses this way of doing things).

And now for a dazzling example of my thinking-I know-it-all stupidity that occurred today. I've been promising myself for some time now that I would edit all the comments-in-running to determine pace factors and the race descriptions, which include sale details in selling (reclamé) races - which horse, how much and by whom the horse was claimed.

T-SQL allows you to have multiple algorithms in the same tab, so long as you comment out each algorithmic code using the "/*...*/" comments and only run the one you want by highlighting it. Paris-Turf doesn't have the same disciplined comments-in-running and sales reports that you have in the Racing Post. It's all very free-flowing and dependent on the style of who's writing them. Right, so I wanted to change the race description for a particular race to show that the horse was bought-in ("défendu") by the owner and not claimed (reclamé) by the current owner. This was the code I used:
/*
USE FrenchDB
UPDATE Links SET [Description] = 'A reclamer - 9.500 € - 1.600 metres - Corde a droite - Bon terrain - Temps total 1''44.67 - Reduction km 1''05.42. Adorian a ete defendu pour 12.888 €.'
WHERE RceID = 1145048

*/
Unfortunately, I didn't highlight the last line, so all 250,000+ races were updated with the same race description. If I had made no back-up provision, I would have been rightly fucked, but I was able to restore the complete DB using the last fortnightly copy (stopping SQL Server via Windows Services before I did so) and scraping the results for 13 days from the 1st November, allowing me to update the database up to today, with no problems. Sure, it took me all afternoon to do it, but compared to what I could have lost, that's nothing at all.

I have, arguably, the best French horse-racing database available today, thanks to a lot of hard work over the years. Unfortunately, this does not exonerate me from incredible idiocy when it comes to programming in T-SQL. So, other users of the facility: BE WARNED! You never really know how idiotic you can be!

Ray
Or use BEGIN TRAN then commit or roll it back :)
 
ZX81 with a 16k ram pack was my very first micro - I'd seen ads for the ZX80 in Punch whilst sitting around the mess (about once and a bit a month SAR standby for 24 hrs living in the mess ready to go, in my previous/first proper career, after lunch I'd read every magazine and paper in the ante room just about). Saw an ad in the local paper for the zx81/Ram pack, mono portable TV, tape recorder and one game..... snapped it up for a £100 - then I started buying the computing mags and discovered the Spectrum was about to appear and realised why the guy was so happy to get the asking price.... ho hum.

Never bought the ribbon cable fix for ram pack wobble - just got into the habit of backing up frequently, even if the wobble didn't get you a small typo in a listing or a finger blip in typing and when you told it to run (exec?) you'd have a frozen computer and sod all way out except to turn off and back on. It was at that point you'd really beging to curse the ZX81's other other other big problem - very unreliable loading. I tried to learn z80 on it, and largely succeeded, but I bought a Dragon 32 as soon as they came out and that was a much more straightforward thing to code - 6809 processor, I bought something I am certain was called 'The 6809 cookbook' and learned to code by writing out assembler instructions and working out the opcodes to poke in by looking them up in the book. The people who produced the Dragon software brought out an assembler after a while and I couldn't believe how easy it was - no more having to work out/count backwards from 255 (in hex) to figure out the offset for a branch command, you just had labels! I could count to 255 (or down from 255) in hex for decades after.....

C64 (6510) after that, then the Atari STFM (512 and 1040 in the end), I got some work for the 'magazines on a tape' lot - Argus press, they pretty much subcontracted them all to a guy called Roy to produce each month, I had a very happy 6 months to a year or so writing small programs and reviewing software, as one of my programs was very good at copying software.....ahem. (Thanks to a very very detailed C64 disk OS book). I bought a Spectrum 128 with a built in tape, Speccy plus 2 I think they were called, tried a bit on it then it became the kids video machine.....First OU computer course I added an Amstrad PC, bought a 386 a year or two later, then built my own for the next 20 odd years whilst learning a good handful of languages and all sorts.The airforce were nice enough to let me be a system manager of a good sized network for 6 years to finish my career, so i got to add a bunch of courses on all sorts of other stuff like database management and security.... by the time I retired in 2000 I'd had enough to know I didn't want an IT job in civvy street!

End of an era was last May or so - my first ever graphics card failure and the realisation that there's no point buying a good replacement card for a PC that is more than about 2 weeks old, as you end up paying over the odds to upgrade the other bits, so I bought a new PC from Currys instead of building my own - I just couldn't be bothered DIYing it any more. Now all I do is reload everything after each windows update, as they seem to enjoy making my PC freeze for hours until I give up and pull the plug out. Sneaky bas***ds have done away with the 'no I don't want to load another of your bug ridden so called updates' option, it's been a bit of an epic just to get it set so that it will - hopefully - let me know it wants to restart next time it does it, giving me a little control of a process I'd happily halt. (It runs okay, leave it be, and you can shove 'Edge' where the monkey keeps his nuts, as my dear mother used to say).

Dave
 
Well, my first interaction with a pc was with a Tandy TRS-80 and my first program in BASIC was on a goupil computer!
Loooong before the ZX Spectrum!
 
If you call up the results page on the Racing Post site you can use the date picker to go back - it looks like the limit is back to 2016 - but that'd be a fairly long winded process I'm afraid. You would, I believe, have access to these figures as a subscriber to Raceform Interactive, Mick is probably the chap to ask about that.
Dave
 
If you call up the results page on the Racing Post site you can use the date picker to go back - it looks like the limit is back to 2016 - but that'd be a fairly long winded process I'm afraid. You would, I believe, have access to these figures as a subscriber to Raceform Interactive, Mick is probably the chap to ask about that.
Dave

Turns out I'm a member. I will ask him. Cheers!
 
I have written some very poor vb code to analyze each day's handicaps using downloaded data from RI. It takes me about 15-20 mins for the days racing. I have coded around RI's data inconsistencies wherever necessary though find it very time consuming and frustrating. My sole reason for subscribing to RI is to download the cards and each horses previous form by running two RI queries. Is it feasible for me to try and learn Python to do this for me (given my own limitations) or should I just bite the bullet and renew my subscription (I could get the data from geegeez as well if anyone knows if that would be easier). Alternatively, how much would it cost(approx) to pay someone to do it for me
 
Back
Top