thanks for this. i have started a basic python programming course and have github installed. this is awesome. i just have to try and work through this and solve. Thanks
Scraper/Method to obtain Top Speed and RPR from Racing Post Website into Excel
- Thread starter giuseppe_esq
- Start date
the example shown is for results. i m looking for the racecards so ideally im looking for date time meeting horse trainer jockey ts and rpr to be produced into a csv file and then i can analysis the data. KDW
Attachments
D daveat91 do you have an program that generates a file like the above ie runners or ts and rpr? thanks
Your luck is in L Laugro1968 because the problem is simply the same one I solved last night when I found the cards would no longer download - RP have made a tiny change to their page code and I had to change the programs I use to suit.... so a quick look (I found a copy of the original code) and I realised this was a problem I already fixed.
Delete the existing files from the first download, then follow the install process with this replacement version and you should be okay. I'm attaching a copy of the download i got a few minutes ago using this program for reference.
Dropbox - Error - Simplify your life
www.dropbox.com
Dave
davejb do you have a problem that produces a csv file of runners and ts and rpr? thanks
davejb
Dam
K
kdw - The thread you took that post of mine from is 2 years old mate, if I still have the code somewhere then it hasn't been updated in a long while, clearly the files I uploaded to Dropbox are no longer there. Your best bet is if one of the people I did this for back then reads this thread and still has a copy they can forward to you. I hope you'll understand that I don't 'maintain' scrapers for others, my programs are for my own use and while I've been able to alter some of them to suit others over the years once they are gone they're gone.
Having had a good root round let's see if this works to save further hassle -
www.dropbox.com
Hopefully this version that I produced for another guy on here will work for you, it did when I tried it although there are one or two bits of data that clearly aren't okay... 2 or 3 of the runners tomorrow have a topspeed rating of 6 inches or some such, but the vast majority look okay. If this works for you fine, but it is old code so no guarantees.
Dave
Having had a good root round let's see if this works to save further hassle -
Dropbox - Error - Simplify your life
www.dropbox.com
Hopefully this version that I produced for another guy on here will work for you, it did when I tried it although there are one or two bits of data that clearly aren't okay... 2 or 3 of the runners tomorrow have a topspeed rating of 6 inches or some such, but the vast majority look okay. If this works for you fine, but it is old code so no guarantees.
Dave
hi gbettle. is the git scrapper still there? Karl
thanks Dave. i ran it and it produced a csv file with headings but no data. no worries. I ll carry on by hand. I still have to do the hard work when i have the dataK kdw - The thread you took that post of mine from is 2 years old mate, if I still have the code somewhere then it hasn't been updated in a long while, clearly the files I uploaded to Dropbox are no longer there. Your best bet is if one of the people I did this for back then reads this thread and still has a copy they can forward to you. I hope you'll understand that I don't 'maintain' scrapers for others, my programs are for my own use and while I've been able to alter some of them to suit others over the years once they are gone they're gone.
Having had a good root round let's see if this works to save further hassle -
Dropbox - Error - Simplify your life
Hopefully this version that I produced for another guy on here will work for you, it did when I tried it although there are one or two bits of data that clearly aren't okay... 2 or 3 of the runners tomorrow have a topspeed rating of 6 inches or some such, but the vast majority look okay. If this works for you fine, but it is old code so no guarantees.
Dave
thanks for your help. im not very good with pcs. how do i down load it? thanks and sorry
GitHub - joenano/rpscrape: Scrape horse racing results data and racecards.
Scrape horse racing results data and racecards. Contribute to joenano/rpscrape development by creating an account on GitHub.
github.com
davejb
Dam
Okay, it worked the day I compiled it, but it looks like the abandoned Southwell meeting tomorrow caused problems - the program tries to download cards for an abandoned meeting basically, and then falls over when it can't find the race information.
I've had a go at it (even though I said I wouldn't) and got it to skip that problem, so there's a new link to the same program with the edit... overwrite the old version with this one therefore and you ought to be okay. I'll attach the resultant file for tomorrow that I get when running it so you can check you get the same.
I won't be doing fixing and stuff going forward though, I don't use this program myself and be warned that when the R Post does something different with its web pages, as it does sometimes, or when the cards have a load of silly stuff added (pony races, gazelle sprints, whatever) it may not cope with the silly stuff, so it'd be a good idea to learn Python!
Good luck,
Dave
www.dropbox.com
I've had a go at it (even though I said I wouldn't) and got it to skip that problem, so there's a new link to the same program with the edit... overwrite the old version with this one therefore and you ought to be okay. I'll attach the resultant file for tomorrow that I get when running it so you can check you get the same.
I won't be doing fixing and stuff going forward though, I don't use this program myself and be warned that when the R Post does something different with its web pages, as it does sometimes, or when the cards have a load of silly stuff added (pony races, gazelle sprints, whatever) it may not cope with the silly stuff, so it'd be a good idea to learn Python!
Good luck,
Dave
Dropbox
www.dropbox.com
wow. awesome thank youOkay, it worked the day I compiled it, but it looks like the abandoned Southwell meeting tomorrow caused problems - the program tries to download cards for an abandoned meeting basically, and then falls over when it can't find the race information.
I've had a go at it (even though I said I wouldn't) and got it to skip that problem, so there's a new link to the same program with the edit... overwrite the old version with this one therefore and you ought to be okay. I'll attach the resultant file for tomorrow that I get when running it so you can check you get the same.
I won't be doing fixing and stuff going forward though, I don't use this program myself and be warned that when the R Post does something different with its web pages, as it does sometimes, or when the cards have a load of silly stuff added (pony races, gazelle sprints, whatever) it may not cope with the silly stuff, so it'd be a good idea to learn Python!
Good luck,
Dave
Dropbox
Oatsey1983
Newbie
Hi
davejb I have just stumbled across your posts, you have done some amazing work scraping the racing post! I was wondering if you could give me some help please? I have a list of horses (700+) that I need to pull some information for. I need to know the Trainer, Current handicap mark, highest ever handicap mark, number of furlongs it runs over (average from previous runs) Its all available on the card on RP. I have done something similar before in c# for football but I'm struggling with the RP. Any help appreciated.
davejb I have just stumbled across your posts, you have done some amazing work scraping the racing post! I was wondering if you could give me some help please? I have a list of horses (700+) that I need to pull some information for. I need to know the Trainer, Current handicap mark, highest ever handicap mark, number of furlongs it runs over (average from previous runs) Its all available on the card on RP. I have done something similar before in c# for football but I'm struggling with the RP. Any help appreciated.
davejb
Dam
Sorry,
you might not have noticed but I stopped the programming/speed figure work a year ago and have no intention of restarting.
I would, generally, suggest that anyone wanting to scrape data either find somebody who is currently doing so and willing to share their work - I think I've seen a few using Excel to scrape data elsewhere on the forum over the past year or two - or go the whole hog and learn a programming language, such as Python, which is the one I used and is available free online, as ultimately if you write it yourself then you'll have some idea of how to fix it when it stops working!
All I did was to identify one or more website pages that contained the information I wanted, make note of the page address(es) I wanted, then download the whole of each page to a text file/collection of text files (a file per web page). It was then a matter of writing simple text search algorithms that would identify sections of text containing the information I wanted, so for example a line that contained the name of a horse's jockey would have some formatting info followed by the words '< horse-jockey == > Fred Bloggs' and my program would search for the horse-jockey phrase and copy whatever followed it. Copy all the bits of text into one place for final editing, then just run through it. You simply (!) learn how a page is formatted, identify phrases that identify data items, strip them out then do what you want with the result. It just looked simple because my program did a lot of text shuffling that went on in the background.
I don't even have the code on my PC any more, it's all off on some hard drive somewhere just on the offchance I ever load Python back on and give it another go.... estimate for that being 'hell freezes over plus one month'.
I hope you manage to figure it all out. The racecard info from RP will give you the data you want quite happily, it's just a matter of downloading rhe pages and parsing the text, plus a bit of aggro when things don't quite go as planned.....
Dave
you might not have noticed but I stopped the programming/speed figure work a year ago and have no intention of restarting.
I would, generally, suggest that anyone wanting to scrape data either find somebody who is currently doing so and willing to share their work - I think I've seen a few using Excel to scrape data elsewhere on the forum over the past year or two - or go the whole hog and learn a programming language, such as Python, which is the one I used and is available free online, as ultimately if you write it yourself then you'll have some idea of how to fix it when it stops working!
All I did was to identify one or more website pages that contained the information I wanted, make note of the page address(es) I wanted, then download the whole of each page to a text file/collection of text files (a file per web page). It was then a matter of writing simple text search algorithms that would identify sections of text containing the information I wanted, so for example a line that contained the name of a horse's jockey would have some formatting info followed by the words '< horse-jockey == > Fred Bloggs' and my program would search for the horse-jockey phrase and copy whatever followed it. Copy all the bits of text into one place for final editing, then just run through it. You simply (!) learn how a page is formatted, identify phrases that identify data items, strip them out then do what you want with the result. It just looked simple because my program did a lot of text shuffling that went on in the background.
I don't even have the code on my PC any more, it's all off on some hard drive somewhere just on the offchance I ever load Python back on and give it another go.... estimate for that being 'hell freezes over plus one month'.
I hope you manage to figure it all out. The racecard info from RP will give you the data you want quite happily, it's just a matter of downloading rhe pages and parsing the text, plus a bit of aggro when things don't quite go as planned.....
Dave
TheBluesBrother
Dam
You can get the information you require from my speed figures lists, the current lists go back to 2020.
FLAT2021_2025.xlsx
drive.google.com
NH2020_2025.xlsx
drive.google.com
If you need to know what handicap marks they ran to in a race add 17 lbs to the speed figure for the flat and 28 lbs for the NH.
Base rate flat = 9-0 (100)
Base rate NH = 11.0 (130)
All my speed figures are rail adjusted.
Mike.