Site
Home / Forum / Developers / Using scripts with the website
Home / Forum / Developers / Using scripts with the website
Using scripts with the website |
| Posted: 23 Aug 2020 01:59 | |||
 Posts: 938 Joined: 2020 | I've made a python script that will import data into the website using specially formatted csv files, earlier I managed to import over 100 events earlier (all European Championship tournaments since 1960), but my next use of the script will be larger as I intend to import all FIFA World Cup matches from 1930 into the DB. My concern was that I may get caught up in some spam filter to stop large amounts of accesses to the site, could you let me know if this may be a problem or if I'm fine to go ahead? zag | ||
 | |||
| Posted: 23 Aug 2020 21:58 | |||
 Ovokx  Posts: 1,769 Joined: 2020 | Any chance of something like this for to upload player stats, please? | ||
| |||
| Posted: 23 Aug 2020 22:32 | |||
Posts: 938 Joined: 2020 | It could definitely be used for that too, and I'm very happy to post a script if it's allowed. I'm very much an amateur when it comes to coding, so it's not exactly the most user friendly thing, hopefully you've got some knowledge so that you'll understand it, but I'll comment it out to make things a little clearer hopefully. Here is an example of what the csv will need to look like for stats: https://docs.google.com/spreadsheets/d/1lu-YQb7XmOXu7yQE6l0tSx7imLUwG8wlUYZqz8IN968/edit?usp=sharing I'll just wait to see what Zag says first before I post it. | ||
| |||
| Posted: 24 Aug 2020 18:39 | |||
 zag Posts: 3,515 Joined: 2020 | I trust your skills and dedication so go right ahead  No limits that I am aware of for adding data via form objects.Its essentially what I do with PHP in the background for some of the stranger leagues with flashscore data that is copy and pasted into a csv with some excel wizardry. Any help is much appreciated on the older leagues, cups or even different sports. I think for newer leagues its not needed as I can import directly and keep some of the internal API ID's for future updates. | ||
| |||
| Posted: 24 Aug 2020 19:12 | |||
Posts: 938 Joined: 2020 | No limits that I am aware of for adding data via form objects. Its essentially what I do with PHP in the background for some of the stranger leagues with flashscore data that is copy and pasted into a csv with some excel wizardry. Any help is much appreciated on the older leagues, cups or even different sports. I think for newer leagues its not needed as I can import directly and keep some of the internal API ID's for future updates. That's fantastic, in the end I uploaded it in two parts just to be on the safe side, but it was still about 300 odd entries each. The 1930 and '38 World Cups were added under the 1950 season since there's no option to go back further. Here is the script I made https://drive.google.com/file/d/1b8-t7Prok7Vs7QuqqtPbC1hH-Yb16shI/view?usp=sharing, I just tested it by adding Lukaku's stats for the current season, and it worked fine, please let me know if you have any questions. | ||
| |||
| Posted: 24 Aug 2020 19:19 | |||
Ovokx Posts: 1,769 Joined: 2020 | Here is the script I made https://drive.google.com/file/d/1b8-t7Prok7Vs7QuqqtPbC1hH-Yb16shI/view?usp=sharing, I just tested it by adding Lukaku's stats for the current season, and it worked fine, please let me know if you have any questions. link is not public  | ||
| |||
| Posted: 24 Aug 2020 19:24 | |||
Posts: 938 Joined: 2020 | Here is the script I made https://drive.google.com/file/d/1b8-t7Prok7Vs7QuqqtPbC1hH-Yb16shI/view?usp=sharing, I just tested it by adding Lukaku's stats for the current season, and it worked fine, please let me know if you have any questions. link is not public Ha, sorry about that, try it again now. | ||
| |||
| Posted: 24 Aug 2020 20:20 | |||
zag Posts: 3,515 Joined: 2020 | Nice thanks, I have added all season strings back to 1900 now curswine | ||
| |||
| Posted: 24 Aug 2020 20:25 | |||
Posts: 938 Joined: 2020 | Thanks, I've changed them to the correct seasons now. | ||
| |||
| Posted: 25 Aug 2020 16:43 | |||
Ovokx Posts: 1,769 Joined: 2020 | Here is the script I made https://drive.google.com/file/d/1b8-t7Prok7Vs7QuqqtPbC1hH-Yb16shI/view?usp=sharing, I just tested it by adding Lukaku's stats for the current season, and it worked fine, please let me know if you have any questions. link is not public Ha, sorry about that, try it again now. Great, thanks! First time playing with these things... :P | ||
| |||
| Posted: 25 Aug 2020 18:43 | |||
Ovokx Posts: 1,769 Joined: 2020 | Finally I get it to run by installing all relative things, but when I'm running it, script gets stop once selects first field like this: IDLE (software which I'm using to run it on a Mac) shows this message: Traceback (most recent call last): File "/Users/Never/Downloads/TSDB_Add_Honours.py", line 29, in season = line[1] IndexError: list index out of range Here you can see the CSV file that I created as a test: https://mega.nz/file/A0hiGaLa#VYe3AdF0zgdICwxh2rqepiVSuoSiunDwhURVuMy6YJ8 Any clue, please? TY! | ||
| |||
| Posted: 25 Aug 2020 19:26 | |||
Posts: 938 Joined: 2020 | It may be that you're using semi-colons to separate the values when commas are preferred. | ||
| |||
| Posted: 26 Aug 2020 15:51 | |||
Ovokx Posts: 1,769 Joined: 2020 | Using commas worked perfect! Thank you mate! zag, curswine | ||
| |||
| Posted: 26 Aug 2020 19:08 | |||
Posts: 938 Joined: 2020 | Using commas worked perfect! Thank you mate! Glad it's working for you, hopefully it can speed the process and cause less RSI  | ||
| |||
| Posted: 29 Oct 2020 20:09 | |||
Ovokx Posts: 1,769 Joined: 2020 | Strange issue today while using the script to upload stats. All values are OK of the spreadsheet and all of them are uploaded fine to the site, but season only gets first year. For example, 1996 instead of 1996-1997. Any change on the site or something? | ||
| |||
| Posted: 29 Oct 2020 21:06 | |||
Posts: 938 Joined: 2020 | Any change on the site or something? It doesn't look like there has been any changes. check your CSV to make sure the character between the two years is a hyphen (-) and not any other sort of character (– — ― ‒). these could cause the date not to be entered properly. If that still doesn't work, post a link to the CSV file and I'll take a look. | ||
| |||
| Posted: 30 Oct 2020 04:59 | |||
Ovokx Posts: 1,769 Joined: 2020 | Still having the same issue: https://mega.nz/file/d4I3yACS#vavZWdceHlmmJMzsCl1zmF8W0A7KDZDtTojEkIvaqqo If you want to test it, related player ID is 34162044 | ||
| |||
| Posted: 30 Oct 2020 07:58 | |||
Posts: 938 Joined: 2020 | From what I can see whatever program you're using to create the CSV (LibreOffice?) is adding '+AC0' before the hyphen, it's strange that has only happened once with this file. Try opening it in a plain text editor such as Notepad and you should see the problem yourself, try doing a find and replace for +AC0 and save it again and that should fix it. | ||
| |||
| Posted: 30 Oct 2020 14:55 | |||
Ovokx Posts: 1,769 Joined: 2020 | Solved, thank you very much for the tip! PS: Yes, I was using OpenOffice and first time that happened this :S | ||
|
Who is Online? In total there are 68 users online :: 3 registered, 0 hidden and 65 guests (based on users active over the past 5 minutes) Most users ever online was 424 on Fri Nov 10, 2017 9:02 pm | About Us Discussion forum for TheSportsDB.com site and related topics | Rules - Be Polite - Respect other users - Always post log files with issues - Try to be helpful - No Piracy discussion |