LordHelmchen's price import - BFZ and mkm
by Goblin Hero
Moderators: charmer, CCGHQ Admins
further generalizing underway
by LordHelmchen » 11 Feb 2014, 21:38
This is mostly for anthonybe's sake. No sitescript uses this version yet, so if you're only looking for working scripts, don't donwload this yet. See the wiki for further details.
- Attachments
-
- LHpi-v2.7pre1.lua.zip
- (30.55 KiB) Downloaded 661 times
-
- LHpi.sitescriptTemplate-v2.6.1.lua.zip
- (7.63 KiB) Downloaded 690 times
- LordHelmchen
- Posts: 125
- Joined: 21 Aug 2012, 16:06
- Has thanked: 21 times
- Been thanked: 32 times
2.7 (2014-02-13)
by LordHelmchen » 13 Feb 2014, 15:09
Goblin Hero: Still looking for answers to two questions. Find them two posts above this or at "Problems" in the wiki. Thanks.
2.7 is stable enough and sufficiently advanced over 2.6 for a new release. I hope this version addresses all issues that anthonybe had with 2.6.
You will need both the library and the data file to accompany the sitescripts from now on. Also, those are now expected in Prices\lib subdir, although files in Prices\ will still be found to ease the updating process.
LHpi.tcgplayerPriceGuide-v2.7.1.1.lua is functionally no different to LHpi.tcgplayerPriceGuide-v2.6.1, this is just a proof of concept for the new library version. Other sitescripts will be updated to 2.7 once there are any changes to warrant a new version.
See the wiki for further details.
EDIT: slight problem with option defaults and logging. downloads will be back in a few moments.
EDIT2:I think I have it working again.
2.7 is stable enough and sufficiently advanced over 2.6 for a new release. I hope this version addresses all issues that anthonybe had with 2.6.
You will need both the library and the data file to accompany the sitescripts from now on. Also, those are now expected in Prices\lib subdir, although files in Prices\ will still be found to ease the updating process.
LHpi.tcgplayerPriceGuide-v2.7.1.1.lua is functionally no different to LHpi.tcgplayerPriceGuide-v2.6.1, this is just a proof of concept for the new library version. Other sitescripts will be updated to 2.7 once there are any changes to warrant a new version.
See the wiki for further details.
EDIT: slight problem with option defaults and logging. downloads will be back in a few moments.
EDIT2:I think I have it working again.
- Attachments
-
- LHpi.tcgplayerPriceGuide-v2.7.1.1.lua.zip
- (13.18 KiB) Downloaded 690 times
-
- LHpi.sitescriptTemplate-v2.7.1.1.lua.zip
- (8.46 KiB) Downloaded 778 times
-
- LHpi-v2.7.lua.zip
- (15.3 KiB) Downloaded 700 times
-
- LHpi.Data-v1.lua.zip
- (20.98 KiB) Downloaded 687 times
-
- dummyMA-v0.2.zip
- (6.27 KiB) Downloaded 655 times
- LordHelmchen
- Posts: 125
- Joined: 21 Aug 2012, 16:06
- Has thanked: 21 times
- Been thanked: 32 times
2.8 sitescripts
by LordHelmchen » 25 Feb 2014, 22:34
As usual, see the LHpi wiki page for instructions and changelog.
- Attachments
-
- LHpi.mtgmintcard-v2.8.1.1.lua.zip
- (10.28 KiB) Downloaded 497 times
-
- LHpi.trader-onlineDE-v2.8.1.1.lua.zip
- (12.52 KiB) Downloaded 522 times
-
- LHpi.sitescriptTemplate-v2.8.1.1.lua.zip
- (8.84 KiB) Downloaded 694 times
-
- LHpi.magicuniverseDE-v2.8.1.1.lua.zip
- (12.1 KiB) Downloaded 499 times
-
- LHpi.tcgplayerPriceGuide-v2.8.1.1.lua.zip
- (13.5 KiB) Downloaded 488 times
Last edited by LordHelmchen on 26 Feb 2014, 00:56, edited 3 times in total.
- LordHelmchen
- Posts: 125
- Joined: 21 Aug 2012, 16:06
- Has thanked: 21 times
- Been thanked: 32 times
2.8 library
by LordHelmchen » 25 Feb 2014, 22:35
and the library (5 attachments per post limit)
- Attachments
-
- LHpi-v2.8.lua.zip
- (15.69 KiB) Downloaded 521 times
- LordHelmchen
- Posts: 125
- Joined: 21 Aug 2012, 16:06
- Has thanked: 21 times
- Been thanked: 32 times
Re: LordHelmchen's price import (lua) (updated 2014-02-25)
by Bloodnut » 01 Mar 2014, 19:11
I'm getting an error trying running the script for TCGplayer:
"lua runtime error: (load) LHpi library:549: attempt to call field 'length' (a nil value)"
I set up the script just as advised in the wiki with latest versions of the files. I have no other scripts in the Prices folder too.
I'm trying to pull all the sets in English. here's the log:
LHpi library loaded and executed successfully.
LHpi lib is ready to use.
Importing low prices. Columns available are { [1]="high";[2]="medium";[3]="low";}
LHpi.Data Prices\lib\LHpi.Data-v1.lua loaded and ready for execution.
LHpi.Data is ready to use.
Importing Sets: { [110]="Unlimited";[250]="5th Edition";[440]="Beatdown";[600]="Unhinged";[788]="Magic 2013";[779]="Magic 2012";[770]="Magic 2011";[759]="Magic 2010";[720]="Tenth Edition";[630]="9th Edition";[550]="8th Edition";[360]="6th Edition";[180]="4th Edition";[140]="Revised Edition";[100]="Beta";[90]="Alpha";[802]="Born of the Gods";[800]="Theros";[795]="Dragon’s Maze";[793]="Gatecrash";[791]="Return to Ravnica";[786]="Avacyn Restored";[784]="Dark Ascension";[782]="Innistrad";[776]="New Phyrexia";[775]="Mirrodin Besieged";[773]="Scars of Mirrodin";[767]="Rise of the Eldrazi";[765]="Worldwake";[762]="Zendikar";[758]="Alara Reborn";[756]="Conflux";[754]="Shards of Alara";[752]="Eventide";[200]="Chronicles";[310]="Portal Second Age";[530]="Legions";[751]="Shadowmoor";[750]="Morningtide";[730]="Lorwyn";[710]="Future Sight";[700]="Planar Chaos";[270]="Weatherlight";[540]="Scourge";[690]="Time Spiral Timeshifted";[680]="Time Spiral";[670]="Coldsnap";[660]="Dissension";[160]="The Dark";[320]="Unglued";[757]="Duel Decks: Divine vs. Demonic";[650]="Guildpact";[640]="Ravnica: City of Guilds";[620]="Saviors of Kamigawa";[610]="Betrayers of Kamigawa";[230]="Mirage";[460]="7th Edition";[794]="Duel Decks: Sorin vs. Tibalt";[590]="Champions of Kamigawa";[580]="Fifth Dawn";[520]="Onslaught";[560]="Mirrodin";[210]="Homelands";[330]="Urza’s Saga";[570]="Darksteel";[510]="Judgment";[500]="Torment";[480]="Odyssey";[470]="Apocalypse";[190]="Ice Age";[380]="Portal Three Kingdoms";[796]="Modern Masters";[450]="Planeshift";[400]="Mercadian Masques";[420]="Prophecy";[410]="Nemesis";[170]="Fallen Empires";[430]="Invasion";[797]="Magic 2014";[370]="Urza’s Destiny";[350]="Urza’s Legacy";[799]="Duel Decks: Heroes vs. Monsters";[120]="Arabian Nights";[240]="Visions";[300]="Exodus";[798]="From the Vault: Twenty";[290]="Stronghold";[150]="Legends";[280]="Tempest";[130]="Antiquities";[260]="Portal";[772]="Duel Decks: Elspeth vs. Tezzeret";[220]="Alliances";[785]="Duel Decks: Venser vs. Koth";}
Importing Languages: { [1]="English";}
"lua runtime error: (load) LHpi library:549: attempt to call field 'length' (a nil value)"
I set up the script just as advised in the wiki with latest versions of the files. I have no other scripts in the Prices folder too.
I'm trying to pull all the sets in English. here's the log:
LHpi library loaded and executed successfully.
LHpi lib is ready to use.
Importing low prices. Columns available are { [1]="high";[2]="medium";[3]="low";}
LHpi.Data Prices\lib\LHpi.Data-v1.lua loaded and ready for execution.
LHpi.Data is ready to use.
Importing Sets: { [110]="Unlimited";[250]="5th Edition";[440]="Beatdown";[600]="Unhinged";[788]="Magic 2013";[779]="Magic 2012";[770]="Magic 2011";[759]="Magic 2010";[720]="Tenth Edition";[630]="9th Edition";[550]="8th Edition";[360]="6th Edition";[180]="4th Edition";[140]="Revised Edition";[100]="Beta";[90]="Alpha";[802]="Born of the Gods";[800]="Theros";[795]="Dragon’s Maze";[793]="Gatecrash";[791]="Return to Ravnica";[786]="Avacyn Restored";[784]="Dark Ascension";[782]="Innistrad";[776]="New Phyrexia";[775]="Mirrodin Besieged";[773]="Scars of Mirrodin";[767]="Rise of the Eldrazi";[765]="Worldwake";[762]="Zendikar";[758]="Alara Reborn";[756]="Conflux";[754]="Shards of Alara";[752]="Eventide";[200]="Chronicles";[310]="Portal Second Age";[530]="Legions";[751]="Shadowmoor";[750]="Morningtide";[730]="Lorwyn";[710]="Future Sight";[700]="Planar Chaos";[270]="Weatherlight";[540]="Scourge";[690]="Time Spiral Timeshifted";[680]="Time Spiral";[670]="Coldsnap";[660]="Dissension";[160]="The Dark";[320]="Unglued";[757]="Duel Decks: Divine vs. Demonic";[650]="Guildpact";[640]="Ravnica: City of Guilds";[620]="Saviors of Kamigawa";[610]="Betrayers of Kamigawa";[230]="Mirage";[460]="7th Edition";[794]="Duel Decks: Sorin vs. Tibalt";[590]="Champions of Kamigawa";[580]="Fifth Dawn";[520]="Onslaught";[560]="Mirrodin";[210]="Homelands";[330]="Urza’s Saga";[570]="Darksteel";[510]="Judgment";[500]="Torment";[480]="Odyssey";[470]="Apocalypse";[190]="Ice Age";[380]="Portal Three Kingdoms";[796]="Modern Masters";[450]="Planeshift";[400]="Mercadian Masques";[420]="Prophecy";[410]="Nemesis";[170]="Fallen Empires";[430]="Invasion";[797]="Magic 2014";[370]="Urza’s Destiny";[350]="Urza’s Legacy";[799]="Duel Decks: Heroes vs. Monsters";[120]="Arabian Nights";[240]="Visions";[300]="Exodus";[798]="From the Vault: Twenty";[290]="Stronghold";[150]="Legends";[280]="Tempest";[130]="Antiquities";[260]="Portal";[772]="Duel Decks: Elspeth vs. Tezzeret";[220]="Alliances";[785]="Duel Decks: Venser vs. Koth";}
Importing Languages: { [1]="English";}
Re: LordHelmchen's price import (lua) (updated 2014-02-25)
by LordHelmchen » 02 Mar 2014, 03:10
right. error is a simple typo in the library.
Search line 549 in LHpi-v2.8.lua for "LHpi.length" and change it to "LHpi.Length" (capital "L"). I will upload a fix once I am awake enough to decide if it's more appropriate to just change the 2.8 library file and post a note to redownload, or if a bugfix, however minor, warrants a new file version. Probably the former, considering all sitescripts would need a new version to call the new library version.
EDIT: ok, why wait, can sleep later
Please fix it yourself or redownload. wiki downlaod links will be updated in a minute (which means, they probably are already when you're reading this). I think no further documentation is needed for this.
Search line 549 in LHpi-v2.8.lua for "LHpi.length" and change it to "LHpi.Length" (capital "L"). I will upload a fix once I am awake enough to decide if it's more appropriate to just change the 2.8 library file and post a note to redownload, or if a bugfix, however minor, warrants a new file version. Probably the former, considering all sitescripts would need a new version to call the new library version.
EDIT: ok, why wait, can sleep later
Please fix it yourself or redownload. wiki downlaod links will be updated in a minute (which means, they probably are already when you're reading this). I think no further documentation is needed for this.
- Attachments
-
- LHpi-v2.8.lua.zip
- fixed file, same version.
- (15.69 KiB) Downloaded 500 times
- LordHelmchen
- Posts: 125
- Joined: 21 Aug 2012, 16:06
- Has thanked: 21 times
- Been thanked: 32 times
Re: LordHelmchen's price import (lua) (updated 2014-02-25)
by Bloodnut » 02 Mar 2014, 08:56
Thanks for the fix! unfortunately I'm getting some more issues:LordHelmchen wrote:right. error is a simple typo in the library.
Search line 549 in LHpi-v2.8.lua for "LHpi.length" and change it to "LHpi.Length" (capital "L"). I will upload a fix once I am awake enough to decide if it's more appropriate to just change the 2.8 library file and post a note to redownload, or if a bugfix, however minor, warrants a new file version. Probably the former, considering all sitescripts would need a new version to call the new library version.
EDIT: ok, why wait, can sleep later
Please fix it yourself or redownload. wiki downlaod links will be updated in a minute (which means, they probably are already when you're reading this). I think no further documentation is needed for this.
first of all you have some more typos in lines 550,554,557 where you call "site.fruc" instead of "site.frucs" but I fixed that. you might want to fix it though.
However, while the script executes it doesn't pull the prices and the csv remains emtpy. I think TCGplayer might have changed their API to not serve the prices publicly but I'm not sure about that though, it's the impression I got from http://store.tcgplayer.com/faq.aspx
I've attached the log for reference.
- Attachments
-
- LHpi.tcgplayerPriceGuide-v2.8.1.1.log
- (86.38 KiB) Downloaded 469 times
Re: LordHelmchen's price import (lua) (updated 2014-02-25)
by LordHelmchen » 02 Mar 2014, 13:50
indeed. First and last time I upload without explicitely running a last testBloodnut wrote:Thanks for the fix! unfortunately I'm getting some more issues:LordHelmchen wrote:right. error is a simple typo in the library.
Search line 549 in LHpi-v2.8.lua for "LHpi.length" and change it to "LHpi.Length" (capital "L").
...
Please fix it yourself or redownload. wiki downlaod links will be updated in a minute (which means, they probably are already when you're reading this). I think no further documentation is needed for this.
first of all you have some more typos in lines 550,554,557 where you call "site.fruc" instead of "site.frucs" but I fixed that. you might want to fix it though.
Thanks for the log. I forgot to comment out a few of the options before zip'ing it up for release. In this case, the culprit was having "OFFLINE" enabled but no local copy of the website's pages available.However, while the script executes it doesn't pull the prices and the csv remains emtpy. I think TCGplayer might have changed their API to not serve the prices publicly but I'm not sure about that though, it's the impression I got from http://store.tcgplayer.com/faq.aspx
I've attached the log for reference.
again, either fix it yourself (just add "--" before all options that you're not sure you want to be changed from the default behaviour, starting with "VERBOSE" and up to and including "SAVETABLE") or redownload. wiki links will be changed in a few minutes (and marked as reupload).
EDIT: wiki updated with download links,date ("hotfix" suffixed) and changelog
- Attachments
-
- LHpi.magicuniverseDE-v2.8.1.1.lua.zip
- (12.1 KiB) Downloaded 646 times
-
- LHpi.mtgmintcard-v2.8.1.1.lua.zip
- (10.28 KiB) Downloaded 667 times
-
- LHpi.tcgplayerPriceGuide-v2.8.1.1.lua.zip
- (13.49 KiB) Downloaded 631 times
-
- LHpi.trader-onlineDE-v2.8.1.1.lua.zip
- (12.52 KiB) Downloaded 646 times
-
- LHpi-v2.8.lua.zip
- (15.7 KiB) Downloaded 706 times
- LordHelmchen
- Posts: 125
- Joined: 21 Aug 2012, 16:06
- Has thanked: 21 times
- Been thanked: 32 times
Re: LordHelmchen's price import (lua) (updated 2014-02-25)
by Bloodnut » 02 Mar 2014, 14:49
I'm kinda feeling bad now about reporting this but... "Lua runtime error: (load) LHpi library:1129: attempt to index field 'foilprice' (a nil value)".LordHelmchen wrote:indeed. First and last time I upload without explicitely running a last test
This time I don't know what's wrong with it.
this log is when trying to fetch Theros only but the results are the same when all sets are selected.
- Attachments
-
- LHpi.tcgplayerPriceGuide-v2.8.1.1.log
- (483 Bytes) Downloaded 430 times
Re: LordHelmchen's price import (lua) (updated 2014-03-02)
by LordHelmchen » 02 Mar 2014, 15:13
apparently not my day toaday...
As long as you keep finding bugs on the weekend, we'll get along just fine
EDIT1: interesting find, actually. That bug must have been lurking there for a long time alrady. You probably are the first one to import only nonfoil prices since I implemented conflict handling by (shoddy) averaging.
As a workaround, please do import foil and nonfoil prices. You won't find that many foilprices on tcgplayerPriceguide, anyways It's probably not that difficult to fix, but in light of your last three posts, I think I should take the time to make sure I don't break anything else in the process.
As long as you keep finding bugs on the weekend, we'll get along just fine
EDIT1: interesting find, actually. That bug must have been lurking there for a long time alrady. You probably are the first one to import only nonfoil prices since I implemented conflict handling by (shoddy) averaging.
As a workaround, please do import foil and nonfoil prices. You won't find that many foilprices on tcgplayerPriceguide, anyways It's probably not that difficult to fix, but in light of your last three posts, I think I should take the time to make sure I don't break anything else in the process.
- LordHelmchen
- Posts: 125
- Joined: 21 Aug 2012, 16:06
- Has thanked: 21 times
- Been thanked: 32 times
Re: LordHelmchen's price import (lua) (updated 2014-03-02)
by Bloodnut » 02 Mar 2014, 15:52
It works! well doneLordHelmchen wrote:apparently not my day toaday...
As long as you keep finding bugs on the weekend, we'll get along just fine
EDIT1: interesting find, actually. That bug must have been lurking there for a long time alrady. You probably are the first one to import only nonfoil prices since I implemented conflict handling by (shoddy) averaging.
As a workaround, please do import foil and nonfoil prices. You won't find that many foilprices on tcgplayerPriceguide, anyways It's probably not that difficult to fix, but in light of your last three posts, I think I should take the time to make sure I don't break anything else in the process.
Oh and there are actually lots of foils in TCGplayer but the script didn't import even a single one. almost every card has a foil price. if you look a card up, on the left pane you can check the "foil" option and you will be presented with foil prices of everything (e.g. "near mint foil", "lightly played foil"...)
Re: LordHelmchen's price import (lua) (updated 2014-03-02)
by LordHelmchen » 02 Mar 2014, 19:14
That's a relief. I admit I only ever tested importing both foil and nonfoil, never seperately. I probably will fall back into that habit once I got this bug sorted out, too.Bloodnut wrote:It works! well doneLordHelmchen wrote:As a workaround, please do import foil and nonfoil prices.
...
We might have a misunderstanding here. The script queries http://magic.tcgplayer.com/magic_price_guides.asp, not the store. No foilage information is given on that page(s), thus the only foil prices that are imported are cards from foilonly sets or cards that are explicitely foiltweaked (see LHpi.Data for details.). At least, it should; if not, that's another bug...Oh and there are actually lots of foils in TCGplayer but the script didn't import even a single one. almost every card has a foil price. if you look a card up, on the left pane you can check the "foil" option and you will be presented with foil prices of everything (e.g. "near mint foil", "lightly played foil"...)
Parsing the shop with its multiple pages per set, multiple shop offers per card and dependence on scripting seems like a lot of work. It would probably be easier to write seperate sitescripts for those of the linked shops that you actually are interested in, and use MA's dynamic averaging instead of LHpi's static (and still wrong, it's on my list) one.
If you want to try either, see the wiki and feel free to suggest (or even sumit patches for) any features and/or improvements the library needs to have.
- LordHelmchen
- Posts: 125
- Joined: 21 Aug 2012, 16:06
- Has thanked: 21 times
- Been thanked: 32 times
Re: LordHelmchen's price import (lua) (updated 2014-03-02)
by Bloodnut » 02 Mar 2014, 20:16
oh, yup you're right, disregard my rambles. and no, I'm not going to query the store per card and parse it. a minor change in their layout and it's all useless. we had those kinds of scripts before IIRC.LordHelmchen wrote:We might have a misunderstanding here. The script queries http://magic.tcgplayer.com/magic_price_guides.asp, not the store. No foilage information is given on that page(s), thus the only foil prices that are imported are cards from foilonly sets or cards that are explicitely foiltweaked (see LHpi.Data for details.). At least, it should; if not, that's another bug...
Parsing the shop with its multiple pages per set, multiple shop offers per card and dependence on scripting seems like a lot of work. It would probably be easier to write seperate sitescripts for those of the linked shops that you actually are interested in, and use MA's dynamic averaging instead of LHpi's static (and still wrong, it's on my list) one.
If you want to try either, see the wiki and feel free to suggest (or even sumit patches for) any features and/or improvements the library needs to have.
Re: LordHelmchen's price import (lua) (updated 2014-03-02)
by LordHelmchen » 04 Mar 2014, 22:36
The bug should be fixed in next version. There's some more small fixes and improvements I want to get done before the next release, though.
On a sidenote: I'm migrating from flat files in MA\Prices to a local git repository, and still need to adapt my debugging dummyMA to the newlocation. lua allows for some sick hacks: the dummy has code that loads and on-the-fly-patches code (sitescript) to load and on-the-fly-patch still other code (library) to change how yet another piece of code (data) is loaded. and now that needs to be adapted to work with different paths... I think I have reached six escaping backslashes for each "\" in the path by now
Also, I'm thinking about leaving the version numbers out of the sitescripts' filenames (library and data still will have them, so the sitescripts can load the version they need). If there are no objections, that will probably happen for 2.9 or 2.10 .
On a sidenote: I'm migrating from flat files in MA\Prices to a local git repository, and still need to adapt my debugging dummyMA to the newlocation. lua allows for some sick hacks: the dummy has code that loads and on-the-fly-patches code (sitescript) to load and on-the-fly-patch still other code (library) to change how yet another piece of code (data) is loaded. and now that needs to be adapted to work with different paths... I think I have reached six escaping backslashes for each "\" in the path by now
Also, I'm thinking about leaving the version numbers out of the sitescripts' filenames (library and data still will have them, so the sitescripts can load the version they need). If there are no objections, that will probably happen for 2.9 or 2.10 .
- LordHelmchen
- Posts: 125
- Joined: 21 Aug 2012, 16:06
- Has thanked: 21 times
- Been thanked: 32 times
2.9.2 release
by LordHelmchen » 06 Mar 2014, 01:34
This should stabilize development for a while, unless Bloodnut finds another bug
- Attachments
-
- LHpi-v2.9.lua.zip
- library core
- (16.64 KiB) Downloaded 645 times
-
- LHpi.Data-v2.lua.zip
- libary data file
- (21.55 KiB) Downloaded 630 times
-
- LHpi.sitescriptTemplate-v2.9.2.8.lua.zip
- for dev use
- (9.2 KiB) Downloaded 653 times
-
- dummyMA.zip
- for dev use
- (7 KiB) Downloaded 616 times
- LordHelmchen
- Posts: 125
- Joined: 21 Aug 2012, 16:06
- Has thanked: 21 times
- Been thanked: 32 times
61 posts
• Page 2 of 5 • 1, 2, 3, 4, 5
Who is online
Users browsing this forum: No registered users and 4 guests