LordHelmchen's price import - BFZ and mkm

[LordHelmchen]

This is mostly for anthonybe's sake. No sitescript uses this version yet, so if you're only looking for working scripts, don't donwload this yet. See the wiki for further details.

[LordHelmchen]

Goblin Hero: Still looking for answers to two questions. Find them two posts above this or at "Problems" in the wiki. Thanks.

2.7 is stable enough and sufficiently advanced over 2.6 for a new release. I hope this version addresses all issues that anthonybe had with 2.6.
You will need both the library and the data file to accompany the sitescripts from now on. Also, those are now expected in Prices\lib subdir, although files in Prices\ will still be found to ease the updating process.

LHpi.tcgplayerPriceGuide-v2.7.1.1.lua is functionally no different to LHpi.tcgplayerPriceGuide-v2.6.1, this is just a proof of concept for the new library version. Other sitescripts will be updated to 2.7 once there are any changes to warrant a new version.

See the wiki for further details.

EDIT: slight problem with option defaults and logging. downloads will be back in a few moments.
EDIT2:I think I have it working again.

[LordHelmchen]

As usual, see the LHpi wiki page for instructions and changelog.

[LordHelmchen]

and the library (5 attachments per post limit)

[Bloodnut]

I'm getting an error trying running the script for TCGplayer:
"lua runtime error: (load) LHpi library:549: attempt to call field 'length' (a nil value)"

I set up the script just as advised in the wiki with latest versions of the files. I have no other scripts in the Prices folder too.
I'm trying to pull all the sets in English. here's the log:


LHpi library loaded and executed successfully.

LHpi lib is ready to use.
Importing low prices. Columns available are { [1]="high";[2]="medium";[3]="low";}
LHpi.Data Prices\lib\LHpi.Data-v1.lua loaded and ready for execution.
LHpi.Data is ready to use.
Importing Sets: { [110]="Unlimited";[250]="5th Edition";[440]="Beatdown";[600]="Unhinged";[788]="Magic 2013";[779]="Magic 2012";[770]="Magic 2011";[759]="Magic 2010";[720]="Tenth Edition";[630]="9th Edition";[550]="8th Edition";[360]="6th Edition";[180]="4th Edition";[140]="Revised Edition";[100]="Beta";[90]="Alpha";[802]="Born of the Gods";[800]="Theros";[795]="Dragon’s Maze";[793]="Gatecrash";[791]="Return to Ravnica";[786]="Avacyn Restored";[784]="Dark Ascension";[782]="Innistrad";[776]="New Phyrexia";[775]="Mirrodin Besieged";[773]="Scars of Mirrodin";[767]="Rise of the Eldrazi";[765]="Worldwake";[762]="Zendikar";[758]="Alara Reborn";[756]="Conflux";[754]="Shards of Alara";[752]="Eventide";[200]="Chronicles";[310]="Portal Second Age";[530]="Legions";[751]="Shadowmoor";[750]="Morningtide";[730]="Lorwyn";[710]="Future Sight";[700]="Planar Chaos";[270]="Weatherlight";[540]="Scourge";[690]="Time Spiral Timeshifted";[680]="Time Spiral";[670]="Coldsnap";[660]="Dissension";[160]="The Dark";[320]="Unglued";[757]="Duel Decks: Divine vs. Demonic";[650]="Guildpact";[640]="Ravnica: City of Guilds";[620]="Saviors of Kamigawa";[610]="Betrayers of Kamigawa";[230]="Mirage";[460]="7th Edition";[794]="Duel Decks: Sorin vs. Tibalt";[590]="Champions of Kamigawa";[580]="Fifth Dawn";[520]="Onslaught";[560]="Mirrodin";[210]="Homelands";[330]="Urza’s Saga";[570]="Darksteel";[510]="Judgment";[500]="Torment";[480]="Odyssey";[470]="Apocalypse";[190]="Ice Age";[380]="Portal Three Kingdoms";[796]="Modern Masters";[450]="Planeshift";[400]="Mercadian Masques";[420]="Prophecy";[410]="Nemesis";[170]="Fallen Empires";[430]="Invasion";[797]="Magic 2014";[370]="Urza’s Destiny";[350]="Urza’s Legacy";[799]="Duel Decks: Heroes vs. Monsters";[120]="Arabian Nights";[240]="Visions";[300]="Exodus";[798]="From the Vault: Twenty";[290]="Stronghold";[150]="Legends";[280]="Tempest";[130]="Antiquities";[260]="Portal";[772]="Duel Decks: Elspeth vs. Tezzeret";[220]="Alliances";[785]="Duel Decks: Venser vs. Koth";}
Importing Languages: { [1]="English";}

[LordHelmchen]

right. error is a simple typo in the library.
Search line 549 in LHpi-v2.8.lua for "LHpi.length" and change it to "LHpi.Length" (capital "L"). I will upload a fix once I am awake enough to decide if it's more appropriate to just change the 2.8 library file and post a note to redownload, or if a bugfix, however minor, warrants a new file version. Probably the former, considering all sitescripts would need a new version to call the new library version.

EDIT: ok, why wait, can sleep later
Please fix it yourself or redownload. wiki downlaod links will be updated in a minute (which means, they probably are already when you're reading this). I think no further documentation is needed for this.

[Bloodnut]

right. error is a simple typo in the library.
Search line 549 in LHpi-v2.8.lua for "LHpi.length" and change it to "LHpi.Length" (capital "L"). I will upload a fix once I am awake enough to decide if it's more appropriate to just change the 2.8 library file and post a note to redownload, or if a bugfix, however minor, warrants a new file version. Probably the former, considering all sitescripts would need a new version to call the new library version.

EDIT: ok, why wait, can sleep later
Please fix it yourself or redownload. wiki downlaod links will be updated in a minute (which means, they probably are already when you're reading this). I think no further documentation is needed for this.

Thanks for the fix! unfortunately I'm getting some more issues:
first of all you have some more typos in lines 550,554,557 where you call "site.fruc" instead of "site.frucs" but I fixed that. you might want to fix it though.
However, while the script executes it doesn't pull the prices and the csv remains emtpy. I think TCGplayer might have changed their API to not serve the prices publicly but I'm not sure about that though, it's the impression I got from http://store.tcgplayer.com/faq.aspx
I've attached the log for reference.

[LordHelmchen]

right. error is a simple typo in the library.
Search line 549 in LHpi-v2.8.lua for "LHpi.length" and change it to "LHpi.Length" (capital "L").
...
Please fix it yourself or redownload. wiki downlaod links will be updated in a minute (which means, they probably are already when you're reading this). I think no further documentation is needed for this.

Thanks for the fix! unfortunately I'm getting some more issues:
first of all you have some more typos in lines 550,554,557 where you call "site.fruc" instead of "site.frucs" but I fixed that. you might want to fix it though.

indeed. First and last time I upload without explicitely running a last test

However, while the script executes it doesn't pull the prices and the csv remains emtpy. I think TCGplayer might have changed their API to not serve the prices publicly but I'm not sure about that though, it's the impression I got from http://store.tcgplayer.com/faq.aspx
I've attached the log for reference.

Thanks for the log. I forgot to comment out a few of the options before zip'ing it up for release. In this case, the culprit was having "OFFLINE" enabled but no local copy of the website's pages available.

again, either fix it yourself (just add "--" before all options that you're not sure you want to be changed from the default behaviour, starting with "VERBOSE" and up to and including "SAVETABLE") or redownload. wiki links will be changed in a few minutes (and marked as reupload).

EDIT: wiki updated with download links,date ("hotfix" suffixed) and changelog

[Bloodnut]

indeed. First and last time I upload without explicitely running a last test

I'm kinda feeling bad now about reporting this but... "Lua runtime error: (load) LHpi library:1129: attempt to index field 'foilprice' (a nil value)".
This time I don't know what's wrong with it.
this log is when trying to fetch Theros only but the results are the same when all sets are selected.

[LordHelmchen]

apparently not my day toaday...
As long as you keep finding bugs on the weekend, we'll get along just fine

EDIT1: interesting find, actually. That bug must have been lurking there for a long time alrady. You probably are the first one to import only nonfoil prices since I implemented conflict handling by (shoddy) averaging.

As a workaround, please do import foil and nonfoil prices. You won't find that many foilprices on tcgplayerPriceguide, anyways It's probably not that difficult to fix, but in light of your last three posts, I think I should take the time to make sure I don't break anything else in the process.

[Bloodnut]

apparently not my day toaday...
As long as you keep finding bugs on the weekend, we'll get along just fine

EDIT1: interesting find, actually. That bug must have been lurking there for a long time alrady. You probably are the first one to import only nonfoil prices since I implemented conflict handling by (shoddy) averaging.

As a workaround, please do import foil and nonfoil prices. You won't find that many foilprices on tcgplayerPriceguide, anyways It's probably not that difficult to fix, but in light of your last three posts, I think I should take the time to make sure I don't break anything else in the process.

It works! well done
Oh and there are actually lots of foils in TCGplayer but the script didn't import even a single one. almost every card has a foil price. if you look a card up, on the left pane you can check the "foil" option and you will be presented with foil prices of everything (e.g. "near mint foil", "lightly played foil"...)

[LordHelmchen]

As a workaround, please do import foil and nonfoil prices.
...

It works! well done

That's a relief. I admit I only ever tested importing both foil and nonfoil, never seperately. I probably will fall back into that habit once I got this bug sorted out, too.

Oh and there are actually lots of foils in TCGplayer but the script didn't import even a single one. almost every card has a foil price. if you look a card up, on the left pane you can check the "foil" option and you will be presented with foil prices of everything (e.g. "near mint foil", "lightly played foil"...)

We might have a misunderstanding here. The script queries http://magic.tcgplayer.com/magic_price_guides.asp, not the store. No foilage information is given on that page(s), thus the only foil prices that are imported are cards from foilonly sets or cards that are explicitely foiltweaked (see LHpi.Data for details.). At least, it should; if not, that's another bug...

Parsing the shop with its multiple pages per set, multiple shop offers per card and dependence on scripting seems like a lot of work. It would probably be easier to write seperate sitescripts for those of the linked shops that you actually are interested in, and use MA's dynamic averaging instead of LHpi's static (and still wrong, it's on my list) one.

If you want to try either, see the wiki and feel free to suggest (or even sumit patches for) any features and/or improvements the library needs to have.

[Bloodnut]

We might have a misunderstanding here. The script queries http://magic.tcgplayer.com/magic_price_guides.asp, not the store. No foilage information is given on that page(s), thus the only foil prices that are imported are cards from foilonly sets or cards that are explicitely foiltweaked (see LHpi.Data for details.). At least, it should; if not, that's another bug...

Parsing the shop with its multiple pages per set, multiple shop offers per card and dependence on scripting seems like a lot of work. It would probably be easier to write seperate sitescripts for those of the linked shops that you actually are interested in, and use MA's dynamic averaging instead of LHpi's static (and still wrong, it's on my list) one.

If you want to try either, see the wiki and feel free to suggest (or even sumit patches for) any features and/or improvements the library needs to have.

oh, yup you're right, disregard my rambles. and no, I'm not going to query the store per card and parse it. a minor change in their layout and it's all useless. we had those kinds of scripts before IIRC.

[LordHelmchen]

The bug should be fixed in next version. There's some more small fixes and improvements I want to get done before the next release, though.

On a sidenote: I'm migrating from flat files in MA\Prices to a local git repository, and still need to adapt my debugging dummyMA to the newlocation. lua allows for some sick hacks: the dummy has code that loads and on-the-fly-patches code (sitescript) to load and on-the-fly-patch still other code (library) to change how yet another piece of code (data) is loaded. and now that needs to be adapted to work with different paths... I think I have reached six escaping backslashes for each "\" in the path by now

Also, I'm thinking about leaving the version numbers out of the sitescripts' filenames (library and data still will have them, so the sitescripts can load the version they need). If there are no objections, that will probably happen for 2.9 or 2.10 .

[LordHelmchen]

This should stabilize development for a while, unless Bloodnut finds another bug