LordHelmchen's price import - BFZ and mkm

[LordHelmchen]

sitescripts
see the wikifor notes about filenamesand versions.

[LordHelmchen]

I'll stop now
More or less only a developer convenience release, 2.9 is still fine.
This implements anthonybe's feature request. Apart from LHpi.mtgmintprice.lua with STRICTEXPECTED=true, no sitescript is affected. If you want, you can just change

Code: Select all

libver="2.9"

to

Code: Select all

libver="2.10"

in the other scripts to update them, or simply keep both library versions for now.

[Bloodnut]

This should stabilize development for a while, unless Bloodnut finds another bug

I have a feeling you have a dartboard at home with "Bloodnut" placed in the center

[LordHelmchen]

Here's the sitescripts, with Journey into Nyx. Also, tcgplayer-PriceGuide have improved their variant marking, so I could make more educated guesses there. Sadly, this meant a small fix was needed in the library to match their variant suffix, so my grand plan of only needing to update the Data file for this set's release failed All other scripts have been bumped to the latest library version to have them all at the same version again.

As usual, find documentation in the wiki, and post comments here.

I even tested them a final time just before attaching Time to set Bloodnut loose on them. Whose reports were much appreciated and who's not at all involvend in any flinging of sharp objects from me

edit: Whose vs who's

[LordHelmchen]

And once again I need more than five attachments.

[Bloodnut]

All scripts seems to work well... surprisingly!

EDIT: well, if you don't count that bug where you only download either foil-only or non-foil only but it's not "mission critical".

[LordHelmchen]

All scripts seems to work well... surprisingly!

EDIT: well, if you don't count that bug where you only download either foil-only or non-foil only but it's not "mission critical".

If you are referring to the one you reported [2nd March, that should have been fixed in libver 2.9:

fixed and improved MergeCardrows (was broken if importfoil=="N" or "O" - reported by Bloodnut)

In all other cases: Would you care to elaborate? I don't see anything unusual in either of the three modes:

Code: Select all

magicuniverseDE.lua:
regular only: 17341 set,  11 failed English cards   15886 set,  16 failed German cards   19 set, 0 failed Italian cards   ; 1116 dropped, 140 namereplaced and 0 foiltweaked.
foil only   :  8010 set, 407 failed English cards    8001 set, 416 failed German cards    0 set, 0 failed Italian cards   ;    0 dropped,  35 namereplaced and 0 foiltweaked.
both        : 17342 set, 418 failed English cards   15887 set, 423 failed German cards   19 set, 0 failed Italian cards   ; 1116 dropped, 175 namereplaced and 0 foiltweaked.
expected    : 17342 set, 418 failed English cards   15887 set, 423 failed German cards   19 set, 0 failed Italian cards   ; 1116 dropped, 175 namereplaced and 0 foiltweaked.

mtgmintcard.lua:
regular only: 19774 set, 2 failed English cards   6684 set, 186 failed Simplified Chinese cards   ; 847 dropped, 86 namereplaced and 0 foiltweaked.
foil only   : 10390 set, 0 failed English cards   5750 set,   0 failed Simplified Chinese cards   ;   1 dropped, 13 namereplaced and 2 foiltweaked.
both        : 19851 set, 2 failed English cards   6697 set, 186 failed Simplified Chinese cards   ; 846 dropped, 99 namereplaced and 4 foiltweaked.
expected    : 19851 set, 2 failed English cards   6697 set, 186 failed Simplified Chinese cards   ; 846 dropped, 99 namereplaced and 4 foiltweaked.

tcgplayerPriceGuide.lua: *(see below)
regular only: 20351 set, 5 failed English cards   ; 165 dropped, 446 namereplaced and  2 foiltweaked.
foil only   :     0 set, 0 failed English cards   ;   0 dropped,   0 namereplaced and  0 foiltweaked.
both        : 20360 set, 5 failed English cards   ; 156 dropped, 446 namereplaced and 11 foiltweaked.
expected    : 20365 set, 5 failed English cards   ; 154 dropped, 446 namereplaced and 11 foiltweaked.

trader-onlineDE.lua:
regular only: 19788 set, 2 failed English cards   17939 set, 13 failed German cards   429 set, 0 failed Italian cards   ;  823 dropped, 235 namereplaced and 0 foiltweaked.
foil only   :  8310 set, 0 failed English cards    6488 set,  0 failed German cards     0 set, 0 failed Italian cards   ;  531 dropped,  74 namereplaced and 0 foiltweaked.
both        : 19797 set, 2 failed English cards   17940 set, 13 failed German cards   429 set, 0 failed Italian cards   ; 1345 dropped, 311 namereplaced and 9 foiltweaked.
expected    : 19797 set, 2 failed English cards   17940 set, 13 failed German cards   429 set, 0 failed Italian cards   ; 1347 dropped, 311 namereplaced and 9 foiltweaked.


*: there are, of course, some foil cards mixed into a few of the pages, but there's no way of telling from the url. The only solution for foil-only would be to fetch all pages anyway, and then import just the few cards that were parsed as foil and drop all others. the library is already equipped to disregard unwanted foilstat cards when it finishes parsing the cards's entry, so it should not take more than redefine the single fruc definition in the sitescript. Probably, this behaviour is closer to what would be expected from the Magic Album Price Manager UI. It would, however, result in a lot of pages fetched and entries parsed, only to be disregarded afterwards:

Code: Select all

Total counted : 9 set, 0 failed English cards   ; 20507 dropped, 0 namereplaced and 9 foiltweaked.

Opinions are most welcome.

[Bloodnut]

Take a look at the screen shots. this error happens on all scripts except on the tcgplayer script.

notice in the screenshots that when I select to download 'regular' the error says 'foilprice' (line 1022) but when I select 'foil' the error says 'regprice' (line 1021)

[LordHelmchen]

notice in the screenshots that when I select to download 'regular' the error says 'foilprice' (line 1022) but when I select 'foil' the error says 'regprice' (line 1021)

Ok, I can see what might be happenning and it's easy to construct it so that there's no reason for the script to error out here. I will change the lib to prevent this error in the future (without changing the version number).
redownload or change lines 1020 to 1022:

Code: Select all

if card.names then card.names[lid]=nil end
if card.regprice then card.regprice[lid]=nil end
if card.foilprice then card.foilprice[lid]=nil end


What does make me curious is that I explicitely did run all scripts in all three modes (as my previous post should prove), and had no error. So pardon me for asking: did you download the latest version of all files? Did you make any changes before running it? Which MA version are you at?

Ah! Strike that. You selected English only, whereas I test with all languages selected (not that there is much besides English and German to import, yet).
And that's why I ask to see a logfile if the script misbehaves. The first 9 lines would have told me the difference between my test and your usage Allthough I admit that the screenshots did show the error (that would not have showed up in the log), and you cleverly moved the error message to include your settings

for example | Open

LHpi library 2.11 loaded and executed successfully.

LHpi lib is ready to use.
LHpi.Data Prices\lib\LHpi.Data-v3.lua loaded and ready for execution.
LHpi.Data is ready to use.
Importing Sets: { [806]="Journey into Nyx";[797]="Magic 2014";[802]="Born of the Gods";[800]="Theros";[795]="Dragon’s Maze";[793]="Gatecrash";[791]="Return to Ravnica";}
Importing Languages: { [1]="English";}
Importing Foil Only Card Prices
9%: "Collecting Magic 2014 into table (id 797)"

[Bloodnut]

I completely forgot about the logs. you're right about that.
I don't bother with other languages even though I have some non-English cards, I don't use it for trading really, only to sort by price or see if some card is an anomaly (by thinking it's crap but suddenly seeing a high price for it).

the scripts work now. good job.

[LordHelmchen]

I did not plan to include it in the sitescript collection, but now that the cat is out of the bag and it has been downloaded a dual-digit number of times, I can at least upload the newer version to make it compatible with the latest lib and data...

It still needs a lot of namereplace tables. I'm not that much interested in $ prices, so if you are, consider voluteering to finish and maintain it.

[LordHelmchen]

Its time for another big update. Most prominent change is the addition of the 2015 Coreset. Also includes changes to be compatible to the new mtgmintcard site layout.
Check the wiki for full changelog.

Once again there were a few changes to be made to the library, so maybe Khans of Tharkir will finally be a data-only update

Note that the M15 Beast tokens (5 and 9) are currently missing version numbers in MA, so their price will be averaged (where available).

[LordHelmchen]

And here's the content for your Prices\lib folder.

Goblin Hero: is there a chance to grant me an exception to the 5-attachments-per-post-limit, so I won't have to double-post on new releases anymore?

[LordHelmchen]

Finally I got around to finish the new version. Besides Khans of Tarkir, this release features object type awareness, so I can finally distinguish between regular cards and oversized commanders having the same cardname and empty variant field. Of course, that means I was wring when I predicted a Data-only update
Starting with Prices\lib\ contents in this post, sitescripts in the next.

[LordHelmchen]

and here's as much sitescripts that I can fit into one posting