Difference between revisions of "LHpi"

Revision as of 21:32, 18 July 2015

LHpi is LordHelmchen's price import library and sitescripts for Magic Album.

LHpi is a collection of lua scripts to be used by Magic Album, but is not part of Magic Album and thus updated and supported seperately.

Discuss, ask for help and leave feedback at the corresponding Forum Thread.

Sitescripts

The sitescripts are the actual price import scripts, written in lua. They use the Magic Album Lua API and the LHpi library to query and parse a website and import the price information into Magic Album. Most of the program logic is contained in the library, so the sitescript needs to hold only site-specific information and a few functions that are not easily generalizable.

In theory, you should not be dependent on a new version once a new set is available: just add new entries to site.sets table in the sitescript. Being easy to update yourself was one of the design goals.

Sitescripts will need to provide

• a table of supported sets with language and url infix fields
• a table of supported languages with url infix field
• a function to build urls
• a function to parse the html raw data into data fields via regular expressions (and the regex itself)
• optionally, they can also provide
  • functions for site specific data manipulation, in case the site has some strange formatting the library does not understand
  • a one-to-one name map for names that differ between the site and Magic Album's database
  • tables to map card names to picture versions, in case the one-to-one map and LHpi's defaults are not sufficient
  • a table with expected card count numbers for LHpi's sanity checks.

magicuniverse.de

Magic Universe is a German site with German and English cards, priced in €.

This site started it all, since I wanted prices in local currency, and my FLG apparently uses them as price reference.

mtgmintcard.com

Functionally the same as "MTG Mint Card.lua" by Goblin Hero and Stromglad1.

When I wanted to generalize my script for easy addaption to other sites, this provided me with a good baseline, since I could search MA for cards that my script missed but should have set. Also, the site is easy to querry and parse. They do have a currency chooser, so I started by trying to make a mtgmintcard-Euro script. I failed, but salvaged the wreck for default $ prices.

magic.tcgplayer.com PriceGuide

Tcgplayer Price Guide claims to accumulate prices from 75 online vendors. They are priced in $. Their tables give a high, low and average price; the sitescript contains a variable to select which one of those it should use.
To change the column to import from, open the script in a text editor, find himelo = 3 and change it to himelo = 1 for high or himelo = 2 for medium prices. You can have multiple copies of the same script
As requested by Golob here.
Also, most Promo sets have not been tested yet and are commented out.
Note that cards with multiple versions are averaged in some sets and set to "variant 1"'s price in others, as the source list does not clearly identify the version. Even when it does, it is unclear if the entry without version suffix is meant to be all unspecified or version 1 only. In most cases, "CARD" price is set for all versions and averaged with "CARD (version)" price for given versions now. Lately, their version handling has somewhat improved, but there's still a lot of guesswork involved...

trader-online.de

Trader Online is a German site with German and English cards, priced in €.

I found this site while looking for Token images, found it to be parseable by sitescript, and decided to use it as a test on how well the library was adapted to support additional sites.

mtgprice.com

I did not plan to include it in the sitescript collection, but now that the cat is out of the bag and it has been downloaded a dual-digit number of times, I can at least upload the newer version to make it compatible with the latest lib and data... Contains a variable to select between "Fair Trade" and "best Buy" price column. It still needs a lot of namereplace tables. I'm not that much interested in $ prices, so if you are, consider volunteering to finish and maintain it. Post in the thread if you want to do it but are unsure what to do.

The LHpi Library

It is not expected to be fast, but as reliable as possible:
It first compares the user requested sets and languages with what the sitescript has available and builds a table of urls to querry, seperated by set.
Then it loops though the sets that are both available and selected:
The raw html data is fetched, parsed and entered into an intermediate table of all prices for the current edition, merging regular and foil prices to minimize calls to ma.SetPrice.
Should the same card (defined by set,card,language,(picture-)version ) be encountered again, it compares the prices. If they are equal, the new price is discarded; if they differ (Yes, magicuniverse.de's database is quirky), it uses the average price. Cards that are suffxed with a condition description are currently disregarded.
Once that cardsetTable is completed (all urls belonging to the set are parsed), it sends the prices to MA. Finally, it compares the count of set prices with the number of cards in the set (or, if given, the expectation given in the sitescript). See How-To_sitescriptTemplate#procedure_of_function_calls for details. Logging to a seperate file is now disabled by default again (but still configurable by the variable SAVELOG in the script). The log (\Prices\LHpi.log) is overwritten on subsequent re-runs of a script.

LHpi.Data

Since library version 2.7, site-independent set data (number of cards,foilage, default variant and foiltweak tables) were moved into a seperate data file that gets loaded by the library on initialization. This will allow to distinguish between code updates (to the library) and set data updates (which are usually needed when a new set is released). Version numbering of the sitescripts will reflect both library and data versions used.

Installation and usage

1. Download at least one sitescript and both the LHpi libray and data file in the version the sitescript uses.
   • note the change to version numbers and file names in 2.9.2 , see #version_history for details.
2. Unzip into the "Prices" subdirectory of your Magic Album installation.
3. (recommended, but not yet strictly necessary) Create a subfolder called "lib" under "Prices" and move both library and data file into the subfolder.
4. (optional) If you want to use the SAVEHTML or SAVETABLE feature, you need to create an empty subdirectory, named like the sitescript without file extension (for example "Prices\LHpi.magicuniverseDE").
5. (re)start Magic Album and use the sitescript (not the library nor the data file. Step 3 makes this warning obsolete.) from Magic Album's Price Manager dialog.

If something does not happen as expected

• make sure the script is saved utf-8 encoded. It's customary for lua scripts to be saved ANSI encoded, but as MA expects utf input, this is what I considered the most wysiwyg approach to set cards with special characters in their names.
• open the lua file in a text editor and change "--VERBOSE = true" to "VERBOSE = true", likewise enable "LOGDROPS","LOGNAMEREPLACE","LOGFOILTWEAK","STRICTCHECKEXPECTED" and "SAVELOG". This will make the script provide more feedback to help track down any problems (or just to see what it was doing) and also enable additional checks. The logging will then be sent to "[MA Folder]\Prices\[scriptname].log".

I'll try to answer any questions you might have. If the script misbehaves please follow the two steps above and attach a copy of the .log to your post.

Development

How-to write your sitescript file

Detailed instructions on the properties and functions to define in your sitescript file can be found at How-To sitescriptTemplate.

sitescriptTemplate

This is just an empty, commented template to help write new sitescripts.

dummyMA.lua

Magic Album API dummy to test and debug LHpi within an IDE (such as eclipse) and without needing Magic Album. Now fixed for backwards compatibility with Lua 5.1 and paths outside of MA's "Prices" directory.

Planned features

in no particular order

• Promo and special sets for magicuniverseDE,mtgmintcards, tcgplayerPriceguide
  • add all special and promo sets (cardcount,variants,foiltweak) to LHpi.Data
• tweak log : better differentiation of log levels (normal,verbose,debug); rewrite normal-level logentries to be more pleasing to read.
  I'd love to get users' feedback on this: how silent should normal level be; what should not be logged unless in debug mode?
• (((a+b)/2)+c)/2 != (a+b+c)/3 = (((a+b)/2*2)+c)/3
  For mathematically correct averaging, need to attach a counter to averaged prices, then on next averaging, do if counter then newaverage=(oldaverage*(counter+1) + newprice) / (counter+2)

Possible features

in no particular order

• passing importsets to BuildCardData could sort out the mess that is magicuniverse's promo page, at the cost of lots looping through sets. Should thus be done in site.BCDPlugins
• ideally, most information in LHpi.Data.sets could be read from MA via API getter functions.
• add hardcoded exchange rate for crude currency conversion between parsing and importing.
• handle conflicting prices differently. Is averaging ok, or should the script discard one (which?) of the prices and/or throw an error ? Until further notice, I'm ok with how averaging works.averaging conflicts. How it's done needs some work.
• import prices from condition-suffixed cards instead of dropping them.
  • patch to accept entries with a condition description if no other entry with better condition is in the table:
    • buildCardData will need to add condition to carddata
    • global conditions{} to define priorities
    • then fillCardsetTable needs a new check before overwriting existing data
• add another sitescript (and adapt library if needed)

add feature requests here, please.

Problems

• number of cards in set must be hardcoded into library. An [url=http://www.slightlymagic.net/forum/tracker.php?p=2&t=303 ma.GetCardsInSet( setid,langid,regular|tokens )] would help, but the numbers are easy to hardcode anyways.
• can't use mtgmintcard to get € prices. see [1] for details.

Hints for developers

Some information that might be useful if you want to write/improve sitescripts. Please add questions related to developing here, so I know what needs to be documented.

moved to How-To_sitescriptTemplate#Hints_for_developing

version history

I decided to start all versions at 2.0, since it's the continuation of magicuniverseDEv1.6.lua, but with enough changes to warrant the next major version number. Do not expect revision numbers to stay synched. Starting with 2.2.1, sitescript minor version will conform to used library version, and the sitecripts will have their version suffixed.Starting with 2.7 sitescript versioning will be libversion.dataversion.sitescriptrevision. This should make it more clear which versions to keep and what can be deleted after updating. The sitescripts will know which library and data version they need and attempt to load the right one or exit gracefully.

Staring with 2.9, the versioning was changed yet again: libversion.dataversion.sitescriptrevision still stands, but the version number will no longer be suffixed to the sitescripts' filename. They still are kept inside the script. This means that the same .csv will be used by Magic album to store the imported prices through updates, and also makes drop-in updates of library and/or datafile simpler. Sitescript revision numbers have been raised to reflect the number of past releases.

In case you need an old version or just want to see how the scripts have evolved: The version numbers below are also download links.

pre-2.0

1.2 (Downloaded 22 times)

first public release

1.3 (Downloaded 20 times)

started externalizing site-specific data to seperate .config . exeptions for some sets still remain in the code.

1.4 (Downloaded 29 times)

sanity checks should be sane again :)

config externalization progressed far enough that adding new sets should be easy

1.5 (Downloaded 5 times)

added RtR to avsets

updated expected fail and drop counts

continued config externalization and script generalization efforts

moved gpl.txt from within the zip to seperate attachment

1.6 (Downloaded 1 time)

last release under the name magicuniverseDEv1.x.lua

added Gatecrash to avsets and expectedcounts (shop does not yet provide foil)

renamed .config to .lconf to ease syntax highlighting

got rid of utf-8 BOM

changed some comments to luadoc-like

LHpi lib

2.0

completely refactored, I'll call the externalization complete:

config became sitescript, old script (and work-in-progress mtgmintcardEURO script) became the library.

improved language handling

slightly improved conflict handling: same price is merged, different price is averaged.

2.1

added number of cards per set

if no expectation is defined in sitescript, expect all cards and 0 tokens to be set successfully

minor fix to debug loging

moved handling of undefined sitescript fields to LHpi.DoImport

adapted to new sitescripts and confirmed that nothing broke in the old ones

DOING format logging more readable

reorganized LHpi.sets cardcount and variants

for kicks and giggles, have SAVETABLE generate a csv usable by woogerboys importprices :-)

2.2

fixed handling of unset cardcounts

updated for MA 1.5.2.264b (Dragon's Maze cardcount added)

fixed savepath handling for new versioning scheme

2.3

updated for MA 1.5.2.267 (M14 and MMA added)

2.4

added Theros

2.5

changed site.BCDpluginName(name, setid) to site.BCDpluginPre(card, setid) and site.BCDpluginCard(card, setid) to :site.BCDpluginPost(card, setid)

fixes to Token handling

GetSourceData now actually preserves all data from site.ParseHtmlData

adjusted handling of multiple pages per set

2.6

added BNG and Commander2013

started adding missing special and promo sets

expanded version tables for old expansions

count foiltweak events

minor improvements and fixes

2.7pre1

fix handling of site.ParseHtmlData supplied sourcerow.* data

pass supImportfoil from MainImportCycle to BuildCardData

improved card dropping in BuildCardData

ListSources now drops foilonly urls if foil import is deselected

2.7

externalized static set data to LHpi.Data

look for LHpi and LHpi.Data in Prices\lib\

set default values for unset sitescript options

fixed luadoc comments

BuildCardData drops unwanted foil/nonfoil cards/prices to make sure user wishes are honoured

2.8

DoImport: added savepath writable check to SAVECSV=true

don't assume fruc[1] is foil and all other frucs are nonfoil

• site.fruc: add booleans to have each fruc identify itself as foil and/or nonfoil
• site.BuildUrl: infix changed from site.frucs[frucid] to site.frucs[frucid].url
• ProcessUserParams: disable unwanted frucs in site.sets at runtime, similar to how unwanted langs are already handled
• ListSources: removed check for url.foilonly, ProcessUserParams should have taken care of it.
• BuildCardData: removed @param #boolean urlfoil

BuildCardData: make sure userParams are honoured, even when preset data is present, by setting unwanted prices nil before returning the card

BuildCardData: BCDplugins should know everything BCD knows, so we're passing them importfoil, importlangs

GetSourceData: now expects site.ParseHtmlData to return card(s) wrapped in a container table

new STRICTCHECKEXPECTED option

Toutf8: new optional parameter enc

string.format'ed all nontrivial Log(string)s

read static language fields from LHpi.Data.languages

shortened site.langs

hotfix: two bugs fixed (thanks to Bloodnut for reporting them)

2.9

(all changes are transparent to sitescripts)

fixed and improved MergeCardrows (was broken if importfoil=="N" or "O" - reported by Bloodnut)

got rid of most global variables

• curhtmlnum
  • local in MainImportCycle
• persetcount
  • local in MainImportCycle
  • BuildCardData also returns #boolean namereplaced, #boolean foiltweaked
  • SetPrice also returns #table psetcount, #table failcount
• totalcount,setcountdiffers
  • local in DoImport
  • MainImportCycle returns #table totalcount, #table setcountdiffers

misc. small improvements to code and/or comments

fixed loading of data file from deprecated "Prices" location

fixed SAVETABLE folder writable check

fixed logging resultregex finds

LHpi.Log now only uses ma.Log when loglevel<0 is used

LHpi.SetPrice adds abs(expected-retval) to failcount

2.10

os.clock() loglevel 2 if os is available

fix default logfile "LHpi.Log" -> "LHpi.log"

more intuitive boolean option names

• DEBUGSKIPFOUND -> DEBUGFOUND
• STRICTCHECKEXPECTED -> STRICTEXPECTED
• legacy conversion for old sitescripts

merge site.variants with Data.variants, same with foiltweak

2.11

"(Version %d)" variant suffix from tcgplayer unified

Log library version after loading

2.11hotfix

BuildCardData now works with foil/nonfoil only when only one language is selected. (thanks to Bloodnut for reporting)

2.12

BuildCardData

• replace Simplified Chinese Basic Lands' names for variant checking if names[9]
• improved card.drop logging
• remove trailing 0s from collector numbers

MainImportCycle

• try to log number of unneeded pages for each set (to help optimize LHpi.mtgmintcard.lua)

FillCardsetTable

• now keeps card.names table if VERBOSE, since I cannot properly debug non-latin cards otherwise (SZH in LHpi.mtgmintcard.lua)

DoImport

• removed legacy conversion of STRICTCHECKEXPECTED to STRICTEXPECTED
• removed legacy conversion of DEBUGSKIPFOUND to DEBUGFOUND
• call site.SetExpected if CHECKEXPECTED

2.13

DoImport

• finetune site.expected[setid].pset defaults according to site.expected.EXPECTTOKENS,site.expected.EXPECTNONTRAD and site.expected.EXPECTREPL
• increased default dataver from 2 to 5

BuildCardData

• set card.object type

FillCardsetTable

• also fill with objtype

MergeCardrows

• merge objtypes

SetPrice

• look for special variant names "token", "nontrad", "replica", "insert" and transform them into object types for ma.SetPrice

2.14

cardcount.nontr renamed to cardcount.nontrad

GetSourceData split into GetSourceData and ParseSourceData.

• this change is completely transparent to existing sitescripts, but having GetSourceData seperated allows to use it from the sitescript for special cases.

LHpi.GetSourceData( sourceurl,urldetails )

• defer fetching to sitescript for oauth
• now returns #string sourcedata
• converts url to filename if OFFLINE
• fixed savepath==nil handling (needed when called before DoImport has run)
• Log https status if not ok

LHpi.ParseSourceData( sourcedata,sourceurl,urldetails )

• does the parsing previously done by GetSourceData and returns the #table sourceTable
• shortcut return nil for sourcedata==nil
• now strictly expect site.ParseHtmlData to return card.price as #number (and not #table, as was possible before)
• do not sanitize nor check card.regprice or card.foilpriceBuildCardData
• counts namereplace and foiltweak even if card is dropped

BuildCardData

• remove "(nontrad)","(replica)","(plane)","(scheme)","(conspiracy)" and

"(oversized)" card.name infixes when no longer needed

• set objecttype replica for 105,106,69
• improved Token suffix handling
• keeps "Replica" suffix for variants
• removes "Insert" suffix
• keep (oversized) in set [40]

SetPrice

• objtype infixe handling improved

Logtable

• no longer strictly requires #table arguments

MainImportCycle

• fixed CHECKEXPECTED handling for semi-available languages

DoImport

• changed site.expected.EXPECTTOKENS to site.expected.tokens
• changed site.expected.EXPECTNONTRAD to site.expected.nontrad
• changed site.expected.EXPECTREPL to site.expected.replica
• all three can be #boolean (like before) or #table { [langid]=#boolean,... }
• DoImport passes supImportfoil,supImportlangs,supImportsets to site.SetExpected
• fixed expected defaults and checks

removed some leftover commented-out code

improved some comments and log msgs

2.15

Split Initialize() from DoImport(importfoil , importlangs , importsets)

always run Initialize() before returning LHpi library table.

scriptname,savepath,logfile change from global to local (in site and LHpi)

Log now filters out VERBOSE and DEBUG lines by loglevel param to save "if DEBUG then Log() end" conditionals

• those conditionals have been cut and loglevel has been set for all existing LHpi.Log calls

DEBUGVARIANTS=true no longer sets DEBUG=false, instead remembers and restores previous DEBUG state

new LHpi.OAuthEncode(s) moved here from mkm-helper

workdir fixes

no longer check for Data in deprecated location

LHpi.Data

1

initial release, split from LHpi library 2.6

added all Special Sets

2

(all changes transparent to sitescripts)

minor improvements to some special sets' variant

misc. small improvements to code and/or comments

added 636:Salvat 2011

3

added:

• 806 Journey into Nyx
• 805 Duel Decks: Jace vs. Vraska
• 804 Challenge Deck: Battle the Horde
• 803 Challenge Deck: Face the Hydra

small fixes to some variant tables

4

added: 235,807,808,105,106

fixed: 320,490,590,139,787,26,31,30,10,740,390

added lots of TODO markers

5

renamed Data.sets[setid].cardcount.overs to Data.sets[setid].cardcount.repl

add "Token" suffix to variant tables to enable objtype=2 in LHpi.BuildCardData

add Nontrad,Replica variants to 807,769,787,70,761,786,784,782,801,778,23,21,20

new sets 809,810,811,812,813

fix cardcount, variant and/or foiltweak tables for 807,812,31,30,25,23,21,10

6

new 52,53,55,814,815,816,817,818,819,820,821,822

added lang [17]="Phyrexian"

fixed names for 21,40,814

variant table for 15,22,25,26,27,33,40,45

foiltweak table for 22,26,33

cardcount fixed/updated in 9,105,106,762,791,800,813,816,30,22,21,26

fixed/updated variants for 43,69,201[ITA],758,764,766,781,787,800,813,22

commented variants in 180,140,110,100,90,280,230,220,170,260,105,25

foiltweak fixed/updated in 813

LHpi.magicuniverseDE

2.0

initial release

2.1

updated for MA v1.5.2.264

use LHpi-v2.1

let LHpi lib set savepath

2.2

German Nemesis added

2.2.1

Updated for MA 1.5.2.264b (includes Dragon's Maze)

use LHpi-v2.2

2.3.1

use LHpi-v2.3

preemptively added M14

2.5.1

preemptively added Theros

removed lots of Ae -> Æ replacements

changed BuildUrl to match lib 2.5

2.5.2

fixed Theros

2.6.1

use LHpi-v2.6 (BNG)

2.8.1.1

uses LHpi-v2.8 and LHpi-Data-v1

hotfix: set options to default values

2.9.2.10

synchronized with template

updated site.expected

2.11.3.11

JOU

2.12.4.12

added 801,808

updated 180,791,806

fixed site.langs

synchronized with template

2.13.5.13

new sets 807,813

removed oversized handling from BCDPluginPre

add "Token" suffix to replacement names

update site.expected

fix 140,420,801

synchronized with template

LHpi.mtgmintcard

2.0

initial release

2.1

updated for MA v1.5.2.264

use LHpi-v2.1

let LHpi lib set savepath

2.2.1

Updated for MA 1.5.2.264b (includes Dragon's Maze)

use LHpi-v2.2

2.2.2

updated expected dropcounts

2.3.1

use LHpi-v2.3

added MMA and M14

2.5.1

added Theros

changed BuildUrl (crudely) to multiple pages per set

lots of small fixes

2.6.1

use LHpi-v2.6

added BNG,FTV:20,DD:HvsM,DD:SvT,DD:VvK,DD:EvT,DD:DvD,Beatdown,Unhinged,Unglued,Portal,Chronicles

2.8.1.1

uses LHpi-v2.8 and LHpi-Data-v1

hotfix: set options to default values

2.9.2.9

synchronized with template

updated site.expected

2.10.2.10

lib 2.10

foiltweak[setid].override=true

2.11.3.11

JOU, JaceVsVraska

2.12.4.12

added 807,808

BuildUrl and tables changed for new site structure

site.regex and ParseHtmlData changed for new raw format

BCDPluginPre now handles "SZH (ENG)" card names

added site.pagenumberregex

updated all expected counts

synchronized with template

2.13.5.13

new sets 809,810,812,813

removed oversized handling from BCDPluginPre

add "Token" suffix to replacement names

update site.expected

fix 330,801,787,600

synchronized with template

LHpi.tcgplayerPriceGuide

2.0

initial release

just adapted Golob's avsets to LHpi syntax and tested basic functionality

2.1

updated for MA v1.5.2.264

use LHpi-v2.1

let LHpi lib set savepath

tested all core and expansions up to gatecrash, also 3 portal sets and both "Un-sets"

added namereplace table and expected card counts

commented out remaining untested sets

2.2.1

Updated for MA 1.5.2.264b (includes Dragon's Maze)

use LHpi-v2.2

2.3.1

use LHpi-v2.3

added MMA and M14

2.4.1

added Theros

lots of Ae -> Æ replacements

2.5.1

changed BuildUrl to match lib 2.5

2.6.1

use LHpi-v2.6

added BNG,FTV:20,DD:HvsM,DD:SvT,DD:VvK,DD:EvT,DD:DvD,Beatdown

now with less expected fails and slightly better support for card variants

2.7.1.1

use LHpi-v2.7 and LHpi.Data-v1

load lib from \lib subdir

cleaned site.sets' frucs

added an option to just copy English card's prices to other languages

2.8.1.1

use LHpi-v2.8

shortened site.langs

simplified site.frucs,site.sets to single fruc

hotfix: set options to default values

2.9.2.10

synchronized with template

drop all "SOON" prices (higher maintenance for site.expected,but less "0"s in averaging)

updated site.expected (apparently, they don't like Boros: a sizable portion is Plains and Mountains!?)

2.11.3.11

JOU

2.12.4.12

added 808, most special and promo sets

TODO-marked all others

updated expected and namereplace tables

synchronized with template

2.13.5.13

new sets 810,812,813

removed oversized handling from BCDPluginPre

fix 100,170,801,787,600,380,10,23

add "Token" suffix to replacement names

update site.expected

synchronized with template

2.13.5.14

added 814

2.14.5.14

removed url to filename changes that are done by the library if OFFLINE

2.15.6.14

added 816,818,815,817,819,820,821,40,32,27,25,23,22,21,822,340,55,45,822

new features from template/mkm branch

site.BuildUrl

• supports multiple urls per set
• set url.details.setid

special handling for

• Dragonfury cards
• unusual lang cards

updated/fixed/added variants,namereplacements,expected

LHpi.trader-onlineDE

2.0

unreleased

2.1

updated for MA v1.5.2.264

rewrote BuildURL and ParseHTMLData, as site was changed in the meantime.

2.2

German Nemesis added

2.2.1

Updated for MA 1.5.2.264b (includes Dragon's Maze)

use LHpi-v2.2

2.3.1

use LHpi-v2.3

added MMA and preemptively M14

2.5.1

preemptively added Theros

lots of Ae -> Æ replacements

use new 2.5 BuildCardDataPlugins for improved token handling

changed BuildUrl to match lib 2.5

2.5.2

fixed THS

2.6.1

use LHpi-v2.6 (BNG)

added BNG and all available special sets

2.8.1.1

uses LHpi-v2.8 and LHpi-Data-v1

hotfix: set options to default values

2.9.2.10

synchronized with template

updated site.expected

2.11.3.11

JOU, JaceVsVraska

2.12.4.12

added 801,807,808

updated expected 100,680,690,800

synchronized with template

2.13.5.13

new sets 808,812,813

add "Token" suffix to replacement names

update site.expected

fix 784,801,600

synchronized with template

LHpi.mtgpprice.com

2.9.2.1

initial release, unfinished

2.11.3.3

first semi-official release

still largely unmaintained

JOU,DDM

2.12.4.3

site.regex,ParseHtmlData fixed for new site layout

added JOU,DDM,CNS,M15, special sets

synchronized with template

some more of namereplacement for basic land variants

fixed offline urls

2.13.5.4

new set 813

fix 250,180 (except basic lands)

synchronized with template

LHpi.sitescriptTemplate

2.0

initial release as part of LHpi-v2.0.zip

2.1

use LHpi-v2.1

let LHpi lib set savepath

moved to seperate file instead of including it in LHpi lib zip

2.2.1

use LHpi-v2.2

2.5.1

use LHpi-v2.5

2.6.1

use LHpi-v2.6

pre-populated site.sets with all sets

added LOGFOILTWEAK option

2.7.1.1

use LHpi-v2.7 and LHpi.Data-v1

load lib from \lib subdir

cleaned up luadoc and improved documentation

2.8.1.1

uses LHpi-v2.8 and LHpi-Data-v1

site.frucs: new format

sample BuildUrl adapted to new fruc format

site.BCDplugins: added importfoil, importlangs parameters

site.ParseHtmlData: now returns a container, to allow multiple cards

new option STRICTCHECKEXPECTED

2.9.2.8

too much simplification prevented possible functionality: reintroduced site.langs[langid].id field for reverse lookup

split script version from scriptname to ease possible future change to versioning scheme

misc. small improvements to example code and/or comments

2.10.2.9

lib 2.10

2.12.4.10

libver 2.12

dataver 4

BuildUrl: added optional fields to urldetails luadoc

new (optional) field: site.pagenumberregex

wrap site.expected in site.SetExpected()

add missing sets

2.13.5.11

new EXPECTNONTRAD and EXPECTREPL options to site.expected

new sets 809,810,811,812,813

new STRICTOBJTYPE option

2.14.5.12

removed url to filename changes that are done by the library if OFFLINE

2.15.6.13

script configuration variables local instead of global

workdir support

site.LoadLib split from ImportPrice

site.Initialize loads LHpi lib if not yet available (needed if ImportPrice is not called)

keep LHpi library table if already present

no longer check for lib and Data in deprecated location

removed site.CompareSiteSets

optional site.BuildUrl("list") behaviour

new optional function site.FetchExpansionList

site.BuildUrl supports multiple urls per set

dummyMA

0

initial release as part of LHpi-v2.0.zip

0.1

moved to seperate file instead of including it in LHpi lib zip

0.2

Backwards compatible with Lua5.1

tested with seperated LHpi.Data

0.3

misc. small improvements to code and/or comments

patch paths, because dev src has been moved from MA\Prices

dummy.fakesitescript()

0.4

execution timer in main()

global variables changed to dummy.*

new dummy.performancetest(repeats,script,impF,impL,impS,timefile)

more comfort, less code duplication

0.5

optional object type parameter in ma.SetPrice

new sets 801,803,804,805,806,807,808,809,810,811,812,813

0.6

new dummy.CompareDummySets(mapath,libver)

new dummy.CompareDataSets(mapath,libver,dataver)

new dummy.CompareSiteSets (moved from LHpi.magickartenmarkt.lua)

new dummy.ListUnknownUrls(expansions,missing,file) (moved from LHpi.mkm-helper.lua)

error handling in ma.GetFile and ma.Putfile

for libver > 2.14 and Lua 5.2, use dofile instead of homemade loader

CompareDataSets compares with dummy's set tales instead of Database/Sets.txt

use global workdir as introduced in 2.15

added 52,53,55,814,815,816,817,818,819,820,821

fixed dummy.forceEnv

fixed savepath handling

deprecated dummy.loadlibonly and dummy.loadscript

dummy.mergetables no longer changes param #table teins

update helper functions ouput to log instead od stdout

ma.GetUrl now uses luasockets instead of returning nil

License

The LHpi library and all its sitescripts are released as open source under the [GNU General Public License]. Please note that LHpi is thusly licensed seperately from Magic Album proper, which to date is closed source.

Other scripts

Just for reference: These unrelated pricing scripts exist

• "MTG Mint Card.lua" by Goblin Hero and Stromglad1 is included with Magic Album as a working script example.
• "Import Prices" by woogerboy21.