Pricing (Scripts and Notes)

[woogerboy21]

Maybe there is even the possibility to outsource the pricing data onto a server, where it could be fetched with an update function from within MA? Instead of having all users working on a personal script?

It's GH program so its up to him, however the pricing functionality of the application is designed to work with multiple pricing sources so you can do high / average / low price comparisons. To have one source would completely undermine the power of the applications pricing functionality.

Because, you know, I'm not the programming type... And I guess that at least some of the other users aren't, either. So in general, some (updated) scripts to download would be a nice gift for us...

Releasing the scripts isn't the problem (from my experience). Its the maintenance involved. Each script that is produced must be updated for each set that comes out. And on top of that there are different and specific problems with each pricing site. For example, some sites don't have a logical set layout so parsing is difficult, while others sometimes don't spell the card names correctly in each site.

[Jayarih]

EDIT: Scratch my question think i figured out the issue just need to figure out scripting then ill be set lol

EDIT 2: Trying to use the bat/vbs/lua of the MTG Mint script and it doesnt seem to pull anything at all for me to import, am i doing something wrong? Im running the bat and it just runs through very quickly, the other one i got off this thread (FindMagicCards) seems to be working just fine

EDIT 3: Ok bat file ran fine for FindMagicCards but cant seem to get the prices imported get an error of no start function

[LordHelmchen]

EDIT 4:
If you have not yet read this post, don't !
Instead read the post below, which is shorter and hopefully better formatted, too.

Greetings.
Since I'm both patient and like the price import to be easy to use and all in one place, I decided to salvage "MTG Mint Card.lua" for my first attempt at lua
I've asked my FLG (friendly local gameshop) where they get their reference prices from and luckily the site's html is both nicely grepable and includes out-of-stock prices (I'll withhold both my script and the shops identity until I've received their OK, but I decided to wait with asking until I gathered all inconsistencies my script will identify in their database, in the hopes that it will persuade them they'll gain something from it).

The script is progressing nicely so far, and a few questions have cropped up:
Most importantly: Would Goblin Hero be so kind as to provide a list of and documentation for all functions that ma provides ? From this thread, "MTG Mint Cards.lua" and "Import Prices.lua" I gathered:
ma.SetPrice(InternalSetID, CardLanguage, CardName, CardVersion, RegularPriceValue, FoilPriceValue) [setting 0 to a card value tells the ma app no value]
Q: return value is the number of cards set ?
RFE: can SetPrice get a second return value to report whether an existing price was overwritten ? Alternatively, or actually preferred: can we get a ReadPrice function ? The shop lists "[card] (light played)" for older editions, and I want my script to use those as fallback if no other price is available.
Q: What character encoding is expected ? Due to CardName being apparently matched against the OracleName field, I got around lots of problems by

Code: Select all

         name = string.gsub(name, "Æ", "AE")
         name = string.gsub(name, "û", "u")
         name = string.gsub(name, "á", "a")
         name = string.gsub(name, "â", "a")
         

but Junún Efreet is officially using the accented u, and setting my script file's encoding from utf-8 to ansi did not help.
ma.GetURL() <- Fetches URL and Returns TextDump of Webpage
ma.Log(Message) <- Writes information into the ma log. [Not exactly global logging since the data isnt stored any were for future processing]
ma.SetPBar(Progressmessage, NumericalProgress)
Q: was documented in the thread, but does not exists, I assume it got renamed to:
ma.SetProgress(Progressmessage, NumericalProgress)
Q: I see the bar update and fill quite fine, but don't find pmesg displayed anywhere.
ma.GetFile (filename)
I guess this is the local equivalent of GetURL ?

EDIT:
RFE: Can we get more get[field] functions ? getOraclename(InternalSetID, CardLanguage, CardName, CardVersion) would make importing the Revised Limited German set much easier (provided äÄöÖüÜß encodings will work out), instead of having to add every single card to the namereplace table.
RFE: Can we get a function to write data to a file, apart from the volatile ma.Log ? I think it might help to have the script write itself partially by generating one half of the namereplacement table . Besides, I'm not sure that a giant table kept in memory would be a good idea to implement checking for duplicates while setting the prices.

EDIT2:
Does someone have an idea why SetPrice returns 0 for "Liliana of the Dark Realms Emblem", but afterwards the price is set in the database ? Are my assumptions on SetPrice's return value wrong ?

EDIT3:
Ignore edit2... not finding the GER overwrote the retval "1" from setting the ENG... need to rething my handling of setting both languages at once

[Goblin Hero]

Most importantly: Would Goblin Hero be so kind as to provide a list of and documentation for all functions that ma provides ?

http://www.slightlymagic.net/wiki/Magic_Album_Lua_API

[LordHelmchen]

scoured the whole thread for informatin on ma.* but did not think of the wiki. Thanks again for pointing it out.

Only a few questions remain open, which I hope nobody will mind me to repost slightly more readable:

1: Is my assumption correct that ma.SetPrice matches name against the OracleName field ?
What character encoding is expected ?
I got around lots of problems by

Code: Select all

name = string.gsub(name, "Æ", "AE")
name = string.gsub(name, "û", "u")
name = string.gsub(name, "á", "a")
name = string.gsub(name, "â", "a")


but Junún Efreet is officially using the accented u, and simply setting my script file's encoding from utf-8 to ansi did not help.

1 Where is the Progressmessage from ma.SetProgress(Progressmessage, NumericalProgress) supposed to be displayed ? I see the bar update and fill, so the NumericalProgress works, but no message is displayed.

RFE1: Can we get some get[field] functions ?
ma.getOraclename(InternalSetID, CardLanguage, CardName) would make importing the Revised Limited German set much easier (provided äÄöÖüÜß encodings will work out), instead of having to add every single card to the namereplace table.
ma.getPrice(InternalSetID, CardLanguage, CardName, CardVersion, boolean Foil)could be useful for debugging, tests and also for statistic log entries ("100 cards found and set, only 10 prices changed since last update")

RFE2: If getOraclename cannot be done: can we get a function to write data to a file (probably restricted to the script's directory), apart from the volatile ma.Log ?
I think it might help to have the script write itself partially by generating one half of the namereplacement table .

[Goblin Hero]

Is my assumption correct that ma.SetPrice matches name against the OracleName field ?
What character encoding is expected ?

Yes.
UTF-8.

but no message is displayed.

Message is displayed on the progress bar itself.

ma.getOraclename(InternalSetID, CardLanguage, CardName)

I'll think what to do.

can we get a function to write data to a file (probably restricted to the script's directory)

Ok, it's easy and fast to implement.

[Goblin Hero]

I will release MA update later today (wiki is already updated).
1. PutFile function is added.
2. SetPrice function is modified so you can now use both "Oracle Name" or "Name" as cardname argument. No need to define replace table if you have the price with the national card names.
Test it.

[LordHelmchen]

no changes in the script, just updating to the new ma.SetPrice resulted in

Code: Select all

02:58:53 cardsetTable for set Revised Edition (Limited) build with 296 regular and 0 foil prices in 296 rows. set supposedly contains 306 cards and 0 tokens.
(snipped the 25 fails that remained and will be looked at soon)
02:58:54 Set Revised Edition (Limited) contains    306 cards (   306 regular,    0 tokens )
    successfully set new price for 0 English and 271 German cards. 25 German and 0 English cards failed; DROPped 16.


I think I can very much live with that

[LordHelmchen]

Greetings again.
My script is almost complete now. I still want to fallback to cards with a condition descriptor ( "Juggernaut (beta, unterschrieben, exc.)" would be a particularly nasty example) if no other price is available, and want to imlement yet another sanity check to log if any prices are discarded due to duplicate entries in the source html (which looks like a good intro to the conditioned cards)...
Both new API calls are working as expected and were put to good use:
ma.SetPrice's new behaviour saved me tons of namereplacements for Revised Limited.
ma.PutFile ended up making an offline mode possible
I would have liked to offer saving the intermediate temporary per cardset tables of the parsed data, but I'd either need a more efficient table.tostring function [Edit: my currently used one crashed MA until I seperated the first degree of recursion depth into a seperate for .. ma.Log loop] or a third parameter in ma.PutFile to change mode to appending to an existing file.
That would also open up the option of putting lots of (optional) DEBUG logging into one file per set instead of the ma.Log.

The Progressbar still looks empty to me... is the message supposed to stay until overwritten or should I update the bar much more often ?

I also found a minor but annoying bug in the Price Manager: The Update button starts executing the selected row, even if its checkbox is unchecked and another row is checked. Which undermined my efforts to use offline files while debugging to go easy on the vendor's server until I realized what happened

Lastly, my script turned out to double as a missing cards detector... at least it points out cards that might be worth looking into/for Would you like me to compile a seperate list, or will the scripts logging suffice ? here's an example:

13:16:03 ! setPrice "Giant Octopus" for language GER returned unexpected retval "0"; expected was 1
13:16:04 ! setPrice "Enormous Baloth" for language GER returned unexpected retval "0"; expected was 1
13:16:04 ! setPrice "Vizzerdrix" for language GER returned unexpected retval "0"; expected was 1
13:16:04 ! setPrice "Vengeance" for language GER returned unexpected retval "0"; expected was 1
13:16:04 ! setPrice "Coral Eel" for language GER returned unexpected retval "0"; expected was 1
13:16:04 ! setPrice "Spined Wurm" for language GER returned unexpected retval "0"; expected was 1
13:16:05 ! setPrice "Index" for language GER returned unexpected retval "0"; expected was 1
13:16:06 Set 9th Edition contains 359 cards ( 359 regular, 0 tokens )
successfully set new price for 339 English and 332 German cards. 7 German and 0 English cards failed

And since I have yet neglected to state it:
Thanks a lot for a wonderful and apparently well thought-through swiss army knife for MtG archival

Last but not least: Since the script is almost in a state that would warrant its release: I'd like to release my code as GPL.
Since I took the bare bones and inspiration from the included "MTG Mint Card.lua" and woogerboy21's "Import Prices.lua", I thusly need to ask their authors' stance on this. In case the gpl's preventing further use of the code in closed source software posing a potential Problem to Goblin Hero, I'd be prepared to grant an additional license for the code or any parts of it for use within or in relation to Magic Album in whatever way seems appropriate. (ianal, but I want to make sure there's as little potential problems as possible)

If you read all this, thanks for your time. I realize it got a bit long

LordHelmchen

[Goblin Hero]

That would also open up the option of putting lots of (optional) DEBUG logging into one file per set instead of the ma.Log.

Good idea.

The Progressbar still looks empty to me...

Strange. Look at the attached screenshot.

I also found a minor but annoying bug in the Price Manager: The Update button starts executing the selected row, even if its checkbox is unchecked and another row is checked.

Checkbox defines whether the price will be used in min/max/avg calculations. Selection (you can select multiple prices at once) defines what to update.

Would you like me to compile a seperate list, or will the scripts logging suffice ?

No, unless you can prove an existence of the said cards. The cards you've listed in your example does not exist in German. I'm sure.

I'd like to release my code as GPL.

Do as you wish. I do not mind.

[LordHelmchen]

Good idea.

Phew. Glad to hear...read I'm not completely wasting your time with my posts

The Progressbar still looks empty to me...

Strange. Look at the attached screenshot.

No, never seen that. I'll make a screenshot for comparison.

No, unless you can prove an existence of the said cards. The cards you've listed in your example does not exist in German. I'm sure.

Hmm... what would constitute proof, so I can check before getting on your nerves with presumed missing cards ? I did check with Gatherer first, not treating the vendor's list as a reliable source. http://gatherer.wizards.com/Pages/Card/Details.aspx?multiverseid=98548 lists for example the
Riesenoktopus. I had assumed the official page to give correct information until now, but as it denies the existance of the French Ice Age Counterspell http://gatherer.wizards.com/Pages/Card/Languages.aspx?multiverseid=2500 I'm holding in my own hands, I'll have to reevaluate my trust in gatherer.

[woogerboy21]

EDIT: Scratch my question think i figured out the issue just need to figure out scripting then ill be set lol

EDIT 2: Trying to use the bat/vbs/lua of the MTG Mint script and it doesnt seem to pull anything at all for me to import, am i doing something wrong? Im running the bat and it just runs through very quickly, the other one i got off this thread (FindMagicCards) seems to be working just fine

EDIT 3: Ok bat file ran fine for FindMagicCards but cant seem to get the prices imported get an error of no start function

Sounds like you have an old version of the scripts. I don't recall what version MA was changed in that required an update to the import script which in turn required an update to the vbs/lua portion of the scripts. In either case sounds like you figured things out. The original intent of the import script was to offer the ability for a uniformed method for others to write price collection scripts in languages they would know. Secondly it allowed for the price collection portion to be scheduled outside the MA application thus speeding time of price importation up. I'll be the first to say its alot easier to simply drop a .lua file into the price folder and click go.

Since I took the bare bones and inspiration from the included "MTG Mint Card.lua" and woogerboy21's "Import Prices.lua", I thusly need to ask their authors' stance on this.

Do as you wish with my script code as well. I don't mind and is there for others to learn from.

I also found a minor but annoying bug in the Price Manager: The Update button starts executing the selected row, even if its check box is unchecked and another row is checked.

I am experiencing the same issue. Appears to be an app bug.

The Progressbar still looks empty to me... is the message supposed to stay until overwritten or should I update the bar much more often ?

I have seen this as well and believe it has to do with Windows and the theme in use. I have ran the app on Linux in wine for example and the text appears. I have also ran this in Windows in the past and the text appears. Needless to say I haven't been able to consistent determine why the text doesn't show up but I can say the text doesn't show up more often than not for me.

I'll have to reevaluate my trust in gatherer.

I've learned that there are multiple sources you can collect data from but at some point you have to pick a source and stick with it (mainly for sanity reasons). In my mind I default to the manufacturer but others generally choose otherwise.

[LordHelmchen]

Greetings.

I think my import script has arrived at a state that warrants its release, if only for it to serve as a warning example
I created a new thread to avoid cluttering this one at Pricing script (lua) for magicuniverse.de

footnote in reference to my commenting on waiting for the vendor's ok: As they've silently corrected the two inconsistency examples I gave when I asked them, I'll consider their lack of reply to me as given consent

[woogerboy21]

Greetings.

I think my import script has arrived at a state that warrants its release, if only for it to serve as a warning example
I created a new thread to avoid cluttering this one at Pricing script (lua) for magicuniverse.de

footnote in reference to my commenting on waiting for the vendor's ok: As they've silently corrected the two inconsistency examples I gave when I asked them, I'll consider their lack of reply to me as given consent

I have added a link to your post in the first price script post. When I have time I will have to reorganize the first post (I don't like the layout.)

GH -
Is there a way you can delete your post that is right below the first post of this thread? This one: (http://www.slightlymagic.net/forum/viewtopic.php?p=30099#p30099)

[LordHelmchen]

Greetings again.
I have a problem with mtgmincard.com and/or MA's GetUrl ...
mtgmintcard offers a dropdown selection for "Currency Reference", which translates into "&currency_reference=EUR" in the url. When opened in firefox, the result is

tablerow as seen through firefox view page source | Open

<tr class="darkRow">
<td align="center">
<a class="opacityit" href="javascript:popupWindow('http://www.mtgmintcard.com/acidic-slime-p-310256.html?popup=1','_blank')"><img src="images/products/161/MAGIC-THE-GATHERING-ACIDIC-SLIME-M13U002F_S.jpg" border="0" alt="Acidic Slime" title=" Acidic Slime " width="20" height="28" class="cardBorderBlack"></a> </td>
<td>
<a class="opacityit" href="http://www.mtgmintcard.com/acidic-slime-p-310256.html">Acidic Slime</a> </td>
<td align="center"><img src="images/editions/exp_sym_M13_U.gif" border="0" alt="2013 Core set" title=" 2013 Core set " width="39" height="21"></td>
<td align="center"><img src="images/icon_mode_eng_foil.gif" border="0" alt="English Foil" title=" English Foil " width="21" height="21" border="0"></td>
<td><img src="images/symbols/Symbol_3_mana.gif" alt="3" title="3"><img src="images/symbols/Green_mana.gif" alt="G" title="G"><img src="images/symbols/Green_mana.gif" alt="G" title="G"></td>
<td>
Creature - Ooze </td>
<td align="center">2</td>
<td align="center">2</td>
<td align="right">
&nbsp;<span class="productListPrice">$0.79</span>&nbsp;<span class="textNote">(€0.61)</span> </td>
<td align="center">
<select name="multiple_quantity_310256" id="multiple_quantity_310256"><option value="1">1</option><option value="2">2</option><option value="3">3</option><option value="4">4</option><option value="5">5</option><option value="6">6</option><option value="7">7</option><option value="8">8</option><option value="9">9</option><option value="10">10</option><option value="11">11</option><option value="12">12</option><option value="13">13</option><option value="14">14</option><option value="15">15</option><option value="16">16</option><option value="17">17</option><option value="18">18</option><option value="19">19</option><option value="20">20</option><option value="21">21</option><option value="22">22</option><option value="23">23</option><option value="24">24</option><option value="25">25</option><option value="26">26</option><option value="27">27</option><option value="28">28</option><option value="29">29</option><option value="30">30</option><option value="31">31</option><option value="32">32</option><option value="33">33</option><option value="34">34</option><option value="35">35</option><option value="36">36</option></select> </td>
<td align="center">
<input type="button" value="Add to Cart" class="btn" onmouseover="this.className='btn btnhov'" onmouseout="this.className='btn'" onmouseup="this.className='btn'" onmousedown="this.className='btn btnclick'" onclick="AddMultipleCartGet('http://www.mtgmintcard.com/ajax_shopping_cart.php?currency_reference=EUR&amp;search_result=500&amp;edition=2013%20Core%20set&amp;mode=Foils&amp;action=buy_now&amp;products_id=310256', 310256); return false;"/> </td>
</tr>

but if GetUrl-ed and PutFile-ed the same section is

tablerow as seen through GetUrl | Open

<tr class="darkRow">
<td align="center">
<a class="opacityit" href="javascript:popupWindow('http://www.mtgmintcard.com/acidic-slime-p-310256.html?popup=1&amp;osCsid=2af083dfd0f0aff4708d7e696d5a93e7','_blank')"><img src="images/products/161/MAGIC-THE-GATHERING-ACIDIC-SLIME-M13U002F_S.jpg" border="0" alt="Acidic Slime" title=" Acidic Slime " width="20" height="28" class="cardBorderBlack"></a> </td>
<td>
<a class="opacityit" href="http://www.mtgmintcard.com/acidic-slime-p-310256.html?osCsid=2af083dfd0f0aff4708d7e696d5a93e7">Acidic Slime</a> </td>
<td align="center"><img src="images/editions/exp_sym_M13_U.gif" border="0" alt="2013 Core set" title=" 2013 Core set " width="39" height="21"></td>
<td align="center"><img src="images/icon_mode_eng_foil.gif" border="0" alt="English Foil" title=" English Foil " width="21" height="21" border="0"></td>
<td><img src="images/symbols/Symbol_3_mana.gif" alt="3" title="3"><img src="images/symbols/Green_mana.gif" alt="G" title="G"><img src="images/symbols/Green_mana.gif" alt="G" title="G"></td>
<td>
Creature - Ooze </td>
<td align="center">2</td>
<td align="center">2</td>
<td align="right">
&nbsp;<span class="productListPrice">$0.79</span>&nbsp; </td>
<td align="center">
<select name="multiple_quantity_310256" id="multiple_quantity_310256"><option value="1">1</option><option value="2">2</option><option value="3">3</option><option value="4">4</option><option value="5">5</option><option value="6">6</option><option value="7">7</option><option value="8">8</option><option value="9">9</option><option value="10">10</option><option value="11">11</option><option value="12">12</option><option value="13">13</option><option value="14">14</option><option value="15">15</option><option value="16">16</option><option value="17">17</option><option value="18">18</option><option value="19">19</option><option value="20">20</option><option value="21">21</option><option value="22">22</option><option value="23">23</option><option value="24">24</option><option value="25">25</option><option value="26">26</option><option value="27">27</option><option value="28">28</option><option value="29">29</option><option value="30">30</option><option value="31">31</option><option value="32">32</option><option value="33">33</option><option value="34">34</option><option value="35">35</option><option value="36">36</option></select> </td>
<td align="center">
<input type="button" value="Add to Cart" class="btn" onmouseover="this.className='btn btnhov'" onmouseout="this.className='btn'" onmouseup="this.className='btn'" onmousedown="this.className='btn btnclick'" onclick="AddMultipleCartGet('http://www.mtgmintcard.com/ajax_shopping_cart.php?currency_reference=EUR&amp;search_result=500&amp;edition=2013%20Core%20set&amp;mode=Foils&amp;action=buy_now&amp;products_id=310256&amp;osCsid=2af083dfd0f0aff4708d7e696d5a93e7', 310256); return false;"/> </td>
</tr>

Note that the section

Code: Select all

&nbsp;<span class="textNote">(€0.61)</span>                                    </td>

is replaced with

Code: Select all

&nbsp;                                    </td>

, which puts a stop on my efforts to adapt my sript to mtmintcard... Any ideas or even remedies would be appreciated.