Added decode script with two supported formats.

Other misc fixes as well, we get all of the choice cards right in the encoding.

data/output.txt

@@ -1280,7 +1280,7 @@
 
 |famine||sorcery||||{^^^BBBB}|@ deals &^^^ damage to each creature and each player.|
 
-|profane command||sorcery||||{XXBBBB}|choose two ~ \= target player loses X life. \= return target creature card with converted mana cost X or less from your graveyard to the battlefield.\= target creature gets -X/-X until end of turn.\= up to X target creatures gain fear until end of turn.|
+|profane command||sorcery||||{XXBBBB}|[&^^ = target player loses X life.  = return target creature card with converted mana cost X or less from your graveyard to the battlefield. = target creature gets -X/-X until end of turn. = up to X target creatures gain fear until end of turn.]\|
 
 |kill~suit cultist||creature||goblin berserker|&^/&^|{RR}|@ attacks each turn if able.\{BB}, sacrifice @: the next time damage would be dealt to target creature this turn, destroy that creature instead.|
 
@@ -2500,7 +2500,7 @@
 
 |goblin sharpshooter||creature||goblin|&^/&^|{^^RR}|@ doesn't untap during your untap step.\whenever a creature dies, untap @.\T: @ deals &^ damage to target creature or player.|
 
-|misfortune||sorcery||||{^BBRRGG}|an opponent chooses one ~\= you put a +&^/+&^ counter on each creature you control and gain &^^^^ life.\= you put a -&^/-&^ counter on each creature that player controls and @ deals &^^^^ damage to him or her.|
+|misfortune||sorcery||||{^BBRRGG}|an opponent [&^ = you put a +&^/+&^ counter on each creature you control and gain &^^^^ life. = you put a -&^/-&^ counter on each creature that player controls and @ deals &^^^^ damage to him or her.]|
 
 |mind extraction||sorcery||||{^^BB}|as an additional cost to cast @, sacrifice a creature.\target player reveals his or her hand and discards all cards of each of the sacrificed creature's colors.|
 
@@ -5870,7 +5870,7 @@
 
 |hallowed healer||creature||human cleric|&^/&^|{^^WW}|T: prevent the next &^^ damage that would be dealt to target creature or player this turn.\threshold ~ T: prevent the next &^^^^ damage that would be dealt to target creature or player this turn. activate this ability only if seven or more cards are in your graveyard.|
 
-|library of lat~nam||sorcery||||{^^^^UU}|an opponent chooses one ~\= you draw three cards at the beginning of the next turn's upkeep.\= you search your library for a card, put that card into your hand, then shuffle your library.|
+|library of lat~nam||sorcery||||{^^^^UU}|an opponent [&^ = you draw three cards at the beginning of the next turn's upkeep. = you search your library for a card, put that card into your hand, then shuffle your library.]|
 
 |kor skyfisher||creature||kor soldier|&^^/&^^^|{^WW}|flying\when @ enters the battlefield, return a permanent you control to its owner's hand.|
 
@@ -10108,7 +10108,7 @@
 
 |chaos moon||enchantment||||{^^^RR}|at the beginning of each upkeep, count the number of permanents. if the number is odd, until end of turn, red creatures get +&^/+&^ and whenever a player taps a mountain for mana, that player adds {RR} to his or her mana pool . if the number is even, until end of turn, red creatures get -&^/-&^ and if a player taps a mountain for mana, that mountain produces colorless mana instead of any other type.|
 
-|frontier siege||enchantment||||{^^^GG}|as @ enters the battlefield, choose khans or dragons.\= khans ~ at the beginning of each of your main phases, add {GGGG} to your mana pool.\= dragons ~ whenever a creature with flying enters the battlefield under your control, you may have it fight target creature you don't control.|
+|frontier siege||enchantment||||{^^^GG}|as @ enters the battlefield, [&^ = khans ~ at the beginning of each of your main phases, add {GGGG} to your mana pool. = dragons ~ whenever a creature with flying enters the battlefield under your control, you may have it fight target creature you don't control.]|
 
 |mobile fort||artifact creature||wall|&/&^^^^^^|{^^^^}|defender \{^^^}: @ gets +&^^^/-&^ until end of turn and can attack this turn as though it didn't have defender. activate this ability only once each turn.|
 
@@ -14440,7 +14440,7 @@
 
 |forgotten harvest||enchantment||||{^GG}|at the beginning of your upkeep, you may exile a land card from your graveyard. if you do, put a +&^/+&^ counter on target creature.|
 
-|outpost siege||enchantment||||{^^^RR}|as @ enters the battlefield, choose khans or dragons.\= khans ~ at the beginning of your upkeep, exile the top card of your library. until end of turn, you may play that card.\= dragons ~ whenever a creature you control leaves the battlefield, @ deals &^ damage to target creature or player.|
+|outpost siege||enchantment||||{^^^RR}|as @ enters the battlefield, [&^ = khans ~ at the beginning of your upkeep, exile the top card of your library. until end of turn, you may play that card. = dragons ~ whenever a creature you control leaves the battlefield, @ deals &^ damage to target creature or player.]|
 
 |vigorous charge||instant||||{GG}|kicker {WW} \target creature gains trample until end of turn. whenever that creature deals combat damage this turn, if @ was kicked, you gain life equal to that damage.|
 
@@ -18615,7 +18615,7 @@
 
 |font of vigor||enchantment||||{^WW}|{^^WW}, sacrifice @: you gain &^^^^^^^ life.|
 
-|citadel siege||enchantment||||{^^WWWW}|as @ enters the battlefield, choose khans or dragons.\= khans ~ at the beginning of combat on your turn, put two +&^/+&^ counters on target creature you control.\= dragons ~ at the beginning of combat on each opponent's turn, tap target creature that player controls.|
+|citadel siege||enchantment||||{^^WWWW}|as @ enters the battlefield, [&^ = khans ~ at the beginning of combat on your turn, put two +&^/+&^ counters on target creature you control. = dragons ~ at the beginning of combat on each opponent's turn, tap target creature that player controls.]|
 
 |gigantomancer||creature||human shaman|&^/&^|{^^^^^^^GG}|{^}: target creature you control has base power and toughness &^^^^^^^/&^^^^^^^ until end of turn.|
 
@@ -18689,7 +18689,7 @@
 
 |heroes' bane||creature||hydra|&/&|{^^^GGGG}|@ enters the battlefield with four +&^/+&^ counters on it.\{^^GGGG}: put X +&^/+&^ counters on @, where X is its power.|
 
-|monastery siege||enchantment||||{^^UU}|as @ enters the battlefield, choose khans or dragons.\= khans ~ at the beginning of your draw step, draw an additional card, then discard a card.\= dragons ~ spells your opponents cast that target you or a permanent you control cost {^^} more to cast.|
+|monastery siege||enchantment||||{^^UU}|as @ enters the battlefield, [&^ = khans ~ at the beginning of your draw step, draw an additional card, then discard a card. = dragons ~ spells your opponents cast that target you or a permanent you control cost {^^} more to cast.]|
 
 |cabal trainee||creature||human minion|&^/&^|{BB}|sacrifice @: target creature gets -&^^/-& until end of turn.|
 
@@ -26025,7 +26025,7 @@
 
 |gisela, blade of goldnight|legendary|creature||angel|&^^^^^/&^^^^^|{^^^^RRWWWW}|flying, first strike\if a source would deal damage to an opponent or a permanent an opponent controls, that source deals double that damage to that player or permanent instead.\if a source would deal damage to you or a permanent you control, prevent half that damage, rounded up.|
 
-|fatal lore||sorcery||||{^^BBBB}|an opponent chooses one ~\= you draw three cards.\= you destroy up to two target creatures that player controls. they can't be regenerated. that player draws up to three cards.|
+|fatal lore||sorcery||||{^^BBBB}|an opponent [&^ = you draw three cards. = you destroy up to two target creatures that player controls. they can't be regenerated. that player draws up to three cards.]|
 
 |puncture bolt||instant||||{^RR}|@ deals &^ damage to target creature. put a -&^/-&^ counter on that creature.|
 
@@ -27334,7 +27334,7 @@
 
 |charging badger||creature||badger|&^/&^|{GG}|trample|
 
-|palace siege||enchantment||||{^^^BBBB}|as @ enters the battlefield, choose khans or dragons.\= khans ~ at the beginning of your upkeep, return target creature card from your graveyard to your hand.\= dragons ~ at the beginning of your upkeep, each opponent loses &^^ life and you gain &^^ life.|
+|palace siege||enchantment||||{^^^BBBB}|as @ enters the battlefield, [&^ = khans ~ at the beginning of your upkeep, return target creature card from your graveyard to your hand. = dragons ~ at the beginning of your upkeep, each opponent loses &^^ life and you gain &^^ life.]|
 
 |second guess||instant||||{^UU}|uncast target spell that's the second spell cast this turn.|
 

decode.py

@@ -0,0 +1,85 @@
+#!/usr/bin/env python
+import sys
+import os
+
+libdir = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib')
+sys.path.append(libdir)
+import utils
+import jdecode
+import cardlib
+
+def main(fname, oname = None, verbose = True, gatherer = False, for_forum = False):
+    cards = []
+    valid = 0
+    invalid = 0
+    unparsed = 0
+
+    if fname[-5:] == '.json':
+        if verbose:
+            print 'This looks like a json file: ' + fname
+        json_srcs = jdecode.mtg_open_json(fname, verbose)
+        for json_cardname in sorted(json_srcs):
+            if len(json_srcs[json_cardname]) > 0:
+                jcards = json_srcs[json_cardname]
+                card = cardlib.Card(json_srcs[json_cardname][0])
+                if card.valid:
+                    valid += 1
+                    cards += [card]
+                elif card.parsed:
+                    invalid += 1
+                else:
+                    unparsed += 1
+
+    # fall back to opening a normal encoded file
+    else:
+        if verbose:
+            print 'Opening encoded card file: ' + fname
+        with open(fname, 'rt') as f:
+            text = f.read()
+        for card_src in text.split(utils.cardsep):
+            if card_src:
+                card = cardlib.Card(card_src)
+                if card.valid:
+                    valid += 1
+                    cards += [card]
+                elif card.parsed:
+                    invalid += 1
+                else:
+                    unparsed += 1
+    if verbose:
+        print (str(valid) + ' valid, ' + str(invalid) + ' invalid, ' 
+               + str(unparsed) + ' failed to parse.')
+
+    if oname:
+        if verbose:
+            print 'Writing output to: ' + oname
+        with open(oname, 'w') as ofile:
+            for card in cards:
+                ofile.write((card.format(gatherer = gatherer, for_forum = for_forum) 
+                            + '\n').encode('utf-8'))
+    else:
+        for card in cards:
+            sys.stdout.write((card.format(gatherer = gatherer, for_forum = for_forum) 
+                              + '\n').encode('utf-8'))
+        sys.stdout.flush()
+
+
+if __name__ == '__main__':
+    import argparse
+    parser = argparse.ArgumentParser()
+    
+    parser.add_argument('infile', #nargs='?'. default=None,
+                        help='encoded card file or json corpus to encode')
+    parser.add_argument('outfile', nargs='?', default=None,
+                        help='output file, defaults to stdout')
+    parser.add_argument('-g', '--gatherer', action='store_true',
+                        help='emulate Gatherer visual spoiler')
+    parser.add_argument('-f', '--forum', action='store_true',
+                        help='use pretty mana encoding for mtgsalvation forum')
+    parser.add_argument('-v', '--verbose', action='store_true', 
+                        help='verbose output')
+    
+    args = parser.parse_args()
+    main(args.infile, args.outfile, verbose = args.verbose, 
+         gatherer = args.gatherer, for_forum = args.forum)
+    exit(0)

encode.py

@@ -46,7 +46,7 @@ def main(fname, oname = None, verbose = True, dupes = 0, encoding = 'std', stabl
             dupes = 10
         fmt_labeled = cardlib.fmt_labeled_default
         randomize_fields = True
-        randomize_mana = True
+        #randomize_mana = True
         final_sep = False
     else:
         raise ValueError('encode.py: unknown encoding: ' + encoding)
@@ -97,6 +97,7 @@ def main(fname, oname = None, verbose = True, dupes = 0, encoding = 'std', stabl
                     invalid += 1
                 else:
                     unparsed += 1
+
     # fall back to opening a normal encoded file
     else:
         if verbose:

lib/cardlib.py

@@ -214,10 +214,12 @@ def fields_from_format(src_text, fmt_ordered, fmt_labeled, fieldsep):
         labels = {fmt_labeled[k] : k for k in fmt_labeled}
         field_label_regex = '[' + ''.join(labels.keys()) + ']'
     def addf(fields, fkey, fval):
-        if fkey in fields:
+        # make sure you pass a pair
-            fields[fkey] += [fval]
+        if fval and fval[1]:
-        else:
+            if fkey in fields:
-            fields[fkey] = [fval]
+                fields[fkey] += [fval]
+            else:
+                fields[fkey] = [fval]
 
     textfields = src_text.split(fieldsep)
     idx = 0
@@ -240,6 +242,7 @@ def fields_from_format(src_text, fmt_ordered, fmt_labeled, fieldsep):
             # use the first label if we saw any at all
             if len(labs) > 0:
                 lab = labs[0]
+                textfield = textfield.replace(lab, '', 1)
         # try to use the field label if we got one
         if lab and lab in labels:
             fname = labels[lab]
@@ -279,7 +282,7 @@ class Card:
     '''card representation with data'''
 
     def __init__(self, src, fmt_ordered = fmt_ordered_default, 
-                            fmt_labeled = None, 
+                            fmt_labeled = fmt_labeled_default, 
                             fieldsep = utils.fieldsep):
         # source fields, exactly one will be set
         self.json = None
@@ -466,3 +469,107 @@ class Card:
                                           initial_sep = initial_sep, final_sep = final_sep))
 
         return outstr
+
+    def format(self, gatherer = False, for_forum = False):
+        outstr = ''
+        if gatherer:
+            cardname = self.__dict__[field_name].title()
+            if for_forum:
+                outstr += '[b]'
+            outstr += cardname
+            if for_forum:
+                outstr += '[/b]'
+
+            outstr += ' ' + self.__dict__[field_cost].format(for_forum = for_forum)
+
+            if self.__dict__[field_rarity]:
+                outstr += '(' + self.__dict__[rarity] + ')'
+                
+            outstr += '\n'
+
+            outstr += ' '.join(self.__dict__[field_supertypes] + self.__dict__[field_types]).title()
+            if self.__dict__[field_subtypes]:
+                outstr += (' ' + utils.dash_marker + ' ' + 
+                           ' '.join(self.__dict__[field_subtypes]).title())
+
+            if self.__dict__[field_pt]:
+                outstr += ' (' + utils.from_unary(self.__dict__[field_pt]) + ')'
+
+            if self.__dict__[field_loyalty]:
+                outstr += ' ((' + utils.from_unary(self.__dict__[field_loyalty]) + '))'
+
+            outstr += '\n'
+
+            if self.__dict__[field_text].text:
+                mtext = self.__dict__[field_text].text
+                mtext = transforms.text_unpass_1_choice(mtext, delimit = False)
+                mtext = transforms.text_unpass_2_counters(mtext)
+                mtext = transforms.text_unpass_3_unary(mtext)
+                mtext = transforms.text_unpass_4_cardname(mtext, cardname)
+                mtext = transforms.text_unpass_5_symbols(mtext, for_forum)
+                mtext = transforms.text_unpass_6_newlines(mtext)
+                newtext = Manatext('')
+                newtext.text = mtext
+                newtext.costs = self.__dict__[field_text].costs
+                outstr += newtext.format(for_forum = for_forum)
+            
+                outstr += '\n'
+
+            if self.__dict__[field_other]:
+                if for_forum:
+                    outstr += '[i]'
+                else:
+                    outstr += utils.dash_marker * 2
+                    outstr += '\n'
+                for idx, value in self.__dict__[field_other]:
+                    outstr += '<' + str(idx) + '> ' + str(value)
+                    outstr += '\n'
+                if for_forum:
+                    outstr = outstr[:-1] # hack off the last newline
+                    outstr += '[/i]'
+                    outstr += '\n'
+
+        else:
+            cardname = self.__dict__[field_name]
+            outstr += cardname
+            if self.__dict__[field_rarity]:
+                outstr += '(' + self.__dict__[field_rarity] + ')'
+            outstr += '\n'
+            
+            outstr += self.__dict__[field_cost].format(for_forum = for_forum)
+            outstr += '\n'
+
+            outstr += ' '.join(self.__dict__[field_supertypes] + self.__dict__[field_types])
+            if self.__dict__[field_subtypes]:
+                outstr += ' ' + utils.dash_marker + ' ' + ' '.join(self.__dict__[field_subtypes])
+            outstr += '\n'
+            
+            if self.__dict__[field_text].text:
+                mtext = self.__dict__[field_text].text
+                mtext = transforms.text_unpass_1_choice(mtext, delimit = True)
+                #mtext = transforms.text_unpass_2_counters(mtext)
+                mtext = transforms.text_unpass_3_unary(mtext)
+                #mtext = transforms.text_unpass_4_cardname(mtext, cardname)
+                mtext = transforms.text_unpass_5_symbols(mtext, for_forum)
+                mtext = transforms.text_unpass_6_newlines(mtext)
+                newtext = Manatext('')
+                newtext.text = mtext
+                newtext.costs = self.__dict__[field_text].costs
+                outstr += newtext.format(for_forum = for_forum) + '\n'
+
+            if self.__dict__[field_pt]:
+                outstr += '(' + utils.from_unary(self.__dict__[field_pt]) + ')'
+                outstr += '\n'
+
+            if self.__dict__[field_loyalty]:
+                outstr += '((' + utils.from_unary(self.__dict__[field_loyalty]) + '))'
+                outstr += '\n'
+                
+            if self.__dict__[field_other]:
+                outstr += utils.dash_marker * 2
+                outstr += '\n'
+                for idx, value in self.__dict__[field_other]:
+                    outstr += '<' + str(idx) + '> ' + str(value)
+                    outstr += '\n'
+
+        return outstr

lib/config.py

@@ -17,6 +17,8 @@ this_marker = '@'
 counter_marker = '%'
 reserved_marker = '\v'
 reserved_mana_marker = '$'
+choice_open_delimiter = '['
+choice_close_delimiter = ']'
 x_marker = 'X'
 tap_marker = 'T'
 untap_marker = 'Q'
@@ -39,7 +41,7 @@ unary_exceptions = {
 
 # field labels, to allow potential reordering of card format
 field_label_name = '1'
-field_label_rarity = '2'
+field_label_rarity = 'Y' # 2 is part of some mana symbols {2/B} ...
 field_label_cost = '3'
 field_label_supertypes = '4'
 field_label_types = '5'

lib/manalib.py

@@ -107,8 +107,8 @@ class Manacost:
                                       + utils.mana_close_delimiter)
 
     def format(self, for_forum = False):
-        return utils.mana_untranslate(utils.mana_open_delimiter + ''.join(self.sequence, for_forum)
+        return utils.mana_untranslate(utils.mana_open_delimiter + ''.join(self.sequence)
-                                      + utils.mana_close_delimiter)
+                                      + utils.mana_close_delimiter, for_forum)
 
     def encode(self, randomize = False):
         if self.none:

lib/transforms.py

@@ -15,6 +15,8 @@ bullet_marker = utils.bullet_marker
 this_marker = utils.this_marker
 counter_marker = utils.counter_marker
 reserved_marker = utils.reserved_marker
+choice_open_delimiter = utils.choice_open_delimiter
+choice_close_delimiter = utils.choice_close_delimiter
 x_marker = utils.x_marker
 tap_marker = utils.tap_marker
 untap_marker = utils.untap_marker
@@ -331,17 +333,21 @@ def text_pass_7_choice(s):
             newchoice = newchoice.replace(prefix, unary_marker + (unary_counter * count))
             newchoice = newchoice.replace('\n', ' ')
             if newchoice[-1:] == ' ':
-                newchoice = '[' + newchoice[:-1] + ']\n'
+                newchoice = choice_open_delimiter + newchoice[:-1] + choice_close_delimiter + '\n'
             else:
-                newchoice = '[' + newchoice + ']'
+                newchoice = choice_open_delimiter + newchoice + choice_close_delimiter
             s_helper = s_helper.replace(choice[0], newchoice)
         return s_helper
 
     s = choice_formatting_helper(s, ur'choose one \u2014', 1)
     s = choice_formatting_helper(s, ur'choose one \u2014 ', 1) # ty Promise of Power
     s = choice_formatting_helper(s, ur'choose two \u2014', 2)
+    s = choice_formatting_helper(s, ur'choose two \u2014 ', 2) # ty Profane Command
     s = choice_formatting_helper(s, ur'choose one or both \u2014', 0)
     s = choice_formatting_helper(s, ur'choose one or more \u2014', 0)
+    s = choice_formatting_helper(s, ur'choose khans or dragons.', 1)
+    # this is for 'an opponent chooses one', which will be a bit weird but still work out
+    s = choice_formatting_helper(s, ur'chooses one \u2014', 1)
 
     return s
 
@@ -388,3 +394,78 @@ def text_pass_9_newlines(s):
 
 def text_pass_10_symbols(s):
     return utils.to_symbols(s)
+
+
+# Text unpasses, for decoding. All assume the text inside a Manatext, so don't do anything
+# weird with the mana cost symbol.
+
+
+def text_unpass_1_choice(s, delimit = False):
+    choice_regex = (re.escape(choice_open_delimiter) + re.escape(unary_marker)
+                    + r'.*' + re.escape(bullet_marker) + r'.*' + re.escape(choice_close_delimiter))
+    choices = re.findall(choice_regex, s)
+    for choice in sorted(choices, lambda x,y: cmp(len(x), len(y)), reverse = True):
+        fragments = choice[1:-1].split(bullet_marker)
+        countfrag = fragments[0]
+        optfrags = fragments[1:]
+        choicecount = int(utils.from_unary(re.findall(utils.number_unary_regex, countfrag)[0]))
+        newchoice = ''
+
+        if choicecount == 0:
+            if len(countfrag) == 2:
+                newchoice += 'choose one or both '
+            else:
+                newchoice += 'choose one or more '
+        elif choicecount == 1:
+            newchoice += 'choose one '
+        elif choicecount == 2:
+            newchoice += 'choose two '
+        else:
+            newchoice += 'choose ' + utils.to_unary(str(choicecount)) + ' '
+        newchoice += dash_marker
+        
+        for option in optfrags:
+            option = option.strip()
+            if option:
+                newchoice += newline + bullet_marker + ' ' + option
+
+        if delimit:
+            s = s.replace(choice, choice_open_delimiter + newchoice + choice_close_delimiter)
+            s = s.replace('an opponent ' + choice_open_delimiter + 'choose ', 
+                          'an opponent ' + choice_open_delimiter + 'chooses ')
+        else:
+            s = s.replace(choice, newchoice)
+            s = s.replace('an opponent choose ', 'an opponent chooses ')
+    
+    return s
+
+
+def text_unpass_2_counters(s):
+    countertypes = re.findall(r'countertype ' + re.escape(counter_marker) 
+                              + r'[^' + re.escape(newline) + r']*' + re.escape(newline), s)
+    # lazier than using groups in the regex
+    countertypes += re.findall(r'countertype ' + re.escape(counter_marker) 
+                              + r'[^' + re.escape(newline) + r']*$', s)
+    if len(countertypes) > 0:
+        countertype = countertypes[0].replace('countertype ' + counter_marker, '')
+        countertype = countertype.replace(newline, '\n').strip()
+        s = s.replace(countertypes[0], '')
+        s = s.replace(counter_marker, countertype)
+    
+    return s
+
+
+def text_unpass_3_unary(s):
+    return utils.from_unary(s)
+    
+
+def text_unpass_4_cardname(s, name):
+    return s.replace(this_marker, name)
+
+
+def text_unpass_5_symbols(s, for_forum):
+    return utils.from_symbols(s, for_forum = for_forum)
+
+
+def text_unpass_6_newlines(s):
+    return s.replace(newline, '\n')

lib/utils.py

@@ -19,6 +19,8 @@ this_marker = config.this_marker
 counter_marker = config.counter_marker
 reserved_marker = config.reserved_marker
 reserved_mana_marker = config.reserved_mana_marker
+choice_open_delimiter = config.choice_open_delimiter
+choice_close_delimiter = config.choice_close_delimiter
 x_marker = config.x_marker
 tap_marker = config.tap_marker
 untap_marker = config.untap_marker
@@ -423,16 +425,14 @@ json_symbol_trans = {
     mana_json_open_delimiter + json_symbol_untap + mana_json_close_delimiter : untap_marker,
     mana_json_open_delimiter + json_symbol_untap.lower() + mana_json_close_delimiter : untap_marker,
 }
-json_forum_trans = {
-    mana_forum_open_delimiter + json_symbol_tap + mana_forum_close_delimiter : tap_marker,
-    mana_forum_open_delimiter + json_symbol_tap.lower() + mana_forum_close_delimiter : tap_marker,
-    mana_forum_open_delimiter + json_symbol_untap + mana_forum_close_delimiter : untap_marker,
-    mana_forum_open_delimiter + json_symbol_untap.lower() + mana_forum_close_delimiter : untap_marker,
-}
 symbol_trans = {
     tap_marker : mana_json_open_delimiter + json_symbol_tap + mana_json_close_delimiter,
     untap_marker : mana_json_open_delimiter + json_symbol_untap + mana_json_close_delimiter,
 }
+symbol_forum_trans = {
+    tap_marker : mana_forum_open_delimiter + json_symbol_tap + mana_forum_close_delimiter,
+    untap_marker : mana_forum_open_delimiter + json_symbol_untap + mana_forum_close_delimiter,
+}
 json_symbol_regex = (re.escape(mana_json_open_delimiter) + '['
                      + json_symbol_tap + json_symbol_tap.lower()
                      + json_symbol_untap + json_symbol_untap.lower()
@@ -441,14 +441,22 @@ symbol_regex = '[' + tap_marker + untap_marker + ']'
 
 def to_symbols(s):
     jsymstrs = re.findall(json_symbol_regex, s)
-    for jsymstr in sorted(jsymstrs, lambda x,y: cmp(len(x), len(y)), reverse = True):
+    #for jsymstr in sorted(jsymstrs, lambda x,y: cmp(len(x), len(y)), reverse = True):
-        s = s.replace(jsymstr, json_symbol_trans[jsymstr])
+    # See below.
+    for jsymstr in jsymstrs:
+        s = s.replace(jsymstr, json_symbol_trans[jsymstr], 1)
     return s
 
 def from_symbols(s, for_forum = False):
     symstrs = re.findall(symbol_regex, s)
-    for symstr in sorted(symstrs, lambda x,y: cmp(len(x), len(y)), reverse = True):
+    #for symstr in sorted(symstrs, lambda x,y: cmp(len(x), len(y)), reverse = True):
-        s = s.replace(symstr, symbol_trans[symstr])
+    # Since replacing doesn't remove the original match, have to do the right thing and go one
+    # at a time. We should probably use this method everywhere.
+    for symstr in symstrs:
+        if for_forum:
+            s = s.replace(symstr, symbol_forum_trans[symstr], 1)
+        else:
+            s = s.replace(symstr, symbol_trans[symstr], 1)
     return s
 
 unletters_regex = r"[^abcdefghijklmnopqrstuvwxyz']"

unscramble.py

@@ -1,232 +0,0 @@
-import re
-import codecs
-import sys
-
-# there should really be a separate file to store the character choices and such
-
-def from_unary(s):
-    numbers = re.findall(r'&\^*', s)
-    for number in sorted(numbers, cmp = lambda x,y: cmp(len(x), len(y)), reverse = True):
-        i = len(number) - 1
-        s = s.replace(number, str(i))
-    return s
-
-def cleanup_mana(s, pretty = False):
-    untranslations = {
-        'WW' : '{W}',
-        'UU' : '{U}',
-        'BB' : '{B}',
-        'RR' : '{R}',
-        'GG' : '{G}',
-        'PP' : '{P}',
-        'WP' : '{W/P}',
-        'UP' : '{U/P}',
-        'BP' : '{B/P}',
-        'RP' : '{R/P}',
-        'GP' : '{G/P}',
-        'VW' : '{2/W}',
-        'VU' : '{2/U}',
-        'VB' : '{2/B}',
-        'VR' : '{2/R}',
-        'VG' : '{2/G}',
-        'WU' : '{W/U}',
-        'WB' : '{W/B}',
-        'RW' : '{R/W}',
-        'GW' : '{G/W}',
-        'UB' : '{U/B}',
-        'UR' : '{U/R}',
-        'GU' : '{G/U}',
-        'BR' : '{B/R}',
-        'BG' : '{B/G}',
-        'RG' : '{R/G}',
-        'SS' : '{S}',
-        'XX' : '{X}',
-    }
-
-    untranslations_pretty = {
-        'WW' : 'W',
-        'UU' : 'U',
-        'BB' : 'B',
-        'RR' : 'R',
-        'GG' : 'G',
-        'PP' : 'P',
-        'SS' : 'S',
-        'XX' : 'X',
-    }
-
-    if pretty:
-        ldelim = ''
-        rdelim = ''
-    else:
-        ldelim = '{'
-        rdelim = '}'
-
-    manacosts = re.findall(r'\{[WUBRGPVSX\^]*\}', s)
-    for cost in manacosts:
-        if cost == '{}':
-            s = s.replace(cost, ldelim + '0' + rdelim)
-            continue
-
-        innercost = cost[1:-1]
-        newcost = ''
-        colorless_total = 0
-
-        # pull out unary countingses
-        colorless_counts = re.findall(r'\^+', innercost)
-        for count in colorless_counts:
-            innercost = innercost.replace(count, '')
-            colorless_total += len(count)            
-        if colorless_total > 0:
-            newcost += ldelim + str(colorless_total) + rdelim
-
-        # now try to read the remaining characters in pairs
-        success = True
-        while len(innercost) > 1:
-            fragment = innercost[0:2]
-            if pretty and fragment in untranslations_pretty:
-                newcost += untranslations_pretty[fragment]
-            elif fragment in untranslations:
-                newcost += untranslations[fragment]
-            else:
-                success = False
-                break
-            innercost = innercost[2:]
-
-        if pretty:
-            newcost = '[mana]' + newcost + '[/mana]'
-        
-        if len(innercost) == 0 and success:
-            s = s.replace(cost, newcost)
-        # else:
-        #     print cost
-        #     print newcost
-    
-    return s
-
-
-def unreplace_newlines(s):
-    return s.replace('\\', '\n')
-
-
-def cleanup_choice(s):
-    openbrackets = re.findall(r'\[[0123456789]+', s)
-    for openbracket in openbrackets:
-        number = openbracket[1:]
-        i = int(number)
-        if i == 0:
-            s = s.replace(number, 'choose one or more ~')
-        elif i == 1:
-            s = s.replace(number, 'choose one ~')
-        elif i == 2:
-            s = s.replace(number, 'choose two ~')
-        else:
-            s = s.replace(number, 'choose ' + number + ' ~')
-
-    clauses = re.findall(r'\[choose.*\]', s)
-    for clause in clauses:
-        newclause = clause.replace('=', '\n=')
-        s = s.replace(clause, newclause)
-
-    return s
-
-def forum_reorder(s):
-    fields = s.split('|')
-    # should see ten of em
-    if not len(fields) >= 10:
-        #print 'badlen ' + str(len(fields))
-        return s
-    # first and last should be empty, if we had | on the ends
-    if not (fields[0] == '' and fields [-1] == '\n'):
-        #print 'badfields ' + repr(fields[0]) + ', ' + repr(fields[-1]) 
-        return s
-    name = fields[1]
-    supertypes = fields[2]
-    types = fields[3]
-    loyalty = fields[4]
-    subtypes = fields[5]
-    pt = fields[6]
-    cost = fields[7]
-    text = fields[8]
-    if len(fields) > 10:
-        cost2 = fields[9]
-    else:
-        cost2 = None
-
-    new_s = ''
-    if not name == '':
-        new_s += name + '\n'
-    if not cost == '':        
-        new_s += cost
-        if cost2:
-            new_s += ' ~ ' + cost2
-        new_s += '\n'
-
-    if not supertypes == '':
-        new_s += supertypes + ' '
-    if not types == '':
-        new_s += types
-    if not subtypes == '':
-        new_s += ' ~ ' + subtypes + '\n'
-    else:
-        new_s += '\n'
-    # super special case, doubt it will come up
-    if types == '' and subtypes == '':
-        new_s += '\n'
-    
-    if not text == '':
-        new_s += text + '\n'
-    if not pt == '':
-        new_s += pt + '\n'
-    if not loyalty == '':
-        new_s += '(' + loyalty + ')\n'
-
-    return new_s
-
-def unscramble(s, pretty = False):
-    s = from_unary(s)
-    s = cleanup_choice(s)
-    s = cleanup_mana(s, pretty)
-    s = unreplace_newlines(s)
-    s = forum_reorder(s)
-    return s
-    
-
-def main(fname, oname = None, verbose = True, pretty = False):
-    if verbose:
-        print 'Opening encoded card file: ' + fname
-
-    if pretty:
-        print 'Using pretty [mana][/mana] encoding for mtgsalvation forum'
-
-    f = open(fname, 'r')
-    lines = f.readlines()
-    f.close()
-
-    if not oname == None:
-        if verbose:
-            print 'Writing output to: ' + oname
-        ofile = codecs.open(oname, 'w', 'utf-8')
-
-    for line in lines:
-        val = unscramble(line, pretty)
-        if oname == None:
-            sys.stdout.write(val)
-        else:
-            ofile.write(val)
-
-    if not oname == None:
-        ofile.close()
-
-    
-if __name__ == '__main__':
-    import sys
-    if len(sys.argv) == 2:
-        main(sys.argv[1])
-    elif len(sys.argv) == 3:
-        main(sys.argv[1], oname = sys.argv[2])
-    elif len(sys.argv) == 4 and sys.argv[3] in ['p', '-p', 'pretty', '-pretty', '--pretty']:
-        main(sys.argv[1], oname = sys.argv[2], pretty = True)
-    else:
-        print 'Usage: ' + sys.argv[0] + ' ' + '<encoded file> [output filename [p]]'
-        exit(1)
-