The testing suite

The windows version of Wagic comes with a basic testing suite for..testing purposes.

Basically, testing a new card or the rules engine is painful because you have to try the game several times, hoping that the situation you want to test will occur. For example, if you want to test what happens if you block a white creature with your black knight, you have to cross fingers so that the AI plays with a white deck, and attacks you with one of its creatures while you have a black knight in play.

Granted, you can trick the game, create some special decks with only black knights, but it still takes a long time.

The test suite allows me to test all of this automatically. I write simple test files that I feed as many times as I want to the testing suite, until I am satisfied with the result. One more advantage of the testing suite is that it allows to do regression tests. In other words, I can make sure that a new feature or a bug fix that I add to the game doesn’t break any previous feature.

Here is how it looks like when it runs :

Overall, the test suite is FAR FROM PERFECT, but it is a very useful tool.

If you learn to write some tests, you may also send them to me as “bug reports”, telling me : “hey, here’s this test file, it should be succesful, but obviously it fails because there is a bug in your engine, please fix it !”

This is the goal of this article. I’ll start with a simple example, then give you a list of keywords that can be used in the test suite.

Let’s get started

Ok, so if you want to use the test suite, you need the windows version, that can be either downloaded, or compiled by yourself from the sources. In order to run correctly, there must be a Res directory in the same folder as the exe file. You can copy the Res directory from your PSP wagic game in this Res Directory.

Actually, the only difference between the windows “Res” directory and the PSP “Res” directory is the following : a “test” subdirectory is included in the windows Res directory. If it’s not there, you can create it, but it’s better if you already have it, as you will be able to see some examples that will help you.

If you have all of this setup correctly, you should be able to run the windows version of Wagic, click on “play” and select “test suite” from the menu, and it should run something, and generate a file called “results.txt” in the test directory.

Let’s create our own test

Ok, say you want to test if Shock actually does 2 damages to its target. This is a real life example, because in one beta version of Wagic, Shock was actually dealing 0 damage because of a bug.

The test suite has lots of flaws, so sometimes you have to think “out of the box” to create tests. To test if Shock deals 2 damage, we are going to cast it on a target 3/2 creature, and make sure the creature ends up in the graveyard.

If you think about it, that’s not enough to see if Shock actually deals 2 damage. It would put the 3/2 creature into a graveyard even if it was dealing 3 damages… but at least, we will be sure it doesn’t deal less than 2 damage, which is already a good thing !

Go into the test directory, and create a new text file, that you will call for example, shock.txt

You will also create a file named _tests.txt. Open this file with notepad, and in it, just write

shock.txt

This is to tell the test suite that we want it to run the test for our file shock.txt

In shock.txt, copy and paste the following :

#Testing Does shock really do 2 damages (against a /2 creature) ?
[INIT]
FIRSTMAIN
[PLAYER1]
hand:129732
manapool:{R}
[PLAYER2]
inplay:129471
[DO]
129732
129471
[ASSERT]
FIRSTMAIN
[PLAYER1]
graveyard:129732
manapool:{0}
[PLAYER2]
graveyard:129471
[END]

You can now try to run the test suite in the application, and it should play a Shock card on a poor Snapping Drake, then end just like a normal Wagic game.

It should also create a text file named “results.txt”, which contains :

=============================
shock.txt
==Test Succesful !==

Which means that the test ran correctly and no problem was found.

That’s great, but what does our test actually do ?

A test is divided into 3 states : intialization state, action, and verification state. These 3 states are described with the following tags :

[INIT], [DO], and [ASSERT]

Let’s have a look at our INIT state :

[INIT]
FIRSTMAIN
[PLAYER1]
hand:129732
manapool:{R}
[PLAYER2]
inplay:129471

So what does it say ?

FIRSTMAIN means we want the game to start directly at the first main phase of player 1

all information below [PLAYER1] gives information on how player1 starts the game, and the same goes for [PLAYER2].

Player 1 starts the game with a manapool containing one Red mana. This will allow to pay for the Shock spell without having to tap some lands. He also has card id 129732 in his hand. If you look at _cards.dat in the 10th edition set, you wil see that is the id for Shock :)

Player 2 starts the game with card id 129471 (Snapping Drake) in play.

So that’s it, player 1 has 1 red mana in his pool, a shock card in hand, and player 2 has a Snapping Drake (3/2 creature) in play. what’s gonna happen ? Well, that’s the job of the action state, described after the [DO] tag :

[DO]
129732
129471

What does that do ? Well it tells the engine to click on card 129732 (shock), then on card 129471 (snapping drake) . Because we have one red mana in our mana pool, it casts the Shock ,then selects the creature as its target. At least, that’s what we expect it to do.

So how do we make sure that it works ? Well we tell Wagic what we expect the game to look like after the actions occured, and Wagic will compare what we expect to the actual state of the game. We write our expectations after the [ASSERT] tag :

[ASSERT]
FIRSTMAIN
[PLAYER1]
graveyard:129732
manapool:{0}
[PLAYER2]
graveyard:129471
[END]

it is still the first main phase.

We expect player 1 to have an empty manapool (the red mana was spent for Shock), and Shock in his graveyard.

We expect player2′s Drake to be in his graveyard.

When the test runs, the Test suite will compare that to what really happened. If it fails, the “results.txt” file will contain information on what failed. If the final state of the game matches what we were expecting, then the test is succesful !

finally, all tests ends with an [END] tag.

More information

Acceptable values for the phase names are the following:

untap,upkeep,draw,firstmain,combatbegin,combatattackers,combatblockers,combatdamage,combatend,secondmain,endofturn,cleanup

Acceptable values for the zone names are the following: inplay,hand,library,graveyard

Other acceptable keys for the players’ state descriptions are life and manapool. The default value for life is 20, and the default manapool is empty.

Card ids must be separated by comas. Due to the current limitations of the engine, you cannot have several instances of the same id in the game. For example, you cannot have 2 plague rats in your test, unless you use one from the 10th edition and one from the Revised edition.

keywords accepted in the [DO] state are the following:

  • any card id that was introduced in the INIT state
  • next (to go to next phase)
  • eot (to go to next player’s turn)
  • yes (to interrupt)
  • no (to refuse to interrupt)

The yes/no thing is a bit complicated. The default behaviour is that there is no interruption unless you specify it with “yes” at some point.

The _tests.txt file can contain as many test files in it (separated by a newline). The test suite will run all of them sequentially and put all the results in results.txt

every line starting with a # in the test files is a comment

To target an ability or something that is on the stack, use the id of its source card.

Have a look at the tests directory provided with the sources or the windows version for more examples

  1. Incantus’s avatar

    Hey, that looks like a neat idea! The way I’ve been testing Incantus is setting up the game situation with a special toolbox card and recording the full replay. Then each of these replays acts as a test -but I like how you can specify a final condition. Also, based on your description of actions, it sounds like we’ve implemented the coupling between rules engine and UI the same way.

    Reply

  2. wololo’s avatar

    Hey Incantus, always nice to have some feedback from you.
    I remember seeing your “testing card” somewhere on a forum.
    I’m guessing it would be easy to add some kind of “final condition” to your recordings as well. for example, when you end the recording, you can save the state of the game, and use that as a reference to compare with replays.
    Then when you launch the replay, rather than making sure that everything went fine “with your eyes”, you just let your app do the job :)

    Reply

Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>