Friday, December 23, 2011

Please read: a personal appeal from the FOnline: 2238 Developer Lisac Twok


I feel like living the last of my days in a polluted, unclean world, infested with bugs, glitches and crashes.

Imagine the world where every human being on the planet could wipe their virtual reality as they want, whenever they want. Although we're currently far away from that vision, it doesn't mean we have to give up on our dreams! Wipe, my brothers and sisters, wipe is our reality.

With the latest features being tested and discussed, with those last bugs being squashed or pulverised with pulse rifle, with each and every stinking breath of the stuck NPCs that break your quests - we are one step closer to our vision! As I write these words, the critical amount of bits and bytes are being accumulated in Paris, waiting to come into contact with a void function that shall release the Big Bang and bring us the next wipe, the salvation! The last of our days are coming, beware!

Not soon, but very soon, the wipe will bring the end of days as we know it and bring a new life to the brave world that has to be born on the ashes of the old one. Very soon.

Gather your forces. Wake up your sleeping moles. Send a telegram to Rangers. Take your minigun from the rusty tribal chest. Get your granpa stand by your side. Because, very, very soon...

It's about to wipe.

Saturday, November 19, 2011

Wanted: Donations - Dead or Alive

Like the title says, we are in dire need of donations for keeping the server running for the upcoming three months. The deadline is 6th December and the server fundings are currently short for ~80 €. Any amount is greatly appreciated and shall help the developers to concentrate on finishing the new features for the upcoming wipe, rather than to see their efforts being hampered by failing to oblige a quarter invoice for the server maintenance.

As always, Cvet & Rotators are grateful for your donations, as well as for any other great work and feedback provided by the FOnline communities.

Friday, October 14, 2011

Improving the vintage

Our old and rusty faction terminal, the very first feature of 2238 could use some improvements. While we started the project few years ago, we couldn't hope for client scripts, nor for easy scripting client-side GUI. The only possibility was to use existing dialog functionalities.

Over the time it became obvious, that using such 'interface' is really big pain, and the usability of our game is not up to modern standards by a mile or two. But we have left that part unchanged, as there were other things to do (and still are). The upcoming wipe will not change it drastically, but it will surely improve the way factions are managed.

Multiple bases

There were so many factions created since the last wipe that we've run out of names. Several times. Most of them were only created to alleviate one problem - a faction is meant to have only one base, and all members have access to it, some were created when factions were creating an alliance, and some only because some rich player wanted private workbench and whores.

Of course, not all problems will be fixed immediately, but we're going to allow one faction to own multiple hideouts.

Access


To simplify things, the access will be resolved on per-rank and per-status basis. This means it won't be possible to do it per-player and per-faction (not yet at least), but still, that simple modification should make life of faction leaders much, much easier, and it should increase the importance of proper rank assignment.

Easier status change


We've also added the ability to create list of known factions. Such list is then used to alter the status of all members of said faction at once.It should work well with per-status access for bases. While we still have basic statuses: friend/neutral/enemy/invited, the system should allow for more so we could improve one that. Question is, whether it's gonna be worked on, or whether web-faction-terminal will takeover.

Summary


So why we haven't reworked it from the scratch, possibly creating client-side interface for it? Well, there are few reasons. First of course are, the plans for web faction terminal (which does not rule out easy client-side interface alongside with web interface). The other is that we're still working on GUI internals, our GUI code differs from the default SDK, and we're working on a ways to make them compatible. Sometimes it's good to work on internals first.

Wednesday, September 14, 2011

Living without client

Recently I've written about the possibility to test the features by writing code, that test them. As I've started doing some refactoring in our code, I immediately wanted to use that feature. Let's see how it worked out...

Gathering, once again


As we already mentioned, we are changing (again) the way resources will be gathered. This of course required yet-another-refactor of that part of code. Last time I was doing this, it was rather boring task: refactor script of one facility, start server, login, spawn some of those facilities, spawn myself needed item, use it, check. Rinse and repeat (sometimes you may hot-reload the script, but you need to still test it manually).

This time I decided to write set of test cases for every 'use event' that results in resource being acquired. And then, those tests will check the stuff for me, so that I do not even need to fire up the client. Nifty!

class Plant : IFacility
{
Item@ item;
uint16 resource;
uint batch;

Plant(Item& item, uin16 resource, uint batch)
{
item.SetEvent(ITEM_EVENT_USE_ON_ME, "_UseItem");
item.SetEvent(ITEM_EVENT_SKILL, "_UseSkill");
item.SetEvent(ITEM_EVENT_FINISH, "_Finish");

@this.item = item;
this.batch = batch;
this.resource = resource;
this.batch = batch;
}

int get_Amount() { return item.Val1; }
void set_Amount(int val) { item.Val1 = val; }

bool UseItem(Critter& cr, Item& usedItem)
{
uint pid = usedItem.GetProtoId();
if(pid == PID_KNIFE || pid == PID_COMBAT_KNIFE || pid == PID_LIL_JESUS_WEAPON || pid == PID_THROWING_KNIFE)
{
if(IsOverweighted(cr))
{
cr.SayMsg(SAY_NETMSG, TEXTMSG_GAME, STR_OVERWEIGHT);
return true;
}
else
{
cr.AddItem(resource, batch);
cr.SayMsg(SAY_NETMSG, TEXTMSG_TEXT, text);
}
return true;
}
else return false;
}
}


This is an excerpt from a class that represents plant facility that we may cut with knife to use the needed resources. It's represented in-game as an item, and we are initializing it in item script in following way:
void item_init(Item& item, bool firstTime)
{
AddFacility(item, Plant(item, PID_FIBER, 2));
}

It uses the pattern I've described in following post. The Plant constructor assigns the events to the item:
// critter uses skill on facility
bool _UseSkill(Item& item, Critter& cr, int skill)
{
IFacility@ facility = GetFacility(item);
if(valid(facility)) return facility.UseSkill(cr, skill);
return false;
}
// critter uses item on facility
bool _UseItem(Item& item, Critter& cr, Item@ usedItem)
{
if(valid(usedItem))
{
IFacility@ facility = GetFacility(item);
if(valid(facility)) return facility.UseItem(cr, usedItem);
}
return false;
}
// :>
void _Finish(Item& item, bool deleted)
{
IFacility@ facility = GetFacility(item);
if(valid(facility)) RemoveFacility(item);
}


Testing stuff


First thing we would probably like to test is, whether critter really receives the items if using proper tool on that facility.Let's first prepare a helper that's gonna greatly reduce the amount of repeated code:
void mock_CritterAddItem(Critter& cr, uint16 pid, uint count)
{
CallExpectation("CritterAddItem_" + pid + "_" + count);
}

class Fixture
{
Critter@ cr;
Item@ tool;
Item@ item;

Fixture(uint16 tool, string@ script)
{
@cr = MockCritter(1);
Mock("Critter::AddItem", "mock_CritterAddItem");
@this.tool = MockItem(tool);
@this.item = MockItem(1); // this pid does not matter
item.SetScript(script);
}

IFacility@ get_Facility() { return GetFacility(item); }
}

And now, the first test:
void test_PlantFruitGivesProperResource()
{
Fixture fix(PID_KNIFE, "prod_plant_fiber@item_init");
fix.Facility.Amount = 100;
ExpectOnce("CritterAddItem_" + PID_FIBER + "_" + 2);

fix.item.EventUseOnMe(fix.cr, fix.tool);

VerifyExpectations();
}

The most important part is the mock for Critter::AddItem, which just stores the expectation named in following manner: CritterAddItem_PID_COUNT. In above test function, we're preparing a context consisting of:
  • critter
  • knife
  • item representing the plant
We invoke the event that would be normally invoked when player or npc would use a knife on the plant (EventUseOnMe). The call process normal behavior, but when it encounters call to Critter:AddItem it calls our mock instead. So we expect that our function is gonna call this function with PID_FIBER and 2 as parameters. Looking into the code of Plant::UseItem, we see that indeed it should work. Plant::batch is equal to 2, so it's gonna execute exactly what we wanted. Running the test confirms that. Yay!

Code is already bugged


Oh great:( I'm writing some code snippets that already contain bugs. So what? Let's fix them, but first let's write a test that confirms them, check if it fails, and then fix the bug and re-check the test:
void test_PlantFruitEmpty()
{
Fixture fix(PID_KNIFE, "prod_plant_fiber@item_init");
fix.Facility.Amount = 0;
ExpectNonce("CritterAddItem_" + PID_FIBER);

fix.item.EventSkill(fix.cr, -1);

VerifyExpectations();
}

Since there are gonna be no timeouts on characters, the facilities itself should somehow limit the amount of resources that we can obtain (and they will be regenerated over time). We've already had implemented some Amount property in our Plant class, but the code didn't care about it. And this test proves that - we expect that AddItem is not gonna be called with PID_FIBER (and whatever count) at all, but it is, because we do not check for amount left in UseItem method. Let's fix it:

            // ...
else
{
if(this.Amount > 0)
{
cr.AddItem(resource, batch);
cr.SayMsg(SAY_NETMSG, TEXTMSG_TEXT, text);
}
}
// ...

And let's run the test again. It succeeds! We've fixed a bug, we tested it. Without doing manual labor.

The method I'm describing here, really makes me happy developer, as I do not need to spend time on manual testing, and also I am preparing more and more tests that's gonna test whether existing features are still working in the future (you know, some change in other area of code can really break the other part of code, in an unexpected way, better catch at least some of such bugs).

There is still one bug left in the code (probably much more, I wrote this post without compiler). I'm leaving it for readers to find out and write test code that proves it!

Friday, September 9, 2011

Walking the way of bombs

It isn't exactly news that various towns in the wasteland are racked with the crazy nature of suicide bombers. In real life and in FOnline: 2238. This particular way of entertainment is in almost all cases only entertaining for a very small minority of people and due to this, discussions internally and externally are kind of an unofficial sporting event by now.

Anyway, to make it short: In the future we want to change the way explosives are working and used in the game, to limit (but not totally remove) the amount of suicide runs. To accomplish this, we internally agreed on the following ideas:
  • Guards can "sniff" activated explosives (C4, Dynamite, and placed Landmines).
  • The player can't activate explosives (C4, Dynamite, and placing Landmines) whenever a guard is around or if the player is in guard range. (It simply doesn't work, with an entry in the message log.)
  • The player can activate explosives (C4, Dynamite and placing Landmines on the ground) whenever a guard is not around or if the player is not in guard range.
The few rules should result in harder suicide runs, without making them impossible, because the players action now will be as the following:
  1. The PC has to activate the explosives far away from guard NPCs,
  2. then the PC has to run up to their targets (NCR bazaar....),
  3. and then hopefully explode there before a guard can kill the bomber and take the explosives (disarming them).
These simple changes should make the game better than with the current system (guard NPCs attack players with active explosives), which is pretty much useless, as especially dynamite can be triggered in around one second and nobody is able to react on it this fast.

Of course, the subject is still open for discussion and suggestions in the forum (for example, there were ideas about involving demolition expert profession in this process). Also we aren't quite sure about when exactly we are able to take this over into the game. It probably won't happen before the next wipe, but I'll hope we can push it afterwards.

Tuesday, September 6, 2011

Testing your server scripts

I've finally gotten around the way to unit test FOnline scripts. I'm happy about it, though it's still a prototype, and I've only written few tests just to try it out.

Why test your code and why should we care? Well, it's very popular approach nowadays, to write a code that tests your code, just to be sure everything works as expected (and it's hard to keep track of everything with large codebase). It's common misconception that's not feasible for games, after all, how can you test something as complex as interactive program - and even worse - with multiple players?

The answer is that you're not writing a code to test your whole game. You're not gonna write a bot that plays it, does all the quests and check if everything still works (and even looks for holes in the walls, though that'd actually be cool). You write the code to test only portions of it, the small pieces, the units. Hence the name: unit testing. There's of course lots of materials on it over the internet, so we're not going to dive into the details, I just describe what were the problems, and how we solved it, so that we can finally try unit tests.

Isolating the code


When you look at the code you'd like to test, first thing that come to mind is that it's probably too complex and involves too much dependencies to be easily testable. Let's look at the example:
bool critter_use_item(Critter& cr, Item& item, Critter@ targetCr, Item@ targetItem, Scenery@ targetScen, uint param)
{
bool isPlayer=cr.IsPlayer();
bool useOnSelf=(not valid(targetCr) && not valid(targetItem) && not valid(targetScen))

// Radio
if(FLAG(item.Flags,ITEM_RADIO) && useOnSelf)
{
if(isPlayer) EditRadioSettings(cr,item);
return true;
}
// ...
}

Say we need to test that when player uses the radio item, he can edit its settings. Unfortunately, to invoke critter_use_skill int this context we need:
  • player
  • item
OK, we may easily spawn radio, but how can we spawn player? Scripting subsystem does not allow us to do this. We could just spawn critter, but then we see script cares about IsPlayer() condition. And then, how do we test whether player actually saw the edit radio interface or not? There is that elusive EditRadioSettings function being invoked there, maybe we could somehow detect that it's been called? But how?
Due to above limitations, unit testing your code seems just not feasible. Surely, there must be some solutions, we just need to go out and learn how the world is doing this.

Mocks


The concept is simple. If you need to provide something for the tested code, that's not the part of the test itself but constitutes to the test context, provide a mock. A substitute for real object, a substitute for a function that's gonna allow us to isolate the tested code to the form that may be safely run from within unit test context. As far as above example goes, we would need three mocks:
  • player - Critter object with overriden IsPlayer() method that's returning true (another mock actually), or with fields set in a way that IsPlayer() is returning true
  • an item - Item object with fields telling us it's a radio
  • a function to detect that EditRadioSettings has been called
But we can't do this in AngelScript. Not without any modifications.

Mock library

At first I wanted to write some library that would be able to load the script, compile it and execute test functions providing overriden implementations for functions we wanted to mock. However, the AngelScript engine does not allow to re-register any function, so that once we've got our engine set up, we can't register any mocks. We thought of some workarounds, but after some time I decided to modify the server source directly (its angelscript source, to be precise), I suspected it would be very minor modification - so that maintain costs are minimal in the future (keeping it up to date with every server update).

Tests Runner server

After some fiddling in AngelScript engine code, I found out that I can easily 'redirect' function call to another script function (whether the original call was meant to invoke script function, or native (engine registered) one. Moreover, it turned out that method calls can be simply redirected to function calls (providing the first argument is the object passed), without any extra work! This way call to bool Critter::IsDead() could be handled by bool critter_IsDead(Critter& cr). Nice!
How does it affect our testing capabilities? To put it simply, we ended up with solution that may be used for testing generally, not only for unit tests. Unit tests are designed to test your small pieces of code out there, but hence we are running our tests in full fledged server, we may as well test the system more broadly, we may check how different pieces interact together - whether it works as a whole or not (though I admit, it wasn't my initial goal - I just wanted simple unit testing facility, not integration tests).

But let's get back to the example. Let's start from the last mock we needed, namely, a function substitute for EditRadioSettings. Say, we just want to know, whether the function has been called or not. Let's mock it, and make it so our substitute will indicate that's been called:
void mock_EditRadioSettings()
{
CallExpectation("EditRadioSettings");
}

For above example to work, we need to know what CallExpectation function is doing. It simply increases the call counter, that's stored in some dictionary under the index that's been passed as argument. Later on, we may check that the counter is equal to 1 - that means our function has been called as expected. With such dictionary at hand, we should also define more helpers: Expect(funcname) and VerifyExpectations(). First one is used to remember the fact, that we want the function funcname to be called, and the latter (called at the end of the test), will check it. In fact, we should have the ability to specify the numbers of the call we're expecting:
  • ExpectOnce(funcname) - VerifyExpectations will succeed only if function has been called once
  • Expect(funcname, count) - success only if called count times
  • ExpectNonce(funcname) - success if hasn't been called at all
Once we've got our mock and our helpers ready, we're gonna substitute our real function:
Mock("EditRadioSettings", "mock_EditRadioSettings")

This function will remember that we want to redirect the call to EditRadioSettings to a function mock_EditRadioSettings.

Ok, what about other mocks? We said that we want an item, and a critter with specific properties and/or we may provide mocks for their methods as well. For this I've implemented simple MockCritter and MockItem functions that spawn the needed objects with only basic properties filled, but for the case of this example, let's assume those are just clean objects - with all properties zeroed out, rest will be handled by method mocks:
bool mock_True()
{
return true;
}


And now, our test function (with needed mocks) in full glory:
bool mock_True()
{
return true;
}
void mock_EditRadioSettings()
{
CallExpectation("EditRadioSettings");
}

void IfPlayerUsesRadioEditRadioSettingsShouldBeCalled()
{
// prepare
Critter@ player = MockCritter();
Mock("Critter::IsPlayer", "mock_True");
Item@ radio = MockItem();
radio.Flags = ITEM_RADIO;
Mock("EditRadioSettings", "mock_EditRadioSettings");

// call tested function
critter_use_item(cr, radio, null, null, null, 0);

// verify
VerifyExpectations();
}

Voila! Now, when we call critter_use_item, it will first call Critter::IsPlayer(), which returns true, even if our Critter structure might have not indicated this, but our mock did. Later, it will check the flags (we've set it on our mocked item) and call the EditRadioSettings, which in fact calls mock_EditRadioSettings. This sets our expectations counter to 1, which is then verified by VerifyExpectations. And then it announces success - we've got our first trivial unit test passed!

The solution we've got is in very early stage, I'm gonna try and involve it in some of the 2238 code testing. We will see if it turns out to be useful.

Enough of this mockery for now!

Sunday, August 28, 2011

Server issues

We're having some problems with the server. We hope to solve it today, but there's no guarantee.

Thanks for your patience.

Thursday, August 25, 2011

Faction data - asynchrony

This is gonna be another post from the series on new approach to the faction data storage, and most technical one. It's gonna describe process of creating a native extension for FOnline server application, that will allow communication with external database, in our case, CouchDB. It's already implemented and is being tested, so I guess I can safely write about it.
The endeavour was quite an interesting one with lots of crazy ideas, few dead-ends, but eventually the solution turned out to be nice and clean.

So, for those interested in hacking the guts of server application, extending it to perform things you haven't imagine, let's read!

Data synchronization

First, we need to make brief summary, of what we want to accomplish. It's been described in previous posts, basically, we need a way to synchronize some game data (players properties, variables) with the data stored in CouchDB. As with every communication, there are two directions we may want to communicate:
  • server asks CouchDB for data it needs to synchronize
  • CouchDB tells server that it may want to synchronize
Since we'd rather extend server application, than to write additional tier for CouchDB that would communicate with server, we of course choose the first approach. The data we need to synchronize are:
  • factions
  • players' documents from factions databases
So basically we may need two functions to do this:
  • GetFactions - gets all factions with their properties
  • GetPlayerFile - gets player file stored in certain faction database
Let's focus now on the first one.

Main factions database

Info on our fac tions is going to reside in central database, called factions. This database is going to contain documents for each of the registered faction (for simplicity sake, we're not going to write about registering new faction process). Example document:
{

"_id": "Brotherhood Of Steel",
"id": 2,
"database": "brotherhood_of_steel"
}

The _id field is the unique identifier for the document, so it's also the faction name. The id (without leading underscore) is the number assigned during registration process, and is the number any member of that faction carries on his player character, so that game logic may react appropriately. The database field is the name of the faction database, that reside on the same CouchDB server. It needs to be different than the faction name, cause CouchDB does not allow certain characters in database names.

In-game, our factions are going to be represented by following class:
class Faction

{
int id;
string name;
string database;

Faction(int id, const string& name, const string& database_name)
{
this.id = id;
this.name = name;
this.database = database_name;
}

int get_Id() const { return this.id; }
string get_Name() const { return this.name; }
string get_Database() const { return this.database; }
};

And we're storing the objects of that class in some array. So, the only thing we need, is indeed a GetFactions function, that would just fetch data from factions database and fill up our array.

But before we dive into code, let's pause for a moment. It is external storage, it's http protocol. By no means it's going to be fast. We can't just write an extension function that is going to perform http request and return the results so that we can process them further in scripts. It would block the server during the call for at least few miliseconds, but it could even be few seconds, it depends. We need a way for asynchronous calls.

Asynchronicity

What's that, and why it's not The Police album? The principles of asynchronous calls are very simple. The call is being performed, and the function immediately returns to the place from where it's been called, allowing main thread to resume its job without waiting for function results (while the function that's been called asynchronously is being executed in other thread). Great, but we are interested in those results, so we need a way to operate on them. Traditionally, this issue is being solved by callbacks. To put it simply, you are defining another function, that will be called when our function that has been called asynchronously finishes its execution. Such callback takes what's been returned by the asynchronous function as its argument, and perform whatever logic we wanted to be performed on that result in the first place.

But we're not at home still, let's check it. This is going to be our hypothetical extension function, that is going to perform http requets to CouchDB, and return data for further processing (pseudocode, native):
void AsyncGetFactions(callback)

{
QueueThreadPoolTask(task, callback);
}
void task(callback)
{
string res = CouchDB::Get("factions");
callback(res);
}

This is how asynchronous functions look like. They only queue the task to be performed later on other thread. The function is taking a callback as an argument, performs http GET request, and calls the callback to operate on data. But we want our callback to be AngelScript function, what we may do about it? Common solution in FOnline scripting, is to pass the name of the function as string(pseudocode, AngelScript):
AsyncGetFactions("callback");


void callback(const string& result)
{
// operate on what's been returned. It's JSON, but we're not going to dive into that matter now
}

Familiar? Should be - CreateTimeEvent works this way. We may rework our native extension to something like this:
void AsyncGetFactions(const string& callback)

void AsyncGetFactions(callback)
{
QueueTask(task, callback);
}
void task(callback)
{
string res = CouchDB::Get("factions");
CallAngelScriptFunction(callback, res);
}

The last line uses a function, that would fetch the function from AngelScript engine, knowing its name, and would call it passing our result as argument.

But hey, something is still wrong here. Remember the task is being executed in another thread. Whoops, and a big one. That means our callback function will be executed on that other thread as well, which means simultaneously with whatever other game logic is being executed at that time. This of course may lead to the data corruption, hard to track errors, unexpected behaviour, all that fun. What can we do about it? Traditionally, you may assure that the function that's going to be run in parallel with main code, does not operate on the same variables that the main logic thread does. But that would be very hard to achieve, after all, for our callback we would probably like to reuse whatever code we alredy have in our codebase, and I bet most of it is not thread-safe at all. Is there something that can save our asses in our quest to achieve our goal?

Message passing concurrency

Some clever folks found out, that to avoid problems with multithreaded code, it's best to avoid the code that's simultaneously executed and operates on same set of data - brilliant, isn't it. Instead, it's better to have totally separated units, and only have them communicating between by passing messages to each other.
Notice, that we may safely run our game logic in parallel with CouchDB http request, as those are totally separated. It's when we want to operate on data returned, when we're running into problems. So why not return the data somehow to the server application, and let server logic be the one responsible for reading it and performing the funcionality? That way, the callback logic will be executed by main thread, so no problems here. For that, we need a queue with messages, that we will be using to exchange data between our asynchronous functions, and the server logic. Then we're gonna be able to leave the message (from the other thread), and fetch the message (from main logic thread) to further operate on it (still in main logic thread). That way, the only structure shared between threads will be the queue itself, but it's not a problem to write such thing to be perfectly thread safe:
void PushMessage(msg)

{
lock(messages);
messages.push(msg);
unlock(messages);
}
msg FetchMessage()
{
lock(messages);
msg = messages.pop();
unlock(messages);
return msg;
}

Above pseudocode shows us, how we can synchronize the reads and writes on our queue, to assure that only one thread is accessing them at a time. The way we do it depends on libraries we're gonna use, I do not want to dive into the details here, but the principles are the same:
  • lock puts a lock on some structure
  • unlock takes that lock away from it
  • there may be only one lock at the structure at a given time, so next call to lock (performed from other thread) is going to be blocked and wait till it's being unlocked
That gives us thread-safety for the queue. Let's use it now:

void AsyncGetFactions()

{
QueueThreadPoolTask(task, callback);
}
void task()
{
string res = CouchDB::Get("factions");
// iterate over res content, to send message for every faction contained there
for(...)
PushMessage(new Message(MESSAGE_FACTION, res));
}


And in script:
void UpdateFactions() // we could call it from main@loop() for example

{
AsyncGetFactions();

while(true)
{
Message@ msg = FetchMessage();
if(!valid(msg)) break;

if(msg.type == MESSAGE_FACTION)
ProcessFaction(msg.res);
}
}
void ProcessFaction(string res)
{
// parse our input, determine faction properties, check if already in array
// if not, add it there, otherwise - update
}
Notice, that we're calling AsyncGetFactions in each loop and after that we're fetching all messages. But the messages won't probably arrive at that moment, for that we will have to wait. And, while we are waiting, there is no point in calling AsyncGetFactions over and over again. We need to orchestrate somehow our calls, we can do this with simple boolean switches:

bool GettingFactions = false;


void UpdateFactions()
{
if(!GettingFactions)
AsyncGetFactions();

while(true)
{
Message@ msg = FetchMessage();
if(!valid(msg)) break;

if(msg.type == MESSAGE_FACTION)
ProcessFaction(msg.res);
}
}

It's that simple. But we need a way to notice the game logic about the fact, that our asynchronous extension has finished with getting factions. For this, we will extend it to send yet another message, after all faction messages have been sent:
void task()

{
string res = CouchDB::Get("factions");
// iterate over res content, to send message for every faction contained there
for(...)
PushMessage(new Message(MESSAGE_FACTION, res));
PushMessage(new Message(MESSAGE_GET_FACTIONS_DONE));
}


And then in scripts, we will switch our variable:
void UpdateFactions()

{
if(!GettingFactions)
AsyncGetFactions();

while(true)
{
Message@ msg = FetchMessage();
if(!valid(msg)) break;

if(msg.type == MESSAGE_FACTION)
ProcessFaction(msg.res);
if(msg.type == MESSAGE_GET_FACTIONS_DONE)
GettingFactions = false;
}
}

By setting GettingFactions to false, we're indicating that we are no longer running AsyncGetFactions in the background, so we can safely call it again next time. And our loop is chewing whatever messages arrive there all the time. All in parallel, all in thread-safety.

I hope it was interesting read, maybe not too detailed and with lots of pseudocode, but I did want to show the idea, not the implementation specifics.

Sunday, August 21, 2011

Tremendous bug!

It didn't take you long to find out what was wrong in the solution I presented in http://fonline2238.blogspot.com/2011/08/what-would-be-point-of-having.html. In fact, you were faster than me fixing it, after the bug introduced by it occured on 2238 server.

Let's start from beginning. In first iteration of gathering (called production) back then, we needed a notion of facility that player could use to obtain some resource. A facility could be one item (item pretending to be scenery object actually, it had to be item because of the scripting possibilities), or collection of items. So, a barrel in Modoc was one facility and all the plants somewhere on the crop could be one facility as well. And when the player obtained some resources from it, we needed to apply timeout for it (store it on the item somewhere). Also, we needed even more scripting possibilities - rotgut still for example. Perfect situation to use the wrapping pattern:

interface IFacility

{
bool CheckTimeout();
void Use(Critter& cr);
}
class BarrelFacility : IFacility
{
Item@ barrel;
BarrelFacility(Item& item)
{
this.item = item;
}
bool CheckTimeout()
{
return item.Val1 > __FullSecond;
}
void Use(Critter& cr)
{
if(CheckTimeout())
{
// give junk
item.Val1 = __FullSecond + 60; // 60 seconds of barrel cooldown
}
}
}


Of course above code is simplification, and we needed to wrap the item for other reasons as well - for example having one facility object per many items, but it's not the point of that post to explain all of it.
So, we were adding the BarrelFacility objects in barrel script:
void item_init(Item& item, bool)

{
AddFacility(BarrelFacility(item));
}

And we had that global array of IFacility objects. I didn't implement removal procedure, simply because, all those facilities were in towns and other fixed locations. So, they were destroyed only when server was being shut down, and were created only on server start. Additionaly, because of the fact there was only few of them, and I think I haven't got any unused Item::ValX field, I haven't been storing the index to the array for fast fetch (like I wrote in previous blogpost on that), but I was iterating over whole array to find proper object. It hadn't got big performance impact, but one day we decided we need to change the system: we introduced facilities into encounters. Boom!

Yes, that was it - the IFacility objects were created very often, and never being destroyed. The array was growing and growing, and the fetching procedure was O(n) in complexity instead of O(1). Terrible stuff. When players started to jumping into encounters, after some time the array was so big that every time someone wanted to use facility, the algorithm had to go through thousands of iterations to find proper index in the array. And the appending was quite painful as well - the underlying array object needs to copy all elements on resize, as it does not reserve space up-front.

And that caused a lag. The Lag in fact, it was so big that it had destroyed the fun from early beta days. And we haven't fixed it that fast. To put it simply - I forgot that the objects are never removed, and found that out later... A little embarassing, I know, but now, when looking at it after such amount of time - it's quite funny to recall.

And quite funny story about 2238 development I think.

Wednesday, August 17, 2011

Wrapping a wrap

What would be the point of having developer's blog, if there wouldn't be anything on programming. Oh yeah, that's where the things got hot. This is where it works, this is where it fails.

Since launch of the SDK, probably lots of people started struggling with that fancy scripting language - AngelScript. I remember I did. Experienced some successes, failures, helped finding some bugs, worked my way out through this language, started appreciating it, started even liking it (now I see it was indeed quite a good choice). That's why I think we could from time to time talk about scripting in FOnline SDK, talk about the AngelScript, share some common pitfalls, solutions that worked out good, and even the ones that weren't.

Of course basics of angelscripting is required, there won't be any tutorials whatsoever.

Objects

Blah blah blah, everything you can model in your OO language is an object, yeah, we already know that and it sucks (I'm getting a little bit off topic right here). Ok, it does not suck, in fact we're gonna often rely heavily on that fact, and we're gonna put large part of our logic into objects. And we're gonna tie those objects to the in-game objects (critters, items, scenery, maps, locations).

Say we've got a class for objects that are gonna make us some nice abstraction layer over default item functionality:

class SuperFoo

{
Item@ item;
SuperFoo(Item& item)
{
@this.item = item;
}

void Bar()
{
item.Val0++;
}
}

What's going on here? We wrap an item type in some class, and we're adding some useless code to operate on it? Why the hell would we want it? Well, think about the interface you've just gained and you can use throughout your scripts. You may now call: something.Bar() instead of item.Val0++, which shows what you want to do, and not how are you going to do. That's the difference if you want to write good code. Less imperative, less verbosity, less bugs (yes, I know that example suck, and you would want that for far more reasons - think about having some additional data, that's not stored on the wrapped item, but it's one of the class' fields - it's not saved anywhere, but still may be useful).

Wait, did someone say interface?
interface ISuperFoo

{
void Bar();
}

Oh great, now we've got the interface that allows us to call Bar(). And we don't even know what's called in the implementation class (in fact, we don't care, this is what interfaces are for).

So far so good, but FOnline scripting engine works in a way, that you can attach your functionality to the handlers for default events.

Let's start of course, from creating an object. If we want to attach it to the item, we need to do this in the item_init handler.
void item_init(Item& item, bool firstTime)

{
SuperFoo@ foo = SuperFoo(item);
}

Yeah, whoops. We're gonna loose the reference to the newly created foo object as soon as we leave the function. Unacceptable, we need to store it somewhere.

Object Managers

This is also common solution. If you've got many objects in your program, you need to manage them somehow, store the references to them, maintain their lifetime, give the references(via interaface) to those interested.
We're gonna use basic generic array type for that.

array<superfoo@> Foos;

ISuperFoo@ GetFoo(const Item& item)
{
for(uint i = 0; i < Foos.length(); i++)
{
if(Foos[i].item.Id == item.Id)
return Foos[i];
}
return null;
}
void AddFoo(Item item)
{
Foos.insertLast(SuperFoo(item));
}

Oh what a horrible piece of code we've just made. Yes, we've got our array, and function to add an object for an item (which we can use in item_init) and get the interface of such object for existing item. But there is that horrible loop in GetFoo, we need to scrap that. The best would be, if we could perform fetch with direct index to our array. We can do that, if we're able to store the index, at which our Foo is located in the array. Let's store it in Item::Val1 field:
void AddFoo(Item& item)

{
Foos.insertLast(SuperFoo(item));
item.Val1 = Foos.length() - 1; // here we've got index to our array stored
}
ISuperFoo@ GetFoo(const Item& item)
{
if(item.Val1 < Foos.length())
return Foos[item.Val1];
else
return null;
}

and our item_init will now look like:
void item_init(Item& item, bool)

{
AddFoo(item);
}

So, we're now adding our SuperFoo object every time we call item_init, and we can fetch it in any script module (any module that imports this function of course), by obtaining an interface, that's not coupled in any way to the implementation class, nor coupled to the item object. Nifty.

We can use our interface in following manner:
void item_init(Item& item, bool)

{
AddFoo(item);
item.SetEvent(ITEM_EVENT_USE, "_FooUse");
}
bool _FooUse(Item& item, Critter& crit, Critter@ onCritter, Item@ onItem, Scenery@ onScenery)
{
ISuperFoo@ foo = GetFoo(item);
if(valid(foo)) foo.Bar();
}

The _FooUse function is where this solution shines. We call our method via interface(we may even pass some parameters given to us by the engine), and we do not care of what's going on inside.

Now, there is still something wrong with that solution, first who find out what, gets a candy!

Tuesday, August 16, 2011

Gathering? Waiting?

Oh, that's certainly hot topic. Countless debates, unlimited flames, and cause of infinite hate, frustration, suicides, recession.


Changes are iminent, be it cooldown time reduction or even changing approach to whole system. We'd like to present you couple of ideas that you might want to discuss - yeah like it hasn't been done before countless of times! Jokes aside, let's check it out.


Scavenging

During early times of open beta, the gathering for resources worked in slightly different manner. You were looking for various materials on the worldmap, there was a chance for particular resource being generated. After you've acquired it, there was no cooldown. This ended in constant entering/looking around/leaving, which was not only rather boring, but also highly farmable. However, the worldmap and encounters are playing big part in Fallout games, so we could tweak that approach, and use it once again:
  • attach a chance to various zones to spawn various materials (be it junk, ore or whatever)
  • disable ability to reroll material spawn when you force-enter from worldmap (that means you need to wander around and wait for next encounter check)
That leaves us with a system in which we still could calculate the average time in which a player is obtaining particular resource: encounter check is being performed every 5 sec, it's easy to calculate values on that.

What are the pros of such approach?
  • no cooldown - or rather, it's being hidden
  • corelating materials with encounters, that means, due to the level of difficulty, some zones could spawn high quality resources
How about the cons?
  • it's all about encounters once again, may be frustrating to look around with no luck

Hunting

Well, but we could as well stay with current cooldowns, but we need a way for players to reduce them. Somewhat entertaining way, though that's subjective. For that, we could add items that reduce timeout - say food. You could obtain food by hunting (encounters again), or you could buy it at traders. That's additional task your player needs to do, but that's optional (well, some may find it mandatory, if they want to have the most out of it).

Pros:
  • additional activities
  • a way to reduce cooldown
Cons:
  • The cooldowns stay (albeit reduced)
That's all for now. I think we don't need to encourage discussion, it's unavoidable!

Sunday, August 14, 2011

Bases visibility

In previous posts I've explained how we're gonna use the documents stored in faction database (a.k.a. files) to remember various info, such as what bases are accessible, and what are not the the given player (whether it's a member or not).

I've focused on synchronization aspect, and I will follow that in this post, as the technical side of it affects how it's gonna work for you.

This is gonna be our hypothetical player's file:
{

"_id": "scypior",
"faction": "The Unity",
"bases": [ "bunker" ]
}

In this case it's of no importance whether this record is stored in The Unity's database and it's record for a member, or whether it's in some other database and we just want to specify the bases this player can access.

With such file, server is gonna synchronize its data in following manner:
- compare the incoming file with previous one, detect if any new base appeared on the list, or if any base got removed from that list
- take the list of new entries in bases list, fetch their location id numbers knowing their names (checking if such base really exist and belong to that faction), mark those locations as visible for given player
- take the list of removed entries, do the above magic for location id and mark those locations as invisible

Notice the difference to the current system, that was caused solely by the technical changes - the visibility is set/unset when the server synchronize the data, while currently, on the following events:
- invited member enters location (visibility set)
- non-member slain in the base location (visibility unset)

The change in first case, may make players happy actually - no more tedious inviting, everything is done automagically. While I certainly do not like when magic happens in game, it's often needed to avoid frustration (and you've got already shattered nerves...), so it's good.

The other case however, might also be liked (oh no, he may come and kill us in our base, better quickly revoke his access!!!), but I very much liked the fact you needed to kill that unwanted player on your own ground to settle things, so I could think of solution for this, if it is really needed.

What are your thoughts on that matter?

Thursday, August 11, 2011

Revolutionizing Faction Terminal Data

This is the second part of the series on crazy ideas for factions terminal. In the previous post I described why the changes were needed, also I've shown why we could be potentially interested in externalizing faction databases and make them sitting on top of CouchDB, thus 'freeing' the data and handing it to the players, while providing html interface for everyone to use.

Let's think now about the faction terminal/database, and the data that lives there. Basically, it's all about players. The main part of it is just a list of players - members, and non-members as well. Currently, each player record (let's call it file for future reference) contains:
- faction to which described player belongs to
- status, i.e. his relation to the faction that owns the database
- rank - call it a 'level of importance' for the faction the player belongs to

Notice how some of that data does not really affect gameworld. For example, the status - if the player is the enemy for your gang, the gameworld (npcs, monsters and other script-driven aspects of the game) does not care about that fact. But should it be the terminal of an NPC faction, then such status matters.

Ok, so what we've got here is some kind of file for a player. Essentially, a document, so it should really be a breeze trying to put that in the CouchDB. Using JSON, as this is the notation used for CouchDB, our file would look like:

{
"_id": "scypior",
"faction": "The Unity"
"status": "enemy",
"rank": 4
}*

*Of course the interface will be reponsible for handling this format, the UI layer should look better:)

I think I don't need to get into the details of JSON format, it really looks intuitive.

The first field in our file structure is the _id. It's identified in such way, because this is how CouchDB handle documents, it does not allow you to have more than one document with same id. So, choosing a player name for that field seems like good idea in FOnline case.
The other fields - this is where we are trying to describe the faction the player belongs to, his status within our ranks and his rank in his own structures. For purpose of this example, assume it's record for non-member.

We may fill up our faction database (remember, each faction holds its own database in CouchDB ecosystem) with thousands of files, but what does it really do? Surely, some aspect of it is to store data for players information, but it cannot only serve that case. Game itself might be interested in some of that data. Of course, the most important should be the faction field. Ok, what we should do about it? We need a way for a server to fetch the data it's interested in, and react accordingly. We need synchronization. And we need the server to decide what kind of data it needs.

It definitely needs data describing the faction membership. So how we could work with that data? Surely we cannot just fetch the file document from database, and act upon it. We cannot change the player's faction, just because it was said so in some database, can we? Well, in one case: if the player's faction is the same as the one we're currently grabbing data from, and the document on that player states otherwise. That means someone (someone with permission for that database) has changed the faction on his member hence we need to clear faction on that player (not change it to the one that's in file now).

What about acquiring faction membership? Currently, we need to accept an invitation. This won't change - if player attempts to join some faction, we first are gonna check if his file in that faction database describes his status as invited. Then we can proceed with membership change.

If we're keeping with current system, there is also one field left - rank. As we've said previously, it does not really matter when the database belongs to a gang. After all, those files are read by players, so they might want to set those ranks to any values they want. But say, we're now talking about NPC faction database. The mechanics for reputations in game decrease your reputation with regard to some faction, if you attack its members. The amount of reputation deduced depends on the rank of the victim. So in that case, we should impose restrictions on the values that may be used for members in such NPC faction.

But so far, that's the very same functionality we've already had, let's see how we could implement new functionality.

Multiple bases. Oh yeah, that's something needed. Say, we acquire the base somehow (it doesn't matter how, whether it will be just 'buying' from some NPC (like now), or whether it will be built like a house from resources (not saying something like that is there:) ). Of course we would like to manage the players, that have access to those bases. To recognize a base, we need to name it. Once we've got a name, we may just fill some additional field in the player file, that's showing us to what bases he's got access to:


{
"_id": "scypior",
"faction": "The Unity",
"status": "enemy",
"rank": 4,
"bases": [ "dacha", "bunker", "depot" ]
}

Oh great, not only we are stating that player scypior is the enemy, but also we're giving him an access to our bases? Well, our bad. But that's not important here, we've filled that data, and we want the server to react (show the green circles on his worldmap). For this, the data needs to keep up with the standard of course, otherwise, server won't have any idea what we mean. Synchronization procedure in this case might look like this:
- take the bases list
- check if the names of the bases are describing locations that really belong to given faction we're fetching the file from
- take the player that the file describes
- mark those locations as visible for him
Similarily it should work for bases that are no longer visible for him (some kind of blacklist), it needs to be in different list.

Ok, so far - great. But you may ask - I was saying a lots about customization, about freeing the data and handing it to the players. But so far, the data was needed to be laid down in some standard format, otherwise server would have not know what to do with it, and hence, such additional data was useless.

That's true, because players cannot customize the way server works. What they can customize however, is what the terminal is showing and what it does for them, players. I already said that we may put whatever data we want there, display it in the terminal (by customizing some html) - various messages, notes, other info, but that's not very connected to the gameworld. Would be good, if such database would contain info on some gameworld elements that are of interest to us (and within our permissions). Following ideas are hypothetical, but we could just fill the faction database with lots of data and let players decide what to use and how to use.

For example, imagine we would like to know what items are in which base. If the server would tirelessly help us filling it, we could use such info, and make various inventories etc.

But that may be a lots of data to cope with. What do you think? What would be useful to see out there in those terminals? Or maybe it's not needed at all?

PS. Next time, I may describe some inner details of how one can implement communication with such external source, that could be useful for those of you interested in FOnline modding, or maybe I will follow the musings about the customization, we will see...

Wednesday, August 10, 2011

Revolutionizing Faction Terminal

Faction management, isn't it the main and central thing of this game? Or wait, shouldn't it be? After all it's common knowledge, that it's been called Faction Mod from the beginning. And one of the first thing implemented, was that 'handy' faction terminal that allowed you to invite other players to the base, build structures of your gang, store data about outer world (other players).

But it has not worked as expected, and many things changed since then. First and foremost, it was build around the standard dialog interface functionality, due to the limitations that was imposed during that time. Because of that, managing faction became tedious and annoying task. Also, it turned out that players do not need all those functionality - not always they need to have four ranks for their members, sometimes maybe they would like to have more than one leader, maybe they would also like to have their radio channel changing randomly automatically. Maybe they also would like to have customized messaging system, in which everyone could post a message on the terminal, to be later read by others. Lots of possibilities, but the main issue was usability. That meant it required changes, or even complete overhaul.

Lots of time has passed, and the engine has grown up to the point almost everything is possible (with varying amount of work). But still, we have not improved that core functionality - chasing other stuff, or simply because of lack of time.

I started to think about the changes quite time ago, and wanted also to expose that stuff to the world by providing web interface for players (along with account management). First, I wanted to put emphasis on that, so that once registered playername always belong to some account, and it cannot be taken after wipe. And on top of that, I wanted to reimplement the very same functionality we've got in faction terminal, but also expose it to the web interface and store data in some persistent, wipe-safe storage (SQL database to be precise).

I started implementing it, but soon I realized it needs some amount of customization, otherwise it might just fail the same way as old system did - providing some useless functionality, being used only partially etc. Of course, I began to think how to inject some flexibility into it - that seemed to be lot of work. Then I stumbled upon something, that entirely changed my ideas and made me to start implementing it almost from a scratch again.

Document databases, namely - CouchDB.

To those interested, CouchDB is non-relational database, that allows you to store your data in form of documents in JSON format (Javascript Object Notation). That means - no relations, only documents. Hey - that sounds good, after all, faction terminal is just a database where you store documents, right? In traditional SQL approach, you would have something like:

table Factions
table Players


connected with relation 1 to many. Then, if you want to fetch all players records stored in faction A, you need to join those 2 tables to prepare output records. Oh nice, this is what the world is doing since ages, it works. But why to impose such rigidness on that, in non-relational database, you would just create database for faction A, and store documents about players there. Nice!

But that's not all. It turned out that CouchDB, because of the fact its protocol is HTTP, is able to serve HTML (and other HTTP content) directly. So I can build web interface (web application) in it, no need for additional tier, no need for additional dependencies. Great? Yeah, I started to love it.

Now, another surprise - it's designed in a way, that each web application is stored in some database, right? And previously, we stated that we will do it the way that each faction would have its own database. That means, each faction could have its own, fully customized web interface (of course, default one is gonna be provided). And it requires only HTML + Javascript knowledge.

Ok, that all sounds fine, but also it feels a bit dangerous. We are allowing players to store the data they want, and we also want to server (gameworld) react to it in proper way. So we need some way of communication, between CouchDB and FOnline server. Seems hard, but part of it already works.

Hopefully, this is the first part of the series about new factions interface. Please note, that's not yet stated when it's gonna be finalized and introduced. To my excuse, I've got real reasons why it's taking so long:)

In next post we will dissect how it might work and how the data will be synchronized, or maybe I'm gonna throw in some technical details, just for fun.

Tuesday, April 19, 2011

The Vault Dweller dissects FOnline: 2238!

It took months after the launch of FOnline: 2238 before The Vault Dweller of the NMA decided to try it out. One way or another, his FOnline: 2238 review is finally up on the NMA. The review itself did get old in the meantime, simply because everlasting new features were added recently, while publishing of the review was delayed for the reasons unknown to us - nonetheless, it is good to hear an opinion of such an unbiased reviewer, no matter what.


We appreciate the effort the VD made in order to write this article; the least we can do to show our appreciation is to read it ourselves, as well as to recommend it to you. In the end, we also do hope that our endeavors shall inspire him during his perilous article writing journeys in the days to come.

Monday, February 28, 2011

Development Diary: These damn animations

So, this one is a small one. Everyone is waiting for the next smaller update-- hey, we too! Anyway, to not abandon this place, here is some insight on what is the reason why it takes and took us so long:


Right now, we are fighting some little ugly problems with the animations of killed critters. That is, flipping around on death, not working flamethrower attack animations and so on and so on. This is, in fact, the last thing we have to do until the next update can get out of the cage. We aren't sure if we can handle the flamer attack animation in the short time, but at least killed critters will stop flipping around while getting killed.

Also as the one or the other might have heard already, we are adding a new armor item. It's a sand colored robe, created by x'il (who also worked on the long hair and bald head dude), some of you might know it already from the Fallout modding community.


The robe will be available for cheap money at most of the traders in the wasteland. Let's not be that blue anymore all together! Switch yer bluesuit to the new sandsuit and be a hipster too!

Additionally and as always, we made lots of map changes, dialog fixes and so on. I'll guess various changes on the maps will be the most obvious ones. You for sure will recognize it in the game after the update has been done.

So far, that's it for now.

Stay tuned for more.

Monday, February 14, 2011

WIP: 3D Model Screenshots

To post something new in here, we got three screenshots of some of the new work in progress 3d models for you guys. Animations and models are 100% community work.