Server issues

We're having some problems with the server. We hope to solve it today, but there's no guarantee.

Thanks for your patience.

Faction data - asynchrony

This is gonna be another post from the series on new approach to the faction data storage, and most technical one. It's gonna describe process of creating a native extension for FOnline server application, that will allow communication with external database, in our case, CouchDB. It's already implemented and is being tested, so I guess I can safely write about it.
The endeavour was quite an interesting one with lots of crazy ideas, few dead-ends, but eventually the solution turned out to be nice and clean.

So, for those interested in hacking the guts of server application, extending it to perform things you haven't imagine, let's read!

Data synchronization

First, we need to make brief summary, of what we want to accomplish. It's been described in previous posts, basically, we need a way to synchronize some game data (players properties, variables) with the data stored in CouchDB. As with every communication, there are two directions we may want to communicate:

• server asks CouchDB for data it needs to synchronize
• CouchDB tells server that it may want to synchronize

Since we'd rather extend server application, than to write additional tier for CouchDB that would communicate with server, we of course choose the first approach. The data we need to synchronize are:

• factions
  
• players' documents from factions databases

So basically we may need two functions to do this:

• GetFactions - gets all factions with their properties
  
• GetPlayerFile - gets player file stored in certain faction database

Let's focus now on the first one.

Main factions database

Info on our fac tions is going to reside in central database, called factions. This database is going to contain documents for each of the registered faction (for simplicity sake, we're not going to write about registering new faction process). Example document:

{

 "_id": "Brotherhood Of Steel",

 "id": 2,

 "database": "brotherhood_of_steel"

}

The _id field is the unique identifier for the document, so it's also the faction name. The id (without leading underscore) is the number assigned during registration process, and is the number any member of that faction carries on his player character, so that game logic may react appropriately. The database field is the name of the faction database, that reside on the same CouchDB server. It needs to be different than the faction name, cause CouchDB does not allow certain characters in database names.

In-game, our factions are going to be represented by following class:

class Faction

{

 int id;

 string name;

 string database;



 Faction(int id, const string& name, const string& database_name)

 {

     this.id = id;

     this.name = name;

     this.database = database_name;

 }



 int get_Id() const { return this.id; }

 string get_Name() const { return this.name; }

 string get_Database() const { return this.database; }

};

And we're storing the objects of that class in some array. So, the only thing we need, is indeed a GetFactions function, that would just fetch data from factions database and fill up our array.

But before we dive into code, let's pause for a moment. It is external storage, it's http protocol. By no means it's going to be fast. We can't just write an extension function that is going to perform http request and return the results so that we can process them further in scripts. It would block the server during the call for at least few miliseconds, but it could even be few seconds, it depends. We need a way for asynchronous calls.

Asynchronicity

What's that, and why it's not The Police album? The principles of asynchronous calls are very simple. The call is being performed, and the function immediately returns to the place from where it's been called, allowing main thread to resume its job without waiting for function results (while the function that's been called asynchronously is being executed in other thread). Great, but we are interested in those results, so we need a way to operate on them. Traditionally, this issue is being solved by callbacks. To put it simply, you are defining another function, that will be called when our function that has been called asynchronously finishes its execution. Such callback takes what's been returned by the asynchronous function as its argument, and perform whatever logic we wanted to be performed on that result in the first place.

But we're not at home still, let's check it. This is going to be our hypothetical extension function, that is going to perform http requets to CouchDB, and return data for further processing (pseudocode, native):

void AsyncGetFactions(callback)

{

 QueueThreadPoolTask(task, callback);

}

void task(callback)

{

 string res = CouchDB::Get("factions");

 callback(res);

}

This is how asynchronous functions look like. They only queue the task to be performed later on other thread. The function is taking a callback as an argument, performs http GET request, and calls the callback to operate on data. But we want our callback to be AngelScript function, what we may do about it? Common solution in FOnline scripting, is to pass the name of the function as string(pseudocode, AngelScript):

AsyncGetFactions("callback");



void callback(const string& result)

{

 // operate on what's been returned. It's JSON, but we're not going to dive into that matter now

}

Familiar? Should be - CreateTimeEvent works this way. We may rework our native extension to something like this:

void AsyncGetFactions(const string& callback)

void AsyncGetFactions(callback)

{

 QueueTask(task, callback);

}

void task(callback)

{

 string res = CouchDB::Get("factions");

 CallAngelScriptFunction(callback, res);

}

The last line uses a function, that would fetch the function from AngelScript engine, knowing its name, and would call it passing our result as argument.

But hey, something is still wrong here. Remember the task is being executed in another thread. Whoops, and a big one. That means our callback function will be executed on that other thread as well, which means simultaneously with whatever other game logic is being executed at that time. This of course may lead to the data corruption, hard to track errors, unexpected behaviour, all that fun. What can we do about it? Traditionally, you may assure that the function that's going to be run in parallel with main code, does not operate on the same variables that the main logic thread does. But that would be very hard to achieve, after all, for our callback we would probably like to reuse whatever code we alredy have in our codebase, and I bet most of it is not thread-safe at all. Is there something that can save our asses in our quest to achieve our goal?

Message passing concurrency

Some clever folks found out, that to avoid problems with multithreaded code, it's best to avoid the code that's simultaneously executed and operates on same set of data - brilliant, isn't it. Instead, it's better to have totally separated units, and only have them communicating between by passing messages to each other.
Notice, that we may safely run our game logic in parallel with CouchDB http request, as those are totally separated. It's when we want to operate on data returned, when we're running into problems. So why not return the data somehow to the server application, and let server logic be the one responsible for reading it and performing the funcionality? That way, the callback logic will be executed by main thread, so no problems here. For that, we need a queue with messages, that we will be using to exchange data between our asynchronous functions, and the server logic. Then we're gonna be able to leave the message (from the other thread), and fetch the message (from main logic thread) to further operate on it (still in main logic thread). That way, the only structure shared between threads will be the queue itself, but it's not a problem to write such thing to be perfectly thread safe:

void PushMessage(msg)

{

 lock(messages);

 messages.push(msg);

 unlock(messages);

}

msg FetchMessage()

{

 lock(messages);

 msg = messages.pop();

 unlock(messages);

 return msg;

}

Above pseudocode shows us, how we can synchronize the reads and writes on our queue, to assure that only one thread is accessing them at a time. The way we do it depends on libraries we're gonna use, I do not want to dive into the details here, but the principles are the same:

• lock puts a lock on some structure
• unlock takes that lock away from it
• there may be only one lock at the structure at a given time, so next call to lock (performed from other thread) is going to be blocked and wait till it's being unlocked

That gives us thread-safety for the queue. Let's use it now:

void AsyncGetFactions()

{

 QueueThreadPoolTask(task, callback);

}

void task()

{

string res = CouchDB::Get("factions");

// iterate over res content, to send message for every faction contained there

for(...)

   PushMessage(new Message(MESSAGE_FACTION, res));

}

And in script:

void UpdateFactions() // we could call it from main@loop() for example

{

 AsyncGetFactions();



 while(true)

 {

     Message@ msg = FetchMessage();

     if(!valid(msg)) break;



     if(msg.type == MESSAGE_FACTION)

         ProcessFaction(msg.res);

 }

}

void ProcessFaction(string res)

{

 // parse our input, determine faction properties, check if already in array

 // if not, add it there, otherwise - update

}

Notice, that we're calling AsyncGetFactions in each loop and after that we're fetching all messages. But the messages won't probably arrive at that moment, for that we will have to wait. And, while we are waiting, there is no point in calling AsyncGetFactions over and over again. We need to orchestrate somehow our calls, we can do this with simple boolean switches:

bool GettingFactions = false;



void UpdateFactions()

{

if(!GettingFactions)

    AsyncGetFactions();



while(true)

{

    Message@ msg = FetchMessage();

    if(!valid(msg)) break;



    if(msg.type == MESSAGE_FACTION)

        ProcessFaction(msg.res);

}

}

It's that simple. But we need a way to notice the game logic about the fact, that our asynchronous extension has finished with getting factions. For this, we will extend it to send yet another message, after all faction messages have been sent:

void task()

{

 string res = CouchDB::Get("factions");

 // iterate over res content, to send message for every faction contained there

 for(...)

   PushMessage(new Message(MESSAGE_FACTION, res));

 PushMessage(new Message(MESSAGE_GET_FACTIONS_DONE));

}

And then in scripts, we will switch our variable:

void UpdateFactions()

{

if(!GettingFactions)

AsyncGetFactions();



while(true)

{

    Message@ msg = FetchMessage();

    if(!valid(msg)) break;



    if(msg.type == MESSAGE_FACTION)

        ProcessFaction(msg.res);

    if(msg.type == MESSAGE_GET_FACTIONS_DONE)

        GettingFactions = false;

}

}

By setting GettingFactions to false, we're indicating that we are no longer running AsyncGetFactions in the background, so we can safely call it again next time. And our loop is chewing whatever messages arrive there all the time. All in parallel, all in thread-safety.

I hope it was interesting read, maybe not too detailed and with lots of pseudocode, but I did want to show the idea, not the implementation specifics.

Tremendous bug!

It didn't take you long to find out what was wrong in the solution I presented in http://fonline2238.blogspot.com/2011/08/what-would-be-point-of-having.html. In fact, you were faster than me fixing it, after the bug introduced by it occured on 2238 server.

Let's start from beginning. In first iteration of gathering (called production) back then, we needed a notion of facility that player could use to obtain some resource. A facility could be one item (item pretending to be scenery object actually, it had to be item because of the scripting possibilities), or collection of items. So, a barrel in Modoc was one facility and all the plants somewhere on the crop could be one facility as well. And when the player obtained some resources from it, we needed to apply timeout for it (store it on the item somewhere). Also, we needed even more scripting possibilities - rotgut still for example. Perfect situation to use the wrapping pattern:

interface IFacility

{

bool CheckTimeout();

void Use(Critter& cr);

}

class BarrelFacility : IFacility

{

Item@ barrel;

BarrelFacility(Item& item)

{

    this.item = item;

}

bool CheckTimeout()

{

    return item.Val1 > __FullSecond;

}

void Use(Critter& cr)

{

    if(CheckTimeout())

    {

        // give junk

        item.Val1 = __FullSecond + 60; // 60 seconds of barrel cooldown

    }

}

}

Of course above code is simplification, and we needed to wrap the item for other reasons as well - for example having one facility object per many items, but it's not the point of that post to explain all of it.
So, we were adding the BarrelFacility objects in barrel script:

void item_init(Item& item, bool)

{

AddFacility(BarrelFacility(item));

}

And we had that global array of IFacility objects. I didn't implement removal procedure, simply because, all those facilities were in towns and other fixed locations. So, they were destroyed only when server was being shut down, and were created only on server start. Additionaly, because of the fact there was only few of them, and I think I haven't got any unused Item::ValX field, I haven't been storing the index to the array for fast fetch (like I wrote in previous blogpost on that), but I was iterating over whole array to find proper object. It hadn't got big performance impact, but one day we decided we need to change the system: we introduced facilities into encounters. Boom!

Yes, that was it - the IFacility objects were created very often, and never being destroyed. The array was growing and growing, and the fetching procedure was O(n) in complexity instead of O(1). Terrible stuff. When players started to jumping into encounters, after some time the array was so big that every time someone wanted to use facility, the algorithm had to go through thousands of iterations to find proper index in the array. And the appending was quite painful as well - the underlying array object needs to copy all elements on resize, as it does not reserve space up-front.

And that caused a lag. The Lag in fact, it was so big that it had destroyed the fun from early beta days. And we haven't fixed it that fast. To put it simply - I forgot that the objects are never removed, and found that out later... A little embarassing, I know, but now, when looking at it after such amount of time - it's quite funny to recall.

And quite funny story about 2238 development I think.

Wrapping a wrap

What would be the point of having developer's blog, if there wouldn't be anything on programming. Oh yeah, that's where the things got hot. This is where it works, this is where it fails.

Since launch of the SDK, probably lots of people started struggling with that fancy scripting language - AngelScript. I remember I did. Experienced some successes, failures, helped finding some bugs, worked my way out through this language, started appreciating it, started even liking it (now I see it was indeed quite a good choice). That's why I think we could from time to time talk about scripting in FOnline SDK, talk about the AngelScript, share some common pitfalls, solutions that worked out good, and even the ones that weren't.

Of course basics of angelscripting is required, there won't be any tutorials whatsoever.

Objects

Blah blah blah, everything you can model in your OO language is an object, yeah, we already know that and it sucks (I'm getting a little bit off topic right here). Ok, it does not suck, in fact we're gonna often rely heavily on that fact, and we're gonna put large part of our logic into objects. And we're gonna tie those objects to the in-game objects (critters, items, scenery, maps, locations).

Say we've got a class for objects that are gonna make us some nice abstraction layer over default item functionality:

class SuperFoo

{

 Item@ item;

 SuperFoo(Item& item)

 {

     @this.item = item;

 }



 void Bar()

 {

     item.Val0++;

 }

}

What's going on here? We wrap an item type in some class, and we're adding some useless code to operate on it? Why the hell would we want it? Well, think about the interface you've just gained and you can use throughout your scripts. You may now call: something.Bar() instead of item.Val0++, which shows what you want to do, and not how are you going to do. That's the difference if you want to write good code. Less imperative, less verbosity, less bugs (yes, I know that example suck, and you would want that for far more reasons - think about having some additional data, that's not stored on the wrapped item, but it's one of the class' fields - it's not saved anywhere, but still may be useful).

Wait, did someone say interface?

interface ISuperFoo

{

    void Bar();

}

Oh great, now we've got the interface that allows us to call Bar(). And we don't even know what's called in the implementation class (in fact, we don't care, this is what interfaces are for).

So far so good, but FOnline scripting engine works in a way, that you can attach your functionality to the handlers for default events.

Let's start of course, from creating an object. If we want to attach it to the item, we need to do this in the item_init handler.

void item_init(Item& item, bool firstTime)

{

    SuperFoo@ foo = SuperFoo(item);

}

Yeah, whoops. We're gonna loose the reference to the newly created foo object as soon as we leave the function. Unacceptable, we need to store it somewhere.

Object Managers

This is also common solution. If you've got many objects in your program, you need to manage them somehow, store the references to them, maintain their lifetime, give the references(via interaface) to those interested.
We're gonna use basic generic array type for that.

array<superfoo@> Foos;



ISuperFoo@ GetFoo(const Item& item)

{

 for(uint i = 0; i < Foos.length(); i++)  

 {

     if(Foos[i].item.Id == item.Id)

         return Foos[i];

 }

 return null;

}

void AddFoo(Item item)

{  

 Foos.insertLast(SuperFoo(item));

}

Oh what a horrible piece of code we've just made. Yes, we've got our array, and function to add an object for an item (which we can use in item_init) and get the interface of such object for existing item. But there is that horrible loop in GetFoo, we need to scrap that. The best would be, if we could perform fetch with direct index to our array. We can do that, if we're able to store the index, at which our Foo is located in the array. Let's store it in Item::Val1 field:

void AddFoo(Item& item)

{

    Foos.insertLast(SuperFoo(item));

    item.Val1 = Foos.length() - 1; // here we've got index to our array stored

}

ISuperFoo@ GetFoo(const Item& item)

{

    if(item.Val1 < Foos.length())

        return Foos[item.Val1];

    else

        return null;

}

and our item_init will now look like:

void item_init(Item& item, bool)

{

    AddFoo(item);

}

So, we're now adding our SuperFoo object every time we call item_init, and we can fetch it in any script module (any module that imports this function of course), by obtaining an interface, that's not coupled in any way to the implementation class, nor coupled to the item object. Nifty.

We can use our interface in following manner:

void item_init(Item& item, bool)

{

    AddFoo(item);

    item.SetEvent(ITEM_EVENT_USE, "_FooUse");

}

bool _FooUse(Item& item, Critter& crit, Critter@ onCritter, Item@ onItem, Scenery@ onScenery)

{

    ISuperFoo@ foo = GetFoo(item);

    if(valid(foo)) foo.Bar();

}

The _FooUse function is where this solution shines. We call our method via interface(we may even pass some parameters given to us by the engine), and we do not care of what's going on inside.

Now, there is still something wrong with that solution, first who find out what, gets a candy!

Gathering? Waiting?

Oh, that's certainly hot topic. Countless debates, unlimited flames, and cause of infinite hate, frustration, suicides, recession.


Changes are iminent, be it cooldown time reduction or even changing approach to whole system. We'd like to present you couple of ideas that you might want to discuss - yeah like it hasn't been done before countless of times! Jokes aside, let's check it out.

Scavenging

During early times of open beta, the gathering for resources worked in slightly different manner. You were looking for various materials on the worldmap, there was a chance for particular resource being generated. After you've acquired it, there was no cooldown. This ended in constant entering/looking around/leaving, which was not only rather boring, but also highly farmable. However, the worldmap and encounters are playing big part in Fallout games, so we could tweak that approach, and use it once again:

• attach a chance to various zones to spawn various materials (be it junk, ore or whatever)
• disable ability to reroll material spawn when you force-enter from worldmap (that means you need to wander around and wait for next encounter check)

That leaves us with a system in which we still could calculate the average time in which a player is obtaining particular resource: encounter check is being performed every 5 sec, it's easy to calculate values on that.

What are the pros of such approach?

• no cooldown - or rather, it's being hidden
• corelating materials with encounters, that means, due to the level of difficulty, some zones could spawn high quality resources

How about the cons?

• it's all about encounters once again, may be frustrating to look around with no luck

Hunting

Well, but we could as well stay with current cooldowns, but we need a way for players to reduce them. Somewhat entertaining way, though that's subjective. For that, we could add items that reduce timeout - say food. You could obtain food by hunting (encounters again), or you could buy it at traders. That's additional task your player needs to do, but that's optional (well, some may find it mandatory, if they want to have the most out of it).

Pros:

• additional activities
• a way to reduce cooldown
  

Cons:

• The cooldowns stay (albeit reduced)
  

That's all for now. I think we don't need to encourage discussion, it's unavoidable!

Bases visibility

In previous posts I've explained how we're gonna use the documents stored in faction database (a.k.a. files) to remember various info, such as what bases are accessible, and what are not the the given player (whether it's a member or not).

I've focused on synchronization aspect, and I will follow that in this post, as the technical side of it affects how it's gonna work for you.

This is gonna be our hypothetical player's file:

{

"_id": "scypior",

"faction": "The Unity",

"bases": [ "bunker" ]

}

In this case it's of no importance whether this record is stored in The Unity's database and it's record for a member, or whether it's in some other database and we just want to specify the bases this player can access.

With such file, server is gonna synchronize its data in following manner:
- compare the incoming file with previous one, detect if any new base appeared on the list, or if any base got removed from that list
- take the list of new entries in bases list, fetch their location id numbers knowing their names (checking if such base really exist and belong to that faction), mark those locations as visible for given player
- take the list of removed entries, do the above magic for location id and mark those locations as invisible

Notice the difference to the current system, that was caused solely by the technical changes - the visibility is set/unset when the server synchronize the data, while currently, on the following events:
- invited member enters location (visibility set)
- non-member slain in the base location (visibility unset)

The change in first case, may make players happy actually - no more tedious inviting, everything is done automagically. While I certainly do not like when magic happens in game, it's often needed to avoid frustration (and you've got already shattered nerves...), so it's good.

The other case however, might also be liked (oh no, he may come and kill us in our base, better quickly revoke his access!!!), but I very much liked the fact you needed to kill that unwanted player on your own ground to settle things, so I could think of solution for this, if it is really needed.

What are your thoughts on that matter?

Revolutionizing Faction Terminal Data

This is the second part of the series on crazy ideas for factions terminal. In the previous post I described why the changes were needed, also I've shown why we could be potentially interested in externalizing faction databases and make them sitting on top of CouchDB, thus 'freeing' the data and handing it to the players, while providing html interface for everyone to use.

Let's think now about the faction terminal/database, and the data that lives there. Basically, it's all about players. The main part of it is just a list of players - members, and non-members as well. Currently, each player record (let's call it file for future reference) contains:
- faction to which described player belongs to
- status, i.e. his relation to the faction that owns the database
- rank - call it a 'level of importance' for the faction the player belongs to

Notice how some of that data does not really affect gameworld. For example, the status - if the player is the enemy for your gang, the gameworld (npcs, monsters and other script-driven aspects of the game) does not care about that fact. But should it be the terminal of an NPC faction, then such status matters.

Ok, so what we've got here is some kind of file for a player. Essentially, a document, so it should really be a breeze trying to put that in the CouchDB. Using JSON, as this is the notation used for CouchDB, our file would look like:

{

   "_id": "scypior",

   "faction": "The Unity"  

   "status": "enemy",

   "rank": 4

}*

*Of course the interface will be reponsible for handling this format, the UI layer should look better:)

I think I don't need to get into the details of JSON format, it really looks intuitive.

The first field in our file structure is the _id. It's identified in such way, because this is how CouchDB handle documents, it does not allow you to have more than one document with same id. So, choosing a player name for that field seems like good idea in FOnline case.
The other fields - this is where we are trying to describe the faction the player belongs to, his status within our ranks and his rank in his own structures. For purpose of this example, assume it's record for non-member.

We may fill up our faction database (remember, each faction holds its own database in CouchDB ecosystem) with thousands of files, but what does it really do? Surely, some aspect of it is to store data for players information, but it cannot only serve that case. Game itself might be interested in some of that data. Of course, the most important should be the faction field. Ok, what we should do about it? We need a way for a server to fetch the data it's interested in, and react accordingly. We need synchronization. And we need the server to decide what kind of data it needs.

It definitely needs data describing the faction membership. So how we could work with that data? Surely we cannot just fetch the file document from database, and act upon it. We cannot change the player's faction, just because it was said so in some database, can we? Well, in one case: if the player's faction is the same as the one we're currently grabbing data from, and the document on that player states otherwise. That means someone (someone with permission for that database) has changed the faction on his member hence we need to clear faction on that player (not change it to the one that's in file now).

What about acquiring faction membership? Currently, we need to accept an invitation. This won't change - if player attempts to join some faction, we first are gonna check if his file in that faction database describes his status as invited. Then we can proceed with membership change.

If we're keeping with current system, there is also one field left - rank. As we've said previously, it does not really matter when the database belongs to a gang. After all, those files are read by players, so they might want to set those ranks to any values they want. But say, we're now talking about NPC faction database. The mechanics for reputations in game decrease your reputation with regard to some faction, if you attack its members. The amount of reputation deduced depends on the rank of the victim. So in that case, we should impose restrictions on the values that may be used for members in such NPC faction.

But so far, that's the very same functionality we've already had, let's see how we could implement new functionality.

Multiple bases. Oh yeah, that's something needed. Say, we acquire the base somehow (it doesn't matter how, whether it will be just 'buying' from some NPC (like now), or whether it will be built like a house from resources (not saying something like that is there:) ). Of course we would like to manage the players, that have access to those bases. To recognize a base, we need to name it. Once we've got a name, we may just fill some additional field in the player file, that's showing us to what bases he's got access to:

{

    "_id": "scypior",

    "faction": "The Unity",

    "status": "enemy",

    "rank": 4,

    "bases": [ "dacha", "bunker", "depot" ]

}

Oh great, not only we are stating that player scypior is the enemy, but also we're giving him an access to our bases? Well, our bad. But that's not important here, we've filled that data, and we want the server to react (show the green circles on his worldmap). For this, the data needs to keep up with the standard of course, otherwise, server won't have any idea what we mean. Synchronization procedure in this case might look like this:
- take the bases list
- check if the names of the bases are describing locations that really belong to given faction we're fetching the file from
- take the player that the file describes
- mark those locations as visible for him
Similarily it should work for bases that are no longer visible for him (some kind of blacklist), it needs to be in different list.

Ok, so far - great. But you may ask - I was saying a lots about customization, about freeing the data and handing it to the players. But so far, the data was needed to be laid down in some standard format, otherwise server would have not know what to do with it, and hence, such additional data was useless.

That's true, because players cannot customize the way server works. What they can customize however, is what the terminal is showing and what it does for them, players. I already said that we may put whatever data we want there, display it in the terminal (by customizing some html) - various messages, notes, other info, but that's not very connected to the gameworld. Would be good, if such database would contain info on some gameworld elements that are of interest to us (and within our permissions). Following ideas are hypothetical, but we could just fill the faction database with lots of data and let players decide what to use and how to use.

For example, imagine we would like to know what items are in which base. If the server would tirelessly help us filling it, we could use such info, and make various inventories etc.

But that may be a lots of data to cope with. What do you think? What would be useful to see out there in those terminals? Or maybe it's not needed at all?

PS. Next time, I may describe some inner details of how one can implement communication with such external source, that could be useful for those of you interested in FOnline modding, or maybe I will follow the musings about the customization, we will see...

Revolutionizing Faction Terminal

Faction management, isn't it the main and central thing of this game? Or wait, shouldn't it be? After all it's common knowledge, that it's been called Faction Mod from the beginning. And one of the first thing implemented, was that 'handy' faction terminal that allowed you to invite other players to the base, build structures of your gang, store data about outer world (other players).

But it has not worked as expected, and many things changed since then. First and foremost, it was build around the standard dialog interface functionality, due to the limitations that was imposed during that time. Because of that, managing faction became tedious and annoying task. Also, it turned out that players do not need all those functionality - not always they need to have four ranks for their members, sometimes maybe they would like to have more than one leader, maybe they would also like to have their radio channel changing randomly automatically. Maybe they also would like to have customized messaging system, in which everyone could post a message on the terminal, to be later read by others. Lots of possibilities, but the main issue was usability. That meant it required changes, or even complete overhaul.

Lots of time has passed, and the engine has grown up to the point almost everything is possible (with varying amount of work). But still, we have not improved that core functionality - chasing other stuff, or simply because of lack of time.

I started to think about the changes quite time ago, and wanted also to expose that stuff to the world by providing web interface for players (along with account management). First, I wanted to put emphasis on that, so that once registered playername always belong to some account, and it cannot be taken after wipe. And on top of that, I wanted to reimplement the very same functionality we've got in faction terminal, but also expose it to the web interface and store data in some persistent, wipe-safe storage (SQL database to be precise).

I started implementing it, but soon I realized it needs some amount of customization, otherwise it might just fail the same way as old system did - providing some useless functionality, being used only partially etc. Of course, I began to think how to inject some flexibility into it - that seemed to be lot of work. Then I stumbled upon something, that entirely changed my ideas and made me to start implementing it almost from a scratch again.

Document databases, namely - CouchDB.

To those interested, CouchDB is non-relational database, that allows you to store your data in form of documents in JSON format (Javascript Object Notation). That means - no relations, only documents. Hey - that sounds good, after all, faction terminal is just a database where you store documents, right? In traditional SQL approach, you would have something like:

table Factions
table Players

connected with relation 1 to many. Then, if you want to fetch all players records stored in faction A, you need to join those 2 tables to prepare output records. Oh nice, this is what the world is doing since ages, it works. But why to impose such rigidness on that, in non-relational database, you would just create database for faction A, and store documents about players there. Nice!

But that's not all. It turned out that CouchDB, because of the fact its protocol is HTTP, is able to serve HTML (and other HTTP content) directly. So I can build web interface (web application) in it, no need for additional tier, no need for additional dependencies. Great? Yeah, I started to love it.

Now, another surprise - it's designed in a way, that each web application is stored in some database, right? And previously, we stated that we will do it the way that each faction would have its own database. That means, each faction could have its own, fully customized web interface (of course, default one is gonna be provided). And it requires only HTML + Javascript knowledge.

Ok, that all sounds fine, but also it feels a bit dangerous. We are allowing players to store the data they want, and we also want to server (gameworld) react to it in proper way. So we need some way of communication, between CouchDB and FOnline server. Seems hard, but part of it already works.

Hopefully, this is the first part of the series about new factions interface. Please note, that's not yet stated when it's gonna be finalized and introduced. To my excuse, I've got real reasons why it's taking so long:)

In next post we will dissect how it might work and how the data will be synchronized, or maybe I'm gonna throw in some technical details, just for fun.