For the last 2+ years, Corey Frang and I have labored away on building the engine that powers R2-Db.com (and EpicAdvice.com). I can honestly say that I'm proud of the work we have done and the progress we made. We developed some revolutionary ideas for the traditional "Item Database" model, we had our moments in the spotlight, and made some mistakes along the way.

Unfortunately, the passion behind R2-Db.com has diminished a lot since the launch of the game and neither of us play Star Wars: The Old Republic anymore. I could write an 10-page article on what went right, wrong and why we stopped/slowed development, but that's not what this is about.

What I want to talk about is the most positive thing (well, besides the experience) to come out of the R2-Db.com project.

The EpicDB Engine

Back in April of 2010 we had a decision to make, do we find a pre-made engine or start our own for EpicAdvice.com? We decided to make our own, and 2 years later, here we are, with an engine named to honor our first site.

EpicDB is the engine running everything behind the scenes of EpicAdvice.com and R2-Db.com, it's also being used to power this blog and everything you see on this site. Now the best question is, how would I describe to you what this engine is about? It's rough, but I'll give it a shot:

EpicDB is an engine that allows users to create and share information in the form of posts and/or records. Posts can be written and displayed publicly or privately, and be intelligently associated to any other post, record or user in the database. Users can also create records in the database that house factual information about a specific non-user and non-post topic.

It's pretty generic, but without diving into an incredible amount of detail, sums it up as best I can.

So, looking back, why did we actually have to design this engine from the ground up? Well, it all boils down to the types of data that most engines out there support.

First World Problems: Traditional DB Structures

Most of the engines/CMSs out there (like the ever popular wordpress) are divided into 2 major "realms" of data:

• Posts/Pages
• Users

Users can create posts, users can administer other users, pages could be created as non-dated posts, posts would have an author related to the user, and posts could be labeled with a category or tag. These are all very necessary things when designing a content driven website in terms of organizing data and maintaining ownership.

We wanted a system with all of these features in mind, but the problem in these systems is that the tags are 2nd class citizens. Sure you could look at a page to find "Articles tagged with XXXXXX", but you never could physically look at information about the tag in specific, only posts related to it.

We wanted to expand upon the category/tag systems so that the things being tagged where physical, real life database records with information of their own. This is why we had to break the mold and is we needed a data type specifically designed to hold non-post information.

Adding a new "Prime" Data Type

With this problem in mind, we set out to design an engine with 3 "prime" data types, the two traditional types and our new 3rd type, which we called records. Our data model now looked something like this:

• Records
• Posts
• Profiles

Every piece of data within our database falls into one of these types. We also designed these primary types to be expandable for more specific types of data. For this blog, using EpicDB, I extended the Record data type and created a new data type called Project, which I've been using to list out information of projects I'm involved in.

Under the title of this post, you'll notice the "about R2-Db.com". During the creation of this post, I used our "tagging engine" to select the record (specifically, a project) that this post is about. It physically links the post to the actual record for the R2-Db.com project! You'll also notice this post appears on the project page under related posts.

With this expanded set of data types, it opened up a whole new world of meta-information possibilities.

Relating the Prime types with Tags

Every data type listed above has a Document Set build on it called "Tags". This field uses the Database Reference feature in MongoDB to related information between different entries in the database.

For example, here's what this post looks like:

> db.posts.findOne();
{
    "_id" : ObjectId("{object_id}"),
    "_created" : 1336762708,
    "_published" : 1336762882,
    "_type" : "post",
    "id" : 1,
    "body" : {html},
    "source" : {markdown},
    "title" : "The power behind R2-Db.com",
    "tags": [
        {
            "ref" : {
                "$ref" : "records",
                "$id" : ObjectId("{object_id}")
            },
            "type" : "tag"
        },
        {
            "ref" : {       
                "$ref" : "profile",
                "$id" : ObjectId("{object_id}")
            }
            "type" : "author"
        }
    ]
}

This allows us to use the tagging engine to relate information in many different ways. The first tag in the tags array is a reference to the project, which is a generic "tag" type. The second tag in the set is a reference to my profile, which it automatically tagged me as the "author" type. By using different types of tags, we are able to relate information in a TON of different ways, here's an example from R2-Db.com, the Imperial Agent:

> db.records.findOne({_type: 'class'});
{
    "_created" : 1288653261,
    "_id" : ObjectId("{object_id}"),
    "_type" : "class",
    "description" : "Agents are masters of stealth, seduction, and assassination.",
    "icon" : "http://media.r2-db.com/icons/2/4ccf49cc358aa75619050000.png",
    "id" : 1,
    "name" : "Imperial Agent",
    "slug" : "imperial-agent",
    "tags" : [
        {
            "ref" : {
                "$ref" : "records",
                "$id" : ObjectId("{object_id}")
            },
            "type" : "required-faction",
            "refType" : "faction"
        },
        {
            "ref" : {
                "$ref" : "records",
                "$id" : ObjectId("{object_id}")
            },
            "type" : "starting-planet"
            "refType" : "place",
        }
    ],
}

We created a record for the Imperial Agent, and used our tagging engine to tag the "Faction" and the "Starting Planet", which actually point to the real records in the database for the Sith Empire faction and the agent's starting planet.

The possibilities of relating information this way are endless! It really lets you break down data into smaller pieces and relate them in any way/shape that you can imagine.

Future uses of the Engine

Right now I'm diving into some HTML5 game development and have been using this engine as the backend for the game. It works pretty well for simplistic online games. I've built a few demo's that I'm not ready to show off yet, but I'm sure will be revealed in the next couple of months.

We also talked about launching a Diablo 3 site, for Q&A, discussion and even for item database purposes, but it's a LOT of work to build up the communities for these sites. I don't have time for the community building aspect of these sites anymore, building and hyping R2-Db.com pre-launch was a HUGE ordeal and took a lot out of me. The other hard part is populating the database, I'm not a guy who can reverse engineer stuff, so finding the data to populate these sites was a huge problem on our end. Again, things I've learned along the way! :)

With that being said, I think I've run out of steam on this MASSIVE blog post. I'll leave it at that. I'll probably be doing a schema/mongo post with more specifics on how we built the database side of things sometime in the future, and maybe even a PHP one. If you have any requests, feel free to sound off in the comments!