|
Hello again! I'm back to discuss the progress on the site rebuild. For this entry I'm just going to talk a little more about the issues I uncovered while performing data migration into the shiny new database.
Before I get into too much details, I would like to say that the migration efforts are actually coming along quite nicely and I think I'm nearing the final steps of moving everything over with the games (aka games & variants) table being the last one I'm in the process of tackling.
Now let's get into the issues during the migration itself and how some of them were resolved. First off, this was a fairly big undertaking. I already talked in detail about the issues with free form fields in the data in the last dev log. All of the company, person, genres, etc have been standardized in the new table. One other free form field proved to be much more complicated - control types (controller on the game page). This was so varied it was hard to even do accurate fuzzy matching to get good results. The original plan was to try to map the listed control to an actual accessory, for instance "standard controller" would map to the Dualshock 2 controller for a PS2 game. This led to me adding a field in the database to set a default controller for the console family. This works for the most part but any specialty controllers get much harder to map without manual intervention. Furthermore, consoles like the 32X and Sega CD add-ons didn't have their own accessories - just using the ones of the genesis. That let to another table to support compatibility mapping between consoles. Of course that's not the last of controller/accessory compatibility, there are instances where some accessory only work with specific consoles within a family. One example is the TurboDuo game pad is not compatible with the original Turbografx-16, this needs another table to support mapping what accessories work on individual platforms within a console family.
This might all sound a bit frustrating or counter-intuitive but it makes the data much more robust while also eventually allowing more searching, sorting, and filtering options for the new site. Having the proper relations between all the data points is a good practice for this kind of data and will also cut down on all those errors and inconsistencies found in the existing data.
Currently, most of the data has been moved over - consoles, accessories, companies, people (shout out to ApolloBoy for helping with the person data).
I have been writing an application that will take JSON files of the data in the existing database and generating SQL queries to run in the new database. I've used it for all of the migrations so far and the game data is no exception. However, its also the most involved table. For each game or variant, up to 15 different tables need to be updated. Its a lot but ideally in the future, this will open up a lot of possibilities. For instance, we'll be able to search for all PlayStation 4 games released in the US that are 1st person and metroidvanias. Or we can find all the games that Square and Nintendo collaborated on. Further, we can add things like collection stats beyond just how many games for a platform you own. There are many more similar examples that I hope we'll be able to support in the long term.
So what's next after data is migrated? I'll be able to start putting together some back end APIs to allow access to this data and work on getting actual site mock-ups on how things can look. Once some mock-ups are created, I'll be sharing them in the discord for community feedback. Its still a little early but once some of these pieces are functional, it'll also be easier to get the community involved by verifying data and looking into proposed features. There is still work needed to migrate the forum pieces as well but I'm mostly focusing my efforts on the main collection tracking portions to start with.
I hope this helps give a glimpse into what's happening behind the scenes. If there are any questions/thoughts/concerns you can put them in the comments below or reach out to me in discord.
I have been developing a lot lately, so I'm sorry for the lack of articles. It has been really good, though. I've been getting a lot of stuff done lately, (and have encountered at least 2,519 bugs), so that's been nice. Hopefully another devlog soon.
One thing I've been doing while not developing has been playing Terraria.
Previously, I had heard of Terraria, thanks to a bunch of my friends really liking it. I had started a few worlds but only got about five minutes into them before eventually just giving up and stating that it was too similar to Minecraft.
About a week ago, however, me and my brother were doing our usual activity of him playing the Xbox 360 and me on my computer. During these he usually plays Split/Second, but a day earlier he had beaten it. After I bombarded him with literally everything happening in my life, he stood there for 2 minutes or so before asking "What's a good Xbox 360 game?".
After a bit of thinking, the only things that came to my mind were Minecraft, which he was on track to beating, Split/Second, which he had just beaten, and Terraria. Out of options, I chose Terraria. He agreed and went to get the game from my father's collection. I had no idea what I was about to get myself into.
We load it up, and I plug in the red Xbox 360 controller I usually use on my PC into the actual console it was made for. He loads up the game after spending some time making his character, and I join.
I guess at this stage in my life things are changing quickly (the repeating narrative), because both Zach and I had gotten way more out of this play session than either of us thought we were going to. We ended up playing for around two hours that day.
Since then, every time we do our daily ritual of loading up the Xbox 360, this is the game we go to instead of Minecraft or Split/Second. There's just something about this game that really clicked with me that a game really hasn't since I got Baba Is You. I have since started my own world on my Switch and play it for a bit every day.
What's even greater about Terraria in particular is that I know diddly squat about it. I like to use the internet and have been regarded as the techy know-it-all in my family. Most of the games I like end up being unfun because after having that initial rush of satisfaction with it, I look up everything there is to know about it. I have seen this happen with most of my favorites, like Minecraft and Persona 5. But since I had previously dashed off Terraria as something too similar to Minecraft, I had not really looked anything up or watched anything about it, and that lets me enjoy Terraria to the fullest extent applicable by law.
I've been koola, and I somehow managed to write code so bad that it softlocked my entire computer for three minutes straight.
(Terraria isn't 2D Minecraft; Minecraft is 3D Terraria.)
|
|
|
|
Blog Navigation
Browse Bloggers | My Blog
|
We are a community of collectors, gamers and the likes, and some of us enjoy to let the world know what is on our mind. For those members, we have the community blogs, a place where they can publish their thoughts and feelings regarding life, universe, and everything. Some of those members might even choose to write about gaming and collecting! Whatever they write about, you can find it on their blog. You can either see the latest community blog entries in the feed you see to the left, or you can browse for your favorite blog using the menu above. Interested in having your own blog hosted on RF Generation? It's rather simple, first be a registered member, and then click the "My Blog" link that you see in the navigation above. Following those two steps will certainly get you on your way to blogging.
Sit back, relax, and enjoy our entries, rantings, and completely unrelated series of thoughts. We write for you to read, so we certainly hope that you enjoy our material.
|
|
Hot Community Entries
|
|
Hot RFG Blog Entries
| |
|
