Hello again! I'm back to discuss the progress on the site rebuild. For this entry I'm just going to talk a little more about the issues I uncovered while performing data migration into the shiny new database.
Before I get into too much details, I would like to say that the migration efforts are actually coming along quite nicely and I think I'm nearing the final steps of moving everything over with the games (aka games & variants) table being the last one I'm in the process of tackling.
Now let's get into the issues during the migration itself and how some of them were resolved. First off, this was a fairly big undertaking. I already talked in detail about the issues with free form fields in the data in the last dev log. All of the company, person, genres, etc have been standardized in the new table. One other free form field proved to be much more complicated - control types (controller on the game page). This was so varied it was hard to even do accurate fuzzy matching to get good results. The original plan was to try to map the listed control to an actual accessory, for instance "standard controller" would map to the Dualshock 2 controller for a PS2 game. This led to me adding a field in the database to set a default controller for the console family. This works for the most part but any specialty controllers get much harder to map without manual intervention. Furthermore, consoles like the 32X and Sega CD add-ons didn't have their own accessories - just using the ones of the genesis. That let to another table to support compatibility mapping between consoles. Of course that's not the last of controller/accessory compatibility, there are instances where some accessory only work with specific consoles within a family. One example is the TurboDuo game pad is not compatible with the original Turbografx-16, this needs another table to support mapping what accessories work on individual platforms within a console family.
This might all sound a bit frustrating or counter-intuitive but it makes the data much more robust while also eventually allowing more searching, sorting, and filtering options for the new site. Having the proper relations between all the data points is a good practice for this kind of data and will also cut down on all those errors and inconsistencies found in the existing data.
Currently, most of the data has been moved over - consoles, accessories, companies, people (shout out to ApolloBoy for helping with the person data).
I have been writing an application that will take JSON files of the data in the existing database and generating SQL queries to run in the new database. I've used it for all of the migrations so far and the game data is no exception. However, its also the most involved table. For each game or variant, up to 15 different tables need to be updated. Its a lot but ideally in the future, this will open up a lot of possibilities. For instance, we'll be able to search for all PlayStation 4 games released in the US that are 1st person and metroidvanias. Or we can find all the games that Square and Nintendo collaborated on. Further, we can add things like collection stats beyond just how many games for a platform you own. There are many more similar examples that I hope we'll be able to support in the long term.
So what's next after data is migrated? I'll be able to start putting together some back end APIs to allow access to this data and work on getting actual site mock-ups on how things can look. Once some mock-ups are created, I'll be sharing them in the discord for community feedback. Its still a little early but once some of these pieces are functional, it'll also be easier to get the community involved by verifying data and looking into proposed features. There is still work needed to migrate the forum pieces as well but I'm mostly focusing my efforts on the main collection tracking portions to start with.
I hope this helps give a glimpse into what's happening behind the scenes. If there are any questions/thoughts/concerns you can put them in the comments below or reach out to me in discord.
[img width=400 height=556]https://imgur.com/E6OwyFF.png[/img]
As you many of you know, our site is currently conducting its annual fund drive to help with the server costs for 2021. So far, we have had a good turnout and I am happy to report that with our donors' generous pledges, we are at 74% of our $1,500.00 goal! This is the last week for donations, and we are SO close to reaching our goal! (you can donate here: http://www.rfgeneration.c...m/index.php?action=paypal), so be sure to donate during this time to get in on the raffle of great prizes donated by our staff. You can now find a listing of all of the raffle prize lots HERE. However, the staff at RF Generation would first like to take a moment to thank those who have donated since the last update:
St0rmTK421
ericeskapade
toze3
blcklblskt
gamereviewgod
singlebanana
Thank you all for your support! Please remember to PM me the email address you donated with, so that we can properly thank you during our weekly updates and provide you with the proper number of raffle tickets. If you donated recently and do not see your name, you may have donated after we pulled numbers. If you don't see it, just send me a PM and I will check on your donation status.
Grand total raised so far: $1,110.00
[img width=400 height=556]https://i.imgur.com/G2hxFpR.png[/img]
As you may of you know, our site is currently conducting its annual fund drive to help with the server costs for 2020. So far, we have had a good turnout and I am happy to report that with our donors' generous pledges, we are just over 77% of our goal! This is the last week for donations, and we are SO close to reaching our goal! (you can donate here: http://www.rfgeneration.c...m/index.php?action=paypal), so be sure to donate during this time to get in on the raffle of great prizes donated by our staff. You can now find a listing of all of the raffle prize lots HERE. However, the staff at RF Generation would first like to take a moment to thank those who have donated since the last update:
shaggy
Pam
Cowtipp Neo
Socialiste
Bear78
MaHaBe
man99helm99
Cobra Blade
St0rmTK421
Marriot_Guy
ericeskapade
Razor Knuckles
Verniet_NH
We also received a very special donation from site member, Mrs C. For those of you who do not know, Mrs C is the mother of the late Michael Collins. I was very touched by the message she sent with her donation, and to paraphrase, she was very appreciative of us keeping her son's site going. If you are unfamiliar with Michael, I suggest you check out the "Michael Collins" tab in our "About" section. We owe a lot to him for putting this great community together.
Thank you all for your support! Please remember to PM me the email address you donated with, so that we can properly thank you during our weekly updates and provide you with the proper number of raffle tickets. If you donated recently and do not see your name, you may have donated after we pulled numbers. If you don't see it, just send me a PM and I will check on your donation status.
Grand total raised so far: $1,159.57
Today is the last day members! We have had several donations over the past week, but as of yesterday, we are still behind in reaching our goal. With our donors' generous pledges, we are just over 71% of our goal! Please donate now to help us reach this goal! (you can donate here now: http://www.rfgeneration.c...m/index.php?action=paypal), so be sure to donate today to get in on the raffle of great prizes donated by our staff. You can check out a list of all the raffle prize lots HERE.
The staff at RF Generation would first like to take a moment to thank those who have donated since the last update:
cowa11
Lazyperson
Zagnorch
Raidou
daraja
OatBob
Duke.Togo
Thanks to all who donated! Please remember to PM me the email address you donated with, so that we can properly thank you when we reveal the final results and I can also provide you with the proper number of raffle tickets. If you donated recently and do not see your name, you may have donated after I pulled numbers. If you don't see it, PLEASE send me a PM and I will check on your donation status. Please remember that RF Generation takes donations throughout the year, so if now is not a good time to donate, you can do so down the road.
Grand total raised so far: $1,068.00
This is the last week of our annual fund drive to help with the server costs for 2019. So far, we have had a decent turnout and I am happy to report that with our donors' generous pledges, we are just under 59% of our goal! We will be taking donations for the fund drive through February 28th (you can donate here: http://www.rfgeneration.c...m/index.php?action=paypal), so be sure to donate during this time to get in on the raffle of great prizes donated by our staff. You can now find a listing of all of the raffle prize lots HERE. The staff at RF Generation would first like to take a moment to thank those who have donated since the last update:
Cowtipp Neo
Goodwill_gamer
Napoli
toze3
monkees19
Marriot_Guy
DrunkenAce
douglie007
blcklblskt
singlebanana
Thank you all! Please remember to PM me the email address you donated with, so that we can properly thank you during our weekly updates and provide you with the proper number of raffle tickets. If you donated recently and do not see your name, you may have donated after we pulled numbers. If you don't see it, just send me a PM and I will check on your donation status.
Grand total raised so far: $883.00
As you may of you know, our site is currently conducting its annual fund drive to help with the server costs for 2019. So far, we have had a good turnout and I am happy to report that with our donors' generous pledges, we are just under 33% of our goal! We will be taking donations for the fund drive through the month of February (you can donate here: http://www.rfgeneration.c...m/index.php?action=paypal), so be sure to donate during this time to get in on the raffle of great prizes donated by our staff. You can now find a listing of all of the raffle prize lots HERE. However, the staff at RF Generation would first like to take a moment to thank those who have donated since the last update:
zophar53
shaggy
drag0nsfyre
Cobra Blade
FireStar
Thank you all! Please remember to PM me the email address you donated with, so that we can properly thank you during our weekly updates and provide you with the proper number of raffle tickets. If you donated recently and do not see your name, you may have donated after we pulled numbers. If you don't see it, just send me a PM and I will check on your donation status.
Grand total raised so far: $493.00
[img width=500 height=400]https://i.makeagif.com/media/3-10-2015/_Pg4Dg.gif[/img]
I am pleased to report that with the generous donations of site members and friends, we obliterated our 2018 fund drive goal! We had an incredible amount of support on our final day and ended at just over 125% of our goal! A HUGE thanks to all of you who supported us in this effort. It was so great getting messages from users who I've never spoken to before that love our site, hearing from old members who popped in to donate and catch up, and seeing the outpouring of love and support over social media this year from many of you. Thank you. Thank you so much for your support of RF Generation!
Please remember that the raffle is just around the corner and I will be posting a video of the drawing on the front page in the upcoming weeks. You can check out a list of all the raffle prize lots HERE.
The staff at RF Generation would first like to take a moment to thank those who have donated since our last update:
MustyHobbit
Wempster
monkees19
@ThePockyX
Crockett
Bearded Gaming Company
Tynstar
Deadman
socialiste
man99helm99
TraderJake
bickman2k
DatenshiBlue
nupoile
If you do not see your name in this list of any of the earlier updates, please PM singlebanana to ensure that your donation was received.
Grand total raised: $1880.00
[img width=596 height=440]https://3.bp.blogspot.com/-6IWVVs-s9b4/WV0jzk5P0LI/AAAAAAAABio/6SqWsKFUMrgPnub76ofCiloB8GIMLzC-wCEwYBhgL/s1600/uhf4.png[/img]
Today is the last day members! We have had a great flood of donations over the past week, but as of yesterday, we are still behind in reaching our goal. With our donors' generous pledges, we are just over 95% of our goal! WE ARE SO CLOSE! Please donate now and help us shatter this goal! (you can donate here now: http://www.rfgeneration.c...m/index.php?action=paypal), so be sure to donate during this time to get in on the raffle of great prizes donated by our staff. You can check out a list of all the raffle prize lots HERE.
The staff at RF Generation would first like to take a moment to thank those who have donated since last week's update:
BuriedOnMars
Abs0luteKa0s
blcklblskt
erikeskapade
NeoMagicWarrior
Pam
Boshamp
Skyrimsamurai
shaggy
DanAdamKOF
Raidou
Marriot_Guy
Misto
neskid
Piles
jaredbat
toze3
neimad
zophar53
A. Westover (podcast listener!)
Thanks to all who donated! Please remember to PM me the email address you donated with, so that we can properly thank you when we reveal the final results and I can also provide you with the proper number of raffle tickets. If you donated recently and do not see your name, you may have donated after I pulled numbers. If you don't see it, PLEASE send me a PM and I will check on your donation status.
Grand total raised so far: $1437.00
We are now less than a week away from the end of the month! After tallying up recent donations, it appears that we are still behind in reaching our goal. With our donors' generous pledges, we are just over 63% of our goal! We will be taking donations for the fund drive through February 28th (you can donate here: http://www.rfgeneration.c...m/index.php?action=paypal), so be sure to donate during this time to get in on the raffle of great prizes donated by our staff. You can check out a list of all the raffle prize lots HERE.
The staff at RF Generation would first like to take a moment to thank those who have donated since last week's update:
Bear78
EngineerMike
Cowtipp Neo
98pacecar
thegirlg33k
slackur
daraja
Wylaf Beulbe
ErbBetaPatched
Silver80
Duke0619
Duke.Togo
Thanks to all who donated! Please remember to PM me the email address you donated with, so that we can properly thank you during our weekly updates and provide you with the proper number of raffle tickets. If you donated recently and do not see your name, you may have donated after I pulled numbers. If you don't see it, PLEASE send me a PM and I will check on your donation status.
Grand total raised so far: $952.00
The RF Generation annual fund drive is moving along and we are now at the midway point of the month. After tallying up recent donations, it appears that we are a little behind in reaching our goal. With our donors' generous pledges, we are just over 39% of our goal! We will be taking donations for the fund drive through the month of February (you can donate here: http://www.rfgeneration.c...m/index.php?action=paypal), so be sure to donate during this time to get in on the raffle of great prizes donated by our staff. You can check out a list of all the raffle prize lots HERE.
The staff at RF Generation would first like to take a moment to thank those who have donated since last week's update:
douglie007
Marriot_Guy
Goodwill_gamer
Tempest242
singlebanana
Guile
gamereviewgod
St0rmTK421
Thanks to all who donated! Please remember to PM me the email address you donated with, so that we can properly thank you during our weekly updates and provide you with the proper number of raffle tickets. If you donated recently and do not see your name, you may have donated after I pulled numbers. If you don't see it, PLEASE send me a PM and I will check on your donation status.
Grand total raised so far: $592.00
As you may of you know, our site is currently conducting its annual fund drive to help with the server costs for 2018. So far, we have had a good turnout and I am happy to report that with our donors' generous pledges, we are just over 18% of our goal! We will be taking donations for the fund drive through the month of February (you can donate here: http://www.rfgeneration.c...m/index.php?action=paypal), so be sure to donate during this time to get in on the raffle of great prizes donated by our staff. You can now find a listing of all of the raffle prize lots HERE. However, the staff at RF Generation would first like to take a moment to thank those who have donated so far:
Voodoo Monkey
Socialiste
disrupture
Bear78
NES_Rules
Undergarmentality
Wempster
bsr241
TheAncientHero
NMErickson
FatherJack
drag0nsfyre
Razor Knuckles
Gamer4Lyfe
Thank you all! Please remember to PM me the email address you donated with, so that we can properly thank you during our weekly updates and provide you with the proper number of raffle tickets. If you donated recently and do not see your name, you may have donated after we pulled numbers. If you don't see it, just send me a PM and I will check on your donation status.
Grand total raised so far: $280.00
I am very happy to report that in the final days of the donation drive, we were able to surpass our $1500 goal. With everyone's generous donations, we raised an incredible $1551 total! This means great things for our site, as it will keep our server's going and allow us to do some special things to help promote our site and improve your experience here. Thank you to all who donated in the final days of the fund drive and to all of you for your support! This is such a fantastic community to be a part of and I am glad that we can continue to help it grow.
A BIG thanks to all of the members who have donated since our last update!:
zophar53
G. Franzen
Fokakis79
techwizard
FatherJack
Duke.Togo
nupoile**
Grand total raised: $1,551.00
Continue reading 2017 Donation Drive - Final Update & Raffle
Since our last update, we have seen an overwhelming amount of generosity from our members. I am incredibly proud to report that with the recent influx of donations, we are now at 95% of our goal! There are only two more days in February, so please help us eclipse this goal (you can donate here: http://www.rfgeneration.c...m/index.php?action=paypal). In order to push us through the finish line, we are giving away two Steam codes to the donor who breaks the $1500 mark! Even if you have already donated, you can donate again and be eligible for these Steam codes.
Also, be sure to donate so that you can get in on the raffle of great prizes donated by our staff. You can find a listing of all the raffle prize lots HERE. A BIG thanks to all of the members who have donated since our last update!:
toze3
Guile
douglie007
Duke.Togo
TheAncientHero
Soia11
NeoMagicWarrior
Sigoya
sydpinkfloyd
Pam
98PaceCar
flyingclimber
Socialiste
bronson66
Lazyperson
Nik the Russian
EngineerMike
Wylaf Beulbe
phalnx
starfox387
neimad
Ze_ro
T. Trycieki
beakerman
Raidou
Thank you all! Please remember to PM me with your screen name and the email address you donated with, (or include your handle with your donation) so that we can properly thank you during our weekly updates and provide you with the proper number of raffle tickets. If you donated recently and do not see your name, you may have donated after we pulled numbers. If you don't see it, just send me a PM and I will check on your donation status.
Grand total raised so far: $1,431.00
RF Generation is currently holding its annual fund drive to help with the server costs for 2017. So far, we have had a good turnout and I am happy to report that with our donors' generous pledges, we are at 54% of our goal! There's only one more week left in February, so if you are considering donating, now is the time (you can donate here: http://www.rfgeneration.c...m/index.php?action=paypal). Be sure to donate so that you can get in on the raffle of great prizes donated by our staff. You can find a listing of all the raffle prize lots HERE. A BIG thanks to all of the members who have donated since our last update!:
bsr241
The Burnin8or
erikescapade
Marriott_Guy
Wempster
Thank you all! Please remember to PM me with your screen name and the email address you donated with, (or include your handle with your donation) so that we can properly thank you during our weekly updates and provide you with the proper number of raffle tickets. If you donated recently and do not see your name, you may have donated after we pulled numbers. If you don't see it, just send me a PM and I will check on your donation status.
Grand total raised so far: $809.00
RF Generation is currently holding its annual fund drive to help with the server costs for 2017. So far, we have had a good turnout and I am happy to report that with our donors' generous pledges, we are nearly 43% of our goal with almost half of the month over! We will be taking donations for the fund drive through the month of February (you can donate here: http://www.rfgeneration.c...m/index.php?action=paypal), so be sure to donate during this time to get in on the raffle of great prizes donated by our staff. You can find a listing of all the raffle prize lots HERE. A BIG thanks to all of the members who have donated since our last update!:
Chris Roberts (N.A.R.C. podcast)
GameReviewGod
Gamer4Lyfe
Goodwill_Gamer
InvadErGII
Malygris
Marcos 'Janauba'
NMErickson
Razor Knuckles
shaggy
StOrmTK421
Voodoo Monkey
Thank you all! Please remember to PM me with your screen name and the email address you donated with, (or include your handle with your donation) so that we can properly thank you during our weekly updates and provide you with the proper number of raffle tickets. If you donated recently and do not see your name, you may have donated after we pulled numbers. If you don't see it, just send me a PM and I will check on your donation status.
Grand total raised so far: $639.00
|
