XBLA library can be redownloaded from your Download History, fwiw.
I did this but there were still a few games that seemed to be removed from Xbox's servers, permanently. I want to say Frogger was one of them but don't quote me on that... possibly Mortal Kombat 3... the trouble is that if say a developer signs a deal with say Sony or Nintendo to have all their IP exclusive to their console, then bam, there goes your ability to play them on Microsoft. not to say that that's what happened with those games but that's just one example of how you can get screwed over.
I bought Shovel Knight last year and that's a game that I particularly wish had a physical copy of.