If you truly have "all the time in the world", you could indeed pull it off.
But that is precisely the main problem you'd face. If you want to use a small budget, that means most likely working alone or with a very small team, which will inevitably slow down the production.
Sure, you may be willing to invest 10 or 20 years in your project, but by the time you are done, the industry moved on, trends changed, your graphics are outdated, etc... it may even be that someone else beat you to the punch and made something really similar but much quicker.
Well, that's just my take on it. Despite all the cheap/free tools available nowadays that massively speed things up, creating a game is still quite a laborious process.