So recently, I noticed that some Japanese devs are starting to use licensed game engines rather than custom game engines such as how Nintendo is using Havok for Botw, or how Shenmue 3 is using Unreal 4 engine I think....

I always had this belief that custom engines are better than licensed engines simply because said engines don't require royalty fees and can be reused and modified over again saving a lot of time and effort and a good way to port games to newer systems unlike licensed engines which not only does the opposite but just makes devs like Epic games less reason to develop games since the engines they give for rent make more money than the games themselves....

Idk..I guess I have to accept that custom engines will start to become irrelevant or something of sort.

I'm sure this question will get nabbed by the mods quick-fast, and in a hurry, but I'll still try to give you a sensible answer. 

It sounds like you're expressing an opinion more than asking a question. The fact is that engines are hard to write. For some individuals / studios, the economics are a no-brainer: using a ready-made/licensed engine can be far more cost-effective than writing a custom one.

And yet, some studios still do write their own engines for one reason or another. The decision has nothing to do with "custom engines starting to become irrelevant"; it has everything to do with cost-benefit analysis. 

You seem to have a bit of a false dichotomy in your mind. There's developing a game from scratch, not using any engine at all. There's using a temporary license for a proprietary engine. But there's also a third option: using a libre engine, like Godot Engine or Irrlicht. Then you get the benefit of not having to do everything from the ground up, but you also get the benefit of being able to continue using the engine as long as you want and even maintain it yourself if necessary.

The problem using custom engines is that you require resources in-house to maintain, develop and extend your custom engine. This is simply a cost factor where companies like Epic (Unreal Engine), Havok or Cry-Tech (Cry Engine) use there fully development resources to rapidly progress there engines; because it is there business. A good example why this might work on Unreal Engine (because they are on the market since the 90's) or it might not work (looking at Cry-Tech and Unity) is simply a good product, support and monetarization program that dosent let your customers feel like paying all there income as license fees.

Unity is a good example while Versions 3 and 4 might have worked stable more or less but up to 5.4 has had new bugs with every upgrade, this engine is in my opinion still difficult to use not for there simplicity but technical and development aspects from Unity itself. You dont know if the next update dosent crush your entire game or works as expected.

Another story is Cry-Tech that partnered with Amazon to set up Lumberyard but was more than once on the edge of insolvency. Reason? Dont know but I would assume that the docs have been not that simple or there fees might be too heavy, as I mentioned, dont know so this are guessworks.

I personally would prefer an in-house solution for most parts of our software but at the point that I worked a dozen years on the go for learning, understanding and developing game engines in several languages. This is also the reason I know that something like switching from OpenGL to Vulkan will result in getting into a new API with totally different approaches and adapt that into an existing system that worked with a different assumption on an approach how rendering works, so a company would need experts using there time on research and development of test versions to get to a working change. Same here, you would need to spend money and resources to keep your solution up to date. But still there are many companies that act on this balance between sharing resources for development of the engine and the product. Ubisoft uses its Snowdrop Engine for titles like Rainbow Six while other companies like Guerrilla Games (Sony Studio) with there title Horizon also push into the direction of using custom engine for there game.

The last point I think might take a big role in this decision process is you state as studio. Publishers do not want to pay much these days for smaller studios or indie companies that is why those would high propable not go into the direction of using in-house software rather than look for an arbitary license where more established studios or contracted ones will have the capital to do so

I think the reasoning is actually pretty straightforward. Basically, the complexity of what consumers expect in a game is continually increasing, meaning that engines also need to become increasingly complex. That means the cost of maintaining a custom engine even through a single development cycle is also going up. Meanwhile, accessible third-party engines are not only keeping up with this complexity, they're also becoming much more mature (and generally less expensive, too) than they were even a few years ago.  All in all, in a lot of cases, it just costs less time and money to use a third-party engine.

Furthermore, there's no reason I can think of that this trend won't continue. Even if a custom engine seems like a good idea for a project now, it probably won't be for some future project, so using a third party engine now means you'll already have valuable experience with the engine you will use in that future project.

Also in in favour for 3rd party engines is training.

Where with in house engines new personnel and temporary personnel constantly need to be trained in using the engine, wasting months and often leading to problems, with engines like Unreal and Unity people can train them self or go to a University for training.

Game developers use licensed engines for the same reason I use a web browser made by someone else. It's faster and easier if someone else has already done the work. In theory I could code up a functional web browser and it might even work better for me in some ways but why waste that time?

Pretty much been said already, but it's far easier to download a licensed engine and get to work instead of having to spend who knows how much time making my own engine myself. This is especially true when you're like me and have to currently do everything on your own.

11 hours ago, masskonfuzion said:

The decision has nothing to do with "custom engines starting to become irrelevant"; it has everything to do with cost-benefit analysis. 

Perfectly said.

Loads of companies still use their own custom engines. Note that this isn't "not using an engine" - they're often just as big and complex as something like Unreal or Unity

Unity and Unreal are both horribly bloated and aren't hard to beat in any specific area. Unreal's strength is that it covers so many areas that it's impossible to catch up. Unreal is also interesting that they give out the source these days, but many parts are simply too complicated to customise in a way that would typically be required, or doing so would lock you out of upgrading.... 

e.g. I've never worked on two games that have used identical rendering pipelines, but doing these changes in Unreal, while technically possible, isn't really feasible. Instead you just use what you're given.

Unreal is cheap noe for hobbyists, but it's still just as expensive as ever for big games.

I'm guessing that Nintendo has switched from internal tech to Unreal because-

1) management died and was refreshed

2) they've no experience writing modern engines at all. All they've ever done has been ad-hoc game-specific tricks and hardware that's totally different to the mainstream.

3) they're finally making games for other people's hardware now too.