This is something I have kind of avoided until now because I just did not want to bother with it, but how do I test what is the minimum android version my app should support in regards of performance? Until now I have just set minimum at the lowest and the target at the highest and hoped for the best. I understand that some players on the very low end might not be able to even play the game though because the performance will be so bad so that is really not so optimal for me.

I assume I should just go out and buy one android device of roughly each version, like one android device that is 2.3, one that is 3.0, one that is 4.0 etc. But are even devices of the same version of the same performance? Say I buy an android phone that is 4.0, can I count on it having roughly the same performance as other devices and tablets that are also 4.0?

It is all very confusing and I don't know how I can test out my app to get the performance right. Is there some kind of list of devices you should test on, on any given point in time to get the best ratio between userbase/performance to target? I assume the big companies, like Rovio and Supercell etc must have some cut of point where they decide what performance to target to maximice their userbase but at the same time not sacrifice all the eyecandy because they need to keep performance down.

Maybe even some recommendations on a list of devices to buy to test on?

How do you test and what is the lowest version you target?

Please stop posting all of your on-topic questions in our off-topic forum (The Lounge) so that they have to be moved to the correct place. Take your best guess at the correct forum (it's usually reasonably obvious) and a moderator will move them if necessary.

Moving you to Mobile and Console Development (although a case could also be made for this fitting in the Business and Law forum).

Note: I've asked you this at least three times before -- this is just a friendly reminder, but if it continues indefinitely your topics may be closed rather than moved.

Dig around on google for 2 things:

1. User stats, what devices are actually being used in your target region.

2. Testing services who can cover all, or at least most, of the devices in the list you made from the above searching.

Given how many devices there are out there, and how widely the performance and differ, it really can pay to just hire someone else to do your basic testing. Think of how much it would cost you to buy the half dozen or so most popular devices, and then think about how much it will cost you for ten hours of testing to get someone else who already owns all those devices to do an install and basic play sweep. (Pro tip: Design your application for a robust "debug" build system. If the testers can jump through to the different parts of the game from the main menu it really speeds up their testing. Especially if you run into a bug like "App crashes when doing X in level [very high number]", then they can jump right too that spot on the next build. Also means testers can confirm that the error is something with playing the level itself, rather than playing for x amount of time or something, which just happens to end up putting you in the same level consistently.)

Android 2.x and 3.x are so old that I don't feel like it is worthwhile to even mess with them. A couple of our older games at work will start target 2.3 but the Google Play breakdown is something like 97%+ Android 4 and above. Performance of the games just isn't very good since the devices are always low spec and it is just a drain on our support to have to deal with them. That is why we made a blanket rule just to support Android 4+ and iOS 6 (soon to be iOS 7).

As far as testing, it is a work place so we have the money and people to have a range of devices. We probably have 10-12 going from Android 2.3 to 5.1 (phones and tablets) and several manufactures. Games in development and beta testing will get run though a bunch of them to see if the game works alright but once the game is out then generally just the high end devices so QA can quickly do what they need to do and a few spot checks with older devices for major new features. Android is so fragmented that it is pretty much impossible to test on everything. You can pay for a service to test your game but they aren't always cheap and they won't catch everything either (but will have a better chance then you). There are some software emulators you can use that can simulate different hardware and Android versions but they don't really work that well in my experience. For a broke indie I'd say just get a Android 4/5 tablet and call it a day. The crash reporting for Google Play is pretty good (we've fixed more then one issue we couldn't repo from the crash logs) so it can give you a chance to fix things you couldn't test.

Useful numbers.

Gingerbread (2.3.x) was the first major wave of devices. Currently it has about 4% install base. Most of these are devices are three years old or older. In normal real life terms that isn't very long, but the mobile space has been rapidly changing.

Honeycomb (3.x) was the first major round of Android's Tablet-specific system. Most tablet vendors updated to Ice Cream Sandwich (4.0) so this doesn't show up on the stats.

Ice Cream Sandwich (4.0.x) has about 4% install base. It includes many early tablets, and phones made from 2011-2012.

Jelly Bean has three variants, (4.1.x, 4.2.x, 4.3) starting with phones and tablets from early 2012. With 12% market share itself, API level 16 targets about 92% of all Android devices in use. This is what many businesses choose for their target today.

You might also select an older (lower number) API version if you want to target older devices, or chose a newer API (higher number) depending on your personal needs and beliefs. If you only old an old Gingerbread phone, target that. If you only own a new Lollipop device and don't care to target older devices, or if you want to exploit some new feature, target that instead.

I think you are confusing minimum version with performance. The Android version you chose will limit what API you have access to, so unless you are using the latest and greatest( at which point you have no choice but to use the lowest version that support those features ), then you have to do the research and see what features you are using and the lowest API level that support that feature. Features introduced in a particular API level are generally available in higher API level unless the feature becomes deprecated, so targeting a lower API-level does NOT mean your app won't run on later version.

This contains a lot of useful information about Android fragmentation: http://opensignal.com/reports/2015/08/android-fragmentation/

We're about to release on Android and are support 4.0+. Version 3.* was never widely distributed, so you can pretty much pretend it doesn't exist. Version 2.* is still used by about 5% of devices but we decided to ditch it because it doesn't support setPreserveEGLContextOnPause which we're using to keep our OpenGL context alive after multitasking.

Picking an API level has little to do with dealing with low-end performance devices. That's a whole other kettle of fish. I'd use the above link and try to identify a few popular low end devices, and try to hit a variety of GPU manufacturers. Also, borrow as many devices off friends as possible. It took testing on about 20 devices before the flow of weird device specific issues dried up for me, but then I'm writing an engine from scratch. If you're using Unity or something, then you won't have to worry about all the quirky workarounds for various device specific issues.

Do you really have to worry about what build API you use? I am using v 21 now and it works on my 2.* something mobile so all API:s seems to work on all devices?

Are android version, for example 4.0 really tied to performance also though? Can't two phones that both use 4.0 have vastly different performance? It is really hard to know what device to buy.. even if I buy a couple of 4.0 devices to target that version to test and release my game on, I don't know if they are representing the general performance of 4.0?

Do you really have to worry about what build API you use? I am using v 21 now and it works on my 2.* something mobile so all API:s seems to work on all devices?

Are android version, for example 4.0 really tied to performance also though? Can't two phones that both use 4.0 have vastly different performance? It is really hard to know what device to buy.. even if I buy a couple of 4.0 devices to target that version to test and release my game on, I don't know if they are representing the general performance of 4.0?

Isn't it this why the android emulator is created for? (little offtopic, i've written the sentence correctly? I'm not a native speaker so I have some doubts :/)

With emulator not only you can test any API, but the performance offered by different models, you can select RAM, screen size, and more.

Initially API level as they already told you is not related with performance (I guess google made some improvements on performance but I seriously doubt that's the main reason for an API change). you can have a mobile with a 4 gb of RAM and android 2.3, while a low end mobile with android 5.0, it doesn't matter, so yes, two mobiles with same API can have different performance, it depends on their hardware.

Different API should have different functions, some new class and another improvements, if you want to develop a game, frob give you probably the best link you can ever have, look which API is the most used and develop for it, it doesn't have any sense being limited for a level 10 API and below when it's only installed on the 4% of mobiles.

No, the emulator doesn't match real-world performance.

In practice, even real-world devices don't have the same performance. One person can have their phone loaded up with crapware, the other lean and mean, and they'll have very different performance. Beyond that, there are cheap phones and expensive phones, high end and low end, and phones in beteween.

We've found API level is a reasonably good indicator. A 3-year-old phone is going to be far worse performance GENERALLY than a 1-year-old phone. Specific performance needs to be measured on actual devices, preferably with some in-game performance counters.